From owner-freebsd-hardware@FreeBSD.ORG Mon Nov 9 15:48:55 2009 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 368FF1065670 for ; Mon, 9 Nov 2009 15:48:55 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from mx04.pub.collaborativefusion.com (mx04.pub.collaborativefusion.com [206.210.72.84]) by mx1.freebsd.org (Postfix) with ESMTP id 013AD8FC18 for ; Mon, 9 Nov 2009 15:48:54 +0000 (UTC) Received: from [192.168.2.164] ([206.210.89.202]) by mx04.pub.collaborativefusion.com (StrongMail Enterprise 4.1.1.4(4.1.1.4-47689)); Mon, 09 Nov 2009 10:48:54 -0500 X-VirtualServerGroup: Default X-MailingID: 00000::00000::00000::00000::::262 X-SMHeaderMap: mid="X-MailingID" X-Destination-ID: freebsd-hardware@freebsd.org X-SMFBL: ZnJlZWJzZC1oYXJkd2FyZUBmcmVlYnNkLm9yZw== Message-ID: <4AF839E5.10303@comcast.net> Date: Mon, 09 Nov 2009 10:48:53 -0500 From: Steve Polyack User-Agent: Thunderbird 2.0.0.23 (X11/20090902) MIME-Version: 1.0 To: freebsd-hardware@freebsd.org, User Questions Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: mfi(4) lockups and the adapter event log X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 15:48:55 -0000 Hello, We saw an odd crash (more of a lockup) on one of our Dell's with a PERC5/i RAID controller (running 6.3-RELEASE-p10): mfi0: COMMAND 0xffffffff89dab0e8 TIMEOUT AFTER 31 SECONDS (repeated many times with different commands) mfi0: 3325 (310696326s/0x0020/4) - Type 18: Fatal firmware error: Line 1091 in ../../raid/verdeMain.c After some troubleshooting with Dell, they determined that there was no problem. An OpenManage live CD was used to run their diagnostic utilities, which turned up nothing. However, after rebooting the system and checking dmesg(8), it could be seen that the adapter was doing some weird things during initialization. It seemed to be going through the whole initialization cycle (Firmware initialization started....) up to five times. This does not jive with our other mfi/PERC5 systems. In addition to the multiple init cycles it was also printing weeks worth of messages from the adapter's event log (mostly battery relearns). Checking the event log with linux-megacli showed TONS of messages. Trying to clear them using linux-megacli seemed to cause a similar lockup, filled with command timeouts, but no fatal firmware error. We ended up clearing the event log using a linux live CD and the linux native MegaCLI binaries. The system hasn't locked up again since, but we're not sure what caused it in the first place. Has anyone else seen any kind of oddities with the PERC series of controllers and the adapter's event log? Does it just fill up after a while and start to cause trouble? I seem to remember a similar post recommending a cleaning of the event logs every so often. Thanks, Steve