From owner-freebsd-scsi@FreeBSD.ORG Tue Jun 4 11:07:55 2013 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id F1F8150A; Tue, 4 Jun 2013 11:07:54 +0000 (UTC) (envelope-from zi@FreeBSD.org) Received: from exodus.zi0r.com (exodus.zi0r.com [71.245.171.203]) by mx1.freebsd.org (Postfix) with ESMTP id AFCCE1AC4; Tue, 4 Jun 2013 11:07:54 +0000 (UTC) Received: from exodus.zi0r.com (localhost [127.0.0.1]) by exodus.zi0r.com (Postfix) with ESMTP id ADF4F3A097; Tue, 4 Jun 2013 07:07:53 -0400 (EDT) X-Virus-Scanned: amavisd-new at zi0r.com Received: from exodus.zi0r.com ([127.0.0.1]) by exodus.zi0r.com (exodus.zi0r.com [127.0.0.1]) (amavisd-new, port 10026) with LMTP id jjj5niJwGYK7; Tue, 4 Jun 2013 07:07:51 -0400 (EDT) Received: from exodus.zi0r.com (syn.zi0r.com [71.245.171.202]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by exodus.zi0r.com (Postfix) with ESMTPSA id 701E23A094; Tue, 4 Jun 2013 07:07:51 -0400 (EDT) Date: Tue, 4 Jun 2013 07:07:50 -0400 From: Ryan Steinmetz To: Steven Hartland Subject: Re: kern/179118: [mfi] COMMAND 0x.. TIMEOUT AFTER ## SECONDS (Dell H710 Mini (blades)) Message-ID: <20130604110749.GA46145@exodus.zi0r.com> References: <201306032130.r53LU1XU042389@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <201306032130.r53LU1XU042389@freefall.freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jun 2013 11:07:55 -0000 I attached a serial console to the box and got the following: mfi0: COMMAND 0xffffff8002b91178 TIMEOUT AFTER 41 SECONDS mfi0: I/O error, cmd=0xffffff8002b91178, error=0x3c mfid0: hard error cmd=write 61548002-61548065 mfi0: g_vfs_done():COMMAND 0xffffff8002b91068 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=31512494080, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b91068, error=0x3c mfid0: hard error cmd=write 184643042-184643105 mfi0: g_vfs_done():COMMAND 0xffffff8002b90760 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=94537154560, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b90760, error=0x3c mfid0: hard error cmd=write 1500221282-1500221345 mfi0: g_vfs_done():COMMAND 0xffffff8002b90ed0 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=768113213440, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b90ed0, error=0x3c mfid0: hard error cmd=write 1722048802-1722048865 mfi0: g_vfs_done():COMMAND 0xffffff8002b91288 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=881688903680, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b91288, error=0x3c mfid0: hard error cmd=write 1722048866-1722048929 mfi0: g_vfs_done():COMMAND 0xffffff8002b91310 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=881688936448, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b91310, error=0x3c mfid0: hard error cmd=write 1722049442-1722049505 mfi0: g_vfs_done():COMMAND 0xffffff8002b91398 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=881689231360, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b91398, error=0x3c mfid0: hard error cmd=write 1722049506-1722049569 mfi0: g_vfs_done():COMMAND 0xffffff8002b915b8 TIMEOUT AFTER 41 SECONDS mfid0p2[WRITE(offset=881689264128, length=32768)]mfi0: error = 60 I/O error, cmd=0xffffff8002b915b8, error=0x3c mfid0: hard error cmd=write 1722050146-1722050209 g_vfs_done():mfid0p2[WRITE(offset=881689591808, length=32768)]error = 60 mfi0: COMMAND 0xffffff8002b914a8 TIMEOUT AFTER 40 SECONDS mfi0: I/O error, cmd=0xffffff8002b914a8, error=0x3c mfid0: hard error cmd=write 1722219234-1722219281 mfi0: g_vfs_done():COMMAND 0xffffff8002b916c8 TIMEOUT AFTER 30 SECONDS mfid0p2[WRITE(offset=881776164864, length=24576)]mfi0: error = 60 I/O error, cmd=0xffffff8002b916c8, error=0x3c mfid0: hard error cmd=write 1722093946-1722093953 g_vfs_done():mfid0p2[WRITE(offset=881712017408, length=4096)]error = 60 mfi0: cmd_tbolt 0xfffffe004a99a100 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a99a200 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a973e00 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a99a380 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a99a000 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a97be00 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a97bd80 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a97bb80 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a97bc80 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a97ba80 has invalid sync_cmd_idx=128 - skipping mfi0: COMMAND 0xffffff8002b91178 TIMEOUT AFTER 30 SECONDS mfi0: I/O error, cmd=0xffffff8002b91178, error=0x3c mfid0: hard error cmd=write 40610-40657 mfi0: g_vfs_done():COMMAND 0xffffff8002b91068 TIMEOUT AFTER 30 SECONDS mfid0p2[WRITE(offset=20709376, length=24576)]mfi0: error = 60 I/O error, cmd=0xffffff8002b91068, error=0x3c mfid0: hard error cmd=write 290-297 g_vfs_done():mfid0p2[WRITE(offset=65536, length=4096)]error = 60 /dev: got error 60 while accessing filesystem panic: softdep_deallocate_dependencies: unrecovered I/O error cpuid = 22 KDB: stack backtrace: #0 0xffffffff80920c76 at kdb_backtrace+0x66 #1 0xffffffff808eac8e at panic+0x1ce #2 0xffffffff80b09100 at clear_remove+0 #3 0xffffffff809670a0 at brelse+0x60 #4 0xffffffff80969a48 at bufdone+0x68 #5 0xffffffff808561b6 at g_io_schedule_up+0xa6 #6 0xffffffff8085673c at g_up_procbody+0x5c #7 0xffffffff808bbdbf at fork_exit+0x11f #8 0xffffffff80bc3a5e at fork_trampoline+0xe Uptime: 2h0m19s Dump failed. Partition too small. mfi0: cmd_tbolt 0xfffffe004a99a100 has invalid sync_cmd_idx=128 - skipping mfi0: cmd_tbolt 0xfffffe004a99a200 has invalid sync_cmd_idx=128 - skipping Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, --> or switch off the system now. Rebooting... cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 22