From owner-freebsd-current@freebsd.org Sun Nov 13 21:57:40 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2092CC3FFC4; Sun, 13 Nov 2016 21:57:40 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 43AD0198A; Sun, 13 Nov 2016 21:57:38 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id XAA01399; Sun, 13 Nov 2016 23:57:31 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c62mI-000Bew-Uf; Sun, 13 Nov 2016 23:57:30 +0200 Subject: Re: firewire panic To: FreeBSD Current , FreeBSD Hackers References: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> From: Andriy Gapon Message-ID: Date: Sun, 13 Nov 2016 23:56:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <91a1440d-14c7-2cc6-6cbb-2b62bfd2c27d@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Nov 2016 21:57:40 -0000 On 11/11/2016 14:25, Andriy Gapon wrote: > panic: mutex sbp not owned at /usr/src/sys/dev/firewire/sbp.c:967 > cpuid = 2 > curthread: 0xfffff8000ada5000 > stack: 0xfffffe0504ded000 - 0xfffffe0504df1000 > stack pointer: 0xfffffe0504df0a00 > KDB: stack backtrace: > db_trace_self_wrapper() at 0xffffffff80420bbb = db_trace_self_wrapper+0x2b/frame > 0xfffffe0504df0930 > kdb_backtrace() at 0xffffffff80670359 = kdb_backtrace+0x39/frame 0xfffffe0504df09e0 > vpanic() at 0xffffffff8063986c = vpanic+0x14c/frame 0xfffffe0504df0a20 > panic() at 0xffffffff806395b3 = panic+0x43/frame 0xfffffe0504df0a80 > __mtx_assert() at 0xffffffff8061c40d = __mtx_assert+0xed/frame 0xfffffe0504df0ac0 > sbp_cam_scan_lun() at 0xffffffff80474667 = sbp_cam_scan_lun+0x37/frame > 0xfffffe0504df0af0 > xpt_done_process() at 0xffffffff802aacfa = xpt_done_process+0x2da/frame > 0xfffffe0504df0b30 > xpt_done_td() at 0xffffffff802ac2e5 = xpt_done_td+0xd5/frame 0xfffffe0504df0b80 So, it's pretty obvious that the sbp mutex can not be held when sbp_cam_scan_lun() is called. After removing the assertion, just to move further, I do not get any panics and can access the disk. But I see many witness warnings. Some examples: bus_dma_tag_create with the following non-sleepable locks held: exclusive sleep mutex sbp (sbp) r = 0 (0xfffff8000ff04f48) locked @ /usr/src/sys/modules/firewire/sbp/../../../dev/firewire/sbp.c:802 stack backtrace: #0 0xffffffff80ab20a0 at witness_debugger+0x70 #1 0xffffffff80ab3387 at witness_warn+0x3d7 #2 0xffffffff810174dd at bus_dma_tag_create+0x3d #3 0xffffffff83703402 at fwdma_malloc+0x82 #4 0xffffffff836ed759 at sbp_post_explore+0x849 #5 0xffffffff836f9e76 at fw_bus_probe_thread+0x906 #6 0xffffffff80a164c4 at fork_exit+0x84 #7 0xffffffff80ea970e at fork_trampoline+0xe bus_dmamem_alloc with the following non-sleepable locks held: exclusive sleep mutex sbp (sbp) r = 0 (0xfffff8000ff04f48) locked @ /usr/src/sys/modules/firewire/sbp/../../../dev/firewire/sbp.c:802 stack backtrace: #0 0xffffffff80ab20a0 at witness_debugger+0x70 #1 0xffffffff80ab3387 at witness_warn+0x3d7 #2 0xffffffff810175c3 at bus_dmamem_alloc+0x33 #3 0xffffffff83703424 at fwdma_malloc+0xa4 #4 0xffffffff836ed759 at sbp_post_explore+0x849 #5 0xffffffff836f9e76 at fw_bus_probe_thread+0x906 #6 0xffffffff80a164c4 at fork_exit+0x84 #7 0xffffffff80ea970e at fork_trampoline+0xe bus_dmamap_create with the following non-sleepable locks held: exclusive sleep mutex sbp (sbp) r = 0 (0xfffff8000ff04f48) locked @ /usr/src/sys/modules/firewire/sbp/../../../dev/firewire/sbp.c:802 stack backtrace: #0 0xffffffff80ab20a0 at witness_debugger+0x70 #1 0xffffffff80ab3387 at witness_warn+0x3d7 #2 0xffffffff8101755f at bus_dmamap_create+0x2f #3 0xffffffff836ed7d2 at sbp_post_explore+0x8c2 #4 0xffffffff836f9e76 at fw_bus_probe_thread+0x906 #5 0xffffffff80a164c4 at fork_exit+0x84 #6 0xffffffff80ea970e at fork_trampoline+0xe lock order reversal: 1st 0xfffff8000ff04f48 sbp (sbp) @ /usr/src/sys/kern/kern_mutex.c:220 2nd 0xfffffe0001b94870 firewire (firewire) @ /usr/src/sys/modules/firewire/firewire/../../../dev/firewire/firewire.c:302 stack backtrace: #0 0xffffffff80ab20a0 at witness_debugger+0x70 #1 0xffffffff80ab1f94 at witness_checkorder+0xe54 #2 0xffffffff80a33854 at __mtx_lock_flags+0xa4 #3 0xffffffff836f6423 at fw_asyreq+0x2d3 #4 0xffffffff80a68f2c at softclock_call_cc+0x19c #5 0xffffffff80a69327 at softclock+0x47 #6 0xffffffff80a18d76 at intr_event_execute_handlers+0x96 #7 0xffffffff80a193f6 at ithread_loop+0xa6 #8 0xffffffff80a164c4 at fork_exit+0x84 #9 0xffffffff80ea970e at fork_trampoline+0xe lock order reversal: 1st 0xfffff8000ff04f48 sbp (sbp) @ /usr/src/sys/kern/kern_mutex.c:220 2nd 0xfffff8000a0b4460 CAM device lock (CAM device lock) @ /usr/src/sys/cam/scsi/scsi_xpt.c:2349 stack backtrace: #0 0xffffffff80ab20a0 at witness_debugger+0x70 #1 0xffffffff80ab1f94 at witness_checkorder+0xe54 #2 0xffffffff80a33854 at __mtx_lock_flags+0xa4 #3 0xffffffff8031af12 at scsi_scan_lun+0x122 #4 0xffffffff836eef40 at sbp_cam_scan_target+0x100 #5 0xffffffff80a68f2c at softclock_call_cc+0x19c #6 0xffffffff80a69327 at softclock+0x47 #7 0xffffffff80a18d76 at intr_event_execute_handlers+0x96 #8 0xffffffff80a193f6 at ithread_loop+0xa6 #9 0xffffffff80a164c4 at fork_exit+0x84 #10 0xffffffff80ea970e at fork_trampoline+0xe -- Andriy Gapon