From owner-freebsd-current@freebsd.org Mon Aug 10 00:24:16 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3858999E84A for ; Mon, 10 Aug 2015 00:24:16 +0000 (UTC) (envelope-from tim@kientzle.com) Received: from monday.kientzle.com (kientzle.com [142.254.26.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E9E0235E for ; Mon, 10 Aug 2015 00:24:15 +0000 (UTC) (envelope-from tim@kientzle.com) Received: (from root@localhost) by monday.kientzle.com (8.14.4/8.14.4) id t7A0OaUi010990; Mon, 10 Aug 2015 00:24:36 GMT (envelope-from tim@kientzle.com) Received: from [192.168.2.100] (192.168.1.101 [192.168.1.101]) by kientzle.com with SMTP id ibh6vx8yxkeimubhk6d4gtk3ew; Mon, 10 Aug 2015 00:24:36 +0000 (UTC) (envelope-from tim@kientzle.com) Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\)) Subject: Re: Consistent crash of BeagleBone kernel From: Tim Kientzle In-Reply-To: <20150809181004.GK2072@kib.kiev.ua> Date: Sun, 9 Aug 2015 17:24:13 -0700 Cc: FreeBSD Current Message-Id: References: <20150809064704.GH2072@kib.kiev.ua> <350C3412-910F-4B08-A903-A51D18474DAF@kientzle.com> <20150809181004.GK2072@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.2102) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2015 00:24:16 -0000 > On Aug 9, 2015, at 11:10 AM, Konstantin Belousov = wrote: >=20 > On Sun, Aug 09, 2015 at 10:53:20AM -0700, Tim Kientzle wrote: >>=20 >> I suspect the LOR is new. >>=20 >> It looks like the panic is occurring when WITNESS tries to print the = backtrace for the LOR. I???m not familiar with that code; does it use = the kernel linker? >>=20 >=20 > It indeed locks the linker lock to resolve symbols. So it seems to be > even more useful to make the linker lock recursive locally, then you > should be able to see the backtrace for LOR. Changing the kld_sx lock to recursive, I now see a backtrace for the = ufs/kernel linker LOR. Full trace pasted below. This is displayed just before the network interfaces; I suspect it=E2=80=99= s being triggered when my startup initializes the urtwn wireless adapter = (which does indeed load a number of kernel modules). lock order reversal: 1st 0xc083ef40 kernel linker (kernel linker) @ = /Users/tim/projects/crochet/src-head/sys/kern/kern_linker.c:1030 2nd 0xc2d63c94 ufs (ufs) @ = /Users/tim/projects/crochet/src-head/sys/kern/vfs_lookup.c:529 KDB: stack backtrace: db_trace_self() at db_trace_self pc =3D 0xc0627ec0 lr =3D 0xc023fd34 = (db_trace_self_wrapper+0x30) sp =3D 0xde2be720 fp =3D 0xde2be838 r10 =3D 0xc069f42b db_trace_self_wrapper() at db_trace_self_wrapper+0x30 pc =3D 0xc023fd34 lr =3D 0xc03eb0ec (witness_checkorder+0xf30) sp =3D 0xde2be840 fp =3D 0xde2be888 r4 =3D 0xc2725be8 r5 =3D 0xc2d63c94 r6 =3D 0xc06ba387 r7 =3D 0xc069f42b witness_checkorder() at witness_checkorder+0xf30 pc =3D 0xc03eb0ec lr =3D 0xc0375bcc (__lockmgr_args+0x24c) sp =3D 0xde2be890 fp =3D 0xde2be8e8 r4 =3D 0xc2d63cb4 r5 =3D 0x00202400 r6 =3D 0x00000211 r7 =3D 0x00000000 r8 =3D 0xc2d63c94 r9 =3D 0x00000000 r10 =3D 0xc06ba387 __lockmgr_args() at __lockmgr_args+0x24c pc =3D 0xc0375bcc lr =3D 0xc05db328 (ffs_lock+0x80) sp =3D 0xde2be8f0 fp =3D 0xde2be920 r4 =3D 0xde2be948 r5 =3D 0x00202400 r6 =3D 0xc2d63c60 r7 =3D 0xc2d63c94 r8 =3D 0xc2d63cb4 r9 =3D 0x00000000 r10 =3D 0x00000008 ffs_lock() at ffs_lock+0x80 pc =3D 0xc05db328 lr =3D 0xc06609ec (VOP_LOCK1_APV+0x128) sp =3D 0xde2be928 fp =3D 0xde2be940 r4 =3D 0xde2be948 r5 =3D 0xc07a9630 r6 =3D 0x00000000 r7 =3D 0x00202400 r8 =3D 0xde2be948 r9 =3D 0x00000211 r10 =3D 0xde2bed38 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x128 pc =3D 0xc06609ec lr =3D 0xc044dcac (_vn_lock+0x48) sp =3D 0xde2be948 fp =3D 0xde2be978 r4 =3D 0xc2d63c60 r5 =3D 0xde2bece0 r6 =3D 0xc06ba387 r10 =3D 0xde2bed38 _vn_lock() at _vn_lock+0x48 pc =3D 0xc044dcac lr =3D 0xc0433900 (lookup+0xf8) sp =3D 0xde2be980 fp =3D 0xde2be9c8 r4 =3D 0xc2d63c60 r5 =3D 0xde2bece0 r6 =3D 0xde2bed50 r7 =3D 0x00000000 r8 =3D 0xde2bed50 r9 =3D 0xc306e448 r10 =3D 0xde2bed38 lookup() at lookup+0xf8 pc =3D 0xc0433900 lr =3D 0xc0433448 (namei+0x3e8) sp =3D 0xde2be9d0 fp =3D 0xde2bea48 r4 =3D 0xde2bece0 r5 =3D 0x00000000 r6 =3D 0xde2bed50 r7 =3D 0x00000000 r8 =3D 0xc2e526a0 r9 =3D 0xc306e448 r10 =3D 0xde2bed38 namei() at namei+0x3e8 pc =3D 0xc0433448 lr =3D 0xc044d4b8 (vn_open_cred+0x1cc) sp =3D 0xde2bea50 fp =3D 0xde2beb38 r4 =3D 0xde2bece0 r5 =3D 0x00000000 r6 =3D 0xc069f42d r7 =3D 0x00000001 r8 =3D 0x00000000 r9 =3D 0x00000000 r10 =3D 0xde2bed50 vn_open_cred() at vn_open_cred+0x1cc pc =3D 0xc044d4b8 lr =3D 0xc044d2e4 (vn_open+0x24) sp =3D 0xde2beb40 fp =3D 0xde2beb48 r4 =3D 0xde2bece0 r5 =3D 0xc2d0d3c0 r6 =3D 0xc069f42d r7 =3D 0x0000001a r8 =3D 0x00000000 r9 =3D 0xc2e526a0 r10 =3D 0xc078d6f8 vn_open() at vn_open+0x24 pc =3D 0xc044d2e4 lr =3D 0xc0371390 (linker_load_module+0x634) sp =3D 0xde2beb50 fp =3D 0xde2beda8 linker_load_module() at linker_load_module+0x634 pc =3D 0xc0371390 lr =3D 0xc0372ff4 (kern_kldload+0xc8) sp =3D 0xde2bedb0 fp =3D 0xde2bedc8 r4 =3D 0xde2bedd4 r5 =3D 0xc083ef40 r6 =3D 0xc2e75000 r7 =3D 0x00000000 r8 =3D 0xde2bee00 r9 =3D 0xc306b380 r10 =3D 0xbfbff5fc kern_kldload() at kern_kldload+0xc8 pc =3D 0xc0372ff4 lr =3D 0xc03730bc (sys_kldload+0x64) sp =3D 0xde2bedd0 fp =3D 0xde2bede8 r4 =3D 0xc2e526a0 r5 =3D 0xc2e75000 r6 =3D 0x00000000 r7 =3D 0x00000000 sys_kldload() at sys_kldload+0x64 pc =3D 0xc03730bc lr =3D 0xc063e010 (swi_handler+0x2d4) sp =3D 0xde2bedf0 fp =3D 0xde2bee50 r4 =3D 0xc2e526a0 r5 =3D 0x00000000 r6 =3D 0xc08b3d20 r10 =3D 0xbfbff5fc swi_handler() at swi_handler+0x2d4 pc =3D 0xc063e010 lr =3D 0xc06294d8 (swi_exit) sp =3D 0xde2bee58 fp =3D 0xbfbff648 r4 =3D 0x00000000 r5 =3D 0xbfbff600 r6 =3D 0xbfbff600 r7 =3D 0x00000130 r8 =3D 0xbfbff5e9 r9 =3D 0xbfbff5d9 r10 =3D 0xbfbff5fc