Date: Tue, 7 Mar 2017 11:14:33 -0800 From: Mark Millard <markmi@dsl-only.net> To: Mark Johnston <markj@FreeBSD.org> Cc: Justin Hibbits <chmeeedalf@gmail.com>, FreeBSD PowerPC ML <freebsd-ppc@freebsd.org> Subject: Re: powerpc64 head -r314687 (PowerMac G5 so-called "Quad Core", clang based): CAM status: Command timeout (always?) Message-ID: <C931A6BB-3AB3-4C8F-B9FD-2EDB9DA27729@dsl-only.net> In-Reply-To: <20170307172558.GB42761@wkstn-mjohnston.west.isilon.com> References: <98A62E0D-C2A0-40B1-AE6D-5810906208AE@dsl-only.net> <4C78F6AA-5ABD-4445-B5EF-4E6778CE36FE@dsl-only.net> <20170306164341.GA83069@wkstn-mjohnston.west.isilon.com> <466C25ED-0A70-4988-9BB1-3B43BD031E5E@dsl-only.net> <E67A6606-941D-4F00-993D-4347C2A1D332@dsl-only.net> <20170307010204.GA3611@wkstn-mjohnston.west.isilon.com> <2FA8AC16-8108-4FC7-B1E6-788CBD32F372@dsl-only.net> <49DE6E23-948B-4DC1-87A0-047F8A2CE966@dsl-only.net> <20170307172558.GB42761@wkstn-mjohnston.west.isilon.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2017-Mar-7, at 9:25 AM, Mark Johnston <markj at FreeBSD.org> wrote: > On Tue, Mar 07, 2017 at 12:10:31AM -0800, Mark Millard wrote: >> The debug kernel has not failed so far but does report >> sleeping with a non-sleepable lock held (in addition >> to a lor): >>=20 >> Expensive timeout(9) function: 0xe84630(0xdc4a10) 0.026154120 s >=20 > What symbol does this correspond to? In my /boot/kerdbg/kernel : 0xe84630 is for the .opd sub-area for vt_timer 0xdc4a10 is for the .data sub-area for vt_consdev >> Sleeping on "smu" with the following non-sleepable locks held: >=20 > Looks like smu_gettime() calls smu_run_cmd(), which may sleep. This is > specific to powerpc. >=20 >> exclusive sleep mutex tod2rl (tod2rl) r =3D 0 (0x122ef50) locked @ = /usr/src/sys/kern/subr_rtc.c:135 >> stack backtrace: >> #0 0x60e3bc at .get_spr+0x50d4dc >> #1 0x59a200 at .get_spr+0x499320 >> #2 0x91b4cc at .rtascall+0x11dac >> #3 0x91a76c at .rtascall+0x1104c >> #4 0x5f0680 at .get_spr+0x4ef7a0 >> #5 0x68729c at .get_spr+0x5863bc >> #6 0x512af4 at .get_spr+0x411c14 >> #7 0x546600 at .get_spr+0x445720 >> #8 0x935f70 at .fork_trampoline+0x10 >> #9 0x935e44 at blocked_loop+0x38 Just FYI: I looked the above up in /boot/kerdbg/kernel : .witness_warn+0x4fc .sleep+0xb8 .smu_run_cmd+0x130 .smu_gettime+0x50 .inittodr+0xbc .vfs_mountroot+0x1b2c .start_init+0xa8 .fork_exit+0xc0 .fork_trampoline+0x10 block_loop+??? (I did not write that down) >> lock order reversal: (Giant after non-sleepable) >> 1st 0x122ef50 tod2rl (tod2rl) @ /usr/src/sys/kern/subr_rtc.c:135 >> 2nd 0x1200b00 Giant (Giant) @ /usr/src/sys/kern/kern_synch.c:230 >> stack backtrace: >> #0 0x60c9e4 at .get_spr+0x50bb04 >> #1 0x56c504 at .get_spr+0x46b624 >> #2 0x59a5d0 at .get_spr+0x4996f0 >> #3 0x91b4cc at .rtascall+0x11dac >> #4 0x91a76c at .rtascall+0x1104c >> #5 0x5f0680 at .get_spr+0x4ef7a0 >> #6 0x68729c at .get_spr+0x5863bc >> #7 0x512af4 at .get_spr+0x411c14 >> #8 0x546600 at .get_spr+0x445720 >> #9 0x935f70 at .fork_trampoline+0x10 >> #10 0x935e44 at blocked_loop+0x38 >=20 > This looks to be related to the above. vfs_mountroot() calls = inittodr() > with Giant held, so the tsleep() in smu_run_cmd() will acquire Giant > after the (non-sleepable) tod2rl mutex upon waking up. >=20 >> uhub3: 3 ports with 2 removable, bus powered >> Setting hostuuid: 11fe4cb1-3cac-11e4-a5a8-001451685a44. >> Setting hostid: 0x489b688f. >> Starting file system checks: >> /dev/ufs/FBSDG5Lrootfs: FILE SYSTEM CLEAN; SKIPPING CHECKS >> /dev/ufs/FBSDG5Lrootfs: clean, 82126801 free (1058921 frags, 10133485 = blocks, 1.1% fragmentation) >> Mounting local filesystems:. >>=20 >> (The symbolic routine names it shows are not of much use. >> I could use /usr/local/bin/objdump to look things up.) >>=20 >> Later there is another lor: >>=20 >> lock order reversal: >> 1st 0xa046b78 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2600 >> 2nd 0xc0000000b6c90ba0 bufwait (bufwait) @ = /usr/src/sys/ufs/ffs/ffs_vnops.c:280 >> 3rd 0xb22e5f0 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2600 >=20 > This is a known benign LOR. =3D=3D=3D Mark Millard markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?C931A6BB-3AB3-4C8F-B9FD-2EDB9DA27729>