Date: Sat, 10 Mar 2007 01:00:42 +0100 From: =?ISO-8859-1?Q?Eirik_=D8verby?= <ltning@anduin.net> To: Kris Kennaway <kris@obsecurity.org> Cc: stable@freebsd.org Subject: Re: Panic: spin lock smp rendezvous ... held too long Message-ID: <3B8198E4-78E6-41AC-93EB-193529550CBC@anduin.net> In-Reply-To: <20070309024157.GA58736@xor.obsecurity.org> References: <3FCD4E86-40CA-4E27-AF97-DFB94F02077C@anduin.net> <20070309024157.GA58736@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156)
--Apple-Mail-10--243369446
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
On Mar 9, 2007, at 03:41, Kris Kennaway wrote:
> On Fri, Mar 09, 2007 at 12:44:03AM +0100, Eirik ?verby wrote:
>> Hi all,
>>
>> I just installed 6.2-RELEASE on a Supermicro 6013P-8 server, a dual
>> P4-Xeon 2.4ghz with 4GB ECC memory and an asr driven SCSI RAID
>> controller.
>>
>> It has been working OK (although I suspect the asr driven, being
>> giant-locked, is very inefficient) for a little while, but as I was
>> extracting a bunch of tarballs it paniced like so:
>>
>> spin lock smp rendezvous held by 0xc9d54600 for > 5 seconds
>> panic: spin lock held too long
>> cpuid = 0
>>
>> I don't have a dump device (though I'm setting that up for the next
>> reboot). However, I have tried turning off HT, to see if that might
>> help.
>>
>> Does this look familiar to anyone? Or do I need to produce more data
>> if it happens again?
>
> It can mean that something deadlocked. Turning on WITNESS may help to
> debug this, although it has a large performance impact.
Just opened the vmcore file, and this is what I see, with a bt at the
end:
Unread portion of the kernel message buffer:
dev = da0s1f, block = 3802920, fs = /usr
panic: ffs_blkfree: freeing free block
cpuid = 1
Uptime: 23h59m46s
Dumping 3967 MB (3 chunks)
chunk 0: 1MB (158 pages) ... ok
chunk 1: 3966MB (1015280 pages) 3950 3934 3918 3902 3886 3870 3854
3838 3822 3806 3790 3774 3758 3742 3726 3710 3694 3678 3662 3646 3630
3614 3598 3582
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 06
fault virtual address = 0x18c
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc04542f4
stack pointer = 0x28:0xe98a3c88
frame pointer = 0x28:0xe98a3c90
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 18 (swi2: cambio)
trap number = 12
panic: page fault
cpuid = 1
3566 3550 3534 3518 3502 3486 3470 3454 3438 3422 3406 3390 3374 3358
3342 3326 3310 3294 3278 3262 3246 3230 3214 3198 3182 3166 3150 3134
3118 3102 3086 3070 3054 3038 3022 3006 2990 2974 2958 2942 2926 2910
2894 2878 2862 2846 2830 2814 2798 2782 2766 2750 2734 2718 2702 2686
2670 2654 2638 2622 2606 2590 2574 2558 2542 2526 2510 2494 2478 2462
2446 2430 2414 2398 2382 2366 2350 2334 2318 2302 2286 2270 2254 2238
2222 2206 2190 2174 2158 2142 2126 2110 2094 2078 2062 2046 2030 2014
1998 1982 1966 1950 1934 1918 1902 1886 1870 1854 1838 1822 1806 1790
1774 1758 1742 1726 1710 1694 1678 1662 1646 1630 1614 1598 1582 1566
1550 1534 1518 1502 1486 1470 1454 1438 1422 1406 1390 1374 1358 1342
1326 1310 1294 1278 1262 1246 1230 1214 1198 1182 1166 1150 1134 1118
1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878
862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 606
590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334
318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46
30 14 ... ok
chunk 2: 1MB (128 pages)
#0 doadump () at pcpu.h:165
165 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) bt
#0 doadump () at pcpu.h:165
#1 0xc067550a in boot (howto=260) at /usr/src/sys/kern/
kern_shutdown.c:409
#2 0xc0675831 in panic (fmt=0xc0911dc2 "ffs_blkfree: freeing free
block") at /usr/src/sys/kern/kern_shutdown.c:565
#3 0xc07b375e in ffs_blkfree (ump=0xc9607c00, fs=0xc93c5800,
devvp=0xc9625110, bno=3802920, size=16384, inum=39400)
at /usr/src/sys/ufs/ffs/ffs_alloc.c:1869
#4 0xc07c38c6 in indir_trunc (freeblks=0xcb2bfa00, dbn=15210848,
level=0, lbn=12, countp=0xe98b8c6c)
at /usr/src/sys/ufs/ffs/ffs_softdep.c:2894
#5 0xc07c32f6 in handle_workitem_freeblocks (freeblks=0xcb2bfa00,
flags=0) at /usr/src/sys/ufs/ffs/ffs_softdep.c:2744
#6 0xc07c01b1 in process_worklist_item (mp=0xc95d87c8, flags=0) at /
usr/src/sys/ufs/ffs/ffs_softdep.c:967
#7 0xc07bfeb2 in softdep_process_worklist (mp=0xc95d87c8, full=0)
at /usr/src/sys/ufs/ffs/ffs_softdep.c:851
#8 0xc07bfc08 in softdep_flush () at /usr/src/sys/ufs/ffs/
ffs_softdep.c:762
#9 0xc065ec4d in fork_exit (callout=0xc07bfa6c <softdep_flush>,
arg=0x0, frame=0xe98b8d38) at /usr/src/sys/kern/kern_fork.c:821
#10 0xc0879dac in fork_trampoline () at /usr/src/sys/i386/i386/
exception.s:208
--Apple-Mail-10--243369446
content-type: application/pgp-signature; x-mac-type=70674453;
name=PGP.sig
content-description: This is a digitally signed message part
content-disposition: inline; filename=PGP.sig
content-transfer-encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (Darwin)
iD8DBQFF8fUqDRlfnc8VQWcRArJ0AJ481ek9NNls67l420KH9isjMAtyLwCglQxa
Sbjea3/sS8PiEnsuVRHEhj4=
=32Ts
-----END PGP SIGNATURE-----
--Apple-Mail-10--243369446--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3B8198E4-78E6-41AC-93EB-193529550CBC>