Date: Thu, 27 Dec 2007 13:20:02 GMT From: Per olof Ljungmark <peo@intersonic.se> To: freebsd-bugs@FreeBSD.org Subject: Re: kern/118044: panic: spin lock held too long Message-ID: <200712271320.lBRDK2R8006784@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/118044; it has been noted by GNATS.
From: Per olof Ljungmark <peo@intersonic.se>
To: Maxime Henrion <mux@FreeBSD.org>
Cc: bug-followup@FreeBSD.org
Subject: Re: kern/118044: panic: spin lock held too long
Date: Thu, 27 Dec 2007 14:16:16 +0100
Maxime Henrion wrote:
> Hi Per,
>
>
> There is a suspicion that the problem you are experiencing might
> be fixed by a patch that has just been committed to HEAD. Could
> you please give it a try? It's a fairly small patch, you just need
> to apply revision 1.125 of sys/net/route.c and revision 1.68 of
> sys/net/route.h.
>
> Let me know if you need some help in doing this testing.
Hi Maxime,
Thank you for your kind attantion to this problem.
Did as per your advise, result below. System was rebuilt from sources
today + patched route.c/h.
savecore: no dumps found
Initial i386 initialization:.
Additional ABI support:.
Starting named.
Setting date via ntp.
lock order reversal:
1st 0xc58370d8 rtentry (rtentry) @ /usr/src/sys/net/route.c:196
2nd 0xc538d07c radix node head (radix node head) @
/usr/src/sys/net/route.c:692
KDB: stack backtrace:
db_trace_self_wrapper(c0a9fc45,e5797a70,c078792e,c0aa2117,c538d07c,...)
at db_trace_self_wrapper+0x26
kdb_backtrace(c0aa2117,c538d07c,c0aa2178,c0aa2178,c0aaaa4b,...) at
kdb_backtrace+0x29
witness_checkorder(c538d07c,9,c0aaaa4b,2b4,c5214d58,...) at
witness_checkorder+0x6de
_mtx_lock_flags(c538d07c,0,c0aaaa4b,2b4,c0a9bb4b,...) at
_mtx_lock_flags+0xbc
rtrequest1(1,e5797b0c,e5797b3c,e5797b48,c08911ac,...) at rtrequest1+0x4f
rtredirect(e5797bbc,e5797bac,0,16,e5797b9c,...) at rtredirect+0x126
icmp_input(c5738000,14,246,c0b9aca0,e5797c0c,...) at icmp_input+0x50f
ip_input(c5738000,14e,800,c5250400,800,...) at ip_input+0x650
netisr_dispatch(2,c5738000,10,3,0,...) at netisr_dispatch+0x73
ether_demux(c5250400,c5738000,3,0,3,...) at ether_demux+0x1f1
ether_input(c5250400,c5738000,c0a6b43a,bc4,c5250400,...) at
ether_input+0x37f
bge_intr(c526d000,0,c0a999ef,471,c515e564,...) at bge_intr+0x7da
ithread_loop(c5268640,e5797d38,c0a99763,2ea,c5263558,...) at
ithread_loop+0x1b5
fork_exit(c07328a0,c5268640,e5797d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5797d70, ebp = 0 ---
27 Dec 14:09:38 ntpdate[648]: adjust time server 192.168.1.2 offset
-0.408824 sec
Clearing /tmp (X related).
Starting local daemons:.
Updating motd.
Mounting late file systems:.
Starting ntpd.
postfix/postfix-script: starting the Postfix mail system
Starting mysql.
Performing sanity check on apache22 configuration:
Syntax OK
Starting apache22.
Configuring syscons: keymap keyrate font8x16 font8x14 font8x8 blanktime.
Starting sshd.
Starting cron.
Local package initialization:.
Starting default moused:.
Starting inetd.
Starting background file system checks in 60 seconds.
Thu Dec 27 14:09:43 CET 2007
FreeBSD/i386 (ruben.i.inter-sonic.com) (ttyd0)
login: rtfree: 0xc5722bb8 has 1 refs
Dec 27 14:10:19 ruben su: peo to root on /dev/ttyp0
rtfree: 0xc5722bb8 has 1 refs
rtfree: 0xc5722bb8 has 1 refs
rtfree: 0xc5722bb8 has 1 refs
KDB: enter: Line break on console
[thread pid 33 tid 100037 ]
Stopped at kdb_enter+0x32: leave
db> wh
Tracing pid 33 tid 100037 td 0xc5214cc0
kdb_enter(c0a8e6e4,c536f8f8,c0c1b0e8,8,c536f88c,...) at kdb_enter+0x32
siointr1(c0c1b0e8,0,c0ac563d,56f,e57979dc,...) at siointr1+0x11c
siointr(c536f800,c0bb0844,2,c5214cc0,c5111280,...) at siointr+0x32
intr_execute_handlers(c5107cd0,e57979fc,e5797a58,c09eb844,3b,...) at
intr_execute_handlers+0xec
lapic_handle_intr(3b,e57979fc) at lapic_handle_intr+0x3f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc07429c2, esp = 0xe5797a3c, ebp = 0xe5797a58 ---
_mtx_lock_sleep(c538d07c,c5214cc0,0,c0aaaa4b,93,...) at _mtx_lock_sleep+0xa2
_mtx_lock_flags(c538d07c,0,c0aaaa4b,93,c0a9bb4b,...) at _mtx_lock_flags+0xef
rtalloc1(e5797bac,1,0,e5797b20,c07426fc,...) at rtalloc1+0x63
rtredirect(e5797bbc,e5797bac,0,26,e5797b9c,...) at rtredirect+0x35e
icmp_input(c57d1700,14,246,c0b9aca0,e5797c0c,...) at icmp_input+0x50f
ip_input(c57d1700,14e,800,c5250400,800,...) at ip_input+0x650
netisr_dispatch(2,c57d1700,10,3,0,...) at netisr_dispatch+0x73
ether_demux(c5250400,c57d1700,3,0,3,...) at ether_demux+0x1f1
ether_input(c5250400,c57d1700,c0a6b43a,bc4,c5250400,...) at
ether_input+0x37f
bge_intr(c526d000,0,c0a999ef,471,c515e564,...) at bge_intr+0x7da
ithread_loop(c5268640,e5797d38,c0a99763,2ea,c5263558,...) at
ithread_loop+0x1b5
fork_exit(c07328a0,c5268640,e5797d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5797d70, ebp = 0 ---
db> ps
pid ppid pgrp uid state wmesg wchan cmd
954 949 954 0 S+ ttywri 0xc5355838 dmesg
949 948 949 0 S+ pause 0xc56ef30c csh
948 944 948 1001 S+ wait 0xc5af0804 su
944 943 944 1001 Ss+ pause 0xc5540b10 csh
943 940 940 1001 R CPU 1 sshd
940 852 940 0 Ss sbwait 0xc58a8560 sshd
936 835 835 80 S accept 0xc58a866a httpd
935 835 835 80 S accept 0xc58a866a httpd
934 835 835 80 S accept 0xc58a866a httpd
933 835 835 80 S accept 0xc58a866a httpd
932 835 835 80 S accept 0xc58a866a httpd
931 1 931 0 Ss+ ttyin 0xc5357c10 getty
930 1 930 0 Ss+ ttyin 0xc538a810 getty
929 1 929 0 Ss+ ttyin 0xc538ac10 getty
928 1 928 0 Ss+ ttyin 0xc5389810 getty
927 1 927 0 Ss+ ttyin 0xc538b410 getty
926 1 926 0 Ss+ ttyin 0xc538c410 getty
925 1 925 0 Ss+ ttyin 0xc538c010 getty
924 1 924 0 Ss+ ttyin 0xc537fc10 getty
923 1 923 0 Ss+ ttyin 0xc5380410 getty
921 919 54 0 S+ nanslp 0xc0bb0824 sleep
920 1 54 0 S+ piperd 0xc586d948 logger
919 918 54 0 S+ wait 0xc559f2ac sh
918 1 54 0 S+ wait 0xc5598558 sh
902 1 902 0 Ss select 0xc0c01ddc inetd
875 1 875 0 Ss select 0xc0c01ddc moused
859 1 859 0 Ss nanslp 0xc0bb0824 cron
852 1 852 0 Ss select 0xc0c01ddc sshd
835 1 835 0 Ss select 0xc0c01ddc httpd
833 808 805 88 S+ (threaded) mysqld
100139 S ucond 0xc561f2c0 mysqld
100138 S sigwait 0xebe8cbe0 mysqld
100135 S ucond 0xc561f240 mysqld
100137 S select 0xc0c01ddc mysqld
100136 S select 0xc0c01ddc mysqld
100134 S ucond 0xc561f140 mysqld
100133 S ucond 0xc561f100 mysqld
100132 S ucond 0xc55ae740 mysqld
100131 S ucond 0xc55ae700 mysqld
100086 S select 0xc0c01ddc initial thread
808 1 805 88 S+ wait 0xc585fab0 sh
803 795 795 125 S kqread 0xc587da00 qmgr
802 795 795 125 S kqread 0xc55aab80 pickup
795 1 795 0 Ss kqread 0xc5564080 master
728 1 728 0 Ss select 0xc0c01ddc ntpd
643 1 643 53 Ss (threaded) named
100130 S select 0xc0c01ddc named
100129 S ucond 0xc555d240 named
100128 S ucond 0xc555d200 named
100127 S ucond 0xc555d1c0 named
100126 S ucond 0xc555d180 named
100125 S ucond 0xc555d140 named
100107 S sigwait 0xe9b14be0 named
578 1 578 0 Ss select 0xc0c01ddc syslogd
517 1 517 0 Ss select 0xc0c01ddc devd
321 1 321 65 Ss select 0xc0c01ddc dhclient
301 1 54 0 S+ select 0xc0c01ddc dhclient
53 0 0 0 SL sdflush 0xc0c0db44 [softdepflush]
52 0 0 0 SL syncer 0xc0bb064c [syncer]
51 0 0 0 SL vlruwt 0xc5374000 [vnlru]
50 0 0 0 SL psleep 0xc0c02244 [bufdaemon]
49 0 0 0 SL pgzero 0xc0c0e700 [pagezero]
48 0 0 0 SL psleep 0xc0c0e318 [vmdaemon]
47 0 0 0 SL psleep 0xc0c0e2e0 [pagedaemon]
46 0 0 0 SL waiting_ 0xc0c03fac [sctp_iterator]
45 0 0 0 WL [irq7: ppc0]
44 0 0 0 SL - 0xc531ce3c [fdc0]
43 0 0 0 WL [swi0: sio]
42 0 0 0 WL [irq12: psm0]
41 0 0 0 WL [irq1: atkbd0]
40 0 0 0 WL [irq15: ata1]
39 0 0 0 WL [irq14: ata0]
38 0 0 0 SL idle 0xc531e000 [aic_recovery1]
37 0 0 0 WL [irq22: ahd1]
36 0 0 0 SL idle 0xc52d5000 [aic_recovery0]
35 0 0 0 SL aifthd 0xc5263000 [aac0aif]
34 0 0 0 WL [irq18: aac0]
33 0 0 0 RL CPU 0 [irq21: bge0]
32 0 0 0 SL usbevt 0xc5210210 [usb2]
31 0 0 0 WL [irq23: ehci0]
30 0 0 0 SL usbevt 0xc5245210 [usb1]
29 0 0 0 WL [irq19: uhci1 ahd0]
28 0 0 0 SL usbtsk 0xc0bae2f4 [usbtask-dr]
27 0 0 0 SL usbtsk 0xc0bae2e0 [usbtask-hc]
26 0 0 0 SL usbevt 0xc524b210 [usb0]
25 0 0 0 WL [irq16: uhci0]
24 0 0 0 WL [irq9: acpi0]
23 0 0 0 SL - 0xc5212600 [kqueue taskq]
22 0 0 0 WL [swi6: task queue]
21 0 0 0 WL [swi6: Giant taskq]
9 0 0 0 SL - 0xc5212880 [thread taskq]
8 0 0 0 SL - 0xc5212900 [acpi_task_2]
7 0 0 0 SL - 0xc5212900 [acpi_task_1]
6 0 0 0 SL - 0xc5212900 [acpi_task_0]
20 0 0 0 WL [swi5: +]
19 0 0 0 WL [swi2: cambio]
5 0 0 0 SL ccb_scan 0xc0b7f514 [xpt_thrd]
18 0 0 0 SL - 0xc0bb0654 [yarrow]
4 0 0 0 SL - 0xc0bae6cc [g_down]
3 0 0 0 SL - 0xc0bae6c8 [g_up]
2 0 0 0 SL - 0xc0bae6c0 [g_event]
17 0 0 0 WL [swi3: vm]
16 0 0 0 RL CPU 3 [swi4: clock sio]
15 0 0 0 WL [swi1: net]
14 0 0 0 RL [idle: cpu0]
13 0 0 0 RL [idle: cpu1]
12 0 0 0 RL CPU 2 [idle: cpu2]
11 0 0 0 RL [idle: cpu3]
1 0 1 0 SLs wait 0xc5114ab0 [init]
10 0 0 0 SL audit_wo 0xc0c0d5b4 [audit]
0 0 0 0 WLs [swapper]
db> show allpcpu
Current CPU: 0
cpuid = 0
curthread = 0xc5214cc0: pid 33 "irq21: bge0"
curpcb = 0xe5797d90
fpcurthread = none
idlethread = 0xc5117000: pid 14 "idle: cpu0"
APIC ID = 0
currentldt = 0x50
spin locks held:
exclusive spin mutex sio r = 0 (0xc0c1b0e8) locked @
/usr/src/sys/dev/sio/sio.c:1391
cpuid = 1
curthread = 0xc5861440: pid 943 "sshd"
curpcb = 0xebe1fd90
fpcurthread = none
idlethread = 0xc5115cc0: pid 13 "idle: cpu1"
APIC ID = 1
currentldt = 0x50
spin locks held:
cpuid = 2
curthread = 0xc5115aa0: pid 12 "idle: cpu2"
curpcb = 0xe3d32d90
fpcurthread = none
idlethread = 0xc5115aa0: pid 12 "idle: cpu2"
APIC ID = 6
currentldt = 0x50
spin locks held:
cpuid = 3
curthread = 0xc5117cc0: pid 16 "swi4: clock sio"
curpcb = 0xe3d4ad90
fpcurthread = none
idlethread = 0xc5115880: pid 11 "idle: cpu3"
APIC ID = 7
currentldt = 0x50
spin locks held:
db> alltrace
Tracing command dmesg pid 954 tid 100095 td 0xc56ee000
sched_switch(c56ee000,0,1,180,3869c968,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c56ee000,0,c0aa0842,19b,c5355838,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebdafa70,c07423dd,c0bb8700,4,5a,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c5355838,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c5355838,0,15a,c0aa420e,493e0,...) at _sleep+0x331
ttysleep(c5355800,c5355838,15a,c0aa420e,493e0,...) at ttysleep+0x39
ttwrite(c5355800,ebdafc60,0,ebdafbbc,c071e148,...) at ttwrite+0x5aa
ptswrite(c5afb000,ebdafc60,0,174,41,...) at ptswrite+0x38
giant_write(c5afb000,ebdafc60,0,0,c0b56b60,...) at giant_write+0x48
devfs_write_f(c55a4438,ebdafc60,c5af9000,0,c56ee000,...) at
devfs_write_f+0x98
dofilewrite(ebdafc60,ffffffff,ffffffff,0,c55a4438,...) at dofilewrite+0x95
kern_writev(c56ee000,1,ebdafc60,28201041,0,...) at kern_writev+0x58
write(c56ee000,ebdafcfc,c,c0a8d9fa,c0b49ae0,...) at write+0x4f
syscall(ebdafd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x2815b423, esp =
0xbfbfe40c, ebp = 0xbfbfe428 ---
Tracing command csh pid 949 tid 100096 td 0xc56e9cc0
sched_switch(c56e9cc0,0,1,180,371d3b46,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c56e9cc0,0,c0aa0842,19b,c56ef30c,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c56ef30c,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c56ef30c,c56ef33c,168,c0a5fd27,0,...) at _sleep+0x359
kern_sigsuspend(c56e9cc0,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c56e9cc0,ebdaccfc,4,c0aa2849,c0b4ba78,...) at sigsuspend+0x4d
syscall(ebdacd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166efb, esp =
0xbfbfe73c, ebp = 0xbfbfe7a8 ---
Tracing command su pid 948 tid 100143 td 0xc5861660
sched_switch(c5861660,0,1,180,d92ee264,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5861660,0,c0aa0842,19b,c5af0804,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5af0804,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5af0804,c5af0894,15c,c0aa296f,0,...) at _sleep+0x359
kern_wait(c5861660,3b5,ebe22c2c,2,0,...) at kern_wait+0xa66
wait4(c5861660,ebe22cfc,10,ebe22d38,c0b49b28,...) at wait4+0x3b
syscall(ebe22d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2810ad7b, esp =
0xbfbfe79c, ebp = 0xbfbfe7b8 ---
Tracing command csh pid 944 tid 100059 td 0xc5265880
sched_switch(c5265880,0,1,180,77538768,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5265880,0,c0aa0842,19b,c5540b10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5540b10,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5540b10,c5540b40,168,c0a5fd27,0,...) at _sleep+0x359
kern_sigsuspend(c5265880,0,0,0,0,...) at kern_sigsuspend+0xe4
sigsuspend(c5265880,e580bcfc,4,c0aa2849,c0b4ba78,...) at sigsuspend+0x4d
syscall(e580bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x28166efb, esp =
0xbfbfe80c, ebp = 0xbfbfe878 ---
Tracing command sshd pid 943 tid 100144 td 0xc5861440
cpustop_handler(2,ebe1f890,c0a04a50,38,c0aa17fa,...) at cpustop_handler+0x32
ipi_nmi_handler(38,c0aa17fa,6ce,c0c01c00,13,...) at ipi_nmi_handler+0x2f
trap(ebe1f89c) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc07429c2, esp = 0xebe1f8dc, ebp = 0xebe1f8f8 ---
_mtx_lock_sleep(c5722c18,c5861440,0,c0aaaa4b,c4,...) at _mtx_lock_sleep+0xa2
_mtx_lock_flags(c5722c18,0,c0aaaa4b,c4,c0786d22,...) at _mtx_lock_flags+0xef
rtalloc1(ebe1fa04,1,0,100,ebe1fa00,...) at rtalloc1+0x225
rtalloc_ign(ebe1fa00,0,c0aaf933,7f,c5861440,...) at rtalloc_ign+0xe6
ip_output(c5893500,0,ebe1fa00,0,0,...) at ip_output+0x355
tcp_output(c588a570,c5bb3c00,c0ab3c29,30b,c58887a4,...) at tcp_output+0x13f0
tcp_usr_send(c5903318,0,c5bb3c00,0,0,...) at tcp_usr_send+0x25a
sosend_generic(c5903318,0,ebe1fc60,0,0,...) at sosend_generic+0x50d
sosend(c5903318,0,ebe1fc60,0,0,...) at sosend+0x54
soo_write(c5853240,ebe1fc60,c5976800,0,c5861440,...) at soo_write+0x4b
dofilewrite(ebe1fc60,ffffffff,ffffffff,0,c5853240,...) at dofilewrite+0x95
kern_writev(c5861440,3,ebe1fc60,28533110,0,...) at kern_writev+0x58
write(c5861440,ebe1fcfc,c,c0a5633b,c0b49ae0,...) at write+0x4f
syscall(ebe1fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (4, FreeBSD ELF32, write), eip = 0x283e3423, esp =
0xbfbfe5cc, ebp = 0xbfbfe5e8 ---
Tracing command sshd pid 940 tid 100145 td 0xc5861220
sched_switch(c5861220,0,1,180,94a3b5cc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5861220,0,c0aa0842,19b,c58a8560,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a8560,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a8560,c58a8518,158,c0aa5515,0) at _sleep+0x359
sbwait(c58a84f4,4,c0aa562a,5ae,c58a8518,...) at sbwait+0x76
soreceive_generic(c58a84a4,0,ebe1cc60,0,0,...) at soreceive_generic+0x42f
soreceive(c58a84a4,0,ebe1cc60,0,0,...) at soreceive+0x4d
soo_read(c55e76c0,ebe1cc60,c5976300,0,c5861220,...) at soo_read+0x3b
dofileread(ebe1cc60,ffffffff,ffffffff,0,c55e76c0,...) at dofileread+0x96
kern_readv(c5861220,5,ebe1cc60,bfbfe64c,4,...) at kern_readv+0x58
read(c5861220,ebe1ccfc,c,c0ab7dcb,c0b49ac8,...) at read+0x4f
syscall(ebe1cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x283e3443, esp =
0xbfbfe60c, ebp = 0xbfbfe628 ---
Tracing command httpd pid 936 tid 100105 td 0xc5543880
sched_switch(c5543880,0,1,180,21cc628c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5543880,0,c0aa0842,19b,c58a866a,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a866a,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a866a,c0c020c0,158,c0a6ee93,0,...) at _sleep+0x359
kern_accept(c5543880,3,e9b1ac70,e9b1ac6c,e9b1ac68,...) at kern_accept+0x12b
accept(c5543880,e9b1acfc,c,e9b1ad38,c0b49d50,...) at accept+0x8f
syscall(e9b1ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (30, FreeBSD ELF32, accept), eip = 0x282d138f, esp =
0xbfbfeb7c, ebp = 0xbfbfeb98 ---
Tracing command httpd pid 935 tid 100147 td 0xc56eecc0
sched_switch(c56eecc0,0,1,180,216c709c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c56eecc0,0,c0aa0842,19b,c58a866a,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a866a,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a866a,c0c020c0,158,c0a6ee93,0,...) at _sleep+0x359
kern_accept(c56eecc0,3,ebdc1c70,ebdc1c6c,ebdc1c68,...) at kern_accept+0x12b
accept(c56eecc0,ebdc1cfc,c,ebdc1d38,c0b49d50,...) at accept+0x8f
syscall(ebdc1d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (30, FreeBSD ELF32, accept), eip = 0x282d138f, esp =
0xbfbfeb7c, ebp = 0xbfbfeb98 ---
Tracing command httpd pid 934 tid 100148 td 0xc5af5000
sched_switch(c5af5000,0,1,180,207a8436,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5af5000,0,c0aa0842,19b,c58a866a,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a866a,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a866a,c0c020c0,158,c0a6ee93,0,...) at _sleep+0x359
kern_accept(c5af5000,3,ebecec70,ebecec6c,ebecec68,...) at kern_accept+0x12b
accept(c5af5000,ebececfc,c,ebeced38,c0b49d50,...) at accept+0x8f
syscall(ebeced38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (30, FreeBSD ELF32, accept), eip = 0x282d138f, esp =
0xbfbfeb7c, ebp = 0xbfbfeb98 ---
Tracing command httpd pid 933 tid 100149 td 0xc5af4cc0
sched_switch(c5af4cc0,0,1,180,2035600c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5af4cc0,0,c0aa0842,19b,c58a866a,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a866a,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a866a,c0c020c0,158,c0a6ee93,0,...) at _sleep+0x359
kern_accept(c5af4cc0,3,ebecbc70,ebecbc6c,ebecbc68,...) at kern_accept+0x12b
accept(c5af4cc0,ebecbcfc,c,ebecbd38,c0b49d50,...) at accept+0x8f
syscall(ebecbd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (30, FreeBSD ELF32, accept), eip = 0x282d138f, esp =
0xbfbfeb7c, ebp = 0xbfbfeb98 ---
Tracing command httpd pid 932 tid 100150 td 0xc5af4aa0
sched_switch(c5af4aa0,0,1,180,1f5dfefc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5af4aa0,0,c0aa0842,19b,c58a866a,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,58,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c58a866a,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c58a866a,c0c020c0,158,c0a6ee93,0,...) at _sleep+0x359
kern_accept(c5af4aa0,3,ebec8c70,ebec8c6c,ebec8c68,...) at kern_accept+0x12b
accept(c5af4aa0,ebec8cfc,c,ebec8d38,c0b49d50,...) at accept+0x8f
syscall(ebec8d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (30, FreeBSD ELF32, accept), eip = 0x282d138f, esp =
0xbfbfeb7c, ebp = 0xbfbfeb98 ---
Tracing command getty pid 931 tid 100064 td 0xc5543aa0
sched_switch(c5543aa0,0,1,180,aa94c4ec,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5543aa0,0,c0aa0842,19b,c5357c10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5357c10,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5357c10,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c5357c00,c5357c10,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c5357c00,e9b1dc60,0,e9b1dbb8,c071e1c8,...) at ttread+0x4e7
ttyread(c5372100,e9b1dc60,0,168,0,...) at ttyread+0x39
giant_read(c5372100,e9b1dc60,0,0,1,...) at giant_read+0x48
devfs_read_f(c56563f0,e9b1dc60,c50f4300,0,c5543aa0,...) at devfs_read_f+0x71
dofileread(e9b1dc60,ffffffff,ffffffff,0,c56563f0,...) at dofileread+0x96
kern_readv(c5543aa0,0,e9b1dc60,bfbfee4b,1,...) at kern_readv+0x58
read(c5543aa0,e9b1dcfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(e9b1dd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 930 tid 100065 td 0xc559d000
sched_switch(c559d000,0,1,180,ab249a30,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559d000,0,c0aa0842,19b,c538a810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538a810,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538a810,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c538a800,c538a810,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c538a800,ebd39c60,0,ebd39b9c,c067a61d,...) at ttread+0x4e7
ttyread(c538e100,ebd39c60,0,ebd39bb8,c071e1c8,...) at ttyread+0x39
scread(c538e100,ebd39c60,0,168,0,...) at scread+0x2d
giant_read(c538e100,ebd39c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a41b0,ebd39c60,c50f4300,0,c559d000,...) at devfs_read_f+0x71
dofileread(ebd39c60,ffffffff,ffffffff,0,c55a41b0,...) at dofileread+0x96
kern_readv(c559d000,0,ebd39c60,bfbfee4b,1,...) at kern_readv+0x58
read(c559d000,ebd39cfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(ebd39d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 929 tid 100151 td 0xc5af4880
sched_switch(c5af4880,0,1,180,aa9a3df4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5af4880,0,c0aa0842,19b,c538ac10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538ac10,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538ac10,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c538ac00,c538ac10,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c538ac00,ebec5c60,0,ebec5b9c,c067a61d,...) at ttread+0x4e7
ttyread(c5373900,ebec5c60,0,ebec5bb8,c071e1c8,...) at ttyread+0x39
scread(c5373900,ebec5c60,0,168,0,...) at scread+0x2d
giant_read(c5373900,ebec5c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a31f8,ebec5c60,c50f4300,0,c5af4880,...) at devfs_read_f+0x71
dofileread(ebec5c60,ffffffff,ffffffff,0,c55a31f8,...) at dofileread+0x96
kern_readv(c5af4880,0,ebec5c60,bfbfee4b,1,...) at kern_readv+0x58
read(c5af4880,ebec5cfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(ebec5d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 928 tid 100077 td 0xc5599440
sched_switch(c5599440,0,1,180,ab04c1b4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5599440,0,c0aa0842,19b,c5389810,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5389810,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5389810,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c5389800,c5389810,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c5389800,ebd15c60,0,ebd15b9c,c067a61d,...) at ttread+0x4e7
ttyread(c5373a00,ebd15c60,0,ebd15bb8,c071e1c8,...) at ttyread+0x39
scread(c5373a00,ebd15c60,0,168,0,...) at scread+0x2d
giant_read(c5373a00,ebd15c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a4990,ebd15c60,c50f4300,0,c5599440,...) at devfs_read_f+0x71
dofileread(ebd15c60,ffffffff,ffffffff,0,c55a4990,...) at dofileread+0x96
kern_readv(c5599440,0,ebd15c60,bfbfee4b,1,...) at kern_readv+0x58
read(c5599440,ebd15cfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(ebd15d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 927 tid 100055 td 0xc5377220
sched_switch(c5377220,0,1,180,aae49548,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5377220,0,c0aa0842,19b,c538b410,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538b410,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538b410,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c538b400,c538b410,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c538b400,e9adac60,0,e9adab9c,c067a61d,...) at ttread+0x4e7
ttyread(c5373b00,e9adac60,0,e9adabb8,c071e1c8,...) at ttyread+0x39
scread(c5373b00,e9adac60,0,168,0,...) at scread+0x2d
giant_read(c5373b00,e9adac60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a3480,e9adac60,c50f4300,0,c5377220,...) at devfs_read_f+0x71
dofileread(e9adac60,ffffffff,ffffffff,0,c55a3480,...) at dofileread+0x96
kern_readv(c5377220,0,e9adac60,bfbfee4b,1,...) at kern_readv+0x58
read(c5377220,e9adacfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(e9adad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 926 tid 100108 td 0xc5543220
sched_switch(c5543220,0,1,180,aaf9cf5c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5543220,0,c0aa0842,19b,c538c410,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538c410,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538c410,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c538c400,c538c410,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c538c400,e9b11c60,0,e9b11b9c,c067a61d,...) at ttread+0x4e7
ttyread(c537d700,e9b11c60,0,e9b11bb8,c071e1c8,...) at ttyread+0x39
scread(c537d700,e9b11c60,0,168,0,...) at scread+0x2d
giant_read(c537d700,e9b11c60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5617d38,e9b11c60,c50f4300,0,c5543220,...) at devfs_read_f+0x71
dofileread(e9b11c60,ffffffff,ffffffff,0,c5617d38,...) at dofileread+0x96
kern_readv(c5543220,0,e9b11c60,bfbfee4b,1,...) at kern_readv+0x58
read(c5543220,e9b11cfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(e9b11d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 925 tid 100120 td 0xc5863880
sched_switch(c5863880,0,1,180,ab30623c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5863880,0,c0aa0842,19b,c538c010,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c538c010,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c538c010,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c538c000,c538c010,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c538c000,ebe3ac60,0,ebe3ab9c,c067a61d,...) at ttread+0x4e7
ttyread(c537d600,ebe3ac60,0,ebe3abb8,c071e1c8,...) at ttyread+0x39
scread(c537d600,ebe3ac60,0,168,0,...) at scread+0x2d
giant_read(c537d600,ebe3ac60,0,0,1,...) at giant_read+0x48
devfs_read_f(c5853c60,ebe3ac60,c50f4300,0,c5863880,...) at devfs_read_f+0x71
dofileread(ebe3ac60,ffffffff,ffffffff,0,c5853c60,...) at dofileread+0x96
kern_readv(c5863880,0,ebe3ac60,bfbfee4b,1,...) at kern_readv+0x58
read(c5863880,ebe3acfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(ebe3ad38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 924 tid 100054 td 0xc5377440
sched_switch(c5377440,0,1,180,ab0373e0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5377440,0,c0aa0842,19b,c537fc10,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c537fc10,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c537fc10,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c537fc00,c537fc10,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c537fc00,e9addc60,0,e9addb9c,c067a61d,...) at ttread+0x4e7
ttyread(c537d500,e9addc60,0,e9addbb8,c071e1c8,...) at ttyread+0x39
scread(c537d500,e9addc60,0,168,0,...) at scread+0x2d
giant_read(c537d500,e9addc60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55e7cf0,e9addc60,c50f4300,0,c5377440,...) at devfs_read_f+0x71
dofileread(e9addc60,ffffffff,ffffffff,0,c55e7cf0,...) at dofileread+0x96
kern_readv(c5377440,0,e9addc60,bfbfee4b,1,...) at kern_readv+0x58
read(c5377440,e9addcfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(e9addd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command getty pid 923 tid 100089 td 0xc55c4000
sched_switch(c55c4000,0,1,180,ab35ec44,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c4000,0,c0aa0842,19b,c5380410,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,59,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5380410,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5380410,0,159,c0aa41dd,0,...) at _sleep+0x359
ttysleep(c5380400,c5380410,159,c0aa41dd,0,...) at ttysleep+0x39
ttread(c5380400,ebd5fc60,0,ebd5fb9c,c067a61d,...) at ttread+0x4e7
ttyread(c537d400,ebd5fc60,0,ebd5fbb8,c071e1c8,...) at ttyread+0x39
scread(c537d400,ebd5fc60,0,168,0,...) at scread+0x2d
giant_read(c537d400,ebd5fc60,0,0,1,...) at giant_read+0x48
devfs_read_f(c55a3678,ebd5fc60,c50f4300,0,c55c4000,...) at devfs_read_f+0x71
dofileread(ebd5fc60,ffffffff,ffffffff,0,c55a3678,...) at dofileread+0x96
kern_readv(c55c4000,0,ebd5fc60,bfbfee4b,1,...) at kern_readv+0x58
read(c55c4000,ebd5fcfc,c,c0acf182,c0b49ac8,...) at read+0x4f
syscall(ebd5fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28164443, esp =
0xbfbfee2c, ebp = 0xbfbfee58 ---
Tracing command sleep pid 921 tid 100100 td 0xc56e9440
sched_switch(c56e9440,0,1,180,87825700,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c56e9440,0,c0aa0842,19b,c0bb0824,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebda0bc4,c07423dd,c0bb8680,4,5c,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bb0824,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c0bb0824,0,15c,c0a9df4b,ea61,...) at _sleep+0x331
kern_nanosleep(c56e9440,ebda0c64,ebda0c6c,3c,0,...) at kern_nanosleep+0xc1
nanosleep(c56e9440,ebda0cfc,8,ebda0d38,c0b4b100,...) at nanosleep+0x6f
syscall(ebda0d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28135f2f, esp =
0xbfbfee5c, ebp = 0xbfbfee98 ---
Tracing command logger pid 920 tid 100066 td 0xc559ccc0
sched_switch(c559ccc0,0,1,180,8553358a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559ccc0,0,c0aa0842,19b,c586d948,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,4c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c586d948,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c586d948,c586dab8,14c,c0aa2569,0,...) at _sleep+0x359
pipe_read(c58a2dc8,ebd36c60,c50f4300,0,c559ccc0,...) at pipe_read+0x3f6
dofileread(ebd36c60,ffffffff,ffffffff,0,c58a2dc8,...) at dofileread+0x96
kern_readv(c559ccc0,0,ebd36c60,28206000,1000,...) at kern_readv+0x58
read(c559ccc0,ebd36cfc,c,c0aa289e,c0b49ac8,...) at read+0x4f
syscall(ebd36d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x28154443, esp =
0xbfbfe99c, ebp = 0xbfbfe9b8 ---
Tracing command sh pid 919 tid 100076 td 0xc5599660
sched_switch(c5599660,0,1,180,848ec352,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5599660,0,c0aa0842,19b,c559f2ac,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c559f2ac,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c559f2ac,c559f33c,15c,c0aa296f,0,...) at _sleep+0x359
kern_wait(c5599660,ffffffff,ebd18c2c,2,0,...) at kern_wait+0xa66
wait4(c5599660,ebd18cfc,10,c0aa294e,c0b49b28,...) at wait4+0x3b
syscall(ebd18d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149d7b, esp =
0xbfbfd8fc, ebp = 0xbfbfd918 ---
Tracing command sh pid 918 tid 100067 td 0xc559caa0
sched_switch(c559caa0,0,1,180,842dbd5e,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559caa0,0,c0aa0842,19b,c5598558,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5598558,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5598558,c55985e8,15c,c0aa296f,0,...) at _sleep+0x359
kern_wait(c559caa0,ffffffff,ebd33c2c,2,0,...) at kern_wait+0xa66
wait4(c559caa0,ebd33cfc,10,c0aa294e,c0b49b28,...) at wait4+0x3b
syscall(ebd33d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149d7b, esp =
0xbfbfda9c, ebp = 0xbfbfdab8 ---
Tracing command inetd pid 902 tid 100081 td 0xc55c7220
sched_switch(c55c7220,0,1,180,7daee5f4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c7220,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c55c7220,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,ebd77b18,...) at
_cv_wait_sig+0x1fc
kern_select(c55c7220,6,bfbfec8c,0,0,0,c55a44c8,bfbfee48) at
kern_select+0x6cc
select(c55c7220,ebd77cfc,14,c0aa29c2,c0b4a338,...) at select+0x5e
syscall(ebd77d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281773c3, esp =
0xbfbfe08c, ebp = 0xbfbfee78 ---
Tracing command moused pid 875 tid 100068 td 0xc559c880
sched_switch(c559c880,0,1,180,609d1204,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559c880,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c559c880,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,c065602f,...) at
_cv_wait_sig+0x1fc
kern_select(c559c880,400,bfbfeaac,0,0,0,4773a417,0) at kern_select+0x6cc
select(c559c880,ebd30cfc,14,c0acf182,c0b4a338,...) at select+0x5e
syscall(ebd30d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817c3c3, esp =
0xbfbfea2c, ebp = 0xbfbfebc8 ---
Tracing command cron pid 859 tid 100118 td 0xc5863cc0
sched_switch(c5863cc0,0,1,180,b0b05d04,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5863cc0,0,c0aa0842,19b,c0bb0824,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebe40bc4,c07423dd,c0bb8700,4,5c,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0bb0824,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c0bb0824,0,15c,c0a9df4b,ea61,...) at _sleep+0x331
kern_nanosleep(c5863cc0,ebe40c64,ebe40c6c,3c,0,...) at kern_nanosleep+0xc1
nanosleep(c5863cc0,ebe40cfc,8,c0aa2ab3,c0b4b100,...) at nanosleep+0x6f
syscall(ebe40d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x28150f2f, esp =
0xbfbfed0c, ebp = 0xbfbfed38 ---
Tracing command sshd pid 852 tid 100082 td 0xc55c7000
sched_switch(c55c7000,0,1,180,86031ef8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c7000,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c55c7000,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,ebd74afc,...) at
_cv_wait_sig+0x1fc
kern_select(c55c7000,7,2850b0b8,0,0,0,c,ebd74c98) at kern_select+0x6cc
select(c55c7000,ebd74cfc,14,ebd74d38,c0b4a338,...) at select+0x5e
syscall(ebd74d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283e33c3, esp =
0xbfbfe6cc, ebp = 0xbfbfee98 ---
Tracing command httpd pid 835 tid 100109 td 0xc5543000
sched_switch(c5543000,0,1,180,e4d583d8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5543000,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9b0ea84,c074278a,c0c01dc4,0,c5543000,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c01ddc,3e9,c0aba6ad,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c01ddc,c0c01dc4,3e9,30f,e9b0eafc,...) at
_cv_timedwait_sig+0x20b
kern_select(c5543000,0,0,0,0,e9b0ec70,1,0) at kern_select+0x6b4
select(c5543000,e9b0ecfc,14,c0aa278a,c0b4a338,...) at select+0x5e
syscall(e9b0ed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283333c3, esp =
0xbfbfec1c, ebp = 0xbfbfec48 ---
Tracing command mysqld pid 833 tid 100139 td 0xc591eaa0
sched_switch(c591eaa0,0,1,180,9e87e438,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591eaa0,0,c0aa0842,19b,c561f2c0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c561f2c0,c0bb14d0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c561f2c0,c0bb14d0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c591eaa0,ebe89cfc,ebe89d2c,c0a04743,c591eaa0,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c591eaa0,ebe89cfc,14,ebe89d38,c0b4c510,...) at _umtx_op+0x27
syscall(ebe89d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf1c5edc, ebp = 0xbf1c5ef8 ---
Tracing command mysqld pid 833 tid 100138 td 0xc591ecc0
sched_switch(c591ecc0,0,1,180,6dc2a8fc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591ecc0,0,c0aa0842,19b,ebe8cbe0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(ebe8cbe0,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(ebe8cbe0,c55c833c,168,c0a99fde,0,...) at _sleep+0x359
kern_sigtimedwait(c591ecc0,26005,0,0,0,...) at kern_sigtimedwait+0x2af
sigwait(c591ecc0,ebe8ccfc,8,c0aa2849,c0b4c2b8,...) at sigwait+0x77
syscall(ebe8cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (429, FreeBSD ELF32, sigwait), eip = 0x2864e6bb, esp =
0xbf1f6f1c, ebp = 0xbf1f6f48 ---
Tracing command mysqld pid 833 tid 100135 td 0xc591f000
sched_switch(c591f000,0,1,180,1453f504,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591f000,0,c0aa0842,19b,c561f240,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c561f240,c0bb1440,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c561f240,c0bb1440,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c591f000,ebe8fcfc,ebe8fd2c,c0a04743,c591f000,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c591f000,ebe8fcfc,14,ebe8fd38,c0b4c510,...) at _umtx_op+0x27
syscall(ebe8fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf2f7eac, ebp = 0xbf2f7ec8 ---
Tracing command mysqld pid 833 tid 100137 td 0xc591f220
sched_switch(c591f220,0,1,180,801e49e8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591f220,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebe92a84,c074278a,c0c01dc4,0,c591f220,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c01ddc,7d1,c0aba6ad,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c01ddc,c0c01dc4,7d1,30f,0,...) at
_cv_timedwait_sig+0x20b
kern_select(c591f220,0,0,0,0,ebe92c70,2,0) at kern_select+0x6b4
select(c591f220,ebe92cfc,14,16,c0b4a338,...) at select+0x5e
syscall(ebe92d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x286e03c3, esp =
0xbf3f8f0c, ebp = 0xbf3f8f38 ---
Tracing command mysqld pid 833 tid 100136 td 0xc591f440
sched_switch(c591f440,0,1,180,4272d1ae,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591f440,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebe95a84,c074278a,c0c01dc4,0,c591f440,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c01ddc,3e9,c0aba6ad,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c01ddc,c0c01dc4,3e9,30f,0,...) at
_cv_timedwait_sig+0x20b
kern_select(c591f440,0,0,0,0,ebe95c70,1,0) at kern_select+0x6b4
select(c591f440,ebe95cfc,14,16,c0b4a338,...) at select+0x5e
syscall(ebe95d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x286e03c3, esp =
0xbf4f9f0c, ebp = 0xbf4f9f38 ---
Tracing command mysqld pid 833 tid 100134 td 0xc591f880
sched_switch(c591f880,0,1,180,992f4b8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591f880,0,c0aa0842,19b,c561f140,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c561f140,c0bb19b0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c561f140,c0bb19b0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c591f880,ebe9bcfc,ebe9bd2c,c0a04743,c591f880,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c591f880,ebe9bcfc,14,c07871f8,c0b4c510,...) at _umtx_op+0x27
syscall(ebe9bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf6fbcfc, ebp = 0xbf6fbd18 ---
Tracing command mysqld pid 833 tid 100133 td 0xc591faa0
sched_switch(c591faa0,0,1,180,46f7d224,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c591faa0,0,c0aa0842,19b,c561f100,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c561f100,c0bb1710,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c561f100,c0bb1710,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c591faa0,ebe9ecfc,ebe9ed2c,c0a04743,c591faa0,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c591faa0,ebe9ecfc,14,ebe9ed38,c0b4c510,...) at _umtx_op+0x27
syscall(ebe9ed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf7fccfc, ebp = 0xbf7fcd18 ---
Tracing command mysqld pid 833 tid 100132 td 0xc55c7660
sched_switch(c55c7660,0,1,180,40ae3094,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c7660,0,c0aa0842,19b,c55ae740,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c55ae740,c0bb1470,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c55ae740,c0bb1470,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c55c7660,ebd7dcfc,ebd7dd2c,c0a04743,c55c7660,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c55c7660,ebd7dcfc,14,c0aa363a,c0b4c510,...) at _umtx_op+0x27
syscall(ebd7dd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf8fdcfc, ebp = 0xbf8fdd18 ---
Tracing command mysqld pid 833 tid 100131 td 0xc55c7880
sched_switch(c55c7880,0,1,180,9830e02,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c7880,0,c0aa0842,19b,c55ae700,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c55ae700,c0bb1200,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c55ae700,c0bb1200,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c55c7880,ebd80cfc,ebd80d2c,c0a04743,c55c7880,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c55c7880,ebd80cfc,14,c0aa363a,c0b4c510,...) at _umtx_op+0x27
syscall(ebd80d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x2864e39b, esp =
0xbf9fecfc, ebp = 0xbf9fed18 ---
Tracing command mysqld pid 833 tid 100086 td 0xc55c4660
sched_switch(c55c4660,0,1,180,9e866198,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c4660,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c55c4660,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,246,...) at _cv_wait_sig+0x1fc
kern_select(c55c4660,d,bfbfead8,0,0,0,0,2860772c) at kern_select+0x6cc
select(c55c4660,ebd68cfc,14,c0aa29c2,c0b4a338,...) at select+0x5e
syscall(ebd68d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x286e03c3, esp =
0xbfbfe6ac, ebp = 0xbfbfe6d8 ---
Tracing command sh pid 808 tid 100114 td 0xc5542440
sched_switch(c5542440,0,1,180,88d8c3f6,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5542440,0,c0aa0842,19b,c585fab0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c585fab0,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c585fab0,c585fb40,15c,c0aa296f,0,...) at _sleep+0x359
kern_wait(c5542440,ffffffff,e9affc2c,2,0,...) at kern_wait+0xa66
wait4(c5542440,e9affcfc,10,c0aa294e,c0b49b28,...) at wait4+0x3b
syscall(e9affd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x28149d7b, esp =
0xbfbfe7fc, ebp = 0xbfbfe818 ---
Tracing command qmgr pid 803 tid 100056 td 0xc5377000
sched_switch(c5377000,0,1,180,f72875dc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5377000,0,c0aa0842,19b,c587da00,...) at sleepq_switch+0xf0
sleepq_catch_signals(e9ad7aac,c07423dd,c0bb8700,4,58,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c587da00,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c587da00,c587da00,158,c0a99450,493e1,...) at _sleep+0x331
kern_kevent(c5377000,8,0,64,e9ad7c60,...) at kern_kevent+0x3c3
kevent(c5377000,e9ad7cfc,18,c0aa2ab3,c0b4bc88,...) at kevent+0x19b
syscall(e9ad7d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (363, FreeBSD ELF32, kevent), eip = 0x2839c14b, esp =
0xbfbfe16c, ebp = 0xbfbfe978 ---
Tracing command pickup pid 802 tid 100074 td 0xc5599aa0
sched_switch(c5599aa0,0,1,180,efe64654,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5599aa0,0,c0aa0842,19b,c55aab80,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebd1eaac,c07423dd,c0bb8700,4,58,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c55aab80,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c55aab80,c55aab80,158,c0a99450,186a1,...) at _sleep+0x331
kern_kevent(c5599aa0,8,0,64,ebd1ec60,...) at kern_kevent+0x3c3
kevent(c5599aa0,ebd1ecfc,18,c0aa2ab3,c0b4bc88,...) at kevent+0x19b
syscall(ebd1ed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (363, FreeBSD ELF32, kevent), eip = 0x2839014b, esp =
0xbfbfe19c, ebp = 0xbfbfe9a8 ---
Tracing command master pid 795 tid 100079 td 0xc5599000
sched_switch(c5599000,0,1,180,f7267f80,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5599000,0,c0aa0842,19b,c5564080,...) at sleepq_switch+0xf0
sleepq_catch_signals(ebd0faac,c07423dd,c0bb8680,4,58,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c5564080,0,c0a9d757,da,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c5564080,c5564080,158,c0a99450,ea61,...) at _sleep+0x331
kern_kevent(c5599000,c,0,64,ebd0fc60,...) at kern_kevent+0x3c3
kevent(c5599000,ebd0fcfc,18,c0aa2ab3,c0b4bc88,...) at kevent+0x19b
syscall(ebd0fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (363, FreeBSD ELF32, kevent), eip = 0x2838314b, esp =
0xbfbfe3ec, ebp = 0xbfbfebf8 ---
Tracing command ntpd pid 728 tid 100112 td 0xc5542880
sched_switch(c5542880,0,1,180,3ff48692,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5542880,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c5542880,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,0,...) at _cv_wait_sig+0x1fc
kern_select(c5542880,a,bfbfed7c,0,0,0,246,0) at kern_select+0x6cc
select(c5542880,e9b05cfc,14,16,c0b4a338,...) at select+0x5e
syscall(e9b05d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2830d3c3, esp =
0xbfbfed5c, ebp = 0xbfbfee18 ---
Tracing command named pid 643 tid 100130 td 0xc559d220
sched_switch(c559d220,0,1,180,149d8050,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559d220,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c559d220,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,0,...) at _cv_wait_sig+0x1fc
kern_select(c559d220,1a,bf4f9f10,bf4f9e90,0,0,8,2836572c) at
kern_select+0x6cc
select(c559d220,ebd3ccfc,14,c0a5633b,c0b4a338,...) at select+0x5e
syscall(ebd3cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2843e3c3, esp =
0xbf4f9cfc, ebp = 0xbf4f9d28 ---
Tracing command named pid 643 tid 100129 td 0xc559d440
sched_switch(c559d440,0,1,180,532312a8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559d440,0,c0aa0842,19b,c555d240,...) at sleepq_switch+0xf0
sleepq_catch_signals(c077f359,c559d5d4,7530,c077e7d0,0,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c555d240,7530,c0a9e334,100,0,...) at
sleepq_timedwait_sig+0x17
_sleep(c555d240,c0bb0900,100,c0a9e334,7530,...) at _sleep+0x331
__umtx_op_cv_wait(c559d440,ebd3fcfc,ebd3fd2c,c0a04743,c559d440,...) at
__umtx_op_cv_wait+0x430
_umtx_op(c559d440,ebd3fcfc,14,c0aa2df1,c0b4c510,...) at _umtx_op+0x27
syscall(ebd3fd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x283ac39b, esp =
0xbf5fae0c, ebp = 0xbf5fae28 ---
Tracing command named pid 643 tid 100128 td 0xc559d660
sched_switch(c559d660,0,1,180,52c4f6a4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559d660,0,c0aa0842,19b,c555d200,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c555d200,c0bb0ea0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c555d200,c0bb0ea0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c559d660,ebd42cfc,ebd42d2c,c0a04743,c559d660,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c559d660,ebd42cfc,14,c0aa2ab3,c0b4c510,...) at _umtx_op+0x27
syscall(ebd42d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x283ac39b, esp =
0xbf6fbecc, ebp = 0xbf6fbee8 ---
Tracing command named pid 643 tid 100127 td 0xc559d880
sched_switch(c559d880,0,1,180,532425b8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559d880,0,c0aa0842,19b,c555d1c0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c555d1c0,c0bb0ea0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c555d1c0,c0bb0ea0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c559d880,ebd45cfc,ebd45d2c,c0a04743,c559d880,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c559d880,ebd45cfc,14,c0aa2ab3,c0b4c510,...) at _umtx_op+0x27
syscall(ebd45d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x283ac39b, esp =
0xbf7fcecc, ebp = 0xbf7fcee8 ---
Tracing command named pid 643 tid 100126 td 0xc559daa0
sched_switch(c559daa0,0,1,180,14a409e8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559daa0,0,c0aa0842,19b,c555d180,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c555d180,c0bb0ea0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c555d180,c0bb0ea0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c559daa0,ebd48cfc,ebd48d2c,c0a04743,c559daa0,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c559daa0,ebd48cfc,14,c0aa2ab3,c0b4c510,...) at _umtx_op+0x27
syscall(ebd48d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x283ac39b, esp =
0xbf8fdecc, ebp = 0xbf8fdee8 ---
Tracing command named pid 643 tid 100125 td 0xc559dcc0
sched_switch(c559dcc0,0,1,180,1497db68,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c559dcc0,0,c0aa0842,19b,c555d140,...) at sleepq_switch+0xf0
sleepq_catch_signals(c0aa0842,155,0,100,0,...) at sleepq_catch_signals+0x21e
sleepq_wait_sig(c555d140,c0bb0ea0,c0a9e334,100,0,...) at
sleepq_wait_sig+0x14
_sleep(c555d140,c0bb0ea0,100,c0a9e334,0,...) at _sleep+0x359
__umtx_op_cv_wait(c559dcc0,ebd4bcfc,ebd4bd2c,c0a04743,c559dcc0,...) at
__umtx_op_cv_wait+0x354
_umtx_op(c559dcc0,ebd4bcfc,14,c0aa2ab3,c0b4c510,...) at _umtx_op+0x27
syscall(ebd4bd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (454, FreeBSD ELF32, _umtx_op), eip = 0x283ac39b, esp =
0xbf9feecc, ebp = 0xbf9feee8 ---
Tracing command named pid 643 tid 100107 td 0xc5543440
sched_switch(c5543440,0,1,180,ff77ecb0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5543440,0,c0aa0842,19b,e9b14be0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8700,4,c0a9bb4b,68,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(e9b14be0,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(e9b14be0,c585e5e8,168,c0a99fde,0,...) at _sleep+0x359
kern_sigtimedwait(c5543440,4003,0,0,0,...) at kern_sigtimedwait+0x2af
sigwait(c5543440,e9b14cfc,8,c0aa363a,c0b4c2b8,...) at sigwait+0x77
syscall(e9b14d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (429, FreeBSD ELF32, sigwait), eip = 0x283ac6bb, esp =
0xbfbfecdc, ebp = 0xbfbfed08 ---
Tracing command syslogd pid 578 tid 100116 td 0xc5542000
sched_switch(c5542000,0,1,180,b40d7874,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5542000,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c5542000,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,d9191108,...) at
_cv_wait_sig+0x1fc
kern_select(c5542000,b,282200b8,0,0,0,c56f2000,28204000) at
kern_select+0x6cc
select(c5542000,e9af9cfc,14,c0abbe3e,c0b4a338,...) at select+0x5e
syscall(e9af9d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281673c3, esp =
0xbfbfdefc, ebp = 0xbfbfee48 ---
Tracing command devd pid 517 tid 100099 td 0xc56e9660
sched_switch(c56e9660,0,1,180,ff905800,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c56e9660,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c56e9660,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,30f,ebda3afc,...) at
_cv_wait_sig+0x1fc
kern_select(c56e9660,5,bfbfedf8,0,0,0,0,bfbfe99c) at kern_select+0x6cc
select(c56e9660,ebda3cfc,14,c0aa2cb6,c0b4a338,...) at select+0x5e
syscall(ebda3d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x807f96f, esp =
0xbfbfe9bc, ebp = 0xbfbfee98 ---
Tracing command dhclient pid 321 tid 100058 td 0xc5265aa0
sched_switch(c5265aa0,0,1,180,386c9404,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5265aa0,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(e580ead8,c074278a,c0c01dc4,0,c5265aa0,...) at
sleepq_catch_signals+0x21e
sleepq_timedwait_sig(c0c01ddc,362f19,c0aba6ad,101,0,...) at
sleepq_timedwait_sig+0x17
_cv_timedwait_sig(c0c01ddc,c0c01dc4,362f19,3bb,e580eb48,...) at
_cv_timedwait_sig+0x20b
poll(c5265aa0,e580ecfc,c,c0a5633b,c0b4ae18,...) at poll+0x554
syscall(e580ed38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2810f8df, esp =
0xbfbfee1c, ebp = 0xbfbfee48 ---
Tracing command dhclient pid 301 tid 100088 td 0xc55c4220
sched_switch(c55c4220,0,1,180,83d8d1d0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c55c4220,0,c0aa0842,19b,c0c01ddc,...) at sleepq_switch+0xf0
sleepq_catch_signals(c074278a,c0c01dc4,0,c0a9bb4b,c55c4220,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c0c01ddc,c0c01dc4,c0aba6ad,101,0,...) at
sleepq_wait_sig+0x14
_cv_wait_sig(c0c01ddc,c0c01dc4,c0aa23c8,3bb,c0aa2465,...) at
_cv_wait_sig+0x1fc
poll(c55c4220,ebd62cfc,c,c0a8d9fa,c0b4ae18,...) at poll+0x56a
syscall(ebd62d38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x2810f8df, esp =
0xbfbfee1c, ebp = 0xbfbfee48 ---
Tracing command softdepflush pid 53 tid 100061 td 0xc5265440
sched_switch(c5265440,0,1,180,dd9baf10,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,44,...) at mi_switch+0x217
sleepq_switch(c5265440,0,c0aa0842,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c0db44,3e8,c0abc2bb,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c0db44,c0c0db1c,44,c0abc2bb,3e8,...) at _sleep+0x345
softdep_flush(0,e5805d38,c0a99763,2ea,c55412ac,...) at softdep_flush+0x2d2
fork_exit(c0935c40,0,e5805d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5805d70, ebp = 0 ---
Tracing command syncer pid 52 tid 100062 td 0xc5265220
sched_switch(c5265220,0,1,180,7e75a8c6,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5265220,...) at mi_switch+0x217
sleepq_switch(c5265220,0,c0aa0842,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0bb064c,0,c0aa8d55,0,0,...) at sleepq_wait+0x60
_sleep(c0bb064c,c0c02524,68,c0aa8d55,0,...) at _sleep+0x369
sched_sync(0,e5802d38,c0a99763,2ea,c5541558,...) at sched_sync+0x742
fork_exit(c07cba20,0,e5802d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5802d70, ebp = 0 ---
Tracing command vnlru pid 51 tid 100040 td 0xc5264aa0
sched_switch(c5264aa0,0,1,180,dd6b4bf0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,50,...) at mi_switch+0x217
sleepq_switch(c5264aa0,0,c0aa0842,240,50,...) at sleepq_switch+0xf0
sleepq_timedwait(c5374000,3e8,c0aa8e62,0,0,...) at sleepq_timedwait+0x68
_sleep(c5374000,c0c024f8,250,c0aa8e62,3e8,...) at _sleep+0x345
vnlru_proc(0,e57f9d38,c0a99763,2ea,c5374000,...) at vnlru_proc+0x115
fork_exit(c07cd020,0,e57f9d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57f9d70, ebp = 0 ---
Tracing command bufdaemon pid 50 tid 100041 td 0xc5264880
sched_switch(c5264880,0,1,180,dd6ac18a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,44,...) at mi_switch+0x217
sleepq_switch(c5264880,0,c0aa0842,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c02244,3e8,c0aa6a11,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c02244,c0c02248,44,c0aa6a11,3e8,...) at _sleep+0x345
buf_daemon(0,e57f6d38,c0a99763,2ea,c53742ac,...) at buf_daemon+0x21e
fork_exit(c07b7510,0,e57f6d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57f6d70, ebp = 0 ---
Tracing command pagezero pid 49 tid 100042 td 0xc5264660
sched_switch(c5264660,0,1,180,38314fb8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5264660,0,c0aa0842,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c0e700,493e0,c0ac0e41,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c0e700,c0c0e2a4,0,c0ac0e41,493e0,...) at _sleep+0x345
vm_pagezero(0,e57f3d38,c0a99763,2ea,c5374558,...) at vm_pagezero+0xdc
fork_exit(c096f430,0,e57f3d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57f3d70, ebp = 0 ---
Tracing command vmdaemon pid 48 tid 100043 td 0xc5264440
sched_switch(c5264440,0,1,180,3821bd24,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5264440,...) at mi_switch+0x217
sleepq_switch(c5264440,0,c0aa0842,21d,68,...) at sleepq_switch+0xf0
sleepq_wait(c0c0e318,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c0c0e318,c0c0e31c,68,c0aa6a11,0,...) at _sleep+0x369
vm_daemon(0,e57f0d38,c0a99763,2ea,c5374804,...) at vm_daemon+0x59
fork_exit(c096a870,0,e57f0d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57f0d70, ebp = 0 ---
Tracing command pagedaemon pid 47 tid 100044 td 0xc5264220
sched_switch(c5264220,0,1,180,1f4cbb44,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,44,...) at mi_switch+0x217
sleepq_switch(c5264220,0,c0aa0842,240,44,...) at sleepq_switch+0xf0
sleepq_timedwait(c0c0e2e0,1388,c0aa6a11,0,0,...) at sleepq_timedwait+0x68
_sleep(c0c0e2e0,c0c0e2a4,44,c0aa6a11,1388,...) at _sleep+0x345
vm_pageout(0,e57edd38,c0a99763,2ea,c5374ab0,...) at vm_pageout+0x2bb
fork_exit(c096b420,0,e57edd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57edd70, ebp = 0 ---
Tracing command sctp_iterator pid 46 tid 100045 td 0xc5264000
sched_switch(c5264000,0,1,180,770887a0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5264000,...) at mi_switch+0x217
sleepq_switch(c5264000,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c0c03fac,c0c03eb4,c0aafbe2,0,0,...) at sleepq_wait+0x60
_sleep(c0c03fac,c0c03eb4,0,c0aafbe2,0,...) at _sleep+0x369
sctp_iterator_thread(0,e9af6d38,c0a99763,2ea,c5375000,...) at
sctp_iterator_thread+0x60
fork_exit(c0833a90,0,e9af6d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9af6d70, ebp = 0 ---
Tracing command irq7: ppc0 pid 45 tid 100046 td 0xc5222cc0
fork_trampoline() at fork_trampoline
Tracing command fdc0 pid 44 tid 100047 td 0xc5222aa0
sched_switch(c5222aa0,0,1,180,c9a4620e,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5222aa0,0,c0aa0842,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c531ce3c,3e8,c0a94778,0,0,...) at sleepq_timedwait+0x68
_sleep(c531ce3c,c531cef0,4c,c0a94778,3e8,...) at _sleep+0x345
fdc_thread(c531ce00,e57bed38,c0a99763,2ea,c5375558,...) at fdc_thread+0x2b8
fork_exit(c09c82c0,c531ce00,e57bed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57bed70, ebp = 0 ---
Tracing command swi0: sio pid 43 tid 100048 td 0xc5222880
sched_switch(c5222880,0,1,180,da52d0f0,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c535cd64,...) at mi_switch+0x217
ithread_loop(c536d440,e57bbd38,c0a99763,2ea,c5375804,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c536d440,e57bbd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57bbd70, ebp = 0 ---
Tracing command irq12: psm0 pid 42 tid 100049 td 0xc5222660
fork_trampoline() at fork_trampoline
Tracing command irq1: atkbd0 pid 41 tid 100050 td 0xc5377cc0
sched_switch(c5377cc0,0,1,180,c36742f4,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c5111464,...) at mi_switch+0x217
ithread_loop(c536d820,e9ae9d38,c0a99763,2ea,c5376000,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c536d820,e9ae9d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe9ae9d70, ebp = 0 ---
Tracing command irq15: ata1 pid 40 tid 100030 td 0xc5221cc0
sched_switch(c5221cc0,0,1,180,3a187a8c,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c5110c64,...) at mi_switch+0x217
ithread_loop(c53582e0,e57acd38,c0a99763,2ea,c5262000,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c53582e0,e57acd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57acd70, ebp = 0 ---
Tracing command irq14: ata0 pid 39 tid 100031 td 0xc5221aa0
fork_trampoline() at fork_trampoline
Tracing command aic_recovery1 pid 38 tid 100032 td 0xc5221880
sched_switch(c5221880,0,1,180,6ffb8158,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5221880,...) at mi_switch+0x217
sleepq_switch(c5221880,0,c0aa0842,21d,a0,...) at sleepq_switch+0xf0
sleepq_wait(c531e000,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c531e000,c531b634,a0,c0a5fa6c,0,...) at _sleep+0x369
ahd_recovery_thread(c531e000,e57a6d38,c0a99763,2ea,c5262558,...) at
ahd_recovery_thread+0x72
fork_exit(c04c4080,c531e000,e57a6d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57a6d70, ebp = 0 ---
Tracing command irq22: ahd1 pid 37 tid 100033 td 0xc5221660
sched_switch(c5221660,0,1,180,34633be0,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c515e4e4,...) at mi_switch+0x217
ithread_loop(c534e950,e57a3d38,c0a99763,2ea,c5262804,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c534e950,e57a3d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57a3d70, ebp = 0 ---
Tracing command aic_recovery0 pid 36 tid 100034 td 0xc5221440
sched_switch(c5221440,0,1,180,6ffb359c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5221440,...) at mi_switch+0x217
sleepq_switch(c5221440,0,c0aa0842,21d,a0,...) at sleepq_switch+0xf0
sleepq_wait(c52d5000,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c52d5000,c52696b4,a0,c0a5fa6c,0,...) at _sleep+0x369
ahd_recovery_thread(c52d5000,e57a0d38,c0a99763,2ea,c5262ab0,...) at
ahd_recovery_thread+0x72
fork_exit(c04c4080,c52d5000,e57a0d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57a0d70, ebp = 0 ---
Tracing command aac0aif pid 35 tid 100035 td 0xc5221220
sched_switch(c5221220,0,1,180,405b856,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5221220,0,c0aa0842,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5263000,4e20,c0a5af1f,0,0,...) at sleepq_timedwait+0x68
_sleep(c5263000,c5277124,4c,c0a5af1f,4e20,...) at _sleep+0x345
aac_command_thread(c5277000,e579dd38,c0a99763,2ea,c5263000,...) at
aac_command_thread+0x8d
fork_exit(c0496410,c5277000,e579dd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe579dd70, ebp = 0 ---
Tracing command irq18: aac0 pid 34 tid 100036 td 0xc5221000
sched_switch(c5221000,0,1,180,b40a4750,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c515e6e4,...) at mi_switch+0x217
ithread_loop(c5276230,e579ad38,c0a99763,2ea,c52632ac,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c5276230,e579ad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe579ad70, ebp = 0 ---
Tracing command irq21: bge0 pid 33 tid 100037 td 0xc5214cc0
kdb_enter(c0a8e6e4,c536f8f8,c0c1b0e8,8,c536f88c,...) at kdb_enter+0x32
siointr1(c0c1b0e8,0,c0ac563d,56f,e57979dc,...) at siointr1+0x11c
siointr(c536f800,c0bb0844,2,c5214cc0,c5111280,...) at siointr+0x32
intr_execute_handlers(c5107cd0,e57979fc,e5797a58,c09eb844,3b,...) at
intr_execute_handlers+0xec
lapic_handle_intr(3b,e57979fc) at lapic_handle_intr+0x3f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc07429c2, esp = 0xe5797a3c, ebp = 0xe5797a58 ---
_mtx_lock_sleep(c538d07c,c5214cc0,0,c0aaaa4b,93,...) at _mtx_lock_sleep+0xa2
_mtx_lock_flags(c538d07c,0,c0aaaa4b,93,c0a9bb4b,...) at _mtx_lock_flags+0xef
rtalloc1(e5797bac,1,0,e5797b20,c07426fc,...) at rtalloc1+0x63
rtredirect(e5797bbc,e5797bac,0,26,e5797b9c,...) at rtredirect+0x35e
icmp_input(c57d1700,14,246,c0b9aca0,e5797c0c,...) at icmp_input+0x50f
ip_input(c57d1700,14e,800,c5250400,800,...) at ip_input+0x650
netisr_dispatch(2,c57d1700,10,3,0,...) at netisr_dispatch+0x73
ether_demux(c5250400,c57d1700,3,0,3,...) at ether_demux+0x1f1
ether_input(c5250400,c57d1700,c0a6b43a,bc4,c5250400,...) at
ether_input+0x37f
bge_intr(c526d000,0,c0a999ef,471,c515e564,...) at bge_intr+0x7da
ithread_loop(c5268640,e5797d38,c0a99763,2ea,c5263558,...) at
ithread_loop+0x1b5
fork_exit(c07328a0,c5268640,e5797d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5797d70, ebp = 0 ---
Tracing command usb2 pid 32 tid 100038 td 0xc5265000
sched_switch(c5265000,0,1,180,204964e,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5265000,0,c0aa0842,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5210210,ea60,c0a916e6,0,0,...) at sleepq_timedwait+0x68
_sleep(c5210210,0,5c,c0a916e6,ea60,...) at _sleep+0x345
usb_event_thread(c523f800,e57ffd38,c0a99763,2ea,c5263804,...) at
usb_event_thread+0xca
fork_exit(c06cc3b0,c523f800,e57ffd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57ffd70, ebp = 0 ---
Tracing command irq23: ehci0 pid 31 tid 100039 td 0xc5264cc0
fork_trampoline() at fork_trampoline
Tracing command usb1 pid 30 tid 100021 td 0xc5214220
sched_switch(c5214220,0,1,180,235540a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5214220,0,c0aa0842,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c5245210,ea60,c0a916e6,0,0,...) at sleepq_timedwait+0x68
_sleep(c5245210,0,5c,c0a916e6,ea60,...) at _sleep+0x345
usb_event_thread(c5253800,e5788d38,c0a99763,2ea,c52152ac,...) at
usb_event_thread+0xca
fork_exit(c06cc3b0,c5253800,e5788d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5788d70, ebp = 0 ---
Tracing command irq19: uhci1 ahd0 pid 29 tid 100022 td 0xc5214000
sched_switch(c5214000,0,1,180,1d89dcfc,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c515e664,...) at mi_switch+0x217
ithread_loop(c524f1b0,e5785d38,c0a99763,2ea,c5215558,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c524f1b0,e5785d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5785d70, ebp = 0 ---
Tracing command usbtask-dr pid 28 tid 100023 td 0xc5119cc0
sched_switch(c5119cc0,0,1,180,6ff99adc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5119cc0,...) at mi_switch+0x217
sleepq_switch(c5119cc0,0,c0aa0842,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bae2f4,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c0bae2f4,0,5c,c0a91723,0,...) at _sleep+0x369
usb_task_thread(c0bae2f4,e3d5fd38,c0a99763,2ea,c5215804,...) at
usb_task_thread+0x62
fork_exit(c06ccb30,c0bae2f4,e3d5fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d5fd70, ebp = 0 ---
Tracing command usbtask-hc pid 27 tid 100024 td 0xc5119aa0
sched_switch(c5119aa0,0,1,180,6ff957ac,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5119aa0,...) at mi_switch+0x217
sleepq_switch(c5119aa0,0,c0aa0842,21d,5c,...) at sleepq_switch+0xf0
sleepq_wait(c0bae2e0,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c0bae2e0,0,5c,c0a91723,0,...) at _sleep+0x369
usb_task_thread(c0bae2e0,e3d5cd38,c0a99763,2ea,c5215ab0,...) at
usb_task_thread+0x62
fork_exit(c06ccb30,c0bae2e0,e3d5cd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d5cd70, ebp = 0 ---
Tracing command usb0 pid 26 tid 100025 td 0xc5119880
sched_switch(c5119880,0,1,180,235937a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,5c,...) at mi_switch+0x217
sleepq_switch(c5119880,0,c0aa0842,240,5c,...) at sleepq_switch+0xf0
sleepq_timedwait(c524b210,ea60,c0a916e6,0,0,...) at sleepq_timedwait+0x68
_sleep(c524b210,0,5c,c0a916e6,ea60,...) at _sleep+0x345
usb_event_thread(c5254280,e3d59d38,c0a99763,2ea,c5220000,...) at
usb_event_thread+0xca
fork_exit(c06cc3b0,c5254280,e3d59d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d59d70, ebp = 0 ---
Tracing command irq16: uhci0 pid 25 tid 100026 td 0xc5119660
fork_trampoline() at fork_trampoline
Tracing command irq9: acpi0 pid 24 tid 100027 td 0xc5222440
fork_trampoline() at fork_trampoline
Tracing command kqueue taskq pid 23 tid 100028 td 0xc5222220
sched_switch(c5222220,0,1,180,70473070,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5222220,...) at mi_switch+0x217
sleepq_switch(c5222220,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5212600,c521261c,c0a94778,0,0,...) at sleepq_wait+0x60
_sleep(c5212600,c521261c,0,c0a94778,0,...) at _sleep+0x369
taskqueue_thread_loop(c0baeec4,e57b2d38,c0a99763,2ea,c5220804,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780c30,c0baeec4,e57b2d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57b2d70, ebp = 0 ---
Tracing command swi6: task queue pid 22 tid 100029 td 0xc5222000
sched_switch(c5222000,0,1,180,cc82e284,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c52126e4,...) at mi_switch+0x217
ithread_loop(c5206120,e57afd38,c0a99763,2ea,c5220ab0,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c5206120,e57afd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe57afd70, ebp = 0 ---
Tracing command swi6: Giant taskq pid 21 tid 100013 td 0xc5117880
fork_trampoline() at fork_trampoline
Tracing command thread taskq pid 9 tid 100014 td 0xc5117660
sched_switch(c5117660,0,1,180,348742c8,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5117660,...) at mi_switch+0x217
sleepq_switch(c5117660,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5212880,c521289c,c0a94778,0,0,...) at sleepq_wait+0x60
_sleep(c5212880,c521289c,0,c0a94778,0,...) at _sleep+0x369
taskqueue_thread_loop(c0bba7d8,e3d41d38,c0a99763,2ea,c515fab0,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780c30,c0bba7d8,e3d41d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d41d70, ebp = 0 ---
Tracing command acpi_task_2 pid 8 tid 100015 td 0xc5117440
sched_switch(c5117440,0,1,180,700a22a0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5117440,...) at mi_switch+0x217
sleepq_switch(c5117440,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5212900,c521291c,c0a94778,0,0,...) at sleepq_wait+0x60
_sleep(c5212900,c521291c,0,c0a94778,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d7eebc,e3d3ed38,c0a99763,2ea,c5213000,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780c30,c0d7eebc,e3d3ed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d3ed70, ebp = 0 ---
Tracing command acpi_task_1 pid 7 tid 100016 td 0xc5117220
sched_switch(c5117220,0,1,180,7009e0a0,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5117220,...) at mi_switch+0x217
sleepq_switch(c5117220,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5212900,c521291c,c0a94778,0,0,...) at sleepq_wait+0x60
_sleep(c5212900,c521291c,0,c0a94778,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d7eebc,e3d3bd38,c0a99763,2ea,c52132ac,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780c30,c0d7eebc,e3d3bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d3bd70, ebp = 0 ---
Tracing command acpi_task_0 pid 6 tid 100017 td 0xc5214aa0
sched_switch(c5214aa0,0,1,180,70099d0c,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5214aa0,...) at mi_switch+0x217
sleepq_switch(c5214aa0,0,c0aa0842,21d,0,...) at sleepq_switch+0xf0
sleepq_wait(c5212900,c521291c,c0a94778,0,0,...) at sleepq_wait+0x60
_sleep(c5212900,c521291c,0,c0a94778,0,...) at _sleep+0x369
taskqueue_thread_loop(c0d7eebc,e5794d38,c0a99763,2ea,c5213558,...) at
taskqueue_thread_loop+0xb4
fork_exit(c0780c30,c0d7eebc,e5794d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe5794d70, ebp = 0 ---
Tracing command swi5: + pid 20 tid 100018 td 0xc5214880
fork_trampoline() at fork_trampoline
Tracing command swi2: cambio pid 19 tid 100019 td 0xc5214660
sched_switch(c5214660,0,1,180,34663108,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c5212ae4,...) at mi_switch+0x217
ithread_loop(c5206170,e578ed38,c0a99763,2ea,c5213ab0,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c5206170,e578ed38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe578ed70, ebp = 0 ---
Tracing command xpt_thrd pid 5 tid 100020 td 0xc5214440
sched_switch(c5214440,0,1,180,6ff8aad4,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5214440,...) at mi_switch+0x217
sleepq_switch(c5214440,0,c0aa0842,21d,4c,...) at sleepq_switch+0xf0
sleepq_wait(c0b7f514,0,c0a9d757,da,0,...) at sleepq_wait+0x60
_sleep(c0b7f514,c0b7f52c,4c,c0a535da,0,...) at _sleep+0x369
xpt_scanner_thread(0,e578bd38,c0a99763,2ea,c5215000,...) at
xpt_scanner_thread+0x41
fork_exit(c0468360,0,e578bd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe578bd70, ebp = 0 ---
Tracing command yarrow pid 18 tid 100006 td 0xc5115220
sched_switch(c5115220,0,1,180,79be0e56,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5115220,0,c0aa0842,240,0,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bb0654,64,c0a94778,2,0,...) at sleepq_timedwait+0x68
_sleep(c0bb0654,0,0,c0a94778,64,...) at _sleep+0x345
pause(c0a94778,64,c0a88f43,112,0,...) at pause+0x47
random_kthread(0,e3d26d38,c0a99763,2ea,c51182ac,...) at random_kthread+0x1d9
fork_exit(c0656d20,0,e3d26d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d26d70, ebp = 0 ---
Tracing command g_down pid 4 tid 100007 td 0xc5115000
sched_switch(c5115000,0,1,180,7c36f0fe,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5115000,0,c0aa0842,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bae6cc,64,c0a94778,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bae6cc,c0bae5e8,24c,c0a94778,64,...) at _sleep+0x345
g_io_schedule_down(c5115000,0,c0a95db7,74,0,...) at g_io_schedule_down+0x6b
g_down_procbody(0,e3d23d38,c0a99763,2ea,c5118558,...) at
g_down_procbody+0x8d
fork_exit(c07019c0,0,e3d23d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d23d70, ebp = 0 ---
Tracing command g_up pid 3 tid 100008 td 0xc5119440
sched_switch(c5119440,0,1,180,7c372a56,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5119440,0,c0aa0842,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bae6c8,64,c0a94778,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bae6c8,c0bae628,24c,c0a94778,64,...) at _sleep+0x345
g_io_schedule_up(c5119440,0,c0a95db7,5d,0,...) at g_io_schedule_up+0x133
g_up_procbody(0,e3d53d38,c0a99763,2ea,c5118804,...) at g_up_procbody+0x8d
fork_exit(c0701a50,0,e3d53d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d53d70, ebp = 0 ---
Tracing command g_event pid 2 tid 100009 td 0xc5119220
sched_switch(c5119220,0,1,180,79eea34a,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,4c,...) at mi_switch+0x217
sleepq_switch(c5119220,0,c0aa0842,240,4c,...) at sleepq_switch+0xf0
sleepq_timedwait(c0bae6c0,64,c0a94778,0,0,...) at sleepq_timedwait+0x68
_sleep(c0bae6c0,0,4c,c0a94778,64,...) at _sleep+0x345
g_event_procbody(0,e3d50d38,c0a99763,2ea,c5118ab0,...) at
g_event_procbody+0xcb
fork_exit(c0701ae0,0,e3d50d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d50d70, ebp = 0 ---
Tracing command swi3: vm pid 17 tid 100010 td 0xc5119000
fork_trampoline() at fork_trampoline
Tracing command swi4: clock sio pid 16 tid 100011 td 0xc5117cc0
cpustop_handler(8,e3d4abd0,c0a04a50,38,c0aa17fa,...) at cpustop_handler+0x32
ipi_nmi_handler(38,c0aa17fa,6ce,c0c01c00,13,...) at ipi_nmi_handler+0x2f
trap(e3d4abdc) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc07429df, esp = 0xe3d4ac1c, ebp = 0xe3d4ac38 ---
_mtx_lock_sleep(c58887a4,c5117cc0,0,c0ab3291,1b6,...) at
_mtx_lock_sleep+0xbf
_mtx_lock_flags(c58887a4,0,c0ab3291,1b6,10207,...) at _mtx_lock_flags+0xef
tcp_timer_rexmt(c588a570,0,c0a9e01d,ef,0,...) at tcp_timer_rexmt+0x96
softclock(0,0,c0a999ef,471,c515e364,...) at softclock+0x266
ithread_loop(c5113260,e3d4ad38,c0a99763,2ea,c515f2ac,...) at
ithread_loop+0x1b5
fork_exit(c07328a0,c5113260,e3d4ad38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d4ad70, ebp = 0 ---
Tracing command swi1: net pid 15 tid 100012 td 0xc5117aa0
sched_switch(c5117aa0,0,1,180,386ad10c,...) at sched_switch+0x49b
mi_switch(1,0,c0a999ef,46b,c515e3e4,...) at mi_switch+0x217
ithread_loop(c5113270,e3d47d38,c0a99763,2ea,c515f558,...) at
ithread_loop+0x34c
fork_exit(c07328a0,c5113270,e3d47d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d47d70, ebp = 0 ---
Tracing command idle: cpu0 pid 14 tid 100000 td 0xc5117000
sched_switch(c5117000,0,6,17b,385804f8,...) at sched_switch+0x49b
mi_switch(6,0,c0a9ec6c,b6,0,...) at mi_switch+0x217
critical_exit(c515e500,0,2,c5117000,c515e500,...) at critical_exit+0x99
intr_execute_handlers(c5107f34,e3d38c50,e3d38c90,c09eb844,34,...) at
intr_execute_handlers+0x14b
lapic_handle_intr(34,e3d38c50) at lapic_handle_intr+0x3f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc0d6e4a5, esp = 0xe3d38c90, ebp = 0xe3d38c90 ---
acpi_cpu_c1(c0b4fdf8,c5117000,e3d38cac,0,c0bb8680,...) at acpi_cpu_c1+0x5
acpi_cpu_idle(e3d38cf8,c076c949,c0bb8680,0,c0a9e7d2,...) at
acpi_cpu_idle+0x18c
cpu_idle(c0bb8680,0,c0a9e7d2,322,c0bb2280,...) at cpu_idle+0x28
sched_idletd(0,e3d38d38,c0a99763,2ea,c5114000,...) at sched_idletd+0x249
fork_exit(c076c700,0,e3d38d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d38d70, ebp = 0 ---
Tracing command idle: cpu1 pid 13 tid 100001 td 0xc5115cc0
sched_switch(c5115cc0,0,1,17b,37c397f4,...) at sched_switch+0x49b
mi_switch(1,0,c0a9e7d2,320,c0bb28c0,...) at mi_switch+0x217
sched_idletd(0,e3d35d38,c0a99763,2ea,c51142ac,...) at sched_idletd+0x217
fork_exit(c076c700,0,e3d35d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d35d70, ebp = 0 ---
Tracing command idle: cpu2 pid 12 tid 100002 td 0xc5115aa0
cpustop_handler(4,e3d32c44,c0a04a50,38,c0aa17fa,...) at cpustop_handler+0x32
ipi_nmi_handler(38,c0aa17fa,6ce,c0c01c00,13,...) at ipi_nmi_handler+0x2f
trap(e3d32c50) at trap+0x30
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0d6e4a5, esp = 0xe3d32c90, ebp = 0xe3d32c90 ---
acpi_cpu_c1(c0b4fdf8,c5115aa0,e3d32cac,2,c0bb8700,...) at acpi_cpu_c1+0x5
acpi_cpu_idle(e3d32cf8,c076c949,c0bb8700,0,c0a9e7d2,...) at
acpi_cpu_idle+0x18c
cpu_idle(c0bb8700,0,c0a9e7d2,322,c0bb2f00,...) at cpu_idle+0x28
sched_idletd(0,e3d32d38,c0a99763,2ea,c5114558,...) at sched_idletd+0x249
fork_exit(c076c700,0,e3d32d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d32d70, ebp = 0 ---
Tracing command idle: cpu3 pid 11 tid 100003 td 0xc5115880
sched_switch(c5115880,0,1,17b,879e466e,...) at sched_switch+0x49b
mi_switch(1,0,c0a9e7d2,320,c0bb3540,...) at mi_switch+0x217
sched_idletd(0,e3d2fd38,c0a99763,2ea,c5114804,...) at sched_idletd+0x217
fork_exit(c076c700,0,e3d2fd38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d2fd70, ebp = 0 ---
Tracing command init pid 1 tid 100004 td 0xc5115660
sched_switch(c5115660,0,1,180,93649798,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,0,...) at mi_switch+0x217
sleepq_switch(c5115660,0,c0aa0842,19b,c5114ab0,...) at sleepq_switch+0xf0
sleepq_catch_signals(c07423dd,c0bb8680,4,c0a9bb4b,5c,...) at
sleepq_catch_signals+0x21e
sleepq_wait_sig(c5114ab0,0,c0a9d757,da,0,...) at sleepq_wait_sig+0x14
_sleep(c5114ab0,c5114b40,15c,c0aa296f,0,...) at _sleep+0x359
kern_wait(c5115660,ffffffff,e3d2cc2c,0,0,...) at kern_wait+0xa66
wait4(c5115660,e3d2ccfc,10,e3d2cd38,c0b49b28,...) at wait4+0x3b
syscall(e3d2cd38) at syscall+0x2b3
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x805455f, esp =
0xbfbfe97c, ebp = 0xbfbfe998 ---
Tracing command audit pid 10 tid 100005 td 0xc5115440
sched_switch(c5115440,0,1,180,6fef90bc,...) at sched_switch+0x49b
mi_switch(1,0,c0aa0842,1bf,c5115440,...) at mi_switch+0x217
sleepq_switch(c5115440,0,c0aa0842,21d,c5115440,...) at sleepq_switch+0xf0
sleepq_wait(c0c0d5b4,c0c0d594,c0ab9dd3,1,0,...) at sleepq_wait+0x60
_cv_wait(c0c0d5b4,c0c0d594,c0aba790,1c2,0,...) at _cv_wait+0x1fc
audit_worker(0,e3d29d38,c0a99763,2ea,c5118000,...) at audit_worker+0x99
fork_exit(c091b750,0,e3d29d38) at fork_exit+0xb8
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe3d29d70, ebp = 0 ---
Tracing command swapper pid 0 tid 0 td 0xc0baea30
sched_switch(c0baea30,0,1,180,3991d04c,...) at sched_switch+0x49b
mi_switch(1,0,c0abea20,2d9,c0ac973d,...) at mi_switch+0x217
scheduler(0,101ec00,101ec00,101e000,1028000,...) at scheduler+0x2f2
mi_startup() at mi_startup+0x96
begin() at begin+0x2c
db> show allocks
No such command
db> show alllocks
Process 943 (sshd) thread 0xc5861440 (100144)
exclusive sleep mutex radix node head r = 0 (0xc538d07c) locked @
/usr/src/sys/net/route.c:147
exclusive sleep mutex inp (tcpinp) r = 0 (0xc58887a4) locked @
/usr/src/sys/netinet/tcp_usrreq.c:779
exclusive sx so_snd_sx r = 0 (0xc5903430) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 940 (sshd) thread 0xc5861220 (100145)
exclusive sx so_rcv_sx r = 0 (0xc58a8530) locked @
/usr/src/sys/kern/uipc_sockbuf.c:145
Process 33 (irq21: bge0) thread 0xc5214cc0 (100037)
exclusive sleep mutex rtentry r = 0 (0xc5722c18) locked @
/usr/src/sys/net/route.c:1065
exclusive spin mutex sio r = 0 (0xc0c1b0e8) locked @
/usr/src/sys/dev/sio/sio.c:1391
Process 16 (swi4: clock sio) thread 0xc5117cc0 (100011)
exclusive sleep mutex tcp r = 0 (0xc0c0472c) locked @
/usr/src/sys/netinet/tcp_timer.c:423
db> show lockedvnods
Locked vnodes
db> call boot(0)
panic: blockable sleep lock (sleep mutex) eventhandler @
/usr/src/sys/kern/subr_eventhandler.c:212
cpuid = 0
KDB: enter: panic
[thread pid 33 tid 100037 ]
Stopped at kdb_enter+0x32: leave
db> reboot
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200712271320.lBRDK2R8006784>