Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 11 Mar 2007 00:26:37 -0600
From:      Tillman Hodgson <tillman@seekingfire.com>
To:        freebsd-current@freebsd.org
Subject:   Re: Experiencing hangs on SMP box with no console messages given for clues. Details inside.
Message-ID:  <20070311062637.GA1256@seekingfire.com>
In-Reply-To: <20070311044033.GB1256@seekingfire.com>
References:  <20070308125927.GA1265@seekingfire.com> <20070308204041.GA55240@xor.obsecurity.org> <20070310153206.GF1230@seekingfire.com> <3bbf2fe10703100749h14e9b075wb6d730ed7c9189f8@mail.gmail.com> <20070310161423.GA1256@seekingfire.com> <20070310193946.GA96514@xor.obsecurity.org> <20070311044033.GB1256@seekingfire.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Mar 10, 2007 at 10:40:33PM -0600, Tillman Hodgson wrote:
> Shouldn't take more than a day or two to get the info requested in
> 11.9.

As it turns out, a few hours.

After capturing the information below I ran `panic`. While booting, the
following lock messages came up -- I thought it might be related so I'll
post it here:

...
Starting spamd.
lock order reversal:
 1st 0xc44d8c44 pf task mtx (pf task mtx) @ /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:6414
 2nd 0xc0a9df6c tcp (tcp) @ /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:2760
KDB: stack backtrace:
db_trace_self_wrapper(c094da94) at db_trace_self_wrapper+0x25
kdb_backtrace(0,ffffffff,c0a5d7d8,c0a5f330,c09f6c04,...) at kdb_backtrace+0x29
witness_checkorder(c0a9df6c,9,c44d5e24,ac8) at witness_checkorder+0x586
_mtx_lock_flags(c0a9df6c,0,c44d5e24,ac8,c0a9df6c,...) at _mtx_lock_flags+0x84
pf_socket_lookup(e466da84,e466da88,1,e466db40,0,...) at pf_socket_lookup+0x1d3
pf_test_tcp(e466daf0,e466dae8,1,c4505d00,c4467700,...) at pf_test_tcp+0x11e6
pf_test(1,c4098800,e466dbdc,0,0,...) at pf_test+0xae3
pf_check_in(0,e466dbdc,c4098800,1,0) at pf_check_in+0x37
pfil_run_hooks(c0a9db40,e466dc2c,c4098800,1,0) at pfil_run_hooks+0x7f
ip_input(c4467700) at ip_input+0x232
netisr_dispatch(2,c4467700,0,c4098800,c44a0800,...) at netisr_dispatch+0x58
ether_demux(c4098800,c4467700,c4037000,c44a1002,e466dca0,...) at ether_demux+0x28a
ether_input(c4098800,c4467700,c4037014,0,c092325d,...) at ether_input+0x202
fxp_intr_body(c4037000,c4098800,40,ffffffff) at fxp_intr_body+0x1a0
fxp_intr(c4037000) at fxp_intr+0x95
ithread_execute_handlers(c40a0240,c3f46900) at ithread_execute_handlers+0x11e
ithread_loop(c40433a0,e466dd38) at ithread_loop+0x67
fork_exit(c06adf00,c40433a0,e466dd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe466dd70, ebp = 0 ---
Starting courier_authdaemond.
...

And here's the section 11.9 information from the debugger (via serial console
cut 'n paste):

telnet> send brk 
KDB: enter: Line break on console
[thread pid 11 tid 100005 ]
Stopped at      kdb_enter+0x2b: nop

db> where
Tracing pid 11 tid 100005 td 0xc3eff6c0
kdb_enter(c093cb3b) at kdb_enter+0x2b
siointr1(c40b4000,c0abdfac,0,c096e56b,56e,...) at siointr1+0xce
siointr(c40b4000) at siointr+0x21
intr_execute_handlers(c3ef4cc8,e2986c94,4,e2986cd4,c08ab2c4,...) at
intr_execute_handlers+0xe1
lapic_handle_intr(36,e2986c94) at lapic_handle_intr+0x2f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc0c000b5, esp = 0xe2986cd4, ebp = 0xe2986cd4 ---
acpi_cpu_c1(c0a4ea90,c06d85a8,e2986cf4,c06d85a8,c06d85a8,...) at
acpi_cpu_c1+0x5
acpi_cpu_idle(e2986d04,c06d85d1,e2986d24,c06acf40,0,...) at
acpi_cpu_idle+0x162
cpu_idle(e2986d24,c06acf40,0,e2986d38,c3efeb40,...) at cpu_idle+0x28
sched_idletd(0,e2986d38) at sched_idletd+0x29
fork_exit(c06d85a8,0,e2986d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2986d70, ebp = 0 ---

db> ps
  pid  ppid  pgrp   uid   state   wmesg     wchan    cmd
10970 10969 10969    22  S       select   0xc0a9be5c sshd
10969  1133 10969     0  Ss      sbwait   0xc46c99b0 sshd
10967  1133 10967     0  LEs    *pf task  0xc46c7700 sshd
10904  1090  1090     0  S       select   0xc0a9be5c local
10903  1090  1090   125  S       select   0xc0a9be5c cleanup
10902  1090  1090   125  S       select   0xc0a9be5c proxymap
10814     0     0     0  SL      -        0xc0aa366c [nfsiod 19]
10813     0     0     0  SL      -        0xc0aa3668 [nfsiod 18]
10812     0     0     0  SL      -        0xc0aa3664 [nfsiod 17]
10811     0     0     0  SL      -        0xc0aa3660 [nfsiod 16]
10810     0     0     0  SL      -        0xc0aa365c [nfsiod 15]
10809     0     0     0  SL      -        0xc0aa3658 [nfsiod 14]
10808     0     0     0  SL      -        0xc0aa3654 [nfsiod 13]
10807     0     0     0  SL      -        0xc0aa3650 [nfsiod 12]
10806     0     0     0  SL      -        0xc0aa364c [nfsiod 11]
10805     0     0     0  SL      -        0xc0aa3648 [nfsiod 10]
10804     0     0     0  SL      -        0xc0aa3644 [nfsiod 9]
10803     0     0     0  SL      -        0xc0aa3640 [nfsiod 8]
10802     0     0     0  SL      -        0xc0aa363c [nfsiod 7]
10801     0     0     0  SL      -        0xc0aa3638 [nfsiod 6]
10800     0     0     0  SL      -        0xc0aa3634 [nfsiod 5]
10799     0     0     0  SL      -        0xc0aa3630 [nfsiod 4]
10798     0     0     0  SL      -        0xc0aa362c [nfsiod 3]
10797     0     0     0  SL      -        0xc0aa3628 [nfsiod 2]
10796     0     0     0  SL      -        0xc0aa3624 [nfsiod 1]
10791     0     0     0  SL      -        0xc0aa3620 [nfsiod 0]
10787  1090  1090   125  S       select   0xc0a9be5c trivial-rewrite
10393  1090  1090   125  S       select   0xc0a9be5c pickup
 9973  1090  1090   125  S       select   0xc0a9be5c smtpd
 9907  1025  1025     0  S       select   0xc0a9be5c perl5.8.8
 8792  1090  1090   125  S       select   0xc0a9be5c anvil
 1538  1537  1538     0  Ss+     ttyin    0xc401bc10 bash
 1537  1296  1537     0  S+      select   0xc0a9be5c script
 1374  1257  1374   500  S+      select   0xc0a9be5c irssi
 1296  1295  1296     0  S+      wait     0xc4341900 bash
 1295  1270  1295     0  Ss+     wait     0xc4858000 login
 1276  1254  1276   500  Ss+     ttyin    0xc40d7810 bash
 1270  1174  1270     0  Ss      select   0xc0a9be5c telnetd
 1269  1254  1269   500  Ss+     ttyin    0xc40ba410 bash
 1267  1254  1267   500  Ss+     ttyin    0xc40ba810 bash
 1266  1254  1266   500  Ss+     ttyin    0xc4027810 bash
 1264  1254  1264   500  Ss+     ttyin    0xc4027410 bash
 1260  1254  1260   500  Ss+     ttyin    0xc40c4410 bash
 1259  1254  1259   500  Ss+     select   0xc0a9be5c telnet
 1258  1254  1258   500  Ss+     select   0xc0a9be5c telnet
 1257  1254  1257   500  Ss+     wait     0xc484e000 bash
 1256  1254  1256   500  Ss+     select   0xc0a9be5c mutt
 1254     1  1254   500  Ss      select   0xc0a9be5c screen
 1205  1025  1025     0  S       select   0xc0a9be5c perl5.8.8
 1198     1  1198     0  Ss+     ttyin    0xc409ac10 getty
 1197     1  1197     0  Ss+     ttyin    0xc40cc810 getty
 1196     1  1196     0  Ss+     ttyin    0xc40c5010 getty
 1195     1  1195     0  Ss+     ttyin    0xc40c3810 getty
 1194     1  1194     0  Ss+     ttyin    0xc40bb410 getty
 1193     1  1193     0  Ss+     ttyin    0xc40c4010 getty
 1174     1  1174     0  Ss      select   0xc0a9be5c inetd
 1141     1  1141     0  Ss      nanslp   0xc0a4f2a4 cron
 1133     1  1133     0  Ss      select   0xc0a9be5c sshd
 1119  1118  1119     0  S+      select   0xc0a9be5c couriertcpd
 1118     1  1118     0  S+      piperd   0xc46ad318 courierlogger
 1107  1106  1107     0  S+      select   0xc0a9be5c couriertcpd
 1106     1  1106     0  S+      piperd   0xc46ad7bc courierlogger
 1102  1090  1090   125  S       select   0xc0a9be5c qmgr
 1090     1  1090     0  Ss      select   0xc0a9be5c master
 1044  1033  1032     0  S+      select   0xc0a9be5c authdaemond
 1043  1033  1032     0  S+      select   0xc0a9be5c authdaemond
 1042  1033  1032     0  S+      select   0xc0a9be5c authdaemond
 1041  1033  1032     0  S+      select   0xc0a9be5c authdaemond
 1040  1033  1032     0  S+      select   0xc0a9be5c authdaemond
 1033  1032  1032     0  S+      select   0xc0a9be5c authdaemond
 1032     1  1032     0  S+      piperd   0xc4298948 courierlogger
 1025     1  1025     0  Ss      select   0xc0a9be5c perl5.8.8
 1018     1  1018  1001  Ss      select   0xc0a9be5c dhcpd
  924     1   924     1  Ss      sbwait   0xc45b89b0 rwhod
  911     1   911     0  Ss      select   0xc0a9be5c ntpd
  895     1   895     0  Ss      select   0xc0a9be5c lpd
  874   866   866     0  S       nfslockd 0xc0aa3848 rpc.lockd
  866     1   866     0  Ss      select   0xc0a9be5c rpc.lockd
  861     1   861     0  Ss      select   0xc0a9be5c rpc.statd
  827     1   827     0  Ss      select   0xc0a9be5c rpcbind
  817     1   817    53  Ss      select   0xc0a9be5c named
  754     1   754     0  Ss      select   0xc0a9be5c syslogd
  729     1   729     0  Ss      select   0xc0a9be5c watchquagga
  724     1   724     0  Ss      select   0xc0a9be5c ospfd
  718     1   718     0  Ss      select   0xc0a9be5c zebra
  658     1   658     0  Ss      select   0xc0a9be5c devd
  436   431   431    64  S       bpf      0xc4496e00 pflogd
  431     1   431     0  Ss      sbwait   0xc433a1e8 pflogd
  165     1   165     0  Ss      pause    0xc40a06e4 adjkerntz
   41     0     0     0  SL      -        0xe46c8cfc [schedcpu]
   40     0     0     0  SL      sdflush  0xc0aa8a84 [softdepflush]
   39     0     0     0  SL      vlruwt   0xc4249000 [vnlru]
   38     0     0     0  SL      syncer   0xc0a4f06c [syncer]
   37     0     0     0  SL      psleep   0xc0a9c2e8 [bufdaemon]
   36     0     0     0  SL      pgzero   0xc0ab13d0 [pagezero]
   35     0     0     0  SL      psleep   0xc0aa9300 [vmdaemon]
   34     0     0     0  SL      psleep   0xc0aa92c0 [pagedaemon]
   33     0     0     0  WL                          [irq7: ppc0]
   32     0     0     0  WL                          [irq1: atkbd0]
   31     0     0     0  WL                          [swi0: sio]
   30     0     0     0  SL      -        0xc4041200 [em0 taskq]
   29     0     0     0  LL     *tcp      0xc4045b80 [irq18: fxp1 em0]
   28     0     0     0  WL                          [irq17: fxp0]
   27     0     0     0  SL      usbevt   0xc402a210 [usb1]
   26     0     0     0  SL      usbtsk   0xc0a4c854 [usbtask-dr]
   25     0     0     0  SL      usbtsk   0xc0a4c840 [usbtask-hc]
   24     0     0     0  SL      usbevt   0xc3ffb210 [usb0]
   23     0     0     0  WL                          [irq12: uhci0 uhci1]
   22     0     0     0  WL                          [irq15: ata1]
   21     0     0     0  WL                          [irq14: ata0]
   20     0     0     0  WL                          [irq9: acpi0]
    9     0     0     0  SL      -        0xc3f78680 [kqueue taskq]
   19     0     0     0  WL                          [swi2: cambio]
    8     0     0     0  SL      -        0xc3f78800 [acpi_task_2]
    7     0     0     0  SL      -        0xc3f78800 [acpi_task_1]
    6     0     0     0  SL      -        0xc3f78800 [acpi_task_0]
   18     0     0     0  WL                          [swi5: +]
    5     0     0     0  SL      -        0xc3f78980 [thread taskq]
   17     0     0     0  WL                          [swi6: Giant taskq]
   16     0     0     0  WL                          [swi6: task queue]
   15     0     0     0  SL      -        0xc0a391e0 [yarrow]
    4     0     0     0  SL      -        0xc0a4d01c [g_down]
    3     0     0     0  SL      -        0xc0a4d018 [g_up]
    2     0     0     0  SL      -        0xc0a4d010 [g_event]
   14     0     0     0  WL                          [swi1: net]
   13     0     0     0  WL                          [swi3: vm]
   12     0     0     0  LL     *tcp      0xc4045b80 [swi4: clock sio]
   11     0     0     0  RL      CPU 0               [idle: cpu0]
   10     0     0     0  RL      CPU 1               [idle: cpu1]
    1     0     1     0  SLs     wait     0xc3f01000 [init]
    0     0     0     0  WLs                         [swapper]
10968 10967 10967    22  Z                           sshd

db> show pcpu
cpuid        = 0
curthread    = 0xc3eff6c0: pid 11 "idle: cpu0"
curpcb       = 0xe2986d90
fpcurthread  = none
idlethread   = 0xc3eff6c0: pid 11 "idle: cpu0"
APIC ID      = 0
currentldt   = 0x50
spin locks held:

db> show allpcpu 
Current CPU: 0

cpuid        = 0
curthread    = 0xc3eff6c0: pid 11 "idle: cpu0"
curpcb       = 0xe2986d90
fpcurthread  = none
idlethread   = 0xc3eff6c0: pid 11 "idle: cpu0"
APIC ID      = 0
currentldt   = 0x50
spin locks held:

cpuid        = 1
curthread    = 0xc3eff510: pid 10 "idle: cpu1"
curpcb       = 0xe2983d90
fpcurthread  = none
idlethread   = 0xc3eff510: pid 10 "idle: cpu1"
APIC ID      = 1
currentldt   = 0x50
spin locks held:

db> show locks
db> show alllocks
Process 10967 (sshd) thread 0xc45dabd0 (100110)
shared rw PFil hook read/write mutex r = 0 (0xc0a9db58) locked @ /usr/src/sys/net/pfil.c:73
exclusive sleep mutex inp (tcpinp) r = 0 (0xc4587528) locked @ /usr/src/sys/netinet/tcp_usrreq.c:577
exclusive sleep mutex tcp r = 0 (0xc0a9df6c) locked @ /usr/src/sys/netinet/tcp_usrreq.c:574
Process 29 (irq18: fxp1 em0) thread 0xc409ca20 (100036)
exclusive sleep mutex pf task mtx r = 0 (0xc4469c44) locked @ /usr/src/sys/modules/pf/../../contrib/pf/net/pf.c:6414
shared rw PFil hook read/write mutex r = 0 (0xc0a9db58) locked @ /usr/src/sys/net/pfil.c:73

db> show lockedvnods
Locked vnodes
db> alltrace

Tracing command sshd pid 10970 tid 100134 td 0xc4852d80
sched_switch(c4852d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c4852d80,4,28401148,0,0,...) at kern_select+0x4bf
select(c4852d80,e69fbd00) at select+0x44
syscall(e69fbd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283841bf, esp = 0xbfbfc0dc, ebp = 0xbfbfe138 ---

Tracing command sshd pid 10969 tid 100156 td 0xc4857d80
sched_switch(c4857d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c46c99b0) at sleepq_switch+0xc9
sleepq_wait_sig(c46c99b0) at sleepq_wait_sig+0x1d
msleep(c46c99b0,c46c9988,158,c0953043,0,...) at msleep+0x296
sbwait(c46c9964,2,c094ccee,91,c4857d80,...) at sbwait+0x48
soreceive_generic(c46c9914,0,e6a16c60,0,0,...) at soreceive_generic+0x2da
soreceive(c46c9914,0,e6a16c60,0,0,0) at soreceive+0x39
soo_read(c4291c60,e6a16c60,c45d6d00,0,c4857d80) at soo_read+0x3d
dofileread(c4857d80,6,c4291c60,e6a16c60,ffffffff,...) at dofileread+0x85
kern_readv(c4857d80,6,e6a16c60,bfbfe0e8,4,...) at kern_readv+0x36
read(c4857d80,e6a16d00) at read+0x45
syscall(e6a16d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2838423f, esp = 0xbfbfe08c, ebp = 0xbfbfe0b8 ---

Tracing command sshd pid 10967 tid 100110 td 0xc45dabd0
sched_switch(c45dabd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
turnstile_wait(c4469c44,c409ca20,0,c4469c44,2,c094a0d3,166) at turnstile_wait+0x3ea
_mtx_lock_sleep(c4469c44,c45dabd0,0,c4466e24,190e) at _mtx_lock_sleep+0x145
_mtx_lock_flags(c4469c44,0,c4466e24,190e,c4469c44,...) at _mtx_lock_flags+0xab
pf_test(2,c4098800,e698c958,0,c4587498,...) at pf_test+0x77
pf_check_out(0,e698c958,c4098800,2,c4587498) at pf_check_out+0x3d
pfil_run_hooks(c0a9db40,e698c9d4,c4098800,2,c4587498,...) at pfil_run_hooks+0x7f
ip_output(c4766800,0,e698c9a0,0,0,c4587498) at ip_output+0x69b
tcp_output(c53311dc) at tcp_output+0x11d6
tcp_disconnect(c53311dc) at tcp_disconnect+0xf8
tcp_usr_disconnect(c5382298,e698cadc,c070788e,c5382298,c46afc60,...) at tcp_usr_disconnect+0x6c
sodisconnect(c5382298) at sodisconnect+0x26
soclose(c5382298) at soclose+0x3e
soo_close(c46afc60,c45dabd0) at soo_close+0x4b
fdrop_locked(c46afc60,c45dabd0,c3ee1718,0,c09475f6,...) at fdrop_locked+0xb0
fdrop(c46afc60,c45dabd0,6b9,c0a55714,0,...) at fdrop+0x24
closef(c46afc60,c45dabd0) at closef+0x367
fdfree(c45dabd0) at fdfree+0x4a3
exit1(c45dabd0,ff00,e698cd2c,c08c154a,c45dabd0,...) at exit1+0x3cc
exit1(c45dabd0,e698cd00) at exit1
syscall(e698cd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (1, FreeBSD ELF32, sys_exit), eip = 0x28319173, esp = 0xbfbfe0ac, ebp = 0xbfbfe0c8 ---

Tracing command local pid 10904 tid 100183 td 0xc69ab360
sched_switch(c69ab360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,86d7,e6ab8ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,86d7) at cv_timedwait_sig+0x194
kern_select(c69ab360,7,bfbfecf0,bfbfec70,bfbfebf0,...) at kern_select+0x4a9
select(c69ab360,e6ab8d00) at select+0x44
syscall(e6ab8d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2818e1bf, esp = 0xbfbfebbc, ebp = 0xbfbfed88 ---

Tracing command cleanup pid 10903 tid 100143 td 0xc45da360
sched_switch(c45da360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,7302,e697dae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,7302) at cv_timedwait_sig+0x194
kern_select(c45da360,c,bfbfece0,bfbfec60,bfbfebe0,...) at kern_select+0x4a9
select(c45da360,e697dd00) at select+0x44
syscall(e697dd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281931bf, esp = 0xbfbfebac, ebp = 0xbfbfed78 ---

racing command proxymap pid 10902 tid 100154 td 0xc49051b0
sched_switch(c49051b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c49051b0,9,bfbfed10,bfbfec90,bfbfec10,...) at kern_select+0x4bf
select(c49051b0,e6a3ad00) at select+0x44
syscall(e6a3ad38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817a1bf, esp = 0xbfbfebdc, ebp = 0xbfbfeda8 ---

Tracing command nfsiod 19 pid 10814 tid 100108 td 0xc45db000
sched_switch(c45db000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa366c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa366c) at sleepq_timedwait_sig+0x1e
msleep(c0aa366c,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa300c,e6992d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa300c,e6992d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6992d70, ebp = 0 ---

Tracing command nfsiod 18 pid 10813 tid 100132 td 0xc48571b0
sched_switch(c48571b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3668) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3668) at sleepq_timedwait_sig+0x1e
msleep(c0aa3668,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa3008,e6a01d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa3008,e6a01d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a01d70, ebp = 0 ---

Tracing command nfsiod 17 pid 10812 tid 100142 td 0xc45da510
sched_switch(c45da510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3664) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3664) at sleepq_timedwait_sig+0x1e
msleep(c0aa3664,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa3004,e6980d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa3004,e6980d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6980d70, ebp = 0 ---

Tracing command nfsiod 16 pid 10811 tid 100131 td 0xc45dba20
sched_switch(c45dba20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3660) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3660) at sleepq_timedwait_sig+0x1e
msleep(c0aa3660,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa3000,e69a4d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa3000,e69a4d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69a4d70, ebp = 0 ---

Tracing command nfsiod 15 pid 10810 tid 100163 td 0xc4852000
sched_switch(c4852000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa365c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa365c) at sleepq_timedwait_sig+0x1e
msleep(c0aa365c,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2ffc,e69e3d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2ffc,e69e3d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69e3d70, ebp = 0 ---

Tracing command nfsiod 14 pid 10809 tid 100166 td 0xc47c3a20
sched_switch(c47c3a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3658) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3658) at sleepq_timedwait_sig+0x1e
msleep(c0aa3658,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2ff8,e69dad38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2ff8,e69dad38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69dad70, ebp = 0 ---

Tracing command nfsiod 13 pid 10808 tid 100152 td 0xc4905510
sched_switch(c4905510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3654) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3654) at sleepq_timedwait_sig+0x1e
msleep(c0aa3654,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2ff4,e6a40d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2ff4,e6a40d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a40d70, ebp = 0 ---

Tracing command nfsiod 12 pid 10807 tid 100202 td 0xc69aa870
sched_switch(c69aa870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3650) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3650) at sleepq_timedwait_sig+0x1e
msleep(c0aa3650,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2ff0,e6aa6d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2ff0,e6aa6d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6aa6d70, ebp = 0 ---

Tracing command nfsiod 11 pid 10806 tid 100196 td 0xc6a331b0
sched_switch(c6a331b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa364c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa364c) at sleepq_timedwait_sig+0x1e
msleep(c0aa364c,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fec,e6b09d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fec,e6b09d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6b09d70, ebp = 0 ---

Tracing command nfsiod 10 pid 10805 tid 100129 td 0xc45dbd80
sched_switch(c45dbd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3648) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3648) at sleepq_timedwait_sig+0x1e
msleep(c0aa3648,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fe8,e69aad38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fe8,e69aad38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69aad70, ebp = 0 ---

Tracing command nfsiod 9 pid 10804 tid 100117 td 0xc47c3360
sched_switch(c47c3360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3644) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3644) at sleepq_timedwait_sig+0x1e
msleep(c0aa3644,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fe4,e69ced38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fe4,e69ced38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69ced70, ebp = 0 ---

Tracing command nfsiod 8 pid 10803 tid 100150 td 0xc4905870
sched_switch(c4905870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3640) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3640) at sleepq_timedwait_sig+0x1e
msleep(c0aa3640,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fe0,e6a46d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fe0,e6a46d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a46d70, ebp = 0 ---

Tracing command nfsiod 7 pid 10802 tid 100200 td 0xc6a32a20
sched_switch(c6a32a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa363c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa363c) at sleepq_timedwait_sig+0x1e
msleep(c0aa363c,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fdc,e6afdd38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fdc,e6afdd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6afdd70, ebp = 0 ---

Tracing command nfsiod 6 pid 10801 tid 100061 td 0xc4342a20
sched_switch(c4342a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3638) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3638) at sleepq_timedwait_sig+0x1e
msleep(c0aa3638,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fd8,e68d5d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fd8,e68d5d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe68d5d70, ebp = 0 ---

Tracing command nfsiod 5 pid 10800 tid 100176 td 0xc52fbd80
sched_switch(c52fbd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3634) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3634) at sleepq_timedwait_sig+0x1e
msleep(c0aa3634,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fd4,e6a79d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fd4,e6a79d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a79d70, ebp = 0 ---

Tracing command nfsiod 4 pid 10799 tid 100148 td 0xc4905bd0
sched_switch(c4905bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3630) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3630) at sleepq_timedwait_sig+0x1e
msleep(c0aa3630,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fd0,e6a4cd38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fd0,e6a4cd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a4cd70, ebp = 0 ---

Tracing command nfsiod 3 pid 10798 tid 100174 td 0xc52fc1b0
sched_switch(c52fc1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa362c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa362c) at sleepq_timedwait_sig+0x1e
msleep(c0aa362c,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fcc,e6a7fd38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fcc,e6a7fd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a7fd70, ebp = 0 ---

Tracing command nfsiod 2 pid 1077 tid 100130 td 0xc45dbbd0
sched_switch(c45dbbd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3628) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3628) at sleepq_timedwait_sig+0x1e
msleep(c0aa3628,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fc8,e69a7d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fc8,e69a7d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe69a7d70, ebp = 0 ---

Tracing command nfsiod 1 pid 10796 tid 100179 td 0xc4857510
sched_switch(c4857510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3624) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3624) at sleepq_timedwait_sig+0x1e
msleep(c0aa3624,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fc4,e6a07d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fc4,e6a07d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a07d70, ebp = 0 ---
        
Tracing command nfsiod 0 pid 10791 tid 100178 td 0xc52fba20
sched_switch(c52fba20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3620) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0aa3620) at sleepq_timedwait_sig+0x1e
msleep(c0aa3620,c0aa3608,15c,c0943377,1d4c0,...) at msleep+0x26c
nfssvc_iod(c0aa2fc0,e6a73d38) at nfssvc_iod+0xa5
fork_exit(c07b3e84,c0aa2fc0,e6a73d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe6a73d70, ebp = 0 ---

Tracing command trivial-rewrite pid 10787 tid 100186 td 0xc69aad80
sched_switch(c69aad80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,2711,e6aafae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,2711) at cv_timedwait_sig+0x194
kern_select(c69aad80,b,bfbfed00,bfbfec80,bfbfec00,...) at kern_select+0x4a9
select(c69aad80,e6aafd00) at select+0x44
syscall(e6aafd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817e1bf, esp = 0xbfbfebcc, ebp = 0xbfbfed98 ---

Tracing command pickup pid 10393 tid 100109 td 0xc45dad80
sched_switch(c45dad80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,10994,e698fae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,10994) at cv_timedwait_sig+0x194
kern_select(c45dad80,7,bfbfe900,bfbfe880,bfbfe800,...) at kern_select+0x4a9
select(c45dad80,e698fd00) at select+0x44
syscall(e698fd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817d1bf, esp = 0xbfbfe7cc, ebp = 0xbfbfe998 ---

Tracing command smtpd pid 9973 tid 100173 td 0xc52fc360
sched_switch(c52fc360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,7135,e6a82ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,7135) at cv_timedwait_sig+0x194
kern_select(c52fc360,e,bfbfecf0,bfbfec70,bfbfebf0,...) at kern_select+0x4a9
select(c52fc360,e6a82d00) at select+0x44
syscall(e6a82d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2819f1bf, esp = 0xbfbfebbc, ebp = 0xbfbfed88 ---

Tracing command perl5.8.8 pid 9907 tid 100199 td 0xc6a32bd0
sched_switch(c6a32bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,35d3e,e6b00ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,35d3e) at cv_timedwait_sig+0x194
kern_select(c6a32bd0,10,9861c34,0,0,...) at kern_select+0x4a9
select(c6a32bd0,e6b00d00) at select+0x44
syscall(e6b00d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282821bf, esp = 0xbfbfecbc, ebp = 0xbfbfed48 ---

Tracing command anvil pid 8792 tid 100054 td 0xc409da20
sched_switch(c409da20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,c60b,e4688ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,c60b) at cv_timedwait_sig+0x194
kern_select(c409da20,a,bfbfed00,bfbfec80,bfbfec00,...) at kern_select+0x4a9
select(c409da20,e4688d00) at select+0x44
syscall(e4688d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2817c1bf, esp = 0xbfbfebcc, ebp = 0xbfbfed98 ---

Tracing command bash pid 1538 tid 100153 td 0xc4905360
sched_switch(c4905360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c401bc10) at sleepq_switch+0xc9
sleepq_wait_sig(c401bc10) at sleepq_wait_sig+0x1d
msleep(c401bc10,0,159,c0951c65,0) at msleep+0x296
ttysleep(c401bc00,c401bc10,159,c0951c65,0,...) at ttysleep+0x21
ttread(c401bc00,e6a3dc60,0) at ttread+0x48f
ptsread(c46b5100,e6a3dc60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c46b5100,e6a3dc60,0) at giant_read+0x2d
devfs_read_f(c46a3900,e6a3dc60,c490d100,0,c4905360) at devfs_read_f+0x62
dofileread(c4905360,0,c46a3900,e6a3dc60,ffffffff,...) at dofileread+0x85
kern_readv(c4905360,0,e6a3dc60,bfbfe83f,1,...) at kern_readv+0x36
read(c4905360,e6a3dd00) at read+0x45
syscall(e6a3dd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe81c, ebp = 0xbfbfe848 ---

Tracing command script pid 1537 tid 100161 td 0xc4852360
sched_switch(c4852360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,6b1f,e69e9ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,6b1f) at cv_timedwait_sig+0x194
kern_select(c4852360,5,bfbfe470,0,0,...) at kern_select+0x4a9
select(c4852360,e69e9d00) at select+0x44
syscall(e69e9d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281551bf, esp = 0xbfbfe40c, ebp = 0xbfbfed68 ---

Tracing command irssi pid 1374 tid 100141 td 0xc45da6c0
sched_switch(c45da6c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,5c,e6983b28,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,5c) at cv_timedwait_sig+0x194
poll(c45da6c0,e6983d00) at poll+0x3a3
syscall(e6983d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x285dcadf, esp = 0xbfbfe43c, ebp = 0xbfbfe488 ---

Tracing command bash pid 1296 tid 100060 td 0xc4342bd0
sched_switch(c4342bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4341900) at sleepq_switch+0xc9
sleepq_wait_sig(c4341900) at sleepq_wait_sig+0x1d
msleep(c4341900,c4341960,15c,c095061f,0) at msleep+0x296
kern_wait(c4342bd0,ffffffff,e68d8c28,6,0) at kern_wait+0x7c3
wait4(c4342bd0,e68d8d00) at wait4+0x2a
syscall(e68d8d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2819eb1b, esp = 0xbfbfe94c, ebp = 0xbfbfe968 ---

Tracing command login pid 1295 tid 100133 td 0xc4857000
sched_switch(c4857000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4858000) at sleepq_switch+0xc9
sleepq_wait_sig(c4858000) at sleepq_wait_sig+0x1d
msleep(c4858000,c4858060,15c,c095061f,0) at msleep+0x296
kern_wait(c4857000,510,e69fec28,0,0) at kern_wait+0x7c3
wait4(c4857000,e69fed00) at wait4+0x2a
syscall(e69fed38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x280fab1b, esp = 0xbfbfed4c, ebp = 0xbfbfed68 ---

Tracing command bash pid 1276 tid 100074 td 0xc44eca20
sched_switch(c44eca20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40d7810) at sleepq_switch+0xc9
sleepq_wait_sig(c40d7810) at sleepq_wait_sig+0x1d
msleep(c40d7810,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40d7800,c40d7810,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40d7800,e6929c60,0) at ttread+0x48f
ptsread(c4871800,e6929c60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c4871800,e6929c60,0) at giant_read+0x2d
devfs_read_f(c4897828,e6929c60,c48e1700,0,c44eca20) at devfs_read_f+0x62
dofileread(c44eca20,0,c4897828,e6929c60,ffffffff,...) at dofileread+0x85
kern_readv(c44eca20,0,e6929c60,bfbfe13f,1,...) at kern_readv+0x36
read(c44eca20,e6929d00) at read+0x45
syscall(e6929d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---

Tracing command telnetd pid 1270 tid 100123 td 0xc47c2870
sched_switch(c47c2870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c47c2870,10,bfbfdc10,bfbfdb90,bfbfdb10,...) at kern_select+0x4bf
select(c47c2870,e69bcd00) at select+0x44
syscall(e69bcd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282581bf, esp = 0xbfbfdabc, ebp = 0xbfbfe5a8 ---

Tracing command bash pid 1269 tid 100079 td 0xc44ec1b0
sched_switch(c44ec1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40ba410) at sleepq_switch+0xc9
sleepq_wait_sig(c40ba410) at sleepq_wait_sig+0x1d
msleep(c40ba410,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40ba400,c40ba410,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40ba400,e691ac60,0) at ttread+0x48f
ptsread(c486f100,e691ac60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c486f100,e691ac60,0) at giant_read+0x2d
devfs_read_f(c4897318,e691ac60,c48e4280,0,c44ec1b0) at devfs_read_f+0x62
dofileread(c44ec1b0,0,c4897318,e691ac60,ffffffff,...) at dofileread+0x85
kern_readv(c44ec1b0,0,e691ac60,bfbfe13f,1,...) at kern_readv+0x36
read(c44ec1b0,e691ad00) at read+0x45
syscall(e691ad38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---

Tracing command bash pid 1267 tid 100128 td 0xc47c2000
sched_switch(c47c2000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40ba810) at sleepq_switch+0xc9
sleepq_wait_sig(c40ba810) at sleepq_wait_sig+0x1d
msleep(c40ba810,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40ba800,c40ba810,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40ba800,e69adc60,0) at ttread+0x48f
ptsread(c486f000,e69adc60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c486f000,e69adc60,0) at giant_read+0x2d
devfs_read_f(c47e2318,e69adc60,c48de600,0,c47c2000) at devfs_read_f+0x62
dofileread(c47c2000,0,c47e2318,e69adc60,ffffffff,...) at dofileread+0x85
kern_readv(c47c2000,0,e69adc60,bfbfe13f,1,...) at kern_readv+0x36
read(c47c2000,e69add00) at read+0x45
syscall(e69add38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---

Tracing command bash pid 1266 tid 100094 td 0xc4596000
sched_switch(c4596000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4027810) at sleepq_switch+0xc9
sleepq_wait_sig(c4027810) at sleepq_wait_sig+0x1d
msleep(c4027810,0,159,c0951c65,0) at msleep+0x296
ttysleep(c4027800,c4027810,159,c0951c65,0,...) at ttysleep+0x21
ttread(c4027800,e694dc60,0) at ttread+0x48f
ptsread(c486f400,e694dc60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c486f400,e694dc60,0) at giant_read+0x2d
devfs_read_f(c46a39d8,e694dc60,c48d4300,0,c4596000) at devfs_read_f+0x62
dofileread(c4596000,0,c46a39d8,e694dc60,ffffffff,...) at dofileread+0x85
kern_readv(c4596000,0,e694dc60,bfbfe13f,1,...) at kern_readv+0x36
read(c4596000,e694dd00) at read+0x45
syscall(e694dd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---
        
Tracing command bash pid 1264 tid 100070 td 0xc4248a20
sched_switch(c4248a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4027410) at sleepq_switch+0xc9
sleepq_wait_sig(c4027410) at sleepq_wait_sig+0x1d
msleep(c4027410,0,159,c0951c65,0) at msleep+0x296
ttysleep(c4027400,c4027410,159,c0951c65,0,...) at ttysleep+0x21
ttread(c4027400,e46cec60,0) at ttread+0x48f
ptsread(c486f700,e46cec60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c486f700,e46cec60,0) at giant_read+0x2d
devfs_read_f(c46b94c8,e46cec60,c48c7100,0,c4248a20) at devfs_read_f+0x62
dofileread(c4248a20,0,c46b94c8,e46cec60,ffffffff,...) at dofileread+0x85
kern_readv(c4248a20,0,e46cec60,bfbfe13f,1,...) at kern_readv+0x36
read(c4248a20,e46ced00) at read+0x45
syscall(e46ced38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---

Tracing command bash pid 1260 tid 100135 td 0xc4852bd0
sched_switch(c4852bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40c4410) at sleepq_switch+0xc9
sleepq_wait_sig(c40c4410) at sleepq_wait_sig+0x1d
msleep(c40c4410,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40c4400,c40c4410,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40c4400,e69f8c60,0) at ttread+0x48f
ptsread(c4870700,e69f8c60,0,c0a4eaa8,0,...) at ptsread+0x2d
giant_read(c4870700,e69f8c60,0) at giant_read+0x2d
devfs_read_f(c4897990,e69f8c60,c47e0c00,0,c4852bd0) at devfs_read_f+0x62
dofileread(c4852bd0,0,c4897990,e69f8c60,ffffffff,...) at dofileread+0x85
kern_readv(c4852bd0,0,e69f8c60,bfbfe13f,1,...) at kern_readv+0x36
read(c4852bd0,e69f8d00) at read+0x45
syscall(e69f8d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2821223f, esp = 0xbfbfe11c, ebp = 0xbfbfe148 ---

Tracing command telnet pid 1259 tid 100124 td 0xc47c26c0
sched_switch(c47c26c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c47c26c0,10,8071d60,8071de0,8071e60,...) at kern_select+0x4bf
select(c47c26c0,e69b9d00) at select+0x44
syscall(e69b9d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282531bf, esp = 0xbfbfe48c, ebp = 0xbfbfe4c8 ---

Tracing command telnet pid 1258 tid 100125 td 0xc47c2510
sched_switch(c47c2510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c47c2510,10,8071d60,8071de0,8071e60,...) at kern_select+0x4bf
select(c47c2510,e69b6d00) at select+0x44
syscall(e69b6d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282531bf, esp = 0xbfbfe48c, ebp = 0xbfbfe4c8 ---
        
Tracing command bash pid 1257 tid 100126 td 0xc47c2360
sched_switch(c47c2360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c484e000) at sleepq_switch+0xc9
sleepq_wait_sig(c484e000) at sleepq_wait_sig+0x1d
msleep(c484e000,c484e060,15c,c095061f,0) at msleep+0x296
kern_wait(c47c2360,ffffffff,e69b3c28,6,0) at kern_wait+0x7c3
wait4(c47c2360,e69b3d00) at wait4+0x2a
syscall(e69b3d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x2819eb1b, esp = 0xbfbfe1dc, ebp = 0xbfbfe1f8 ---

Tracing command mutt pid 1256 tid 100127 td 0xc47c21b0
sched_switch(c47c21b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,59368,e69b0ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,59368) at cv_timedwait_sig+0x194
kern_select(c47c21b0,4,28159ca0,0,0,...) at kern_select+0x4a9
select(c47c21b0,e69b0d00) at select+0x44
syscall(e69b0d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2854c1bf, esp = 0xbfbfd27c, ebp = 0xbfbfd2b8 ---

Tracing command screen pid 1254 tid 100052 td 0xc409dd80
sched_switch(c409dd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c409dd80,14,bfbfe250,bfbfe1d0,0,...) at kern_select+0x4bf
select(c409dd80,e468ed00) at select+0x44
syscall(e468ed38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281f01bf, esp = 0xbfbfe19c, ebp = 0xbfbfe2e8 ---

Tracing command perl5.8.8 pid 1205 tid 100086 td 0xc4343510
sched_switch(c4343510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,3943b,e68e7ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,3943b) at cv_timedwait_sig+0x194
kern_select(c4343510,10,9a07a90,0,0,...) at kern_select+0x4a9
select(c4343510,e68e7d00) at select+0x44
syscall(e68e7d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282821bf, esp = 0xbfbfecbc, ebp = 0xbfbfed48 ---

Tracing command getty pid 1198 tid 100105 td 0xc45db510
sched_switch(c45db510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c409ac10) at sleepq_switch+0xc9
sleepq_wait_sig(c409ac10) at sleepq_wait_sig+0x1d
msleep(c409ac10,0,159,c0951c65,0) at msleep+0x296
ttysleep(c409ac00,c409ac10,159,c0951c65,0,...) at ttysleep+0x21
ttread(c409ac00,e699bc60,0) at ttread+0x48f
ttyread(c40a1900,e699bc60,0,c0a4eaa8,0,...) at ttyread+0x2f
giant_read(c40a1900,e699bc60,0) at giant_read+0x2d
devfs_read_f(c46a3e58,e699bc60,c3efca80,0,c45db510) at devfs_read_f+0x62
dofileread(c45db510,0,c46a3e58,e699bc60,ffffffff,...) at dofileread+0x85
kern_readv(c45db510,0,e699bc60,bfbfee3f,1,...) at kern_readv+0x36
read(c45db510,e699bd00) at read+0x45
syscall(e699bd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command getty pid 1197 tid 100104 td 0xc45db6c0
sched_switch(c45db6c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40cc810) at sleepq_switch+0xc9
sleepq_wait_sig(c40cc810) at sleepq_wait_sig+0x1d
msleep(c40cc810,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40cc800,c40cc810,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40cc800,e699ec60,0) at ttread+0x48f
ttyread(c40bed00,e699ec60,0,e699ebb8,c069ba39,...) at ttyread+0x2f
scread(c40bed00,e699ec60,0,c0a4eaa8,0,...) at scread+0x22
giant_read(c40bed00,e699ec60,0) at giant_read+0x2d
devfs_read_f(c46b9318,e699ec60,c3efca80,0,c45db6c0) at devfs_read_f+0x62
dofileread(c45db6c0,0,c46b9318,e699ec60,ffffffff,...) at dofileread+0x85
kern_readv(c45db6c0,0,e699ec60,bfbfee3f,1,...) at kern_readv+0x36
read(c45db6c0,e699ed00) at read+0x45
syscall(e699ed38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command getty pid 1196 tid 100055 td 0xc409d870
sched_switch(c409d870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40c5010) at sleepq_switch+0xc9
sleepq_wait_sig(c40c5010) at sleepq_wait_sig+0x1d
msleep(c40c5010,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40c5000,c40c5010,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40c5000,e4685c60,0) at ttread+0x48f
ttyread(c40bec00,e4685c60,0,e4685bb8,c069ba39,...) at ttyread+0x2f
scread(c40bec00,e4685c60,0,c0a4eaa8,0,...) at scread+0x22
giant_read(c40bec00,e4685c60,0) at giant_read+0x2d
devfs_read_f(c46a3d80,e4685c60,c3efca80,0,c409d870) at devfs_read_f+0x62
dofileread(c409d870,0,c46a3d80,e4685c60,ffffffff,...) at dofileread+0x85
kern_readv(c409d870,0,e4685c60,bfbfee3f,1,...) at kern_readv+0x36
read(c409d870,e4685d00) at read+0x45
syscall(e4685d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command getty pid 1195 tid 100046 td 0xc4248870
sched_switch(c4248870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40c3810) at sleepq_switch+0xc9
sleepq_wait_sig(c40c3810) at sleepq_wait_sig+0x1d
msleep(c40c3810,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40c3800,c40c3810,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40c3800,e46cbc60,0) at ttread+0x48f
ttyread(c40beb00,e46cbc60,0,e46cbbb8,c069ba39,...) at ttyread+0x2f
scread(c40beb00,e46cbc60,0,c0a4eaa8,0,...) at scread+0x22
giant_read(c40beb00,e46cbc60,0) at giant_read+0x2d
devfs_read_f(c4292090,e46cbc60,c3efca80,0,c4248870) at devfs_read_f+0x62
dofileread(c4248870,0,c4292090,e46cbc60,ffffffff,...) at dofileread+0x85
kern_readv(c4248870,0,e46cbc60,bfbfee3f,1,...) at kern_readv+0x36
read(c4248870,e46cbd00) at read+0x45
syscall(e46cbd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command getty pid 1194 tid 100082 td 0xc4343bd0
sched_switch(c4343bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40bb410) at sleepq_switch+0xc9
sleepq_wait_sig(c40bb410) at sleepq_wait_sig+0x1d
msleep(c40bb410,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40bb400,c40bb410,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40bb400,e68f3c60,0) at ttread+0x48f
ttyread(c40bea00,e68f3c60,0,e68f3bb8,c069ba39,...) at ttyread+0x2f
scread(c40bea00,e68f3c60,0,c0a4eaa8,0,...) at scread+0x22
giant_read(c40bea00,e68f3c60,0) at giant_read+0x2d
devfs_read_f(c46a3ab0,e68f3c60,c3efca80,0,c4343bd0) at devfs_read_f+0x62
dofileread(c4343bd0,0,c46a3ab0,e68f3c60,ffffffff,...) at dofileread+0x85
kern_readv(c4343bd0,0,e68f3c60,bfbfee3f,1,...) at kern_readv+0x36
read(c4343bd0,e68f3d00) at read+0x45
syscall(e68f3d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command getty pid 1193 tid 100073 td 0xc44ecbd0
sched_switch(c44ecbd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40c4010) at sleepq_switch+0xc9
sleepq_wait_sig(c40c4010) at sleepq_wait_sig+0x1d
msleep(c40c4010,0,159,c0951c65,0) at msleep+0x296
ttysleep(c40c4000,c40c4010,159,c0951c65,0,...) at ttysleep+0x21
ttread(c40c4000,e692cc60,0) at ttread+0x48f
ttyread(c40be900,e692cc60,0,e692cbb8,c069ba39,...) at ttyread+0x2f
scread(c40be900,e692cc60,0,c0a4eaa8,0,...) at scread+0x22
giant_read(c40be900,e692cc60,0) at giant_read+0x2d
devfs_read_f(c46a32d0,e692cc60,c3efca80,0,c44ecbd0) at devfs_read_f+0x62
dofileread(c44ecbd0,0,c46a32d0,e692cc60,ffffffff,...) at dofileread+0x85
kern_readv(c44ecbd0,0,e692cc60,bfbfee3f,1,...) at kern_readv+0x36
read(c44ecbd0,e692cd00) at read+0x45
syscall(e692cd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2815723f, esp = 0xbfbfee1c, ebp = 0xbfbfee48 ---

Tracing command inetd pid 1174 tid 100085 td 0xc43436c0
sched_switch(c43436c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c43436c0,9,bfbfe8b0,0,0,...) at kern_select+0x4bf
select(c43436c0,e68ead00) at select+0x44
syscall(e68ead38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281691bf, esp = 0xbfbfdb8c, ebp = 0xbfbfee78 ---

Tracing command cron pid 1141 tid 100064 td 0xc4342510
sched_switch(c4342510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4f2a4) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a4f2a4) at sleepq_timedwait_sig+0x1e
msleep(c0a4f2a4,0,15c,c094c10d,ea61,...) at msleep+0x26c
kern_nanosleep(c4342510,e68ccc70,e68ccc68) at kern_nanosleep+0xab
nanosleep(c4342510,e68ccd00) at nanosleep+0x4f
syscall(e68ccd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x2813fc3b, esp = 0xbfbfecec, ebp = 0xbfbfed18 ---

Tracing command sshd pid 1133 tid 100095 td 0xc44eed80
sched_switch(c44eed80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c44eed80,a,284c1144,0,0,...) at kern_select+0x4bf
select(c44eed80,e694ad00) at select+0x44
syscall(e694ad38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x283841bf, esp = 0xbfbfe18c, ebp = 0xbfbfee98 ---

Tracing command couriertcpd pid 1119 tid 100066 td 0xc43421b0
sched_switch(c43421b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c43421b0,6,bfbfe630,0,0,...) at kern_select+0x4bf
select(c43421b0,e68c6d00) at select+0x44
syscall(e68c6d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281521bf, esp = 0xbfbfe4ac, ebp = 0xbfbfe6c8 ---

Tracing command courierlogger pid 1118 tid 100059 td 0xc4342d80
sched_switch(c4342d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c46ad318) at sleepq_switch+0xc9
sleepq_wait_sig(c46ad318) at sleepq_wait_sig+0x1d
msleep(c46ad318,c46ad488,14c,c09502df,0) at msleep+0x296
pipe_read(c46a3bd0,e68dbc60,c3efca80,0,c4342d80) at pipe_read+0x377
dofileread(c4342d80,0,c46a3bd0,e68dbc60,ffffffff,...) at dofileread+0x85
kern_readv(c4342d80,0,e68dbc60,28201000,1000,...) at kern_readv+0x36
read(c4342d80,e68dbd00) at read+0x45
syscall(e68dbd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814a23f, esp = 0xbfbfe32c, ebp = 0xbfbfe348 ---

Tracing command couriertcpd pid 1107 tid 100088 td 0xc4596a20
sched_switch(c4596a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c4596a20,6,bfbfe9e0,0,0,...) at kern_select+0x4bf
select(c4596a20,e695fd00) at select+0x44
syscall(e695fd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281521bf, esp = 0xbfbfe85c, ebp = 0xbfbfea78 ---

Tracing command courierlogger pid 1106 tid 100106 td 0xc45db360
sched_switch(c45db360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c46ad7bc) at sleepq_switch+0xc9
sleepq_wait_sig(c46ad7bc) at sleepq_wait_sig+0x1d
msleep(c46ad7bc,c46ad92c,14c,c09502df,0) at msleep+0x296
pipe_read(c46a3558,e6998c60,c3efca80,0,c45db360) at pipe_read+0x377
dofileread(c45db360,0,c46a3558,e6998c60,ffffffff,...) at dofileread+0x85
kern_readv(c45db360,0,e6998c60,28201000,1000,...) at kern_readv+0x36
read(c45db360,e6998d00) at read+0x45
syscall(e6998d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814a23f, esp = 0xbfbfe6ec, ebp = 0xbfbfe708 ---

Tracing command qmgr pid 1102 tid 100080 td 0xc44ec000
sched_switch(c44ec000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,734f,e6917ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,734f) at cv_timedwait_sig+0x194
kern_select(c44ec000,14,bfbfe8d0,bfbfe850,bfbfe7d0,...) at kern_select+0x4a9
select(c44ec000,e6917d00) at select+0x44
syscall(e6917d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281881bf, esp = 0xbfbfe79c, ebp = 0xbfbfe968 ---

Tracing command master pid 1090 tid 100041 td 0xc409c1b0
sched_switch(c409c1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,6d4d,e465eae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,6d4d) at cv_timedwait_sig+0x194
kern_select(c409c1b0,63,bfbfeb60,bfbfeae0,bfbfea60,...) at kern_select+0x4a9
select(c409c1b0,e465ed00) at select+0x44
syscall(e465ed38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281771bf, esp = 0xbfbfea2c, ebp = 0xbfbfebf8 ---

Tracing command authdaemond pid 1044 tid 100099 td 0xc44ee6c0
sched_switch(c44ee6c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,2af99,e693eae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,2af99) at cv_timedwait_sig+0x194
kern_select(c44ee6c0,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c44ee6c0,e693ed00) at select+0x44
syscall(e693ed38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command authdaemond pid 1043 tid 100102 td 0xc44ee1b0
sched_switch(c44ee1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,493e1,e6935ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,493e1) at cv_timedwait_sig+0x194
kern_select(c44ee1b0,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c44ee1b0,e6935d00) at select+0x44
syscall(e6935d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command authdaemond pid 1042 tid 100077 td 0xc44ec510
sched_switch(c44ec510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,493e1,e6920ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,493e1) at cv_timedwait_sig+0x194
kern_select(c44ec510,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c44ec510,e6920d00) at select+0x44
syscall(e6920d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command authdaemond pid 1041 tid 100092 td 0xc4596360
sched_switch(c4596360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,3a1ca,e6953ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,3a1ca) at cv_timedwait_sig+0x194
kern_select(c4596360,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c4596360,e6953d00) at select+0x44
syscall(e6953d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command authdaemond pid 1040 tid 100093 td 0xc45961b0
sched_switch(c45961b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,2af86,e6950ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,2af86) at cv_timedwait_sig+0x194
kern_select(c45961b0,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c45961b0,e6950d00) at select+0x44
syscall(e6950d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command authdaemond pid 1033 tid 100096 td 0xc44eebd0
sched_switch(c44eebd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,21269,e6947ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,21269) at cv_timedwait_sig+0x194
kern_select(c44eebd0,6,bfbfec10,0,0,...) at kern_select+0x4a9
select(c44eebd0,e6947d00) at select+0x44
syscall(e6947d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815f1bf, esp = 0xbfbfe37c, ebp = 0xbfbfed28 ---

Tracing command courierlogger pid 1032 tid 100097 td 0xc44eea20
sched_switch(c44eea20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4298948) at sleepq_switch+0xc9
sleepq_wait_sig(c4298948) at sleepq_wait_sig+0x1d
msleep(c4298948,c4298ab8,14c,c09502df,0) at msleep+0x296
pipe_read(c430d900,e6944c60,c3efca80,0,c44eea20) at pipe_read+0x377
dofileread(c44eea20,0,c430d900,e6944c60,ffffffff,...) at dofileread+0x85
kern_readv(c44eea20,0,e6944c60,28201000,1000,...) at kern_readv+0x36
read(c44eea20,e6944d00) at read+0x45
syscall(e6944d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2814a23f, esp = 0xbfbfe9ac, ebp = 0xbfbfe9c8 ---

Tracing command perl5.8.8 pid 1025 tid 100103 td 0xc45db870
sched_switch(c45db870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,7d1,e69a1ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,7d1) at cv_timedwait_sig+0x194
kern_select(c45db870,10,96c14f0,0,96c1440,...) at kern_select+0x4a9
select(c45db870,e69a1d00) at select+0x44
syscall(e69a1d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282821bf, esp = 0xbfbfecbc, ebp = 0xbfbfed48 ---

Tracing command dhcpd pid 1018 tid 100107 td 0xc45db1b0
sched_switch(c45db1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,2749f3,e6995ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,2749f3) at cv_timedwait_sig+0x194
kern_select(c45db1b0,8,bfbfec30,bfbfebb0,bfbfeb30,...) at kern_select+0x4a9
select(c45db1b0,e6995d00) at select+0x44
syscall(e6995d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281b11bf, esp = 0xbfbfeacc, ebp = 0xbfbfecc8 ---

Tracing command rwhod pid 924 tid 100081 td 0xc4343d80
sched_switch(c4343d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c45b89b0) at sleepq_switch+0xc9
sleepq_wait_sig(c45b89b0) at sleepq_wait_sig+0x1d
msleep(c45b89b0,c45b8988,158,c0953043,0,...) at msleep+0x296
sbwait(c45b8964,c0a9bc4c,e68f6b70,c06ed422,c0a9bc48,...) at sbwait+0x48
soreceive_generic(c45b8914,e68f6be0,e68f6bec,0,0,...) at soreceive_generic+0x2da
soreceive(c45b8914,e68f6be0,e68f6bec,0,0,e68f6c74) at soreceive+0x39
kern_recvit(c4343d80,4,e68f6c5c,0,0) at kern_recvit+0x188
recvit(c4343d80,4,e68f6c5c,bfbfe980) at recvit+0x1b
recvfrom(c4343d80,e68f6d00) at recvfrom+0x6c
syscall(e68f6d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (29, FreeBSD ELF32, recvfrom), eip = 0x28107cbb, esp = 0xbfbfe92c, ebp = 0xbfbfee98 ---

Tracing command ntpd pid 911 tid 100084 td 0xc4343870
sched_switch(c4343870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c4343870,b,bfbfed70,0,0,...) at kern_select+0x4bf
select(c4343870,e68edd00) at select+0x44
syscall(e68edd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282c81bf, esp = 0xbfbfed2c, ebp = 0xbfbfee08 ---

Tracing command lpd pid 895 tid 100042 td 0xc409c000
sched_switch(c409c000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c409c000,14,bfbfead0,0,0,...) at kern_select+0x4bf
select(c409c000,e465bd00) at select+0x44
syscall(e465bd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281561bf, esp = 0xbfbfe0ec, ebp = 0xbfbfee98 ---

Tracing command rpc.lockd pid 874 tid 100065 td 0xc4342360
sched_switch(c4342360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa3848) at sleepq_switch+0xc9
sleepq_wait_sig(c0aa3848) at sleepq_wait_sig+0x1d
msleep(c0aa3848,c0aa382c,158,c0962239,0) at msleep+0x296
nfslock_read(c3f58400,e68c9c60,4) at nfslock_read+0x5f
devfs_read_f(c4291af8,e68c9c60,c4599700,0,c4342360) at devfs_read_f+0x62
dofileread(c4342360,8,c4291af8,e68c9c60,ffffffff,...) at dofileread+0x85
kern_readv(c4342360,8,e68c9c60,bfbfec70,194,...) at kern_readv+0x36
read(c4342360,e68c9d00) at read+0x45
syscall(e68c9d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2816423f, esp = 0xbfbfec3c, ebp = 0xbfbfee18 ---

Tracing command rpc.lockd pid 866 tid 100076 td 0xc44ec6c0
sched_switch(c44ec6c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,403c,e6923ae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,403c) at cv_timedwait_sig+0x194
kern_select(c44ec6c0,7,bfbfed80,0,0,...) at kern_select+0x4a9
select(c44ec6c0,e6923d00) at select+0x44
syscall(e6923d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281641bf, esp = 0xbfbfecbc, ebp = 0xbfbfee18 ---

Tracing command rpc.statd pid 861 tid 100089 td 0xc4596870
sched_switch(c4596870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,403c,e695cae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,403c) at cv_timedwait_sig+0x194
kern_select(c4596870,8,bfbfed90,0,0,...) at kern_select+0x4a9
select(c4596870,e695cd00) at select+0x44
syscall(e695cd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281521bf, esp = 0xbfbfeccc, ebp = 0xbfbfee28 ---

Tracing command rpcbind pid 827 tid 100069 td 0xc4248bd0
sched_switch(c4248bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,403c,e46d1b28,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,403c) at cv_timedwait_sig+0x194
poll(c4248bd0,e46d1d00) at poll+0x3a3
syscall(e46d1d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (209, FreeBSD ELF32, poll), eip = 0x28113adf, esp = 0xbfbfcc6c, ebp = 0xbfbfee38 ---

Tracing command named pid 817 tid 100072 td 0xc44ecd80
sched_switch(c44ecd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,1e0b,e692fae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,1e0b) at cv_timedwait_sig+0x194
kern_select(c44ecd80,26,bfbfed60,bfbfece0,0,...) at kern_select+0x4a9
select(c44ecd80,e692fd00) at select+0x44
syscall(e692fd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2839c1bf, esp = 0xbfbfec9c, ebp = 0xbfbfedf8 ---

Tracing command syslogd pid 754 tid 100057 td 0xc43431b0
sched_switch(c43431b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c43431b0,b,28208144,0,0,...) at kern_select+0x4bf
select(c43431b0,e68e1d00) at select+0x44
syscall(e68e1d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x2815b1bf, esp = 0xbfbfd9fc, ebp = 0xbfbfee48 ---

Tracing command watchquagga pid 729 tid 100050 td 0xc42481b0
sched_switch(c42481b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,117e,e46bfae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,117e) at cv_timedwait_sig+0x194
kern_select(c42481b0,14,bfbfecb0,bfbfec30,bfbfebb0,...) at kern_select+0x4a9
select(c42481b0,e46bfd00) at select+0x44
syscall(e46bfd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281c31bf, esp = 0xbfbfeb4c, ebp = 0xbfbfed48 ---

Tracing command ospfd pid 724 tid 100051 td 0xc4248000
sched_switch(c4248000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_timedwait_sig(c0a9be5c,c0a9be5c,73,e46bcae4,246,...) at sleepq_timedwait_sig+0x1e
cv_timedwait_sig(c0a9be5c,c0a9be44,73) at cv_timedwait_sig+0x194
kern_select(c4248000,14,bfbfed10,bfbfec90,bfbfec10,...) at kern_select+0x4a9
select(c4248000,e46bcd00) at select+0x44
syscall(e46bcd38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x282201bf, esp = 0xbfbfebac, ebp = 0xbfbfeda8 ---

Tracing command zebra pid 718 tid 100056 td 0xc4343360
sched_switch(c4343360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c4343360,14,bfbfed10,bfbfec90,bfbfec10,...) at kern_select+0x4bf
select(c4343360,e68e4d00) at select+0x44
syscall(e68e4d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x281d91bf, esp = 0xbfbfebac, ebp = 0xbfbfeda8 ---

Tracing command devd pid 658 tid 100043 td 0xc4007d80
sched_switch(c4007d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9be5c) at sleepq_switch+0xc9
sleepq_wait_sig(c0a9be5c,246,c0a4ea90,c095006f,308,...) at sleepq_wait_sig+0x1d
cv_wait_sig(c0a9be5c,c0a9be44) at cv_wait_sig+0x18b
kern_select(c4007d80,5,bfbfe9f0,0,0,...) at kern_select+0x4bf
select(c4007d80,e4406d00) at select+0x44
syscall(e4406d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (93, FreeBSD ELF32, select), eip = 0x807a2bf, esp = 0xbfbfe98c, ebp = 0xbfbfee98 ---

Tracing command pflogd pid 436 tid 100039 td 0xc409c510
sched_switch(c409c510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4496e00) at sleepq_switch+0xc9
sleepq_timedwait_sig(c4496e00) at sleepq_timedwait_sig+0x1e
msleep(c4496e00,c4496e74,11a,c09569e8,1f4) at msleep+0x26c
bpfread(c4497000,e4664c60,0,c0a4eaa8,0,...) at bpfread+0x13a
giant_read(c4497000,e4664c60,0) at giant_read+0x2d
devfs_read_f(c4291630,e4664c60,c40d2380,0,c409c510) at devfs_read_f+0x62
dofileread(c409c510,3,c4291630,e4664c60,ffffffff,...) at dofileread+0x85
kern_readv(c409c510,3,e4664c60,28208000,8000,...) at kern_readv+0x36
read(c409c510,e4664d00) at read+0x45
syscall(e4664d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2817c23f, esp = 0xbfbfed9c, ebp = 0xbfbfedf8 ---

Tracing command pflogd pid 431 tid 100044 td 0xc4007bd0
sched_switch(c4007bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c433a1e8) at sleepq_switch+0xc9
sleepq_wait_sig(c433a1e8) at sleepq_wait_sig+0x1d
msleep(c433a1e8,c433a1c0,158,c0953043,0,...) at msleep+0x296
sbwait(c433a19c,2,c094ccee,91,c4007bd0,...) at sbwait+0x48
soreceive_generic(c433a14c,0,e4403c60,0,0,...) at soreceive_generic+0x2da
soreceive(c433a14c,0,e4403c60,0,0,0) at soreceive+0x39
soo_read(c4292d38,e4403c60,c3efca80,0,c4007bd0) at soo_read+0x3d
dofileread(c4007bd0,4,c4292d38,e4403c60,ffffffff,...) at dofileread+0x85
kern_readv(c4007bd0,4,e4403c60,bfbfee08,4,...) at kern_readv+0x36
read(c4007bd0,e4403d00) at read+0x45
syscall(e4403d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (3, FreeBSD ELF32, read), eip = 0x2817c23f, esp = 0xbfbfedec, ebp = 0xbfbfee28 ---

Tracing command adjkerntz pid 165 tid 100038 td 0xc409c6c0
sched_switch(c409c6c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c40a06e4) at sleepq_switch+0xc9
sleepq_wait_sig(c40a06e4) at sleepq_wait_sig+0x1d
msleep(c40a06e4,c40a0720,168,c090f176,0) at msleep+0x296
kern_sigsuspend(c409c6c0,0,0,0,0,...) at kern_sigsuspend+0xa3
sigsuspend(c409c6c0,e4667d00) at sigsuspend+0x33
syscall(e4667d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x280c144f, esp = 0xbfbfedac, ebp = 0xbfbfee88 ---

Tracing command schedcpu pid 41 tid 100047 td 0xc42486c0
sched_switch(c42486c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(e46c8cfc,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(e46c8cfc) at sleepq_timedwait+0x4a
msleep(e46c8cfc,0,0,c0943377,3e8) at msleep+0x281
schedcpu_thread(0,e46c8d38) at schedcpu_thread+0x26
fork_exit(c06d7768,0,e46c8d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe46c8d70, ebp = 0 ---

Tracing command softdepflush pid 40 tid 100048 td 0xc4248510
sched_switch(c4248510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa8a84,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0aa8a84) at sleepq_timedwait+0x4a
msleep(c0aa8a84,c0aa8a5c,44,c09650a2,3e8) at msleep+0x281
softdep_flush(0,e46c5d38) at softdep_flush+0x1f3
fork_exit(c07fda40,0,e46c5d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe46c5d70, ebp = 0 ---

Tracing command vnlru pid 39 tid 100049 td 0xc4248360
sched_switch(c4248360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4249000,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c4249000) at sleepq_timedwait+0x4a
msleep(c4249000,c0a9c5b8,250,c0955f23,3e8,c0a9c618) at msleep+0x281
vnlru_proc(0,e46c2d38) at vnlru_proc+0xdf
fork_exit(c07222a0,0,e46c2d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe46c2d70, ebp = 0 ---

Tracing command syncer pid 38 tid 100027 td 0xc3f02d80
sched_switch(c3f02d80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4f06c,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c0a4f06c,0,c0955e55,641,c0955e55,...) at sleepq_wait+0x46
msleep(c0a4f06c,c0a9c5e4,68,c0956220,0) at msleep+0x2a5
sched_sync(0,e29c8d38) at sched_sync+0x2fe
fork_exit(c0723f28,0,e29c8d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29c8d70, ebp = 0 ---

Tracing command bufdaemon pid 37 tid 100028 td 0xc3f02bd0
sched_switch(c3f02bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a9c2e8,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0a9c2e8) at sleepq_timedwait+0x4a
msleep(c0a9c2e8,c0a9c2ec,44,c0954698,3e8) at msleep+0x281
buf_daemon(0,e29c5d38) at buf_daemon+0x198
fork_exit(c0713ac4,0,e29c5d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29c5d70, ebp = 0 ---

Tracing command pagezero pid 36 tid 100029 td 0xc3f02a20
sched_switch(c3f02a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0ab13d0,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0ab13d0) at sleepq_timedwait+0x4a
msleep(c0ab13d0,c0aa9284,0,c096a141,493e0) at msleep+0x281
vm_pagezero(0,e29c2d38) at vm_pagezero+0x6b
fork_exit(c0830814,0,e29c2d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29c2d70, ebp = 0 ---

Tracing command vmdaemon pid 35 tid 100030 td 0xc3f02870
sched_switch(c3f02870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa9300,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c0aa9300,0,c06b996c,c0a4eaa8,c0969cf8,...) at sleepq_wait+0x46
msleep(c0aa9300,0,68,c0954698,0) at msleep+0x2a5
vm_daemon(0,e29bfd38) at vm_daemon+0x36
fork_exit(c082f44c,0,e29bfd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29bfd70, ebp = 0 ---

Tracing command pagedaemon pid 34 tid 100031 td 0xc3f026c0
sched_switch(c3f026c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0aa92c0,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0aa92c0) at sleepq_timedwait+0x4a
msleep(c0aa92c0,c0aa9284,44,c0954698,1388) at msleep+0x281
vm_pageout(0,e29bcd38) at vm_pageout+0x268
fork_exit(c082f0d4,0,e29bcd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29bcd70, ebp = 0 ---

Tracing command irq7: ppc0 pid 33 tid 100032 td 0xc3f02510
fork_trampoline() at fork_trampoline

Tracing command irq1: atkbd0 pid 32 tid 100033 td 0xc409d000
fork_trampoline() at fork_trampoline

Tracing command swi0: sio pid 31 tid 100034 td 0xc409cd80
sched_switch(c409cd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c40b6200,e4673d38) at ithread_loop+0xda
fork_exit(c06adf00,c40b6200,e4673d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe4673d70, ebp = 0 ---

Tracing command em0 taskq pid 30 tid 100035 td 0xc409cbd0
sched_switch(c409cbd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c4041200,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c4041200,c3ff2204,c3ff2204,e4670cec,c06e8952,...) at sleepq_wait+0x46
msleep_spin(c4041200,c404121c,c0943377,0) at msleep_spin+0x1a1
taskqueue_thread_loop(c3ff2214,e4670d38) at taskqueue_thread_loop+0x61
fork_exit(c06e8c4c,c3ff2214,e4670d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe4670d70, ebp = 0 ---

Tracing command irq18: fxp1 em0 pid 29 tid 100036 td 0xc409ca20
sched_switch(c409ca20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
turnstile_wait(c0a9df6c,c45dabd0,0,c0a9df6c,2,c094a0d3,166) at turnstile_wait+0x3ea
_mtx_lock_sleep(c0a9df6c,c409ca20,0,c4466e24,ac8) at _mtx_lock_sleep+0x145
_mtx_lock_flags(c0a9df6c,0,c4466e24,ac8,c0a9df6c,...) at _mtx_lock_flags+0xab
pf_socket_lookup(e466da84,e466da88,1,e466db40,0,...) at pf_socket_lookup+0x1d3
pf_test_tcp(e466daf0,e466dae8,1,c4295100,c4349e00,...) at pf_test_tcp+0x11e6
pf_test(1,c4098800,e466dbdc,0,0,...) at pf_test+0xae3
pf_check_in(0,e466dbdc,c4098800,1,0) at pf_check_in+0x37
pfil_run_hooks(c0a9db40,e466dc2c,c4098800,1,0) at pfil_run_hooks+0x7f
ip_input(c4349e00) at ip_input+0x232
netisr_dispatch(2,c4349e00,0,c4098800,c43d0800,...) at netisr_dispatch+0x58
ether_demux(c4098800,c4349e00,c4037000,c43da002,e466dca0,...) at ether_demux+0x28a
ether_input(c4098800,c4349e00,c4037014,0,c092325d,...) at ether_input+0x202
fxp_intr_body(c4037000,c4098800,40,ffffffff) at fxp_intr_body+0x1a0
fxp_intr(c4037000) at fxp_intr+0x95
ithread_execute_handlers(c40a0240,c3f46900) at ithread_execute_handlers+0x11e
ithread_loop(c40433a0,e466dd38) at ithread_loop+0x67
fork_exit(c06adf00,c40433a0,e466dd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe466dd70, ebp = 0 ---

Tracing command irq17: fxp0 pid 28 tid 100037 td 0xc409c870
fork_trampoline() at fork_trampoline

Tracing command usb1 pid 27 tid 100017 td 0xc3f006c0
sched_switch(c3f006c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c402a210,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c402a210) at sleepq_timedwait+0x4a
msleep(c402a210,0,5c,c09402e9,ea60,c4020940) at msleep+0x281
usb_event_thread(c4020940,e29a1d38) at usb_event_thread+0x94
fork_exit(c0658948,c4020940,e29a1d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29a1d70, ebp = 0 ---
        
Tracing command usbtask-dr pid 26 tid 100018 td 0xc3f00510
sched_switch(c3f00510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4c854,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c0a4c854,0,c06b996c,c0a4eaa8,c09402c7,...) at sleepq_wait+0x46
msleep(c0a4c854,0,5c,c09402f0,0,...) at msleep+0x2a5
usb_task_thread(c0a4c854,e299ed38) at usb_task_thread+0x47
fork_exit(c06589fc,c0a4c854,e299ed38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe299ed70, ebp = 0 ---

Tracing command usbtask-hc pid 25 tid 100019 td 0xc3f00360
sched_switch(c3f00360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4c840,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c0a4c840,0,c06b996c,c0a4eaa8,c09402c7,...) at sleepq_wait+0x46
msleep(c0a4c840,0,5c,c09402f0,0,...) at msleep+0x2a5
usb_task_thread(c0a4c840,e299bd38) at usb_task_thread+0x47
fork_exit(c06589fc,c0a4c840,e299bd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe299bd70, ebp = 0 ---

Tracing command usb0 pid 24 tid 100020 td 0xc3f001b0
sched_switch(c3f001b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3ffb210,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c3ffb210) at sleepq_timedwait+0x4a
msleep(c3ffb210,0,5c,c09402e9,ea60,c4023440) at msleep+0x281
usb_event_thread(c4023440,e2998d38) at usb_event_thread+0x94
fork_exit(c0658948,c4023440,e2998d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2998d70, ebp = 0 ---

Tracing command irq12: uhci0 uhci1 pid 23 tid 100021 td 0xc4007870
fork_trampoline() at fork_trampoline

Tracing command irq15: ata1 pid 22 tid 100022 td 0xc40076c0
sched_switch(c40076c0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c4022800,e43fad38) at ithread_loop+0xda
fork_exit(c06adf00,c4022800,e43fad38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe43fad70, ebp = 0 ---

Tracing command irq14: ata0 pid 21 tid 100023 td 0xc4007510
sched_switch(c4007510,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c40226b0,e43f7d38) at ithread_loop+0xda
fork_exit(c06adf00,c40226b0,e43f7d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe43f7d70, ebp = 0 ---

Tracing command irq9: acpi0 pid 20 tid 100024 td 0xc4007360
fork_trampoline() at fork_trampoline

Tracing command kqueue taskq pid 9 tid 100025 td 0xc40071b0
sched_switch(c40071b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f78680,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c3f78680,0,c094eb1b,49,8,...) at sleepq_wait+0x46
msleep(c3f78680,c3f7869c,0,c0943377,0) at msleep+0x2a5
taskqueue_thread_loop(c0a4d7dc,e43f1d38) at taskqueue_thread_loop+0x78
fork_exit(c06e8c4c,c0a4d7dc,e43f1d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe43f1d70, ebp = 0 ---

Tracing command swi2: cambio pid 19 tid 100026 td 0xc4007000
fork_trampoline() at fork_trampoline

Tracing command acpi_task_2 pid 8 tid 100008 td 0xc3eff1b0
sched_switch(c3eff1b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f78800,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c3f78800,0,c094eb1b,49,8,...) at sleepq_wait+0x46
msleep(c3f78800,c3f7881c,0,c0943377,0) at msleep+0x2a5
taskqueue_thread_loop(c0c0e214,e297dd38) at taskqueue_thread_loop+0x78
fork_exit(c06e8c4c,c0c0e214,e297dd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe297dd70, ebp = 0 ---
        
Tracing command acpi_task_1 pid 7 tid 100009 td 0xc3eff000
sched_switch(c3eff000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f78800,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c3f78800,0,c094eb1b,49,8,...) at sleepq_wait+0x46
msleep(c3f78800,c3f7881c,0,c0943377,0) at msleep+0x2a5
taskqueue_thread_loop(c0c0e214,e297ad38) at taskqueue_thread_loop+0x78
fork_exit(c06e8c4c,c0c0e214,e297ad38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe297ad70, ebp = 0 ---

Tracing command acpi_task_0 pid 6 tid 100010 td 0xc3f02360
sched_switch(c3f02360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f78800,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c3f78800,0,c094eb1b,49,c3f415e0,...) at sleepq_wait+0x46
msleep(c3f78800,c3f7881c,0,c0943377,0) at msleep+0x2a5
taskqueue_thread_loop(c0c0e214,e29b6d38) at taskqueue_thread_loop+0x78
fork_exit(c06e8c4c,c0c0e214,e29b6d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29b6d70, ebp = 0 ---

Tracing command swi5: + pid 18 tid 100011 td 0xc3f021b0
sched_switch(c3f021b0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c3f854d0,e29b3d38) at ithread_loop+0xda
fork_exit(c06adf00,c3f854d0,e29b3d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29b3d70, ebp = 0 ---

Tracing command thread taskq pid 5 tid 100012 td 0xc3f02000
sched_switch(c3f02000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f78980,c0a4ea90,0,c094e4e3,21e,...) at sleepq_switch+0xc9
sleepq_wait(c3f78980,0,c094eb1b,49,8,...) at sleepq_wait+0x46
msleep(c3f78980,c3f7899c,0,c0943377,0) at msleep+0x2a5
taskqueue_thread_loop(c0a54870,e29b0d38) at taskqueue_thread_loop+0x78
fork_exit(c06e8c4c,c0a54870,e29b0d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29b0d70, ebp = 0 ---

Tracing command swi6: Giant taskq pid 17 tid 100013 td 0xc3f00d80
fork_trampoline() at fork_trampoline

Tracing command swi6: task queue pid 16 tid 100014 td 0xc3f00bd0
sched_switch(c3f00bd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c3f85500,e29aad38) at ithread_loop+0xda
fork_exit(c06adf00,c3f85500,e29aad38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29aad70, ebp = 0 ---

Tracing command yarrow pid 15 tid 100015 td 0xc3f00a20
sched_switch(c3f00a20,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a391e0,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0a391e0) at sleepq_timedwait+0x4a
msleep(c0a391e0,0,0,c0943377,64) at msleep+0x281
random_kthread(0,e29a7d38) at random_kthread+0x180
fork_exit(c05f560c,0,e29a7d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29a7d70, ebp = 0 ---

Tracing command g_down pid 4 tid 100016 td 0xc3f00870
sched_switch(c3f00870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4d01c,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0a4d01c) at sleepq_timedwait+0x4a
msleep(c0a4d01c,c0a4cf08,24c,c0943377,64) at msleep+0x281
g_io_schedule_down(c3f00870) at g_io_schedule_down+0x56
g_down_procbody(0,e29a4d38) at g_down_procbody+0x5a
fork_exit(c06853cc,0,e29a4d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe29a4d70, ebp = 0 ---

Tracing command g_up pid 3 tid 100000 td 0xc3f00000
sched_switch(c3f00000,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4d018,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0a4d018) at sleepq_timedwait+0x4a
msleep(c0a4d018,c0a4cf48,24c,c0943377,64) at msleep+0x281
g_io_schedule_up(c3f00000) at g_io_schedule_up+0x127
g_up_procbody(0,e2995d38) at g_up_procbody+0x5a
fork_exit(c068536c,0,e2995d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2995d70, ebp = 0 ---

Tracing command g_event pid 2 tid 100001 td 0xc3effd80
sched_switch(c3effd80,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c0a4d010,c0a4ea90,0,c094e4e3,243,...) at sleepq_switch+0xc9
sleepq_timedwait(c0a4d010) at sleepq_timedwait+0x4a
msleep(c0a4d010,0,4c,c0943377,64) at msleep+0x281
g_event_procbody(0,e2992d38) at g_event_procbody+0x9e
fork_exit(c068542c,0,e2992d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2992d70, ebp = 0 ---

Tracing command swi1: net pid 14 tid 100002 td 0xc3effbd0
sched_switch(c3effbd0,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
ithread_loop(c3ec48e0,e298fd38) at ithread_loop+0xda
fork_exit(c06adf00,c3ec48e0,e298fd38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe298fd70, ebp = 0 ---

Tracing command swi3: vm pid 13 tid 100003 td 0xc3effa20
fork_trampoline() at fork_trampoline

Tracing command swi4: clock sio pid 12 tid 100004 td 0xc3eff870
sched_switch(c3eff870,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
turnstile_wait(c0a9df6c,c45dabd0,0,c0a9df6c,2,c094a0d3,166) at turnstile_wait+0x3ea
_mtx_lock_sleep(c0a9df6c,c3eff870,0,c095d7ff,73) at _mtx_lock_sleep+0x145
_mtx_lock_flags(c0a9df6c,0,c095d7ff,73,e2989c98,...) at _mtx_lock_flags+0xab
tcp_slowtimo(0,16,e2989ccc,c06cf537,0,...) at tcp_slowtimo+0x27
pfslowtimo(0) at pfslowtimo+0x45
softclock(0) at softclock+0x22f
ithread_execute_handlers(c3efe900,c3f46600) at ithread_execute_handlers+0x11e
ithread_loop(c3ec4900,e2989d38) at ithread_loop+0x67
fork_exit(c06adf00,c3ec4900,e2989d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2989d70, ebp = 0 ---

Tracing command idle: cpu0 pid 11 tid 100005 td 0xc3eff6c0
kdb_enter(c093cb3b) at kdb_enter+0x2b
siointr1(c40b4000,c0abdfac,0,c096e56b,56e,...) at siointr1+0xce
siointr(c40b4000) at siointr+0x21
intr_execute_handlers(c3ef4cc8,e2986c94,4,e2986cd4,c08ab2c4,...) at intr_execute_handlers+0xe1
lapic_handle_intr(36,e2986c94) at lapic_handle_intr+0x2f
Xapic_isr1() at Xapic_isr1+0x34
--- interrupt, eip = 0xc0c000b5, esp = 0xe2986cd4, ebp = 0xe2986cd4 ---
acpi_cpu_c1(c0a4ea90,c06d85a8,e2986cf4,c06d85a8,c06d85a8,...) at acpi_cpu_c1+0x5
acpi_cpu_idle(e2986d04,c06d85d1,e2986d24,c06acf40,0,...) at acpi_cpu_idle+0x162
cpu_idle(e2986d24,c06acf40,0,e2986d38,c3efeb40,...) at cpu_idle+0x28
sched_idletd(0,e2986d38) at sched_idletd+0x29
fork_exit(c06d85a8,0,e2986d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2986d70, ebp = 0 ---
        
Tracing command idle: cpu1 pid 10 tid 100006 td 0xc3eff510
cpustop_handler(e2983c88,c08c07d3,e2983c20,c06b9bed,c3f46900,...) at cpustop_handler+0x31
ipi_nmi_handler(e2983c20,c06b9bed,c3f46900,0,0,...) at ipi_nmi_handler+0x28
trap(e2983c94) at trap+0x3f
calltrap() at calltrap+0x6
--- trap 0x13, eip = 0xc0c000b5, esp = 0xe2983cd4, ebp = 0xe2983cd4 ---
acpi_cpu_c1(c0a4ea90,c06d85a8,e2983cf4,c06d85a8,c06d85a8,...) at acpi_cpu_c1+0x5
acpi_cpu_idle(e2983d04,c06d85d1,e2983d24,c06acf40,0,...) at acpi_cpu_idle+0x162
cpu_idle(e2983d24,c06acf40,0,e2983d38,c3efed80,...) at cpu_idle+0x28
sched_idletd(0,e2983d38) at sched_idletd+0x29
fork_exit(c06d85a8,0,e2983d38) at fork_exit+0xac
fork_trampoline() at fork_trampoline+0x8
--- trap 0, eip = 0, esp = 0xe2983d70, ebp = 0 ---

Tracing command init pid 1 tid 100007 td 0xc3eff360
sched_switch(c3eff360,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
sleepq_switch(c3f01000) at sleepq_switch+0xc9
sleepq_wait_sig(c3f01000) at sleepq_wait_sig+0x1d
msleep(c3f01000,c3f01060,15c,c095061f,0) at msleep+0x296
kern_wait(c3eff360,ffffffff,e2980c28,0,0) at kern_wait+0x7c3
wait4(c3eff360,e2980d00) at wait4+0x2a
syscall(e2980d38) at syscall+0x256
Xint0x80_syscall() at Xint0x80_syscall+0x20
--- syscall (7, FreeBSD ELF32, wait4), eip = 0x80531f7, esp = 0xbfbfe92c, ebp = 0xbfbfe948 ---

Tracing command swapper pid 0 tid 0 td 0xc0a4d320
sched_switch(c0a4d320,0,1) at sched_switch+0xff
mi_switch(1,0) at mi_switch+0x280
scheduler(0,101ec00,101e000,0,c0452345,...) at scheduler+0x195
mi_startup() at mi_startup+0x96
begin() at begin+0x2c

db> panic
panic: from debugger
cpuid = 0
Uptime: 13h55m37s
Physical memory: 1011 MB
Dumping 152 MB: 137 121 105 89 73 57 41 25 9
Dump complete
Automatic reboot in 15 seconds - press a key on the console to abort






Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070311062637.GA1256>