Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 5 Feb 2007 12:40:38 GMT
From:      "Joerg Lehners" <Joerg.Lehners@Informatik.Uni-Oldenburg.DE>
To:        freebsd-bugs@FreeBSD.org
Subject:   Re: kern/107109: [nfs] Netbeans/Eclipse/Java provoke deadlocks when used with NFS
Message-ID:  <200702051240.l15Cecee039442@freefall.freebsd.org>

next in thread | raw e-mail | index | archive | help
The following reply was made to PR kern/107109; it has been noted by GNATS.

From: "Joerg Lehners" <Joerg.Lehners@Informatik.Uni-Oldenburg.DE>
To: bug-followup@FreeBSD.org
Cc:  
Subject: Re: kern/107109: [nfs] Netbeans/Eclipse/Java provoke deadlocks when
 used with NFS
Date: Mon, 5 Feb 2007 13:33:21 +0100 (MET)

 On Fri, 2 Feb 2007, Joerg Lehners wrote:
 > On Tue, 30 Jan 2007, Joerg Lehners wrote:
 >> [...]
 >> I'll try to write a threaded C program to try to show that
 >> this issue is not specific to Java. Perhaps someone ele is faster? :)
 >
 > Done, see attached file.
 > [...]
 
 On request of Craig Rodrigues I put the following ddb ouput here.
 Note: this output is from another machine (testido). The ddb output
 from 22 Dec. 2006 was from the machine cenis, which has slightly
 different hardware and its kernel config did not include all
 debugging options listed in kerneldebug-deadlocks.html.
 
 --- ddb.output begins here ---
 
 FreeBSD/i386 (testido) (ttyd0)
 
 login: [thread pid 10 tid 100006 ]
 Stopped at      kdb_enter+0x2b: nop
 db> ps
    pid  ppid  pgrp   uid   state   wmesg     wchan    cmd
    371   355   371  1002  T+      (threaded)          c
 100078                   s                           c
 100081                   D       nfs      0xc36dc5c8 c
 100080                   s                           c
 100077                   Ss      ksesigwa 0xc3515dc8 c
    355   354   355  1002  S+      pause    0xc370c67c ksh
    354   353   354     0  Ss+     wait     0xc36c1a78 login
    353   152   353     0  Ss      select   0xc07dbf24 rlogind
    352     1   337     0  S       select   0xc07dbf24 xconsole
    331   301   331     0  Ss      select   0xc07dbf24 xdm
    325   304   325     0  S       select   0xc07dbf24 Xorg
    304   301   304     0  Ss      pause    0xc370567c ksh
    301     1   301     0  Ss      select   0xc07dbf24 xdm
    300     1   300     0  Ss+     ttyin    0xc33bb410 getty
    299     1   299     0  Ss+     ttyin    0xc33d2410 getty
    298     1   298     0  Ss+     ttyin    0xc33bd810 getty
    297     1   297     0  Ss+     ttyin    0xc33bd410 getty
    296     1   296     0  Ss+     ttyin    0xc33d2c10 getty
    290     1   290     0  Ss      select   0xc07dbf24 usbd
    287     1    40     0  S+      fifoor   0xc34ee6c8 ksh
    259     1   259     0  Ss      select   0xc07dbf24 nwhod
    248     1   248     0  Ss      select   0xc07dbf24 lpd
    245     1   245     0  Ss      nanslp   0xc07913cc cron
    243     1   243     0  Ss      select   0xc07dbf24 ntpd
    238     1   238   501  Ss      select   0xc07dbf24 exim-4.66-1
    193   188   188     0  S       -        0xc34f0a00 nfsd
    191   188   188     0  S       -        0xc34f0400 nfsd
    190   188   188     0  S       -        0xc34f0c00 nfsd
    189   188   188     0  S       -        0xc34ef200 nfsd
    188     1   188     0  Ss      accept   0xc3600892 nfsd
    183     1   183     0  Ss      select   0xc07dbf24 mountd
    174     1   174     0  Ss      select   0xc07dbf24 ypbind
    165     1   165     0  Ss      select   0xc07dbf24 rpcbind
    157     1   157     0  Ss      select   0xc07dbf24 sshd
    152     1   152     0  Ss      select   0xc07dbf24 inetd
    147     1   147     0  Ss      select   0xc07dbf24 syslogd
    135     1   135     0  Ss      select   0xc07dbf24 devd
     39     0     0     0  SL      -        0xd56d9d04 [schedcpu]
     38     0     0     0  SL      sdflush  0xc07e4f94 [softdepflush]
     37     0     0     0  SL      vlruwt   0xc33be000 [vnlru]
     36     0     0     0  SL      syncer   0xc079113c [syncer]
     35     0     0     0  SL      psleep   0xc07dc48c [bufdaemon]
     34     0     0     0  SL      pgzero   0xc07e5f04 [pagezero]
     33     0     0     0  SL      psleep   0xc07e5a54 [vmdaemon]
     32     0     0     0  SL      psleep   0xc07e5a10 [pagedaemon]
     31     0     0     0  WL                          [irq12: psm0]
     30     0     0     0  WL                          [irq1: atkbd0]
     29     0     0     0  WL                          [swi0: sio]
     28     0     0     0  WL                          [irq7: ppc0]
     27     0     0     0  SL      -        0xc33a623c [fdc0]
     26     0     0     0  WL                          [irq10: atapci1]
     25     0     0     0  SL      usbevt   0xc339c210 [usb1]
     24     0     0     0  SL      usbtsk   0xc078e204 [usbtask]
     23     0     0     0  SL      usbevt   0xc3390210 [usb0]
     22     0     0     0  WL                          [irq5: xl0 uhci0++]
     21     0     0     0  WL                          [irq15: ata1]
     20     0     0     0  WL                          [irq14: ata0]
     19     0     0     0  WL                          [irq9: acpi0]
     18     0     0     0  WL                          [swi6: Giant taskq]
     17     0     0     0  WL                          [swi6: task queue]
     16     0     0     0  WL                          [swi2: cambio]
      9     0     0     0  SL      -        0xc3351300 [acpi_task_2]
      8     0     0     0  SL      -        0xc3351300 [acpi_task_1]
      7     0     0     0  SL      -        0xc3351300 [acpi_task_0]
      6     0     0     0  SL      -        0xc3351380 [kqueue taskq]
     15     0     0     0  WL                          [swi5: +]
      5     0     0     0  SL      -        0xc329f380 [thread taskq]
     14     0     0     0  SL      -        0xc078b120 [yarrow]
      4     0     0     0  SL      -        0xc078e888 [g_down]
      3     0     0     0  SL      -        0xc078e884 [g_up]
      2     0     0     0  SL      -        0xc078e87c [g_event]
     13     0     0     0  WL                          [swi3: vm]
     12     0     0     0  WL                          [swi4: clock sio]
     11     0     0     0  WL                          [swi1: net]
     10     0     0     0  RL      CPU 0               [idle]
      1     0     1     0  SLs     wait     0xc32a5000 [init]
      0     0     0     0  WLs                         [swapper]
 db> show pcpu
 cpuid        = 0
 curthread    = 0xc32a1600: pid 10 "idle"
 curpcb       = 0xd44afd90
 fpcurthread  = none
 idlethread   = 0xc32a1600: pid 10 "idle"
 APIC ID      = 0
 currentldt   = 0x50
 spin locks held:
 exclusive spin mutex sio r = 0 (0xc07e7f80) locked @ dev/sio/sio.c:1390
 db> show allpcpu
 Current CPU: 0
 
 cpuid        = 0
 curthread    = 0xc32a1600: pid 10 "idle"
 curpcb       = 0xd44afd90
 fpcurthread  = none
 idlethread   = 0xc32a1600: pid 10 "idle"
 APIC ID      = 0
 currentldt   = 0x50
 spin locks held:
 exclusive spin mutex sio r = 0 (0xc07e7f80) locked @ dev/sio/sio.c:1390
 
 db> show locks
 exclusive spin mutex sio r = 0 (0xc07e7f80) locked @ dev/sio/sio.c:1390
 db> show alllocks
 db> show lockedvnods
 Locked vnodes
 
 0xc36dc570: tag nfs, type VDIR
      usecount 5, writecount 0, refcount 5 mountedhere 0
      flags (VV_ROOT)
       lock type nfs: EXCL (count 1) by thread 0xc3709900 (pid 371) with 1 pending#0 0xc054ec81 at lockmgr+0x4ed
 #1 0xc05a6c39 at vop_stdlock+0x21
 #2 0xc06da3ff at VOP_LOCK_APV+0x87
 #3 0xc05bbc2c at vn_lock+0xac
 #4 0xc05a885e at lookup+0xd2
 #5 0xc05a850e at namei+0x39a
 #6 0xc05b5f65 at kern_stat+0x35
 #7 0xc05b5f13 at stat+0x1b
 #8 0xc06c49fb at syscall+0x25b
 #9 0xc06b2b3f at Xint0x80_syscall+0x1f
 
  	fileid 2 fsid 0x100ff05
 db> alltrace
 
 Tracing command c pid 371 tid 100078 td 0xc3709600
 sched_switch(c3709600,0,1) at sched_switch+0x177
 mi_switch(1,0,c3515cf8,0,c0717139,335,c3709600,c3515c90) at mi_switch+0x270
 thread_single(0,c3515cf8,0,c07135ea,111) at thread_single+0x24f
 fork1(c3709600,14,0,da9dacd4) at fork1+0x16b
 fork(c3709600,da9dad04) at fork+0x18
 syscall(3b,3b,3b,8807c050,8818aa24,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (2, FreeBSD ELF32, fork), eip = 0x880fd6cf, esp = 0xbf7fcf4c, ebp = 0xbf7fcf98 ---
 
 Tracing command c pid 371 tid 100081 td 0xc3709780
 sched_switch(c3709780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c36dc5c8,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c36dc5c8,0,c36dc638,b7,c071ef72,...) at sleepq_wait+0x46
 msleep(c36dc5c8,c0790268,50,c07257ab,0,...) at msleep+0x279
 acquire(da9dda64,40,60000,c3709780,0,...) at acquire+0x76
 lockmgr(c36dc5c8,3002,c36dc638,c3709780,da9dda88,...) at lockmgr+0x44e
 vop_stdlock(da9ddaa8) at vop_stdlock+0x21
 VOP_LOCK_APV(c0770fa0,da9ddaa8) at VOP_LOCK_APV+0x87
 vn_lock(c36dc570,1002,c3709780) at vn_lock+0xac
 lookup(da9ddc00) at lookup+0xd2
 namei(da9ddc00) at namei+0x39a
 kern_stat(c3709780,bf6fbe98,0,da9ddc74) at kern_stat+0x35
 stat(c3709780,da9ddd04) at stat+0x1b
 syscall(3b,3b,3b,8807c050,bf6fbfec,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (188, FreeBSD ELF32, stat), eip = 0x88159447, esp = 0xbf6fbdec, ebp = 0xbf6fbfb8 ---
 
 Tracing command c pid 371 tid 100080 td 0xc3709900
 sched_switch(c3709900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c37efe00) at sleepq_switch+0xc1
 sleepq_wait_sig(c37efe00) at sleepq_wait_sig+0x1d
 msleep(c37efe00,c07df2e0,153,c072514f,0) at msleep+0x26a
 nfs_reply(c37efe00,0,29d,0,c34e0300,...) at nfs_reply+0x1ec
 nfs_request(c36dc570,c34e2100,3,c3709900,c37efc00,da9e0a04,da9e0a08,da9e0a0c,c3709900,da9e0a18) at nfs_request+0x371
 nfs_lookup(da9e0b14) at nfs_lookup+0x2d4
 VOP_LOOKUP_APV(c0770fa0,da9e0b14) at VOP_LOOKUP_APV+0x87
 lookup(da9e0c00) at lookup+0x46e
 namei(da9e0c00) at namei+0x39a
 kern_stat(c3709900,bf4f9e98,0,da9e0c74) at kern_stat+0x35
 stat(c3709900,da9e0d04) at stat+0x1b
 syscall(3b,3b,3b,8807c050,bf4f9fec,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (188, FreeBSD ELF32, stat), eip = 0x88159447, esp = 0xbf4f9dec, ebp = 0xbf4f9fb8 ---
 
 Tracing command c pid 371 tid 100077 td 0xc351bc00
 sched_switch(c351bc00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3515dc8) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c3515dc8) at sleepq_timedwait_sig+0x1e
 msleep(c3515dc8,c3515cf8,168,c0713c77,7531,...) at msleep+0x240
 kse_release(c351bc00,d97f7d04) at kse_release+0x121
 syscall(3b,3b,3b,804d100,81,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (383, FreeBSD ELF32, kse_release), eip = 0x880a04f7, esp = 0xbf9fef2c, ebp = 0xbf9fef88 ---
 
 Tracing command ksh pid 355 tid 100073 td 0xc3706300
 sched_switch(c3706300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c370c67c) at sleepq_switch+0xc1
 sleepq_wait_sig(c370c67c) at sleepq_wait_sig+0x1d
 msleep(c370c67c,c370c6b0,168,c07166be,0) at msleep+0x26a
 kern_sigsuspend(c3706300,0,0,0,0,...) at kern_sigsuspend+0xa3
 sigsuspend(c3706300,da9b6d04) at sigsuspend+0x33
 syscall(3b,3b,3b,0,8079be8,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x880eb797, esp = 0xbfbfea8c, ebp = 0xbfbfeab8 ---
 
 Tracing command login pid 354 tid 100061 td 0xc3519480
 sched_switch(c3519480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c36c1a78) at sleepq_switch+0xc1
 sleepq_wait_sig(c36c1a78) at sleepq_wait_sig+0x1d
 msleep(c36c1a78,c36c1ae0,15c,c071af18,0) at msleep+0x26a
 kern_wait(c3519480,163,d97cac84,0,0) at kern_wait+0x6b7
 wait4(c3519480,d97cad04) at wait4+0x2a
 syscall(3b,3b,3b,246,1,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x880fe6af, esp = 0xbfbfed2c, ebp = 0xbfbfed48 ---
 
 Tracing command rlogind pid 353 tid 100055 td 0xc3519d80
 sched_switch(c3519d80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c3519d80,4,bfbfe0d8,0,bfbfe0dc,...) at kern_select+0x4bf
 select(c3519d80,d97dcd04) at select+0x44
 syscall(3b,3b,3b,0,3,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8814e2c3, esp = 0xbfbfe06c, ebp = 0xbfbfe908 ---
 
 Tracing command xconsole pid 352 tid 100076 td 0xc351bd80
 sched_switch(c351bd80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,3b7,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 poll(c351bd80,d97fad04) at poll+0x3b9
 syscall(3b,3b,3b,805a9e0,6,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (209, FreeBSD ELF32, poll), eip = 0x882b0dc3, esp = 0xbfbfea5c, ebp = 0xbfbfea78 ---
 
 Tracing command xdm pid 331 tid 100065 td 0xc3709000
 sched_switch(c3709000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,3b7,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 poll(c3709000,da9ced04) at poll+0x3b9
 syscall(3b,3b,3b,0,1,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (209, FreeBSD ELF32, poll), eip = 0x8829fdc3, esp = 0xbfbfe5ec, ebp = 0xbfbfe608 ---
 
 Tracing command Xorg pid 325 tid 100058 td 0xc3519900
 sched_switch(c3519900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07dbf24,c07dbf24,ea4c,d97d3b40,203246,...) at sleepq_timedwait_sig+0x1e
 cv_timedwait_sig(c07dbf24,c07dbf00,ea4c) at cv_timedwait_sig+0x192
 kern_select(c3519900,14,81efe60,0,0,...) at kern_select+0x4a9
 select(c3519900,d97d3d04) at select+0x44
 syscall(3b,3b,3b,0,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x882d52c3, esp = 0xbfbfe6ec, ebp = 0xbfbfe938 ---
 
 Tracing command ksh pid 304 tid 100066 td 0xc3706d80
 sched_switch(c3706d80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c370567c) at sleepq_switch+0xc1
 sleepq_wait_sig(c370567c) at sleepq_wait_sig+0x1d
 msleep(c370567c,c37056b0,168,c07166be,0) at msleep+0x26a
 kern_sigsuspend(c3706d80,0,0,0,0,...) at kern_sigsuspend+0xa3
 sigsuspend(c3706d80,da9cbd04) at sigsuspend+0x33
 syscall(3b,3b,3b,5,8079360,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (341, FreeBSD ELF32, sigsuspend), eip = 0x880eb797, esp = 0xbfbfe9bc, ebp = 0xbfbfe9e8 ---
 
 Tracing command xdm pid 301 tid 100069 td 0xc3706900
 sched_switch(c3706900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c3706900,6,bfbfe960,0,0,...) at kern_select+0x4bf
 select(c3706900,da9c2d04) at select+0x44
 syscall(3b,3b,3b,8050528,bfbfe960,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x882e82c3, esp = 0xbfbfe93c, ebp = 0xbfbfe9e8 ---
 
 Tracing command getty pid 300 tid 100052 td 0xc351b300
 sched_switch(c351b300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33bb410) at sleepq_switch+0xc1
 sleepq_wait_sig(c33bb410) at sleepq_wait_sig+0x1d
 msleep(c33bb410,0,159,c071c433,0) at msleep+0x26a
 ttysleep(c33bb400,c33bb410,159,c071c433,0,...) at ttysleep+0x21
 ttread(c33bb400,d97e5cbc,0) at ttread+0x48f
 ttyread(c33c1000,d97e5cbc,0,c0790ae0,0,...) at ttyread+0x2f
 giant_read(c33c1000,d97e5cbc,0) at giant_read+0x2d
 devfs_read_f(c353dbd0,d97e5cbc,c329ed80,0,c351b300) at devfs_read_f+0x62
 dofileread(c351b300,0,c353dbd0,d97e5cbc,ffffffff,...) at dofileread+0x85
 kern_readv(c351b300,0,d97e5cbc,bfbfee1f,1,...) at kern_readv+0x36
 read(c351b300,d97e5d04) at read+0x45
 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (3, FreeBSD ELF32, read), eip = 0x88150343, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
 
 Tracing command getty pid 299 tid 100070 td 0xc3706780
 sched_switch(c3706780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33d2410) at sleepq_switch+0xc1
 sleepq_wait_sig(c33d2410) at sleepq_wait_sig+0x1d
 msleep(c33d2410,0,159,c071c433,0) at msleep+0x26a
 ttysleep(c33d2400,c33d2410,159,c071c433,0,...) at ttysleep+0x21
 ttread(c33d2400,da9bfcbc,0) at ttread+0x48f
 ttyread(c33c1900,da9bfcbc,0,da9bfc14,c05350bd,...) at ttyread+0x2f
 scread(c33c1900,da9bfcbc,0,c0790ae0,0,...) at scread+0x22
 giant_read(c33c1900,da9bfcbc,0) at giant_read+0x2d
 devfs_read_f(c353d438,da9bfcbc,c329ed80,0,c3706780) at devfs_read_f+0x62
 dofileread(c3706780,0,c353d438,da9bfcbc,ffffffff,...) at dofileread+0x85
 kern_readv(c3706780,0,da9bfcbc,bfbfee1f,1,...) at kern_readv+0x36
 read(c3706780,da9bfd04) at read+0x45
 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (3, FreeBSD ELF32, read), eip = 0x88150343, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
 
 Tracing command getty pid 298 tid 100072 td 0xc3706480
 sched_switch(c3706480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33bd810) at sleepq_switch+0xc1
 sleepq_wait_sig(c33bd810) at sleepq_wait_sig+0x1d
 msleep(c33bd810,0,159,c071c433,0) at msleep+0x26a
 ttysleep(c33bd800,c33bd810,159,c071c433,0,...) at ttysleep+0x21
 ttread(c33bd800,da9b9cbc,0) at ttread+0x48f
 ttyread(c33c1a00,da9b9cbc,0,da9b9c14,c05350bd,...) at ttyread+0x2f
 scread(c33c1a00,da9b9cbc,0,c0790ae0,0,...) at scread+0x22
 giant_read(c33c1a00,da9b9cbc,0) at giant_read+0x2d
 devfs_read_f(c353d5e8,da9b9cbc,c329ed80,0,c3706480) at devfs_read_f+0x62
 dofileread(c3706480,0,c353d5e8,da9b9cbc,ffffffff,...) at dofileread+0x85
 kern_readv(c3706480,0,da9b9cbc,bfbfee1f,1,...) at kern_readv+0x36
 read(c3706480,da9b9d04) at read+0x45
 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (3, FreeBSD ELF32, read), eip = 0x88150343, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
 
 Tracing command getty pid 297 tid 100064 td 0xc3519000
 sched_switch(c3519000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33bd410) at sleepq_switch+0xc1
 sleepq_wait_sig(c33bd410) at sleepq_wait_sig+0x1d
 msleep(c33bd410,0,159,c071c433,0) at msleep+0x26a
 ttysleep(c33bd400,c33bd410,159,c071c433,0,...) at ttysleep+0x21
 ttread(c33bd400,d97c1cbc,0) at ttread+0x48f
 ttyread(c33c1b00,d97c1cbc,0,d97c1c14,c05350bd,...) at ttyread+0x2f
 scread(c33c1b00,d97c1cbc,0,c0790ae0,0,...) at scread+0x22
 giant_read(c33c1b00,d97c1cbc,0) at giant_read+0x2d
 devfs_read_f(c353dcf0,d97c1cbc,c329ed80,0,c3519000) at devfs_read_f+0x62
 dofileread(c3519000,0,c353dcf0,d97c1cbc,ffffffff,...) at dofileread+0x85
 kern_readv(c3519000,0,d97c1cbc,bfbfee1f,1,...) at kern_readv+0x36
 read(c3519000,d97c1d04) at read+0x45
 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (3, FreeBSD ELF32, read), eip = 0x88150343, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
 
 Tracing command getty pid 296 tid 100051 td 0xc351b480
 sched_switch(c351b480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33d2c10) at sleepq_switch+0xc1
 sleepq_wait_sig(c33d2c10) at sleepq_wait_sig+0x1d
 msleep(c33d2c10,0,159,c071c433,0) at msleep+0x26a
 ttysleep(c33d2c00,c33d2c10,159,c071c433,0,...) at ttysleep+0x21
 ttread(c33d2c00,d97e8cbc,0) at ttread+0x48f
 ttyread(c33c1c00,d97e8cbc,0,d97e8c14,c05350bd,...) at ttyread+0x2f
 scread(c33c1c00,d97e8cbc,0,c0790ae0,0,...) at scread+0x22
 giant_read(c33c1c00,d97e8cbc,0) at giant_read+0x2d
 devfs_read_f(c353da68,d97e8cbc,c329ed80,0,c351b480) at devfs_read_f+0x62
 dofileread(c351b480,0,c353da68,d97e8cbc,ffffffff,...) at dofileread+0x85
 kern_readv(c351b480,0,d97e8cbc,bfbfee1f,1,...) at kern_readv+0x36
 read(c351b480,d97e8d04) at read+0x45
 syscall(3b,3b,3b,0,804e580,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (3, FreeBSD ELF32, read), eip = 0x88150343, esp = 0xbfbfedfc, ebp = 0xbfbfee28 ---
 
 Tracing command usbd pid 290 tid 100074 td 0xc3706180
 sched_switch(c3706180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07dbf24,c07dbf24,7531,da9b3b40,246,...) at sleepq_timedwait_sig+0x1e
 cv_timedwait_sig(c07dbf24,c07dbf00,7531) at cv_timedwait_sig+0x192
 kern_select(c3706180,6,bfbfed80,bfbfed00,0,...) at kern_select+0x4a9
 select(c3706180,da9b3d04) at select+0x44
 syscall(3b,3b,3b,bfbfed00,1e,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x881442c3, esp = 0xbfbfecac, ebp = 0xbfbfee58 ---
 
 Tracing command ksh pid 287 tid 100071 td 0xc3706600
 sched_switch(c3706600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c34ee6c8) at sleepq_switch+0xc1
 sleepq_wait_sig(c34ee6c8) at sleepq_wait_sig+0x1d
 msleep(c34ee6c8,c078e400,358,c070e041,0,...) at msleep+0x26a
 fifo_open(da9bca64) at fifo_open+0x3b3
 VOP_OPEN_APV(c0775200,da9bca64) at VOP_OPEN_APV+0x9b
 vn_open_cred(da9bcbcc,da9bcccc,1a4,c329ed80,3,...) at vn_open_cred+0x3f6
 vn_open(da9bcbcc,da9bcccc,1a4,3) at vn_open+0x1e
 kern_open(c3706600,80818e0,0,1,1b6,...) at kern_open+0xb6
 open(c3706600,da9bcd04) at open+0x1a
 syscall(3b,3b,3b,bfbfeac4,80799ec,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (5, FreeBSD ELF32, open), eip = 0x8815c4a7, esp = 0xbfbfea1c, ebp = 0xbfbfeae8 ---
 
 Tracing command nwhod pid 259 tid 100054 td 0xc351b000
 sched_switch(c351b000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07dbf24,c07dbf24,1d4c1,d97dfb40,246,...) at sleepq_timedwait_sig+0x1e
 cv_timedwait_sig(c07dbf24,c07dbf00,1d4c1) at cv_timedwait_sig+0x192
 kern_select(c351b000,14,bfbfe8b0,0,0,...) at kern_select+0x4a9
 select(c351b000,d97dfd04) at select+0x44
 syscall(3b,3b,3b,18,20,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x881442c3, esp = 0xbfbfe82c, ebp = 0xbfbfee58 ---
 
 Tracing command lpd pid 248 tid 100059 td 0xc3519780
 sched_switch(c3519780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c3519780,14,bfbfeab0,0,0,...) at kern_select+0x4bf
 select(c3519780,d97d0d04) at select+0x44
 syscall(3b,3b,3b,805d030,805d030,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8814f2c3, esp = 0xbfbfe0cc, ebp = 0xbfbfee78 ---
 
 Tracing command cron pid 245 tid 100050 td 0xc351b600
 sched_switch(c351b600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07913cc) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07913cc) at sleepq_timedwait_sig+0x1e
 msleep(c07913cc,0,15c,c0717308,ea61,...) at msleep+0x240
 kern_nanosleep(c351b600,d97ebccc,d97ebcc4) at kern_nanosleep+0xab
 nanosleep(c351b600,d97ebd04) at nanosleep+0x4f
 syscall(3b,3b,3b,805008c,3c,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (240, FreeBSD ELF32, nanosleep), eip = 0x88139f03, esp = 0xbfbfecdc, ebp = 0xbfbfed08 ---
 
 Tracing command ntpd pid 243 tid 100053 td 0xc351b180
 sched_switch(c351b180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c351b180,7,bfbfedb0,0,0,...) at kern_select+0x4bf
 select(c351b180,d97e2d04) at select+0x44
 syscall(3b,3b,3b,bfbfedb0,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x882922c3, esp = 0xbfbfed6c, ebp = 0xbfbfee48 ---
 
 Tracing command exim-4.66-1 pid 238 tid 100057 td 0xc3519a80
 sched_switch(c3519a80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c3519a80,4,bfbfea00,0,0,...) at kern_select+0x4bf
 select(c3519a80,d97d6d04) at select+0x44
 syscall(3b,3b,3b,8,3,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8844b2c3, esp = 0xbfbfe94c, ebp = 0xbfbfeab8 ---
 
 Tracing command nfsd pid 193 tid 100060 td 0xc3519600
 sched_switch(c3519600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c34f0a00) at sleepq_switch+0xc1
 sleepq_wait_sig(c34f0a00) at sleepq_wait_sig+0x1d
 msleep(c34f0a00,c07e4ca0,158,c070f1de,0,...) at msleep+0x26a
 nfssvc_nfsd(c3519600) at nfssvc_nfsd+0xe5
 nfssvc(c3519600,d97cdd04) at nfssvc+0x18c
 syscall(3b,3b,3b,1,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x880bd197, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
 
 Tracing command nfsd pid 191 tid 100062 td 0xc3519300
 sched_switch(c3519300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c34f0400) at sleepq_switch+0xc1
 sleepq_wait_sig(c34f0400) at sleepq_wait_sig+0x1d
 msleep(c34f0400,c07e4ca0,158,c070f1de,0,...) at msleep+0x26a
 nfssvc_nfsd(c3519300) at nfssvc_nfsd+0xe5
 nfssvc(c3519300,d97c7d04) at nfssvc+0x18c
 syscall(3b,3b,3b,1,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x880bd197, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
 
 Tracing command nfsd pid 190 tid 100038 td 0xc33c0a80
 sched_switch(c33c0a80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c34f0c00) at sleepq_switch+0xc1
 sleepq_wait_sig(c34f0c00) at sleepq_wait_sig+0x1d
 msleep(c34f0c00,c07e4ca0,158,c070f1de,0,...) at msleep+0x26a
 nfssvc_nfsd(c33c0a80) at nfssvc_nfsd+0xe5
 nfssvc(c33c0a80,d97b8d04) at nfssvc+0x18c
 syscall(3b,3b,3b,1,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x880bd197, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
 
 Tracing command nfsd pid 189 tid 100042 td 0xc33c0480
 sched_switch(c33c0480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c34ef200) at sleepq_switch+0xc1
 sleepq_wait_sig(c34ef200) at sleepq_wait_sig+0x1d
 msleep(c34ef200,c07e4ca0,158,c070f1de,0,...) at msleep+0x26a
 nfssvc_nfsd(c33c0480) at nfssvc_nfsd+0xe5
 nfssvc(c33c0480,d97acd04) at nfssvc+0x18c
 syscall(3b,3b,3b,1,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (155, FreeBSD ELF32, nfssvc), eip = 0x880bd197, esp = 0xbfbfeb1c, ebp = 0xbfbfeb38 ---
 
 Tracing command nfsd pid 188 tid 100044 td 0xc33c0180
 sched_switch(c33c0180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3600892) at sleepq_switch+0xc1
 sleepq_wait_sig(c3600892) at sleepq_wait_sig+0x1d
 msleep(c3600892,c07dc2e0,158,c071af86,0) at msleep+0x26a
 accept1(c33c0180,d97a6d04,0,d97a6d30,c06c49fb,...) at accept1+0x142
 accept(c33c0180,d97a6d04) at accept+0x10
 syscall(3b,3b,3b,bfbfee40,3,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (30, FreeBSD ELF32, accept), eip = 0x880e475f, esp = 0xbfbfeb3c, ebp = 0xbfbfee68 ---
 
 Tracing command mountd pid 183 tid 100043 td 0xc33c0300
 sched_switch(c33c0300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c33c0300,6,bfbfede0,0,0,...) at kern_select+0x4bf
 select(c33c0300,d97a9d04) at select+0x44
 syscall(3b,3b,3b,4,1,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8814a2c3, esp = 0xbfbfed3c, ebp = 0xbfbfee78 ---
 
 Tracing command ypbind pid 174 tid 100045 td 0xc33c0000
 sched_switch(c33c0000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07dbf24,c07dbf24,ea61,d97a3b40,246,...) at sleepq_timedwait_sig+0x1e
 cv_timedwait_sig(c07dbf24,c07dbf00,ea61) at cv_timedwait_sig+0x192
 kern_select(c33c0000,14,804c380,0,0,...) at kern_select+0x4a9
 select(c33c0000,d97a3d04) at select+0x44
 syscall(3b,3b,3b,bfbfea18,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8813b2c3, esp = 0xbfbfe9cc, ebp = 0xbfbfee38 ---
 
 Tracing command rpcbind pid 165 tid 100039 td 0xc33c0900
 sched_switch(c33c0900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_timedwait_sig(c07dbf24,c07dbf24,7531,d97b5b84,246,...) at sleepq_timedwait_sig+0x1e
 cv_timedwait_sig(c07dbf24,c07dbf00,7531) at cv_timedwait_sig+0x192
 poll(c33c0900,d97b5d04) at poll+0x3a3
 syscall(3b,3b,3b,1,4,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (209, FreeBSD ELF32, poll), eip = 0x8810adc3, esp = 0xbfbfcc5c, ebp = 0xbfbfee28 ---
 
 Tracing command sshd pid 157 tid 100041 td 0xc33c0600
 sched_switch(c33c0600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c33c0600,4,8140b20,0,0,...) at kern_select+0x4bf
 select(c33c0600,d97afd04) at select+0x44
 syscall(3b,3b,3b,813f040,a,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x882522c3, esp = 0xbfbfe15c, ebp = 0xbfbfee78 ---
 
 Tracing command inetd pid 152 tid 100040 td 0xc33c0780
 sched_switch(c33c0780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c33c0780,b,bfbfe890,0,0,...) at kern_select+0x4bf
 select(c33c0780,d97b2d04) at select+0x44
 syscall(3b,3b,3b,bfbfe890,161,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x881592c3, esp = 0xbfbfdb6c, ebp = 0xbfbfee58 ---
 
 Tracing command syslogd pid 147 tid 100046 td 0xc335bd80
 sched_switch(c335bd80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c335bd80,8,8056040,0,0,...) at kern_select+0x4bf
 select(c335bd80,d56e2d04) at select+0x44
 syscall(3b,3b,3b,8056044,8056040,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x8814b2c3, esp = 0xbfbfda2c, ebp = 0xbfbfee78 ---
 
 Tracing command devd pid 135 tid 100047 td 0xc335bc00
 sched_switch(c335bc00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dbf24) at sleepq_switch+0xc1
 sleepq_wait_sig(c07dbf24,246,c0790aa0,c071a984,303,...) at sleepq_wait_sig+0x1d
 cv_wait_sig(c07dbf24,c07dbf00) at cv_wait_sig+0x189
 kern_select(c335bc00,5,bfbfe9d0,0,0,...) at kern_select+0x4bf
 select(c335bc00,d56dfd04) at select+0x44
 syscall(3b,3b,3b,0,8,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (93, FreeBSD ELF32, select), eip = 0x80757f7, esp = 0xbfbfe96c, ebp = 0xbfbfee78 ---
 
 Tracing command schedcpu pid 39 tid 100049 td 0xc335b900
 sched_switch(c335b900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(d56d9d04,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(d56d9d04) at sleepq_timedwait+0x4a
 msleep(d56d9d04,0,0,c070f1de,3e8) at msleep+0x255
 schedcpu_thread(0,d56d9d38,0,c056a360,0,...) at schedcpu_thread+0x26
 fork_exit(c056a360,0,d56d9d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd56d9d6c, ebp = 0 ---
 
 Tracing command softdepflush pid 38 tid 100027 td 0xc335b180
 sched_switch(c335b180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07e4f94,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c07e4f94) at sleepq_timedwait+0x4a
 msleep(c07e4f94,c07e4f60,44,c0729939,3e8) at msleep+0x255
 softdep_flush(0,d56cad38,0,c06429dc,0,...) at softdep_flush+0x227
 fork_exit(c06429dc,0,d56cad38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd56cad6c, ebp = 0 ---
 
 Tracing command vnlru pid 37 tid 100028 td 0xc335b000
 sched_switch(c335b000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33be000,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c33be000) at sleepq_timedwait+0x4a
 msleep(c33be000,c07dc8a0,250,c071f95a,3e8,c07dc920) at msleep+0x255
 vnlru_proc(0,d56c7d38,0,c05ad800,0,...) at vnlru_proc+0xdf
 fork_exit(c05ad800,0,d56c7d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd56c7d6c, ebp = 0 ---
 
 Tracing command syncer pid 36 tid 100029 td 0xc32a6d80
 sched_switch(c32a6d80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c079113c,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c079113c,0,c055217e,c07dc8e0,c071f899,...) at sleepq_wait+0x46
 msleep(c079113c,0,68,c071fc68,0) at msleep+0x279
 sched_sync(0,d44fad38,0,c05af30c,0,...) at sched_sync+0x309
 fork_exit(c05af30c,0,d44fad38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44fad6c, ebp = 0 ---
 
 Tracing command bufdaemon pid 35 tid 100030 td 0xc32a6c00
 sched_switch(c32a6c00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07dc48c,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c07dc48c) at sleepq_timedwait+0x4a
 msleep(c07dc48c,c07dc4a0,44,c071e1d7,3e8) at msleep+0x255
 buf_daemon(0,d44f7d38,0,c05a0fec,0,...) at buf_daemon+0x108
 fork_exit(c05a0fec,0,d44f7d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44f7d6c, ebp = 0 ---
 
 Tracing command pagezero pid 34 tid 100031 td 0xc32a6a80
 sched_switch(c32a6a80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07e5f04,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c07e5f04) at sleepq_timedwait+0x4a
 msleep(c07e5f04,c07e59e0,200,c072ea6b,493e0,c07e59e0,0,c072ea51,9d) at msleep+0x255
 vm_pagezero(0,d44f4d38,0,c0674508,0,...) at vm_pagezero+0x6c
 fork_exit(c0674508,0,d44f4d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44f4d6c, ebp = 0 ---
 
 Tracing command vmdaemon pid 33 tid 100032 td 0xc32a6900
 sched_switch(c32a6900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07e5a54,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c07e5a54,0,c05520d8,c0790ae0,c072e727,...) at sleepq_wait+0x46
 msleep(c07e5a54,0,68,c071e1d7,0) at msleep+0x279
 vm_daemon(0,d44f1d38,0,c0673534,0,...) at vm_daemon+0x36
 fork_exit(c0673534,0,d44f1d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44f1d6c, ebp = 0 ---
 
 Tracing command pagedaemon pid 32 tid 100033 td 0xc32a6780
 sched_switch(c32a6780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c07e5a10,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c07e5a10) at sleepq_timedwait+0x4a
 msleep(c07e5a10,c07e59e0,44,c071e1d7,1388) at msleep+0x255
 vm_pageout(0,d44eed38,0,c06731f4,0,...) at vm_pageout+0x260
 fork_exit(c06731f4,0,d44eed38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44eed6c, ebp = 0 ---
 
 Tracing command irq12: psm0 pid 31 tid 100034 td 0xc32a6600
 fork_trampoline() at fork_trampoline
 
 Tracing command irq1: atkbd0 pid 30 tid 100035 td 0xc32a6480
 sched_switch(c32a6480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c33cd0b0,d44e8d38,c33cd0b0,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c33cd0b0,d44e8d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44e8d6c, ebp = 0 ---
 
 Tracing command swi0: sio pid 29 tid 100036 td 0xc33c0d80
 sched_switch(c33c0d80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c33cd0d0,d97bed38,c33cd0d0,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c33cd0d0,d97bed38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd97bed6c, ebp = 0 ---
 
 Tracing command irq7: ppc0 pid 28 tid 100037 td 0xc33c0c00
 sched_switch(c33c0c00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c33b3bc0,d97bbd38,c33b3bc0,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c33b3bc0,d97bbd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd97bbd6c, ebp = 0 ---
 
 Tracing command fdc0 pid 27 tid 100017 td 0xc32a2900
 sched_switch(c32a2900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c33a623c,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c33a623c) at sleepq_timedwait+0x4a
 msleep(c33a623c,c33a62f0,4c,c070f1de,3e8) at msleep+0x255
 fdc_worker(c33a6200,c33a62f0,0,c0730830,48e) at fdc_worker+0x1ca
 fdc_thread(c33a6200,d44d3d38,c33a6200,c068c828,0,...) at fdc_thread+0x54
 fork_exit(c068c828,c33a6200,d44d3d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44d3d6c, ebp = 0 ---
 
 Tracing command irq10: atapci1 pid 26 tid 100018 td 0xc32a2780
 sched_switch(c32a2780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c33ab310,d44d0d38,c33ab310,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c33ab310,d44d0d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44d0d6c, ebp = 0 ---
 
 Tracing command usb1 pid 25 tid 100019 td 0xc32a2600
 sched_switch(c32a2600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c339c210,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c339c210) at sleepq_timedwait+0x4a
 msleep(c339c210,0,5c,c070d0d9,ea60,c3398c40) at msleep+0x255
 usb_event_thread(c3398c40,d44cdd38,c3398c40,c05004bc,0,...) at usb_event_thread+0x70
 fork_exit(c05004bc,c3398c40,d44cdd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44cdd6c, ebp = 0 ---
 
 Tracing command usbtask pid 24 tid 100020 td 0xc32a2480
 sched_switch(c32a2480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c078e204,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c078e204,0,c05520d8,c0790ae0,c070d0cb,...) at sleepq_wait+0x46
 msleep(c078e204,0,5c,c070d0e0,0,...) at msleep+0x279
 usb_task_thread(0,d44cad38,0,c050054c,0,...) at usb_task_thread+0x47
 fork_exit(c050054c,0,d44cad38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44cad6c, ebp = 0 ---
 
 Tracing command usb0 pid 23 tid 100021 td 0xc32a2300
 sched_switch(c32a2300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3390210,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c3390210) at sleepq_timedwait+0x4a
 msleep(c3390210,0,5c,c070d0d9,ea60,c338a900) at msleep+0x255
 usb_event_thread(c338a900,d44c7d38,c338a900,c05004bc,0,...) at usb_event_thread+0x70
 fork_exit(c05004bc,c338a900,d44c7d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44c7d6c, ebp = 0 ---
 
 Tracing command irq5: xl0 uhci0++ pid 22 tid 100022 td 0xc32a2180
 sched_switch(c32a2180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3389140,d44c4d38,c3389140,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3389140,d44c4d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44c4d6c, ebp = 0 ---
 
 Tracing command irq15: ata1 pid 21 tid 100023 td 0xc335b780
 sched_switch(c335b780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c33892e0,d56d6d38,c33892e0,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c33892e0,d56d6d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd56d6d6c, ebp = 0 ---
 
 Tracing command irq14: ata0 pid 20 tid 100024 td 0xc335b600
 sched_switch(c335b600,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3389430,d56d3d38,c3389430,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3389430,d56d3d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd56d3d6c, ebp = 0 ---
 
 Tracing command irq9: acpi0 pid 19 tid 100025 td 0xc335b480
 fork_trampoline() at fork_trampoline
 
 Tracing command swi6: Giant taskq pid 18 tid 100026 td 0xc335b300
 fork_trampoline() at fork_trampoline
 
 Tracing command swi6: task queue pid 17 tid 100008 td 0xc32a1300
 sched_switch(c32a1300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3350520,d44a9d38,c3350520,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3350520,d44a9d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44a9d6c, ebp = 0 ---
 
 Tracing command swi2: cambio pid 16 tid 100009 td 0xc32a1180
 sched_switch(c32a1180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3350530,d44a6d38,c3350530,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3350530,d44a6d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44a6d6c, ebp = 0 ---
 
 Tracing command acpi_task_2 pid 9 tid 100010 td 0xc32a1000
 sched_switch(c32a1000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3351300,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c3351300,0,c07194d2,49,8,...) at sleepq_wait+0x46
 msleep(c3351300,c335131c,0,c070f1de,0) at msleep+0x279
 taskqueue_thread_loop(c08ff200,d44a3d38,c08ff200,c05790d4,0,...) at taskqueue_thread_loop+0x78
 fork_exit(c05790d4,c08ff200,d44a3d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44a3d6c, ebp = 0 ---
 
 Tracing command acpi_task_1 pid 8 tid 100011 td 0xc32a6300
 sched_switch(c32a6300,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3351300,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c3351300,0,c07194d2,49,8,...) at sleepq_wait+0x46
 msleep(c3351300,c335131c,0,c070f1de,0) at msleep+0x279
 taskqueue_thread_loop(c08ff200,d44e5d38,c08ff200,c05790d4,0,...) at taskqueue_thread_loop+0x78
 fork_exit(c05790d4,c08ff200,d44e5d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44e5d6c, ebp = 0 ---
 
 Tracing command acpi_task_0 pid 7 tid 100012 td 0xc32a6180
 sched_switch(c32a6180,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3351300,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c3351300,0,c07194d2,49,c336a1c0,...) at sleepq_wait+0x46
 msleep(c3351300,c335131c,0,c070f1de,0) at msleep+0x279
 taskqueue_thread_loop(c08ff200,d44e2d38,c08ff200,c05790d4,0,...) at taskqueue_thread_loop+0x78
 fork_exit(c05790d4,c08ff200,d44e2d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44e2d6c, ebp = 0 ---
 
 Tracing command kqueue taskq pid 6 tid 100013 td 0xc32a6000
 sched_switch(c32a6000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c3351380,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c3351380,0,c07194d2,49,8,...) at sleepq_wait+0x46
 msleep(c3351380,c335139c,0,c070f1de,0) at msleep+0x279
 taskqueue_thread_loop(c078f120,d44dfd38,c078f120,c05790d4,0,...) at taskqueue_thread_loop+0x78
 fork_exit(c05790d4,c078f120,d44dfd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44dfd6c, ebp = 0 ---
 
 Tracing command swi5: + pid 15 tid 100014 td 0xc32a2d80
 fork_trampoline() at fork_trampoline
 
 Tracing command thread taskq pid 5 tid 100015 td 0xc32a2c00
 sched_switch(c32a2c00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c329f380,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c329f380,0,c07194d2,49,c33852b0,...) at sleepq_wait+0x46
 msleep(c329f380,c329f39c,0,c070f1de,0) at msleep+0x279
 taskqueue_thread_loop(c0794908,d44d9d38,c0794908,c05790d4,0,...) at taskqueue_thread_loop+0x78
 fork_exit(c05790d4,c0794908,d44d9d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44d9d6c, ebp = 0 ---
 
 Tracing command yarrow pid 14 tid 100016 td 0xc32a2a80
 sched_switch(c32a2a80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c078b120,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c078b120) at sleepq_timedwait+0x4a
 msleep(c078b120,0,0,c070f1de,64) at msleep+0x255
 random_kthread(0,d44d6d38,0,c04ca7a8,0,...) at random_kthread+0x180
 fork_exit(c04ca7a8,0,d44d6d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44d6d6c, ebp = 0 ---
 
 Tracing command g_down pid 4 tid 100000 td 0xc32a2000
 sched_switch(c32a2000,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c078e888,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c078e888) at sleepq_timedwait+0x4a
 msleep(c078e888,c078e768,24c,c070f1de,64) at msleep+0x255
 g_io_schedule_down(c32a2000) at g_io_schedule_down+0x56
 g_down_procbody(0,d44c1d38,0,c0524c78,0,...) at g_down_procbody+0x5a
 fork_exit(c0524c78,0,d44c1d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44c1d6c, ebp = 0 ---
 
 Tracing command g_up pid 3 tid 100001 td 0xc32a1d80
 sched_switch(c32a1d80,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c078e884,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c078e884) at sleepq_timedwait+0x4a
 msleep(c078e884,c078e7a8,24c,c070f1de,64) at msleep+0x255
 g_io_schedule_up(c32a1d80) at g_io_schedule_up+0x127
 g_up_procbody(0,d44bed38,0,c0524c18,0,...) at g_up_procbody+0x5a
 fork_exit(c0524c18,0,d44bed38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44bed6c, ebp = 0 ---
 
 Tracing command g_event pid 2 tid 100002 td 0xc32a1c00
 sched_switch(c32a1c00,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c078e87c,c0790aa0,0,c0718ff4,236,...) at sleepq_switch+0xc1
 sleepq_timedwait(c078e87c) at sleepq_timedwait+0x4a
 msleep(c078e87c,0,4c,c070f1de,64) at msleep+0x255
 g_event_procbody(0,d44bbd38,0,c0524cd8,0,...) at g_event_procbody+0x9e
 fork_exit(c0524cd8,0,d44bbd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44bbd6c, ebp = 0 ---
 
 Tracing command swi3: vm pid 13 tid 100003 td 0xc32a1a80
 fork_trampoline() at fork_trampoline
 
 Tracing command swi4: clock sio pid 12 tid 100004 td 0xc32a1900
 sched_switch(c32a1900,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3285710,d44b5d38,c3285710,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3285710,d44b5d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44b5d6c, ebp = 0 ---
 
 Tracing command swi1: net pid 11 tid 100005 td 0xc32a1780
 sched_switch(c32a1780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 ithread_loop(c3285720,d44b2d38,c3285720,c0546cfc,0,...) at ithread_loop+0xd9
 fork_exit(c0546cfc,c3285720,d44b2d38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44b2d6c, ebp = 0 ---
 
 Tracing command idle pid 10 tid 100006 td 0xc32a1600
 kdb_enter(c0732314) at kdb_enter+0x2b
 siointr1(c33bb800,c07e7f80,0,c0732124,56e,...) at siointr1+0xce
 siointr(c33bb800) at siointr+0x21
 intr_execute_handlers(c0785310,d44afc98,0,c3352818,c3352800,...) at intr_execute_handlers+0xe1
 atpic_handle_intr(4) at atpic_handle_intr+0x96
 Xatpic_intr4() at Xatpic_intr4+0x20
 --- interrupt, eip = 0xc08f0b49, esp = 0xd44afcdc, ebp = 0xd44afcdc ---
 acpi_cpu_c1(0,0,1,c054604c,c32a0c90,...) at acpi_cpu_c1+0x5
 acpi_cpu_idle(d44afd0c,c0546075,d44afd24,c0545eac,0,...) at acpi_cpu_idle+0x152
 cpu_idle(d44afd24,c0545eac,0,d44afd38,0,...) at cpu_idle+0x1f
 idle_proc(0,d44afd38,0,c054604c,0,...) at idle_proc+0x29
 fork_exit(c054604c,0,d44afd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44afd6c, ebp = 0 ---
 
 Tracing command init pid 1 tid 100007 td 0xc32a1480
 sched_switch(c32a1480,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c32a5000) at sleepq_switch+0xc1
 sleepq_wait_sig(c32a5000) at sleepq_wait_sig+0x1d
 msleep(c32a5000,c32a5068,15c,c071af18,0) at msleep+0x26a
 kern_wait(c32a1480,ffffffff,d44acc84,0,0) at kern_wait+0x6b7
 wait4(c32a1480,d44acd04) at wait4+0x2a
 syscall(3b,3b,3b,0,0,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (7, FreeBSD ELF32, wait4), eip = 0x8052f47, esp = 0xbfbfed7c, ebp = 0xbfbfed98 ---
 
 Tracing command swapper pid 0 tid 0 td 0xc078eba0
 sched_switch(c078eba0,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 scheduler(0,c1ec00,c1e000,0,c0438ed5,...) at scheduler+0x195
 mi_startup() at mi_startup+0x96
 begin() at begin+0x2c
 db> ps
    pid  ppid  pgrp   uid   state   wmesg     wchan    cmd
    371   355   371  1002  T+      (threaded)          c
 100078                   s                           c
 100081                   D       nfs      0xc36dc5c8 c
 100080                   s                           c
 100077                   Ss      ksesigwa 0xc3515dc8 c
    355   354   355  1002  S+      pause    0xc370c67c ksh
    354   353   354     0  Ss+     wait     0xc36c1a78 login
    353   152   353     0  Ss      select   0xc07dbf24 rlogind
    352     1   337     0  S       select   0xc07dbf24 xconsole
    331   301   331     0  Ss      select   0xc07dbf24 xdm
    325   304   325     0  S       select   0xc07dbf24 Xorg
    304   301   304     0  Ss      pause    0xc370567c ksh
    301     1   301     0  Ss      select   0xc07dbf24 xdm
    300     1   300     0  Ss+     ttyin    0xc33bb410 getty
    299     1   299     0  Ss+     ttyin    0xc33d2410 getty
    298     1   298     0  Ss+     ttyin    0xc33bd810 getty
    297     1   297     0  Ss+     ttyin    0xc33bd410 getty
    296     1   296     0  Ss+     ttyin    0xc33d2c10 getty
    290     1   290     0  Ss      select   0xc07dbf24 usbd
 db> where
 Tracing pid 10 tid 100006 td 0xc32a1600
 kdb_enter(c0732314) at kdb_enter+0x2b
 siointr1(c33bb800,c07e7f80,0,c0732124,56e,...) at siointr1+0xce
 siointr(c33bb800) at siointr+0x21
 intr_execute_handlers(c0785310,d44afc98,0,c3352818,c3352800,...) at intr_execute_handlers+0xe1
 atpic_handle_intr(4) at atpic_handle_intr+0x96
 Xatpic_intr4() at Xatpic_intr4+0x20
 --- interrupt, eip = 0xc08f0b49, esp = 0xd44afcdc, ebp = 0xd44afcdc ---
 acpi_cpu_c1(0,0,1,c054604c,c32a0c90,...) at acpi_cpu_c1+0x5
 acpi_cpu_idle(d44afd0c,c0546075,d44afd24,c0545eac,0,...) at acpi_cpu_idle+0x152
 cpu_idle(d44afd24,c0545eac,0,d44afd38,0,...) at cpu_idle+0x1f
 idle_proc(0,d44afd38,0,c054604c,0,...) at idle_proc+0x29
 fork_exit(c054604c,0,d44afd38) at fork_exit+0xa0
 fork_trampoline() at fork_trampoline+0x8
 --- trap 0x1, eip = 0, esp = 0xd44afd6c, ebp = 0 ---
 db> thread 100081
 [thread pid 371 tid 100081 ]
 sched_switch+0x177:     addl    $0x8,%esp
 db> where
 Tracing pid 371 tid 100081 td 0xc3709780
 sched_switch(c3709780,0,1) at sched_switch+0x177
 mi_switch(1,0) at mi_switch+0x270
 sleepq_switch(c36dc5c8,c0790aa0,0,c0718ff4,211,...) at sleepq_switch+0xc1
 sleepq_wait(c36dc5c8,0,c36dc638,b7,c071ef72,...) at sleepq_wait+0x46
 msleep(c36dc5c8,c0790268,50,c07257ab,0,...) at msleep+0x279
 acquire(da9dda64,40,60000,c3709780,0,...) at acquire+0x76
 lockmgr(c36dc5c8,3002,c36dc638,c3709780,da9dda88,...) at lockmgr+0x44e
 vop_stdlock(da9ddaa8) at vop_stdlock+0x21
 VOP_LOCK_APV(c0770fa0,da9ddaa8) at VOP_LOCK_APV+0x87
 vn_lock(c36dc570,1002,c3709780) at vn_lock+0xac
 lookup(da9ddc00) at lookup+0xd2
 namei(da9ddc00) at namei+0x39a
 kern_stat(c3709780,bf6fbe98,0,da9ddc74) at kern_stat+0x35
 stat(c3709780,da9ddd04) at stat+0x1b
 syscall(3b,3b,3b,8807c050,bf6fbfec,...) at syscall+0x25b
 Xint0x80_syscall() at Xint0x80_syscall+0x1f
 --- syscall (188, FreeBSD ELF32, stat), eip = 0x88159447, esp = 0xbf6fbdec, ebp = 0xbf6fbfb8 ---
 db> reset
 --- ddb.output ends here ---
 
 -- 
 Mail: Joerg.Lehners@Informatik.Uni-Oldenburg.DE    Tel: 2198
 Real: Joerg Lehners, Informatik ARBI, Uni Oldenburg, D-26111 Oldenburg
 Unwoerter: Kostensenkung - Gewinnmaximierung - billig, billig, billig



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200702051240.l15Cecee039442>