Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 07 Aug 2003 07:30:28 +0200
From:      Poul-Henning Kamp <phk@phk.freebsd.dk>
To:        current@freebsd.org
Subject:   reproducible deadlock...
Message-ID:  <51750.1060234228@critter.freebsd.dk>

next in thread | raw e-mail | index | archive | help

I am still able to reproduce a deadlock by running a "make -j 21 buildworld"
on a 48MB physmem configuration.

The first LOR is a false positive according to Alan Cox.

Interestingly, gstat(8) over an ssh session continues to run, so Giant
is presumably not involved.

I can reproduce it with both SCHED_ULE and SCHED_4BSD.

I've left the machine hanging in case anybody has any requests for further
DDB probing.

Poul-Henning


syv# sh t82.sh
lock order reversal
 1st 0xc135db90 vm object (vm object) @ vm/vm_object.c:434
 2nd 0xc082f110 system map (system map) @ vm/vm_kern.c:328
Stack backtrace:
backtrace(c038747a,c082f110,c0392feb,c0392feb,c0392e93) at 0xc01fe7a7 = backtrace+0x17
witness_lock(c082f110,8,c0392e93,148,c1455130) at 0xc02239f2 = witness_lock+0x672
_mtx_lock_flags(c082f110,0,c0392e8a,148,3) at 0xc01f496a = _mtx_lock_flags+0xba
_vm_map_lock(c082f0b0,c0392e8a,148,2b3,c03e61e0) at 0xc02ef006 = _vm_map_lock+0x36
kmem_malloc(c082f0b0,1000,101,c5616ac4,c02fef72) at 0xc02ee1da = kmem_malloc+0x3a
page_alloc(c083a1c0,1000,c5616ab7,101,c03dfc98) at 0xc02ff1e7 = page_alloc+0x27
slab_zalloc(c083a1c0,101,c039477e,682,c1244c54) at 0xc02fef72 = slab_zalloc+0xc2
uma_zone_slab(c083a1c0,101,c0394775,682,0) at 0xc0300138 = uma_zone_slab+0xe8
uma_zalloc_internal(c083a1c0,0,101,706,0) at 0xc030045f = uma_zalloc_internal+0x4f
uma_zfree_arg(c1244c40,c54498a0,0,773,0) at 0xc03007e2 = uma_zfree_arg+0x2c2
swp_pager_meta_free_all(c135db90,c039278c,c03926f5,20f) at 0xc02e9482 = swp_pager_meta_free_all+0xf2
swap_pager_dealloc(c135db90,1,c039472e,104,0) at 0xc02e769a = swap_pager_dealloc+0x11a
vm_pager_deallocate(c135db90,0,c039391c,261,c0394775) at 0xc02fdbed = vm_pager_deallocate+0x3d
vm_object_terminate(c135db90,0,c039391c,1b2,c01f4a50) at 0xc02f5953 = vm_object_terminate+0x1f3
vm_object_deallocate(c135db90,c1205654,c135db90,c1205654,c5616c64) at 0xc02f5701 = vm_object_deallocate+0x371
vm_map_entry_delete(c16bc100,c1205654,c0393059,897,0) at 0xc02f186b = vm_map_entry_delete+0x3b
vm_map_delete(c16bc100,0,bfc00000,c16bc100,c125c000) at 0xc02f1c70 = vm_map_delete+0x3e0
vm_map_remove(c16bc100,0,bfc00000,11d,c0382132) at 0xc02f1ce8 = vm_map_remove+0x58
exit1(c164e260,0,c0382132,65,c5616d40) at 0xc01e6f36 = exit1+0x686
sys_exit(c164e260,c5616d10,c039884a,3ee,1) at 0xc01e68a1 = sys_exit+0x41
syscall(2f,2f,2f,bfbff8dc,0) at 0xc033e213 = syscall+0x273
Xint0x80_syscall() at 0xc032d6bd = Xint0x80_syscall+0x1d
--- syscall (1), eip = 0x80b2d63, esp = 0xbfbff85c, ebp = 0xbfbff878 ---
Debugger("witness_lock")
Stopped at      0xc032bf34 = Debugger+0x54:     xchgl   %ebx,0xc0431044 = in_Debugger.0
db> cont
load: 0.00  cmd: sh 26559 [ufs] 0.00u 0.00s 0% 32k
load: 0.00  cmd: sh 26559 [ufs] 0.00u 0.00s 0% 32k
load: 0.00  cmd: sh 26559 [ufs] 0.00u 0.00s 0% 32k
Stopped at      0xc031845c = siointr1+0xdc:     jmp     0xc0318590 = siointr1+0x210
db> ps
  pid   proc     addr    uid  ppid  pgrp  flag   stat  wmesg    wchan  cmd
26559 c12f6000 c5809000    0 26251   498 0004002 [SLP]ufs 0xc149fd48] sh
26558 c13843c8 c57e5000    0 26546   498 4000002 [SLP]ufs 0xc140b1e0] sh
26557 c195fd3c c59cc000    0 26545   498 4000002 [SLP]ufs 0xc1773670] sh
26556 c12e1000 c5773000    0 26251   498 0004002 [SLP]ufs 0xc149fd48] sh
26555 c12e13c8 c5775000    0 26251   498 0004002 [SLP]ufs 0xc149fd48] sh
26554 c16b9000 c56b4000    0 26251   498 0004002 [SLP]ufs 0xc149fd48] sh
26553 c13583c8 c59e4000    0 26251   498 0004002 [SLP]ufs 0xc14b1d48] sh
26552 c12a23c8 c576d000    0 26543   498 4000002 [SLP]thrd_sleep 0xc09820ec] sh
26551 c12e05ac c56fe000    0 26251   498 0004002 [SLP]ufs 0xc149fd48] sh
26550 c195d3c8 c5969000    0 26251   498 0004002 [SLP]ufs 0xc14b8304] sh
26549 c14b51e4 c5880000    0 26251   498 0004002 [SLP]ufs 0xc14b1b00] sh
26548 c12f6b58 c580f000    0 26539   498 4000002 [SLP]thrd_sleep 0xc09820ec] sh
26547 c164dd3c c563b000    0 26544   498 4000002 [SLP]ufs 0xc1773670] sh
26546 c195d790 c596b000    0 26251   498 0004002 [SLP]wait 0xc195d790] sh
26545 c14b5000 c587f000    0 26251   498 0004002 [SLP]wait 0xc14b5000] sh
26544 c13725ac c590d000    0 26251   498 0004002 [SLP]wait 0xc13725ac] sh
26543 c1372b58 c5910000    0 26251   498 0004002 [SLP]wait 0xc1372b58] sh
26542 c19583c8 c5961000    0     1   498 0004002 [SLP]ufs 0xc1773670] cc
26540 c1491000 c5829000    0 26535   498 0004002 [SLP]ufs 0xc1773670] cc
26539 c12a2974 c5770000    0 26251   498 0004002 [SLP]wait 0xc12a2974][SWAP] sh
26537 c14915ac c587a000    0 26536   498 0004002 [SLP]ufs 0xc1773670] cc
26536 c139b3c8 c5a3f000    0 26531   498 0004002 [SLP]wait 0xc139b3c8][SWAP] sh
26535 c12e1b58 c5779000    0 26251   498 0004002 [SLP]wait 0xc12e1b58][SWAP] sh
26534 c1958974 c5964000    0 26532   498 0004002 [SLP]ufs 0xc1773670] cc
26533 c195dd3c c596e000    0 26530   498 0004002 [SLP]ufs 0xc1773670] cc
26532 c12a2b58 c5771000    0 26251   498 0004002 [SLP]wait 0xc12a2b58][SWAP] sh
26531 c12eed3c c5808000    0 26397   498 0004002 [CV]select 0xc04024b4] make
26530 c13705ac c5905000    0 26251   498 0004002 [SLP]wait 0xc13705ac][SWAP] sh
26529 c12e1d3c c577a000    0 26527   498 0004002 [SLP]piperd 0xc1779f20] as
26528 c12a21e4 c576c000    0 26527   498 0004002 [SLP]ufs 0xc149fd48] cc1
26527 c12ce5ac c57fc000    0     1   498 0004002 [SLP]wait 0xc12ce5ac][SWAP] cc
26524 c12deb58 c56f9000    0 26518   498 0004002 [SLP]piperd 0xc194af20] as
26523 c12a2000 c576b000    0 26518   498 0004002 [SLP]ufs 0xc149fd48] cc1
26522 c160e5ac c588a000    0 26516   498 0004002 [SLP]piperd 0xc194adc0] as
26521 c195e974 c5974000    0 26516   498 0004002 [SLP]ufs 0xc149fd48] cc1
26520 c195f790 c597b000    0 26513   498 0004002 [SLP]piperd 0xc194a000] as
26519 c12f61e4 c580a000    0 26513   498 0004002 [SLP]ufs 0xc149fd48] cc1
26518 c1372974 c590f000    0     1   498 0004002 [SLP]wait 0xc1372974][SWAP] cc
26516 c13675ac c58fd000    0     1   498 0004002 [SLP]wait 0xc13675ac][SWAP] cc
26515 c1958790 c5963000    0 26512   498 0004002 [SLP]piperd 0xc1343b00] as
26514 c12de5ac c56f6000    0 26512   498 0004002 [SLP]ufs 0xc149fd48] cc1
26513 c139b1e4 c5a3e000    0     1   498 0004002 [SLP]wait 0xc139b1e4][SWAP] cc
26512 c168e974 c569d000    0     1   498 0004002 [SLP]wait 0xc168e974][SWAP] cc
26511 c12de790 c56f7000    0 26506   498 0004002 [SLP]piperd 0xc135b210] as
26509 c1358974 c5a32000    0 26506   498 0004002 [SLP]ufs 0xc149fd48] cc1
26506 c12911e4 c5764000    0     1   498 0004002 [SLP]wait 0xc12911e4][SWAP] cc
26505 c16b9d3c c56bb000    0 26501   498 0004002 [SLP]piperd 0xc135b8f0] as
26504 c1384b58 c57e9000    0 26501   498 0004002 [SLP]ufs 0xc149fd48] cc1
26503 c195fb58 c59cb000    0 26497   498 0004002 [SLP]piperd 0xc135b0b0] as
26502 c160e974 c588c000    0 26497   498 0004002 [SLP]ufs 0xc149fd48] cc1
26501 c12ce790 c57fd000    0     1   498 0004002 [SLP]wait 0xc12ce790][SWAP] cc
26499 c13581e4 c59a3000    0 26493   498 0004002 [SLP]piperd 0xc135bf20] as
26498 c168d1e4 c5643000    0 26493   498 0004002 [SLP]ufs 0xc149fd48] cc1
26497 c195e3c8 c5971000    0     1   498 0004002 [SLP]wait 0xc195e3c8][SWAP] cc
26495 c160e790 c588b000    0 26484   498 0004002 [SLP]piperd 0xc1779580] as
26494 c168d5ac c5645000    0 26484   498 0004002 [SLP]ufs 0xc149fd48] cc1
26493 c164c974 c560a000    0     1   498 0004002 [SLP]wait 0xc164c974][SWAP] cc
26492 c1358790 c5a31000    0 26485   498 0004002 [SLP]piperd 0xc1338210] as
26491 c195db58 c596d000    0 26485   498 0004002 [SLP]ufs 0xc14b1c24] cc1
26490 c1372d3c c5911000    0 26478   498 0004002 [SLP]piperd 0xc1338d10] as
26489 c1291790 c5767000    0 26478   498 0004002 [SLP]ufs 0xc149fd48] cc1
26487 c1291000 c5715000    0 26483   498 0004002 [SLP]piperd 0xc1338a50] as
26486 c1370790 c5906000    0 26483   498 0004002 [SLP]ufs 0xc149fd48] cc1
26485 c164d790 c5638000    0     1   498 0004002 [SLP]wait 0xc164d790][SWAP] cc
26484 c139b790 c5a41000    0     1   498 0004002 [SLP]wait 0xc139b790][SWAP] cc
26483 c12913c8 c5765000    0     1   498 0004002 [SLP]wait 0xc12913c8][SWAP] cc
26478 c14b5b58 c5885000    0     1   498 0004002 [SLP]wait 0xc14b5b58][SWAP] cc
26476 c168d000 c5642000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 9
26475 c139b000 c5a3d000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 8
26474 c139ab58 c5a3b000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 7
26473 c16b83c8 c56ae000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 6
26467 c1358d3c c5a34000    0 23685   498 0004002 [SLP]ufs 0xc149fd48] cc1
26466 c168db58 c5648000    0 26456   498 0004002 [SLP]piperd 0xc1333a50] as
26465 c12de1e4 c56f4000    0 26456   498 0004002 [SLP]ufs 0xc149fd48] cc1
26464 c12e0974 c5700000    0 26454   498 0004002 [SLP]piperd 0xc13389a0] as
26463 c12e1974 c5778000    0 26451   498 0004002 [SLP]piperd 0xc1338630] as
26462 c195e000 c596f000    0 26458   498 0004002 [SLP]piperd 0xc18fbdc0] as
26461 c16b9790 c56b8000    0 26454   498 0004002 [SLP]ufs 0xc149fd48] cc1
26460 c168e3c8 c564c000    0 26451   498 0004002 [SLP]ufs 0xc149fd48] cc1
26459 c14913c8 c5879000    0 26458   498 0004002 [SLP]ufs 0xc149fd48] cc1
26458 c12e0d3c c5702000    0 26455   498 0004002 [SLP]wait 0xc12e0d3c][SWAP] cc
26457 c1370974 c5907000    0 26452   498 0004002 [SLP]ufs 0xc1773670] cc
26456 c12ce1e4 c57fa000    0 26450   498 0004002 [SLP]wait 0xc12ce1e4][SWAP] cc
26455 c1491d3c c587e000    0 26251   498 0004002 [SLP]wait 0xc1491d3c][SWAP] sh
26454 c16b91e4 c56b5000    0 26449   498 0004002 [SLP]wait 0xc16b91e4][SWAP] cc
26452 c168d3c8 c5644000    0 26251   498 0004002 [SLP]wait 0xc168d3c8][SWAP] sh
26451 c195eb58 c5975000    0 26447   498 0004002 [SLP]wait 0xc195eb58][SWAP] cc
26450 c1358b58 c5a33000    0 26251   498 0004002 [SLP]wait 0xc1358b58][SWAP] sh
26449 c14b5d3c c5886000    0 26251   498 0004002 [SLP]wait 0xc14b5d3c][SWAP] sh
26448 c164c000 c5605000    0 23611   498 0004002 [SLP]ufs 0xc149fd48] cc1
26447 c1372790 c590e000    0 26251   498 0004002 [SLP]wait 0xc1372790][SWAP] sh
26445 c12e0000 c56fb000    0 23742   498 0004002 [SLP]ufs 0xc149fd48] cc1
26444 c12ee974 c5806000    0 23700   498 0004002 [SLP]ufs 0xc149fd48] cc1
26442 c12f6790 c580d000    0 26438   498 0006002 [CV]allproc 0xc03dc838] as
26441 c1491790 c587b000    0 26438   498 0006002 [CV]allproc 0xc03dc838] cc1
26440 c12ee790 c5805000    0 26432   498 0006002 [CV]allproc 0xc03dc838] as
26439 c1358000 c5991000    0 26432   498 0006002 [CV]allproc 0xc03dc838] cc1
26438 c12ee5ac c5804000    0     1   498 0004002 [SLP]wait 0xc12ee5ac][SWAP] cc
26436 c13723c8 c590c000    0 26434   498 0006002 [CV]allproc 0xc03dc838] as
26435 c195f3c8 c5979000    0 26434   498 0006002 [CV]allproc 0xc03dc838] cc1
26434 c19581e4 c5960000    0     1   498 0004002 [SLP]wait 0xc19581e4][SWAP] cc
26432 c195e790 c5973000    0     1   498 0004002 [SLP]wait 0xc195e790][SWAP] cc
26431 c12ced3c c5800000    0 26428   498 0006002 [CV]allproc 0xc03dc838] as
26430 c14b5790 c5883000    0 26428   498 0006002 [CV]allproc 0xc03dc838] cc1
26428 c1958b58 c5965000    0     1   498 0004002 [SLP]wait 0xc1958b58][SWAP] cc
26426 c139a5ac c5a38000    0 26424   498 0006002 [CV]allproc 0xc03dc838] as
26425 c139a790 c5a39000    0 26424   498 0006002 [CV]allproc 0xc03dc838] cc1
26424 c12de974 c56f8000    0     1   498 0004002 [SLP]wait 0xc12de974][SWAP] cc
26423 c168e000 c564a000    0 26420   498 0006002 [CV]allproc 0xc03dc838] as
26422 c1384790 c57e7000    0 26420   498 0006002 [CV]allproc 0xc03dc838] cc1
26420 c1370b58 c5908000    0     1   498 0004002 [SLP]wait 0xc1370b58][SWAP] cc
26415 c195e1e4 c5970000    0     1   498 0004002 [SLP]ufs 0xc1773670] cc
26397 c164cd3c c560c000    0 26394   498 0004002 [SLP]wait 0xc164cd3c][SWAP] sh
26394 c12e0b58 c5701000    0 26393   498 0004002 [CV]select 0xc04024b4] make
26393 c160e000 c5887000    0 26391   498 0004002 [SLP]wait 0xc160e000][SWAP] sh
26391 c139a3c8 c5a37000    0 23509   498 0004002 [CV]select 0xc04024b4] make
26378 c16b8790 c56b0000    0 26374   498 0006002 [CV]allproc 0xc03dc838] as
26377 c13585ac c59e5000    0 26374   498 0006002 [CV]allproc 0xc03dc838] cc1
26376 c160ed3c c588e000    0 26372   498 0006002 [CV]allproc 0xc03dc838] as
26375 c12ce974 c57fe000    0 26372   498 0006002 [CV]allproc 0xc03dc838] cc1
26374 c12e01e4 c56fc000    0     1   498 0004002 [SLP]wait 0xc12e01e4][SWAP] cc
26372 c1384d3c c57ea000    0     1   498 0004002 [SLP]wait 0xc1384d3c][SWAP] cc
26369 c12a2d3c c5772000    0 26363   498 0006002 [CV]allproc 0xc03dc838] as
26368 c168eb58 c569e000    0 26363   498 0006002 [CV]allproc 0xc03dc838] cc1
26367 c195d974 c596c000    0 26360   498 0006002 [CV]allproc 0xc03dc838] as
26365 c139b974 c5a42000    0 26360   498 0006002 [CV]allproc 0xc03dc838] cc1
26364 c195f5ac c597a000    0 26351   498 0006002 [CV]allproc 0xc03dc838] as
26363 c12f65ac c580c000    0     1   498 0004002 [SLP]wait 0xc12f65ac][SWAP] cc
26360 c168ed3c c569f000    0     1   498 0004002 [SLP]wait 0xc168ed3c][SWAP] cc
26358 c195e5ac c5972000    0 26355   498 0006002 [CV]allproc 0xc03dc838] as
26357 c139a1e4 c5a36000    0 26355   498 0006002 [CV]allproc 0xc03dc838] cc1
26355 c12f6d3c c5810000    0     1   498 0004002 [SLP]wait 0xc12f6d3c][SWAP] cc
26353 c16b9b58 c56ba000    0 26351   498 0006002 [CV]allproc 0xc03dc838] cc1
26351 c168d790 c5646000    0     1   498 0004002 [SLP]wait 0xc168d790][SWAP] cc
26251 c1367d3c c5901000    0 23519   498 0004002 [CV]select 0xc04024b4] make
26230 c1370000 c5902000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 5
26229 c195d000 c5967000    0     1     0 0002204 [CV]allproc 0xc03dc838] nfsiod 4
26119 c1372000 c590a000    0 26108   498 0004002 [SLP]ufs 0xc1773670] cc
26108 c1291974 c5768000    0 26105   498 0004002 [SLP]wait 0xc1291974][SWAP] sh
26105 c164d974 c5639000    0 23499   498 0004002 [SLP]wait 0xc164d974][SWAP] sh
23742 c12ce000 c57f9000    0 23736   498 0004002 [SLP]wait 0xc12ce000][SWAP] cc
23736 c1491974 c587c000    0 23614   498 0004002 [SLP]wait 0xc1491974][SWAP] sh
23700 c16b8d3c c56b3000    0 23549   498 0004002 [SLP]wait 0xc16b8d3c][SWAP] cc
23685 c12e0790 c56ff000    0 23682   498 0004002 [SLP]wait 0xc12e0790][SWAP] cc
23682 c13841e4 c57a1000    0 23676   498 0004002 [SLP]wait 0xc13841e4][SWAP] sh
23676 c164db58 c563a000    0 23529   498 0004002 [SLP]wait 0xc164db58][SWAP] sh
23614 c14b55ac c5882000    0 23493   498 0004002 [SLP]wait 0xc14b55ac][SWAP] sh
23611 c143d000 c55ec000    0 23604   498 0004002 [SLP]wait 0xc143d000][SWAP] cc
23604 c12ceb58 c57ff000    0 23581   498 0004002 [SLP]wait 0xc12ceb58][SWAP] sh
23581 c14911e4 c5878000    0 23496   498 0004002 [SLP]wait 0xc14911e4][SWAP] sh
23549 c12a2790 c576f000    0 23526   498 0004002 [SLP]wait 0xc12a2790][SWAP] sh
23529 c168dd3c c5649000    0 23523   498 0004002 [CV]select 0xc04024b4] make
23526 c13845ac c57e6000    0 23492   498 0004002 [SLP]wait 0xc13845ac][SWAP] sh
23523 c16b81e4 c56ad000    0 23481   498 0004002 [SLP]wait 0xc16b81e4][SWAP] sh
23519 c12ee1e4 c5802000    0 23481   498 0004002 [SLP]wait 0xc12ee1e4][SWAP] sh
23509 c160e1e4 c5888000    0 23481   498 0004002 [SLP]wait 0xc160e1e4][SWAP] sh
23499 c14b5974 c5884000    0 23495   498 0004002 [CV]select 0xc04024b4] make
23496 c164d1e4 c5635000    0 23494   498 0004002 [CV]select 0xc04024b4] make
23495 c12e15ac c5776000    0 23481   498 0004002 [SLP]wait 0xc12e15ac][SWAP] sh
23494 c1291b58 c5769000    0 23481   498 0004002 [SLP]wait 0xc1291b58][SWAP] sh
23493 c12f6974 c580e000    0 23491   498 0004002 [CV]select 0xc04024b4] make
23492 c1384974 c57e8000    0 23490   498 0004002 [CV]select 0xc04024b4] make
23491 c164c3c8 c5607000    0 23481   498 0004002 [SLP]wait 0xc164c3c8][SWAP] sh
23490 c16b95ac c56b7000    0 23481   498 0004002 [SLP]wait 0xc16b95ac][SWAP] sh
23481 c12de000 c56f3000    0 22718   498 0004002 [CV]select 0xc04024b4] make
22718 c168e5ac c564d000    0 22711   498 0004002 [SLP]wait 0xc168e5ac][SWAP] sh
22711 c16b8000 c56ac000    0 22710   498 0004002 [CV]select 0xc04024b4] make
22710 c168d974 c5647000    0   593   498 0004002 [SLP]wait 0xc168d974][SWAP] sh
  712 c168e1e4 c564b000    0   709   712 0004002 [CV]sysctl lock 0xc03dd238] top
  709 c164c790 c5609000    0   703   709 0004002 [SLP]pause 0xc164b000][SWAP] csh
  703 c143db58 c55f2000    0   427   703 0000100 [CV]select 0xc04024b4] sshd
  635 c164c1e4 c5606000    0   582   635 0004002 [SLP]thrd_sleep 0xc16bc63c] systat
  593 c143d974 c55f1000    0   591   498 0004002 [CV]select 0xc04024b4] make
  591 c143dd3c c55f3000    0   562   498 0004002 [SLP]wait 0xc143dd3c][SWAP] sh
  582 c164d5ac c5637000    0   567   582 0004002 [SLP]pause 0xc1646000][SWAP] csh
  567 c164c5ac c5608000    0   427   567 0000100 [CV]select 0xc04024b4] sshd
  562 c164d000 c560d000    0   498   498 0004002 [CV]select 0xc04024b4] make
  556 c11f93c8 c53ae000    0     0     0 0000204 [SLP]mdwait 0xc15bf400] md21
  498 c11f9b58 c53d9000    0   491   498 0004002 [SLP]wait 0xc11f9b58][SWAP] sh
  497 c14385ac c5599000    0   495   497 0004002 [SLP]nanslp 0xc03dd3ec] gstat
  495 c143d1e4 c55ed000    0   493   495 0004002 [SLP]pause 0xc1443000][SWAP] csh
  493 c143d790 c55f0000    0   427   493 0000100 [CV]select 0xc04024b4] sshd
  491 c14381e4 c5597000    0   490   491 0004002 [SLP]pause 0xc1453000][SWAP] csh
  490 c1438b58 c559c000    0     1   490 0004102 [SLP]wait 0xc1438b58][SWAP] login
  489 c11f95ac c53af000    0     1   489 0004002 [SLP]ttyin 0xc13ab210][SWAP] getty
  488 c11f9974 c53b1000    0     1   488 0004002 [SLP]ttyin 0xc13ab010][SWAP] getty
  487 c13e6d3c c5401000    0     1   487 0004002 [SLP]ttyin 0xc13ac410][SWAP] getty
  486 c11f9790 c53b0000    0     1   486 0004002 [SLP]ttyin 0xc13bf810][SWAP] getty
  485 c14383c8 c5598000    0     1   485 0004002 [SLP]ttyin 0xc13bf210][SWAP] getty
  484 c1438974 c559b000    0     1   484 0004002 [SLP]ttyin 0xc1375410][SWAP] getty
  483 c1438000 c5596000    0     1   483 0004002 [SLP]ttyin 0xc13be810][SWAP] getty
  482 c1438790 c559a000    0     1   482 0004002 [SLP]ttyin 0xc11fc010][SWAP] getty
  427 c143d5ac c55ef000    0     1   427 0000100 [CV]select 0xc04024b4][SWAP] sshd
  283 c143d3c8 c55ee000    0     1   283 0000000 [CV]select 0xc04024b4] rpcbind
  269 c1438d3c c55eb000    0     1   269 0000000 [CV]select 0xc04024b4] syslogd
   50 c11f9d3c c53da000    0     0     0 0000204 [SLP]- 0xc04068cc] nfsiod 3
   49 c13e6000 c53fa000    0     0     0 0000204 [SLP]- 0xc04068c8] nfsiod 2
   48 c13e61e4 c53fb000    0     0     0 0000204 [SLP]- 0xc04068c4] nfsiod 1
   47 c13e63c8 c53fc000    0     0     0 0000204 [SLP]- 0xc04068c0] nfsiod 0
   46 c13e65ac c53fd000    0     0     0 0000204 [SLP]vlruwt 0xc13e65ac] vnlru
   45 c13e6790 c53fe000    0     0     0 0000204 [SLP]syncer 0xc03dcda0] syncer
   44 c13e6974 c53ff000    0     0     0 0000204 [SLP]psleep 0xc0402950] bufdaemon
   43 c13e6b58 c5400000    0     0     0 000020c [SLP]pgzero 0xc040cbc8] pagezero
   42 c11b8b58 c538d000    0     0     0 0000204 [SLP]psleep 0xc040cc20] vmdaemon
   41 c11b8d3c c538e000    0     0     0 0000204 [SLP]psleep 0xc040cc0c] pagedaemon
   40 c11f7000 c53a4000    0     0     0 0000204 new [IWAIT] irq8: rtc
   39 c11f71e4 c53a5000    0     0     0 0000204 new [IWAIT] irq0: clk
   38 c11f73c8 c53a6000    0     0     0 0000204 [IWAIT] irq7: ppc0
   37 c11f75ac c53a7000    0     0     0 0000204 new [IWAIT] irq3: sio1
   36 c11f7790 c53a8000    0     0     0 0000204 new [IWAIT] irq4: sio0
   35 c11f7974 c53a9000    0     0     0 0000204 [IWAIT] swi0: tty:sio
   34 c11f7b58 c53aa000    0     0     0 0000204 [IWAIT] irq6: fdc0
   33 c11f7d3c c53ab000    0     0     0 0000204 [IWAIT] irq1: atkbd0
   32 c11f9000 c53ac000    0     0     0 0000204 [CV]isp_kthread_cv 0xc0985874] isp1: fc_thrd
   31 c11f91e4 c53ad000    0     0     0 0000204 [IWAIT] irq10: isp1
   30 c112b5ac c5356000    0     0     0 0000204 [CV]isp_kthread_cv 0xc0985474] isp0: fc_thrd
   29 c112b790 c5357000    0     0     0 0000204 [IWAIT] irq11: isp0
   28 c112b974 c5358000    0     0     0 0000204 [SLP]usbevt 0xc11cf210] usb1
   27 c112bb58 c5359000    0     0     0 0000204 [SLP]usbtsk 0xc03d3c88] usbtask
   26 c112bd3c c535a000    0     0     0 0000204 [SLP]usbevt 0xc11cd210] usb0
   25 c11b8000 c5360000    0     0     0 0000204 [IWAIT] irq12: fxp0 uhci0+
   24 c11b81e4 c5361000    0     0     0 0000204 [IWAIT] irq15: ata1
   23 c11b83c8 c5362000    0     0     0 0000204 [IWAIT] irq14: ata0
    9 c11b85ac c5363000    0     0     0 0000204 [SLP]actask 0xc04e99ac] acpi_task2
    8 c11b8790 c5364000    0     0     0 0000204 [SLP]actask 0xc04e99ac] acpi_task1
    7 c11b8974 c5365000    0     0     0 0000204 [SLP]actask 0xc04e99ac] acpi_task0
   22 c09901e4 c4f24000    0     0     0 0000204 new [IWAIT] irq9: acpi0
   21 c09903c8 c4f25000    0     0     0 0000204 new [IWAIT] irq13:
   20 c09905ac c4f26000    0     0     0 0000204 new [IWAIT] swi5:+
   19 c0990790 c4f27000    0     0     0 0000204 new [IWAIT] swi6: task queue
   18 c0990974 c4f28000    0     0     0 0000204 [IWAIT] swi3: cambio
   17 c0990b58 c4f29000    0     0     0 0000204 new [IWAIT] swi2: camnet
   16 c0990d3c c4f51000    0     0     0 0000204 new [IWAIT] swi6: acpitaskq
   15 c112b000 c5353000    0     0     0 0000204 [SLP]- 0xc03c9660] random
    6 c112b1e4 c5354000    0     0     0 0000204 [SLP]crypto_ret_wait 0xc040bc54] crypto returns
    5 c112b3c8 c5355000    0     0     0 0000204 [SLP]crypto_wait 0xc040bbf4] crypto
    4 c0989000 c4ecd000    0     0     0 0000204 [SLP]- 0xc03d6220] g_down
    3 c09891e4 c4f1c000    0     0     0 0000204 [SLP]- 0xc03d621c] g_up
    2 c09893c8 c4f1d000    0     0     0 0000204 [SLP]- 0xc03d6214] g_event
   14 c09895ac c4f1e000    0     0     0 0000204 new [IWAIT] swi4: vm
   13 c0989790 c4f1f000    0     0     0 000020c [IWAIT] swi7: tty:sio clock
   12 c0989974 c4f20000    0     0     0 0000204 [IWAIT] swi1: net
   11 c0989b58 c4f21000    0     0     0 000020c [CPU 0] idle
    1 c0989d3c c4f22000    0     0     1 0004200 [SLP]wait 0xc0989d3c] init
   10 c0990000 c4f23000    0     0     0 0000204 [CV]ktrace 0xc03d9854] ktrace
    0 c03d6340 c0519000    0     0     0 0000200 [SLP]sched 0xc03d6340] swapper
db> show lock
Ambiguous
db> show locks
db> show witness
Sleep locks:
0 g_xdown -- last acquired @ geom/geom_io.c:359
2  Giant -- last acquired @ kern/kern_timeout.c:216
3   eventhandler -- last acquired @ kern/subr_eventhandler.c:213
4    eventhandler list -- last acquired @ kern/kern_exit.c:210
3   g_disk_done -- last acquired @ geom/geom_disk.c:179
4    bio queue -- last acquired @ geom/geom_io.c:70
10   UMA pcpu -- last acquired @ vm/uma_core.c:255
11    KMAP ENTRY -- last acquired @ vm/uma_core.c:265
11    UMA zone -- last acquired @ vm/uma_core.c:265
3   UMA lock -- last acquired @ vm/uma_core.c:1167
7    system map -- last acquired @ vm/vm_map.c:2228
8     kmem object -- last acquired @ vm/vm_kern.c:436
9      vm page queue mutex -- last acquired @ vm/vm_pageout.c:1464
10      UMA pcpu -- (already displayed)
10      vnode interlock -- last acquired @ kern/vfs_subr.c:3228
11       Syncer mtx -- last acquired @ kern/vfs_subr.c:1798
11       vnode_free_list -- last acquired @ kern/vfs_subr.c:760
11       spechash -- last acquired @ kern/vfs_subr.c:2009
10      CMAPCADDR12 -- last acquired @ i386/i386/pmap.c:2582
8     vm object -- last acquired @ vm/vm_object.c:434
9      swap_pager list -- last acquired @ vm/swap_pager.c:519
9      vm page queue mutex -- (already displayed)
7    Malloc Stats -- last acquired @ kern/kern_malloc.c:324
4    vm object_list -- last acquired @ vm/vm_object.c:621
8     kmem object -- (already displayed)
8     vm object -- (already displayed)
3   malloc -- last acquired @ kern/kern_malloc.c:561
3   md bio queue -- last acquired @ dev/md/md.c:605
3   vm86 lock -- last acquired @ i386/i386/vm86.c:583
3   GEOM orphanage -- last acquired @ geom/geom_event.c:169
3   ithread -- last acquired @ kern/kern_intr.c:265
3   pbuf mutex -- last acquired @ vm/vm_pager.c:443
3   taskqueue list -- last acquired @ kern/subr_taskqueue.c:87
3   sf_bufs list lock -- last acquired @ kern/uipc_syscalls.c:1642
7    Malloc Stats -- (already displayed)
7    system map -- (already displayed)
3   kernel linker -- last acquired @ kern/kern_linker.c:1032
3   rman head -- last acquired @ kern/subr_rman.c:110
3   ACPI semaphore -- last acquired @ /bang/src/sys/dev/acpica/Osd/OsdSynch.c:319
3   acpica subsystem lock -- last acquired @ /bang/src/sys/dev/acpica/Osd/OsdSynch.c:380
3   rman -- last acquired @ kern/subr_rman.c:445
7    Malloc Stats -- (already displayed)
7    system map -- (already displayed)
3   pseudofs_vncache -- last acquired @ fs/pseudofs/pseudofs_vncache.c:227
3   devd -- last acquired @ kern/subr_bus.c:479
7    Malloc Stats -- (already displayed)
10   UMA pcpu -- (already displayed)
3   devstat -- last acquired @ kern/subr_devstat.c:398
3   pseudofs -- last acquired @ fs/pseudofs/pseudofs_fileno.c:86
3   mntid -- last acquired @ kern/vfs_subr.c:583
4    mountlist -- last acquired @ kern/vfs_subr.c:3524
3   isp -- last acquired @ dev/isp/isp_freebsd.c:2696
7    Malloc Stats -- (already displayed)
10   UMA pcpu -- (already displayed)
3   bpf global lock -- last acquired @ net/bpf.c:1382
3   bdone lock -- last acquired @ kern/vfs_bio.c:2953
3   netisr lock -- last acquired @ net/netisr.c:215
4    tcp -- last acquired @ netinet/tcp_timer.c:141
5     inp -- last acquired @ netinet/tcp_input.c:650
6      network driver -- last acquired @ dev/fxp/if_fxp.c:1798
7       mbuf PCPU list lock -- last acquired @ kern/subr_mbuf.c:926
8        mbuf subsystem general lists lock -- last acquired @ kern/subr_mbuf.c:676
7       Malloc Stats -- (already displayed)
7       if send queue -- last acquired @ dev/fxp/if_fxp.c:1317
7       system map -- (already displayed)
6      random reseed -- last acquired @ dev/random/yarrow.c:170
6      sellck -- last acquired @ kern/sys_generic.c:816
6      radix node head -- last acquired @ net/route.c:139
7       Malloc Stats -- (already displayed)
7       ifnet -- last acquired @ net/if.c:1170
7       ifaddr -- last acquired @ net/route.c:671
7       system map -- (already displayed)
11     sleep mtxpool -- last acquired @ kern/kern_descrip.c:1808
4    udp -- last acquired @ netinet/udp_usrreq.c:263
5     inp -- (already displayed)
4    arp_inq -- last acquired @ net/netisr.c:233
4    ip_inq -- last acquired @ net/netisr.c:233
3   sem -- last acquired @ kern/sysv_sem.c:1157
3   mntvnode -- last acquired @ kern/vfs_subr.c:3218
10   vnode interlock -- (already displayed)
3   crypto driver table -- last acquired @ opencrypto/crypto.c:498
3   p_peers -- last acquired @ kern/kern_exit.c:252
3   buffer daemon lock -- last acquired @ kern/vfs_bio.c:2093
3   fdesc -- last acquired @ kern/kern_descrip.c:1516
4    filedesc structure -- last acquired @ kern/sys_generic.c:895
5     pipe mutex -- last acquired @ kern/sys_pipe.c:1303
6      sellck -- (already displayed)
6      sigio lock -- last acquired @ kern/kern_descrip.c:588
7       process group -- last acquired @ kern/tty.c:2407
8        process lock -- last acquired @ kern/kern_condvar.c:149
9         ktrace -- last acquired @ kern/kern_exit.c:353
9         sigacts -- last acquired @ kern/kern_condvar.c:150
9         struct pargs.ref -- last acquired @ kern/kern_proc.c:1065
9         session -- last acquired @ kern/kern_proc.c:790
10         vnode interlock -- (already displayed)
10         uidinfo hash -- last acquired @ kern/kern_resource.c:878
11          sleep mtxpool -- (already displayed)
11          uidinfo struct -- last acquired @ order list:0
12           allprison -- last acquired @ kern/kern_jail.c:414
3   ufs ihash -- last acquired @ ufs/ufs/ufs_ihash.c:124
10   vnode interlock -- (already displayed)
3   needsbuffer lock -- last acquired @ kern/vfs_bio.c:292
3   runningbufspace lock -- last acquired @ kern/vfs_bio.c:310
3   buf queue lock -- last acquired @ kern/vfs_bio.c:1493
10   vnode interlock -- (already displayed)
3   accounting -- last acquired @ kern/kern_acct.c:232
3   dirhash list -- last acquired @ ufs/ufs/ufs_dirhash.c:342
4    dirhash -- last acquired @ ufs/ufs/ufs_dirhash.c:450
0 g_xup -- last acquired @ geom/geom_io.c:378
2  Giant -- (already displayed)
0 ACPI task -- last acquired @ /bang/src/sys/dev/acpica/Osd/OsdSchedule.c:110
0 crypto return queues -- last acquired @ opencrypto/crypto.c:1197
0 crypto op queues -- last acquired @ opencrypto/crypto.c:1073
0 GEOM event stalling -- last acquired @ geom/geom_event.c:157
1  GEOM topology -- last acquired @ geom/geom_event.c:158
2   Giant -- (already displayed)
0 module subsystem sx lock -- last acquired @ kern/kern_module.c:331
0 kernel environment -- last acquired @ kern/kern_environment.c:288
0 sysctl lock -- last acquired @ kern/kern_sysctl.c:1269
1  filelist lock -- last acquired @ kern/kern_descrip.c:1234
4   filedesc structure -- (already displayed)
1  allproc -- last acquired @ kern/sched_4bsd.c:254
2   Giant -- (already displayed)
0 dev_pager create -- last acquired @ vm/device_pager.c:150
1  dev_pager list -- last acquired @ vm/device_pager.c:164
4  vm object_list -- (already displayed)
0 proctree -- last acquired @ kern/kern_exit.c:573
1  allproc -- (already displayed)

Spin locks:

Locks which were never acquired:
rip
pseudofs_fileno
msq
semid
cd9660_ihash
msdosfs dehash
bpf interface lock
ACPI global lock
taskqueue
strategy
bounce pages lock
jumbo mutex
securelevel mutex lock
UUID generator mutex lock
umtx
phys_pager list
vm map sleep mutex
lockmgr
db>

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51750.1060234228>