Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 28 Sep 2004 23:32:40 -0400 (EDT)
From:      Robert Watson <rwatosn@freebsd.org>
To:        Evren Yurtesen <yurtesen@ispro.net.tr>
Cc:        freebsd-current@freebsd.org
Subject:   Re: panic: sorele
Message-ID:  <Pine.NEB.3.96L.1040928232554.15557D-100000@fledge.watson.org>
In-Reply-To: <4159431F.6010502@ispro.net.tr>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 28 Sep 2004, Evren Yurtesen wrote:

> So you say that disabling SMP fixes the problem? I can easily do that :) 
> I dont even have 2 processors, I was trying this hyperthreading. 

If you compile a kernel with NET_WITH_GIANT but keep SMP, does the problem
persist? 

Are you using Netgraph or any other non-default kernel compile options
relating to the network stack?  Do you make moderate or extensive use of
IPv6?

This is a somewhat odd assertion failure: sodealloc() asserts that
so_count is 0, but so does sofree(), and sofree() is only called by
sotryfree() in in_pcbdetach() if so_count is 0.  This suggests that either
(a) we're looking at a race in which so_count is bumped in that window, or
(b) there's a problem with the compile of the kernel where the invariants
checks may be compiled into some objects but not others.  In theory,
locking should prevent (a), so if it is (a) there's a bug in the locking.
I'll start reviewing use of so_count and work my way through the rest of
this thread.  Knowing if compiling with NET_WITH_GIANT helps would be
useful, if possible.

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org      Principal Research Scientist, McAfee Research


> 
> Evren
> 
> Vlad wrote:
> 
> > John,
> > 
> > I've got some DDB output for the exactly the same problem - I think I
> > already sent u in email...  note it crashes with a slitely different
> > panic message than when it's not compiled with kernel debugging stuff.
> > 
> > I've submitted more details in bug report here:
> > http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/72126
> > 
> > here is some DDB stuff:
> > 
> > login: panic: sodealloc(): so_count 1
> > cpuid = 0
> > KDB: enter: panic
> > [thread 100044]
> > Stopped at      kdb_enter+0x30: leave
> > db> tr
> > kdb_enter(c06497b7,0,c064f89d,e8955b18,c27df640) at kdb_enter+0x30
> > panic(c064f89d,1,122,c3a9fd78,c3a9fca8) at panic+0x14e
> > sodealloc(c3a9fca8,0,c064f87d,17d,c3a9fd10) at sodealloc+0x21
> > sofree(c3a9fca8,1,c0653b3a,2b4,c06c8b40) at sofree+0x301
> > in_pcbdetach(c359b9d8,c36f4e00,e8955c68,c058a645,c36f4e00) at in_pcbdetach+0xf8
> > tcp_close(c36f4e00,c28a5034,0,0,0) at tcp_close+0x1d
> > tcp_input(c3286800,14,c283a000,1,374a41cf) at tcp_input+0x1885
> > ip_input(c3286800,0,c065313d,e5,c06c78b8) at ip_input+0x592
> > netisr_processqueue(c06c78b8,c069acc0,1,c0648b1f,c27e6d00) at
> > netisr_processqueue+0x8e
> > swi_net(0,0,c0647164,269,0) at swi_net+0xe9
> > ithread_loop(c275d780,e8955d48,c0646f4f,323,0) at ithread_loop+0x172
> > fork_exit(c04d1970,c275d780,e8955d48) at fork_exit+0xc6
> > fork_trampoline() at fork_trampoline+0x8
> > --- trap 0x1, eip = 0, esp = 0xe8955d7c, ebp = 0 ---
> > db> show object
> > Object 0xc0504cf0: type=0, size=0x7400047b83ffffff, res=-121515895,
> > ref=-1265778039, flags=0x27bc
> > db> show witness
> > Sleep locks:
> > 0 ATAPI CD bioqueue lock -- last acquired @ /usr/src/sys/dev/ata/atapi-cd.c:1100
> > 0 g_xdown -- last acquired @ /usr/src/sys/geom/geom_io.c:374
> > 3  Giant -- last acquired @ /usr/src/sys/vm/vm_fault.c:294
> > 4   standard object -- last acquired @ /usr/src/sys/vm/vm_object.c:454
> > 5    vm object_list -- last acquired @ /usr/src/sys/vm/vm_object.c:222
> > 12   vm page queue mutex -- last acquired @ /usr/src/sys/vm/vm_fault.c:876
> > 13    CMAPCADDR12 -- last acquired @ /usr/src/sys/i386/i386/pmap.c:2486
> > 13    vnode interlock -- last acquired @ /usr/src/sys/kern/vfs_subr.c:2159
> > 14     spechash -- last acquired @ /usr/src/sys/kern/vfs_subr.c:1903
> > 14     cdev -- last acquired @ /usr/src/sys/kern/kern_conf.c:81
> > 14     vnode_free_list -- last acquired @ /usr/src/sys/kern/vfs_subr.c:745
> > 14     Syncer mtx -- last acquired @ /usr/src/sys/kern/vfs_subr.c:1854
> > 14     Name Cache -- last acquired @ /usr/src/sys/kern/vfs_cache.c:478
> > 13    pmap -- last acquired @ /usr/src/sys/i386/i386/pmap.c:2685
> > 14     uma object -- last acquired @ /usr/src/sys/vm/uma_core.c:963
> > 14     UMA pcpu -- last acquired @ /usr/src/sys/vm/uma_core.c:1800
> > 15      KMAP ENTRY -- last acquired @ /usr/src/sys/vm/uma_core.c:2224
> > 16       UMA zone -- last acquired @ /usr/src/sys/vm/uma_core.c:1817
> > 4   malloc -- last acquired @ /usr/src/sys/kern/kern_malloc.c:588
> > 4   struct mount mtx -- last acquired @ /usr/src/sys/kern/vfs_subr.c:901
> > 13   vnode interlock -- (already displayed)
> > 4   bounce pages lock -- last acquired @
> > /usr/src/sys/i386/i386/busdma_machdep.c:860
> > 4   eventhandler -- last acquired @ /usr/src/sys/kern/subr_eventhandler.c:213
> > 5    eventhandler list -- last acquired @ /usr/src/sys/kern/kern_exit.c:199
> > 4   UMA lock -- last acquired @ /usr/src/sys/vm/uma_core.c:889
> > 14   UMA pcpu -- (already displayed)
> > 4   random reseed -- last acquired @ /usr/src/sys/dev/random/yarrow.c:193
> > 4   kobj -- last acquired @ /usr/src/sys/kern/subr_kobj.c:298
> > 4   kernel linker -- last acquired @ /usr/src/sys/kern/kern_linker.c:431
> > 4   TID lock -- last acquired @ /usr/src/sys/kern/kern_thread.c:206
> > 4   GEOM orphanage -- last acquired @ /usr/src/sys/geom/geom_event.c:170
> > 4   vm86 lock -- last acquired @ /usr/src/sys/i386/i386/vm86.c:582
> > 4   ithread -- last acquired @ /usr/src/sys/kern/kern_intr.c:276
> > 4   taskqueue list -- last acquired @ /usr/src/sys/kern/subr_taskqueue.c:85
> > 4   acpica subsystem lock -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/Osd/OsdSynch.c:360
> > 5    ACPI semaphore -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/Osd/OsdSynch.c:300
> > 11   Malloc Stats -- last acquired @ /usr/src/sys/kern/kern_malloc.c:185
> > 5    taskqueue -- last acquired @ /usr/src/sys/kern/subr_taskqueue.c:193
> > 14   UMA pcpu -- (already displayed)
> > 4   if_cloners lock -- last acquired @ /usr/src/sys/net/if_clone.c:199
> > 4   rman head -- last acquired @ /usr/src/sys/kern/subr_rman.c:111
> > 4   arc4_mtx -- last acquired @ /usr/src/sys/libkern/arc4random.c:137
> > 4   devstat -- last acquired @ /usr/src/sys/kern/subr_devstat.c:83
> > 4   rman -- last acquired @ /usr/src/sys/kern/subr_rman.c:448
> > 11   Malloc Stats -- (already displayed)
> > 10   system map -- last acquired @ /usr/src/sys/vm/vm_kern.c:296
> > 11    kmem object -- last acquired @ /usr/src/sys/vm/vm_kern.c:398
> > 12     vm page queue mutex -- (already displayed)
> > 11    kernel object -- last acquired @ /usr/src/sys/vm/vm_object.c:454
> > 12     vm page queue mutex -- (already displayed)
> > 4   sf_buf -- last acquired @ /usr/src/sys/i386/i386/vm_machdep.c:674
> > 4   bdone lock -- last acquired @ /usr/src/sys/kern/vfs_bio.c:3768
> > 4   domain list -- last acquired @ /usr/src/sys/kern/uipc_domain.c:110
> > 4   bpf global lock -- last acquired @ /usr/src/sys/net/bpf.c:1446
> > 5    bpf interface lock -- last acquired @ order list:0
> > 6     bpf cdev lock -- last acquired @ order list:0
> > 4   pbuf mutex -- last acquired @ /usr/src/sys/vm/vm_pager.c:414
> > 4   ttylist -- last acquired @ /usr/src/sys/kern/tty.c:2745
> > 11   tty -- last acquired @ /usr/src/sys/kern/kern_event.c:1451
> > 4   pseudofs -- last acquired @ /usr/src/sys/fs/pseudofs/pseudofs_fileno.c:86
> > 4   if_clone lock -- last acquired @ /usr/src/sys/net/if_clone.c:321
> > 4   ACPI task queue -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/Osd/OsdSchedule.c:118
> > 4   so_glabel -- last acquired @ /usr/src/sys/kern/uipc_socket.c:169
> > 4   g_disk_done -- last acquired @ /usr/src/sys/geom/geom_disk.c:196
> > 5    bio queue -- last acquired @ /usr/src/sys/geom/geom_io.c:65
> > 14   UMA pcpu -- (already displayed)
> > 4   lo_mtx -- last acquired @ /usr/src/sys/net/if_loop.c:154
> > 4   IPFW static rules -- last acquired @ /usr/src/sys/netinet/ip_fw2.c:2510
> > 4   ATA queue lock -- last acquired @ /usr/src/sys/dev/ata/ata-queue.c:172
> > 4   pfil_head_mtx -- last acquired @ /usr/src/sys/net/pfil.c:81
> > 5    pfil_head_list lock -- last acquired @ /usr/src/sys/net/pfil.c:218
> > 4   buffer daemon lock -- last acquired @ /usr/src/sys/kern/vfs_bio.c:401
> > 4   dirhash list -- last acquired @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:348
> > 5    dirhash -- last acquired @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:367
> > 4   needsbuffer lock -- last acquired @ /usr/src/sys/kern/vfs_bio.c:296
> > 4   ufs ihash -- last acquired @ /usr/src/sys/ufs/ufs/ufs_ihash.c:156
> > 13   vnode interlock -- (already displayed)
> > 4   mntid -- last acquired @ /usr/src/sys/kern/vfs_subr.c:407
> > 5    mountlist -- last acquired @ /usr/src/sys/kern/vfs_syscalls.c:386
> > 4   pseudofs_vncache -- last acquired @
> > /usr/src/sys/fs/pseudofs/pseudofs_vncache.c:239
> > 4   buf queue lock -- last acquired @ /usr/src/sys/kern/vfs_bio.c:1505
> > 13   vnode interlock -- (already displayed)
> > 4   fdesc -- last acquired @ /usr/src/sys/kern/kern_descrip.c:1614
> > 5    filedesc structure -- last acquired @ /usr/src/sys/kern/kern_descrip.c:1356
> > 6     devd -- last acquired @ /usr/src/sys/kern/subr_bus.c:496
> > 9      sellck -- last acquired @ /usr/src/sys/kern/sys_generic.c:726
> > 6     accept -- last acquired @ /usr/src/sys/kern/uipc_socket.c:334
> > 7      so_snd -- last acquired @ /usr/src/sys/kern/uipc_socket.c:2091
> > 8       so_rcv -- last acquired @ /usr/src/sys/kern/uipc_socket.c:2092
> > 9        radix node head -- last acquired @ /usr/src/sys/net/route.c:662
> > 10        system map -- (already displayed)
> > 10        ifnet -- last acquired @ /usr/src/sys/net/if.c:1019
> > 10        rtentry -- last acquired @ /usr/src/sys/net/rtsock.c:398
> > 11         rts_inq -- last acquired @ /usr/src/sys/net/netisr.c:229
> > 11         Malloc Stats -- (already displayed)
> > 14         UMA pcpu -- (already displayed)
> > 11         ifaddr -- last acquired @ /usr/src/sys/net/route.c:788
> > 9        sellck -- (already displayed)
> > 9        process lock -- last acquired @ /usr/src/sys/kern/kern_descrip.c:1244
> > 10        ktrace -- last acquired @ /usr/src/sys/kern/kern_fork.c:609
> > 10        struct pargs.ref -- last acquired @ /usr/src/sys/kern/kern_proc.c:1100
> > 10        sigacts -- last acquired @ /usr/src/sys/kern/kern_sig.c:294
> > 10        session -- last acquired @ /usr/src/sys/kern/kern_fork.c:591
> > 11         tty -- (already displayed)
> > 11         uidinfo hash -- last acquired @
> > /usr/src/sys/kern/kern_resource.c:1004
> > 12          sleep mtxpool -- last acquired @ /usr/src/sys/kern/kern_prot.c:1739
> > 12          uidinfo struct -- last acquired @ order list:0
> > 13           allprison -- last acquired @ /usr/src/sys/kern/kern_jail.c:460
> > 6     pipe mutex -- last acquired @ /usr/src/sys/kern/sys_pipe.c:1520
> > 7      sigio lock -- last acquired @ /usr/src/sys/kern/kern_descrip.c:729
> > 8       process group -- last acquired @ /usr/src/sys/kern/kern_fork.c:581
> > 9        process lock -- (already displayed)
> > 4   rtsock route_cb lock -- last acquired @ /usr/src/sys/net/rtsock.c:234
> > 4   accounting -- last acquired @ /usr/src/sys/kern/kern_acct.c:232
> > 4   network driver -- last acquired @ /usr/src/sys/dev/fxp/if_fxp.c:1755
> > 11   Malloc Stats -- (already displayed)
> > 5    knlist lock for lockless objects -- last acquired @
> > /usr/src/sys/kern/kern_event.c:1451
> > 5    if send queue -- last acquired @ /usr/src/sys/dev/fxp/if_fxp.c:1267
> > 10   system map -- (already displayed)
> > 4   runningbufspace lock -- last acquired @ /usr/src/sys/kern/vfs_bio.c:314
> > 4   udp -- last acquired @ /usr/src/sys/netinet/udp_usrreq.c:995
> > 5    udpinp -- last acquired @ /usr/src/sys/netinet/udp_usrreq.c:1001
> > 7     so_snd -- (already displayed)
> > 4   unp -- last acquired @ /usr/src/sys/kern/uipc_usrreq.c:842
> > 7    so_snd -- (already displayed)
> > 0 g_xup -- last acquired @ /usr/src/sys/geom/geom_io.c:428
> > 3  Giant -- (already displayed)
> > 0 igmp_mtx -- last acquired @ /usr/src/sys/netinet/igmp.c:431
> > 0 ipqlock -- last acquired @ /usr/src/sys/netinet/ip_input.c:1096
> > 0 sem -- last acquired @ /usr/src/sys/kern/sysv_sem.c:1174
> > 0 if_afdata -- last acquired @ /usr/src/sys/net/if.c:489
> > 0 GEOM topology -- last acquired @ /usr/src/sys/geom/geom_event.c:202
> > 1  fdc lock -- last acquired @ /usr/src/sys/dev/fdc/fdc.c:772
> > 2   callout_wait_lock -- last acquired @ /usr/src/sys/kern/kern_timeout.c:289
> > 1  swapdev -- last acquired @ /usr/src/sys/vm/swap_pager.c:2124
> > 3  Giant -- (already displayed)
> > 0 p_peers -- last acquired @ /usr/src/sys/kern/kern_exit.c:243
> > 0 ACPI root bus -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi.c:832
> > 4  rman -- (already displayed)
> > 5  ACPI semaphore -- (already displayed)
> > 0 ACPI PCI bus methods -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_pcib.c:123
> > 1  ACPI PCI link -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_pci_link.c:1061
> > 11  Malloc Stats -- (already displayed)
> > 5   ACPI semaphore -- (already displayed)
> > 2   kernel environment -- last acquired @
> > /usr/src/sys/kern/kern_environment.c:285
> > 14  UMA pcpu -- (already displayed)
> > 0 ACPI thermal zone -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_thermal.c:783
> > 0 ACPI embedded controller -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_ec.c:302
> > 0 ACPI power resources -- last acquired @
> > /usr/src/sys/modules/acpi/acpi/../../../dev/acpica/acpi_powerres.c:324
> > 5  ACPI semaphore -- (already displayed)
> > 11 Malloc Stats -- (already displayed)
> > 14 UMA pcpu -- (already displayed)
> > 0 module subsystem sx lock -- last acquired @
> > /usr/src/sys/kern/kern_module.c:313
> > 0 rawcb -- last acquired @ /usr/src/sys/net/raw_usrreq.c:80
> > 8  so_rcv -- (already displayed)
> > 0 sysctl lock -- last acquired @ /usr/src/sys/kern/kern_sysctl.c:1315
> > 1  rip -- last acquired @ /usr/src/sys/netinet/raw_ip.c:844
> > 1  tcp -- last acquired @ /usr/src/sys/netinet/tcp_timer.c:138
> > 2   tcpinp -- last acquired @ order list:0
> > 7    so_snd -- (already displayed)
> > 1  filelist lock -- last acquired @ /usr/src/sys/kern/kern_descrip.c:1330
> > 5   filedesc structure -- (already displayed)
> > 1  allproc -- last acquired @ /usr/src/sys/kern/kern_fork.c:301
> > 2   user map -- last acquired @ /usr/src/sys/vm/vm_map.c:2997
> > 3    Giant -- (already displayed)
> > 0 ddp_list_mtx -- last acquired @ order list:0
> > 1  ddp_mtx -- last acquired @ order list:0
> > 0 slip_mtx -- last acquired @ order list:0
> > 1  slip sc_mtx -- last acquired @ order list:0
> > 0 proctree -- last acquired @ /usr/src/sys/kern/tty.c:2027
> > 1  allproc -- (already displayed)
> > 
> > Spin locks:
> > 0 ap boot -- last acquired @ /usr/src/sys/i386/i386/mp_machdep.c:517
> > 1  sio -- last acquired @ /usr/src/sys/dev/sio/sio.c:1709
> > 2   cy -- last acquired @ order list:0
> > 3    uart_hwmtx -- last acquired @ order list:0
> > 4     sabtty -- last acquired @ order list:0
> > 5      zstty -- last acquired @ order list:0
> > 6       ng_node -- last acquired @ order list:0
> > 7        ng_worklist -- last acquired @ order list:0
> > 8         taskqueue_fast -- last acquired @ order list:0
> > 9          intr table -- last acquired @
> > /usr/src/sys/i386/i386/intr_machdep.c:89
> > 10          ithread table lock -- last acquired @ order list:0
> > 11           sleepq chain -- last acquired @
> > /usr/src/sys/kern/subr_sleepqueue.c:223
> > 12            sched lock -- last acquired @ /usr/src/sys/kern/sched_4bsd.c:1120
> > 13             turnstile chain -- last acquired @
> > /usr/src/sys/kern/subr_turnstile.c:411
> > 14              td_contested -- last acquired @ order list:0
> > 15               callout -- last acquired @ /usr/src/sys/kern/kern_timeout.c:398
> > 16                entropy harvest -- last acquired @
> > /usr/src/sys/dev/random/randomdev_soft.c:304
> > 17                 entropy harvest buffers -- last acquired @
> > /usr/src/sys/dev/random/randomdev_soft.c:313
> > 18                  allpmaps -- last acquired @
> > /usr/src/sys/i386/i386/pmap.c:1127
> > 19                   vm page queue free mutex -- last acquired @
> > /usr/src/sys/vm/vm_page.c:774
> > 20                    icu -- last acquired @ order list:0
> > 21                     smp rendezvous -- last acquired @
> > /usr/src/sys/i386/i386/pmap.c:585
> > 22                      tlb -- last acquired @ order list:0
> > 23                       clk -- last acquired @
> > /usr/src/sys/i386/isa/clock.c:404
> > 24                        mutex profiling lock -- last acquired @ order list:0
> > 25                         kse zombie lock -- last acquired @ order list:0
> > 26                          ALD Queue -- last acquired @ order list:0
> > 27                           pcicfg -- last acquired @
> > /usr/src/sys/i386/pci/pci_cfgreg.c:230
> > 
> > Locks which were never acquired:
> > swap_pager swhash
> > IPFW dynamic rules
> > arp_inq
> > tcp_hc_entry
> > ip_inq
> > pseudofs_fileno
> > ppp_softc_list_mtx
> > tunmtx
> > msq
> > semid
> > cd9660_ihash
> > msdosfs dehash
> > strategy
> > ACPI global lock
> > ACPI cmbat
> > ACPI generic battery
> > ACPI AC adapter
> > ACPI PCI power methods
> > ACPI lid
> > ACPI CPU
> > kqueue order
> > jumbo mutex
> > encapmtx
> > accept_filter_mtx
> > securelevel mutex lock
> > fifo mutex
> > UUID generator mutex lock
> > umtx
> > protect sysfilt_ops
> > phys_pager list
> > dev_pager list
> > dev_pager create
> > swap_pager list
> > vm map sleep mutex
> > lockmgr
> > db> show pciregs
> > hostb0@pci0:0:0:        class=0x060000 card=0x00000000 chip=0x00091166
> > rev=0x05 hdr=0x00
> > hostb1@pci0:0:1:        class=0x060000 card=0x00000000 chip=0x00091166
> > rev=0x05 hdr=0x00
> > none0@pci0:2:0: class=0x030000 card=0x47568086 chip=0x47561002 rev=0x7a hdr=0x00
> > fxp0@pci0:3:0:  class=0x020000 card=0x12298086 chip=0x12298086 rev=0x08 hdr=0x00
> > pcib1@pci0:7:0: class=0x060400 card=0x00000068 chip=0xa5001044 rev=0x02 hdr=0x01
> > asr0@pci0:7:1:  class=0x0e0001 card=0xc03c1044 chip=0xa5011044 rev=0x02 hdr=0x00
> > isab0@pci0:15:0:        class=0x060100 card=0x02001166 chip=0x02001166
> > rev=0x4f hdr=0x00
> > atapci0@pci0:15:1:      class=0x01018a card=0x00000000 chip=0x02111166
> > rev=0x00 hdr=0x00
> > none1@pci2:4:0: class=0x010000 card=0x00cf8086 chip=0x00cf9005 rev=0x01 hdr=0x00
> > none2@pci2:4:1: class=0x010000 card=0x00cf8086 chip=0x00cf9005 rev=0x01 hdr=0x00
> > db> show registers
> > cs                 0x8
> > ds          0xe8950010
> > es                0x10
> > fs                0x18
> > ss                0x10
> > eax               0x12
> > ecx         0xc0c14000
> > edx                  0
> > ebx         0xe8955b18
> > esp         0xe8955ad4
> > ebp         0xe8955adc
> > esi                0x1
> > edi              0x100
> > eip         0xc0504cf0  kdb_enter+0x30
> > efl              0x282
> > dr0                  0
> > dr1                  0
> > dr2                  0
> > dr3                  0
> > dr4         0xffff0ff0
> > dr5              0x400
> > dr6         0xffff0ff0
> > dr7              0x400
> > kdb_enter+0x30: leave
> > db> show threads
> >   100265 (0xc32067d0)  sched_switch(c32067d0,0,1,11d,5aa48b0d) at
> > sched_switch+0x180
> >   100251 (0xc3204190)  sched_switch(c3204190,0,1,11d,5db1e499) at
> > sched_switch+0x180
> >   100118 (0xc2bbdc80)  sched_switch(c2bbdc80,0,1,11d,82553467) at
> > sched_switch+0x180
> >   100158 (0xc2e43c80)  sched_switch(c2e43c80,0,1,11d,5a568cc1) at
> > sched_switch+0x180
> >   100135 (0xc2e1b7d0)  sched_switch(c2e1b7d0,0,1,11d,43f9759f) at
> > sched_switch+0x180
> >   100271 (0xc33fe190)  sched_switch(c33fe190,0,1,11d,1dc845c7) at
> > sched_switch+0x180
> >   100232 (0xc3199320)  sched_switch(c3199320,0,1,11d,e09772f9) at
> > sched_switch+0x180
> >   100100 (0xc2bbb000) 
> > sched_switch(eee41a94,c06c69a0,c06c69a0,437,c064d749) at
> > sched_switch+0x180
> >   100053 (0xc27e24b0)  sched_switch(c27e24b0,0,1,11d,c2d83d01) at
> > sched_switch+0x180
> >   100248 (0xc319ac80)  sched_switch(c319ac80,0,1,11d,7d8d3e1b) at
> > sched_switch+0x180
> >   100279 (0xc33fee10)  sched_switch(c33fee10,0,1,11d,35fbfb91) at
> > sched_switch+0x180
> >   100195 (0xc30107d0)  sched_switch(c30107d0,0,1,11d,9b3f1175) at
> > sched_switch+0x180
> >   100054 (0xc27e2640)  sched_switch(c27e2640,0,1,11d,3a14f887) at
> > sched_switch+0x180
> >   100140 (0xc2e42000)  sched_switch(c2e42000,0,1,11d,c759049f) at
> > sched_switch+0x180
> >   100254 (0xc3204640)  sched_switch(c3204640,0,1,11d,203fa40f) at
> > sched_switch+0x180
> >   100257 (0xc3204af0)  sched_switch(c3204af0,0,1,11d,71139bf5) at
> > sched_switch+0x180
> >   100266 (0xc3206960)  sched_switch(c3206960,0,1,11d,14a393b3) at
> > sched_switch+0x180
> >   100115 (0xc2bbd7d0)  sched_switch(c2bbd7d0,0,1,11d,3140338b) at
> > sched_switch+0x180
> >   100319 (0xc3893e10)  sched_switch(c3893e10,0,1,11d,d00e7fff) at
> > sched_switch+0x180
> >   100290 (0xc3ae4000)  sched_switch(c3ae4000,0,1,11d,f84f391) at
> > sched_switch+0x180
> >   100247 (0xc319aaf0)  sched_switch(c319aaf0,0,1,11d,6b6ace79) at
> > sched_switch+0x180
> >   100187 (0xc2fb7af0)  sched_switch(c2fb7af0,0,1,11d,2525f137) at
> > sched_switch+0x180
> >   100117 (0xc2bbdaf0)  sched_switch(c2bbdaf0,0,1,11d,1064956d) at
> > sched_switch+0x180
> >   100104 (0xc2bbb640)  sched_switch(c2bbb640,0,1,11d,f4dda92f) at
> > sched_switch+0x180
> >   100350 (0xc3a88000)  sched_switch(c3a88000,0,1,11d,ec14fe77) at
> > sched_switch+0x180
> >   100095 (0xc2bb87d0)  sched_switch(c2bb87d0,0,1,11d,ded7426b) at
> > sched_switch+0x180
> >   100160 (0xc2e46000)  sched_switch(c2e46000,0,1,11d,6a3de8a5) at
> > sched_switch+0x180
> >   100274 (0xc33fe640)  sched_switch(c33fe640,0,1,11d,de3a3cdf) at
> > sched_switch+0x180
> >   100310 (0xc3893000)  sched_switch(c3893000,0,1,11d,5fd324ef) at
> > sched_switch+0x180
> >   100337 (0xc3a83af0)  sched_switch(c3a83af0,0,1,11d,8c4f2c1d) at
> > sched_switch+0x180
> >   100108 (0xc2bbbc80)  sched_switch(c2bbbc80,0,1,11d,f71adac1) at
> > sched_switch+0x180
> >   100218 (0xc3057c80)  sched_switch(c3057c80,0,1,11d,3198796b) at
> > sched_switch+0x180
> >   100153 (0xc2e434b0)  sched_switch(c2e434b0,0,1,11d,76cd2cdb) at
> > sched_switch+0x180
> >   100291 (0xc3ae4190)  sched_switch(c3ae4190,0,1,11d,68406d3f) at
> > sched_switch+0x180
> >   100197 (0xc3010af0)  sched_switch(c3010af0,0,1,11d,20da36eb) at
> > sched_switch+0x180
> >   100270 (0xc33fe000)  sched_switch(c33fe000,0,1,11d,5f3a315) at
> > sched_switch+0x180
> >   100294 (0xc3ae4640)  sched_switch(c3ae4640,0,1,11d,37dcbad9) at
> > sched_switch+0x180
> >   100323 (0xc38944b0)  sched_switch(c38944b0,0,1,11d,1b6e368b) at
> > sched_switch+0x180
> >   100217 (0xc3057af0)  sched_switch(c3057af0,0,1,11d,b13a0819) at
> > sched_switch+0x180
> >   100156 (0xc2e43960)  sched_switch(c2e43960,0,1,11d,b2fbcc2d) at
> > sched_switch+0x180
> >   100231 (0xc3199190)  sched_switch(c3199190,0,1,11d,720276ad) at
> > sched_switch+0x180
> >   100286 (0xc3ae2960)  sched_switch(c3ae2960,0,1,11d,c048be07) at
> > sched_switch+0x180
> >   100143 (0xc2e424b0)  sched_switch(c2e424b0,0,1,11d,5fae8b1b) at
> > sched_switch+0x180
> >   100275 (0xc33fe7d0)  sched_switch(c33fe7d0,0,1,11d,77f0f3a3) at
> > sched_switch+0x180
> >   100101 (0xc2bbb190)  sched_switch(c2bbb190,0,1,11d,a4d35a21) at
> > sched_switch+0x180
> >   100179 (0xc2fb6e10)  sched_switch(c2fb6e10,0,1,11d,6a2f0e33) at
> > sched_switch+0x180
> >   100268 (0xc3206c80)  sched_switch(c3206c80,0,1,11d,eca11829) at
> > sched_switch+0x180
> >   100240 (0xc319a000)  sched_switch(c319a000,0,1,11d,5136c437) at
> > sched_switch+0x180
> >   100150 (0xc2e43000)  sched_switch(c2e43000,0,1,11d,2d08bb99) at
> > sched_switch+0x180
> >   100090 (0xc2bb8000)  sched_switch(c2bb8000,0,1,11d,35ede6a7) at
> > sched_switch+0x180
> >   100205 (0xc30117d0)  sched_switch(c30117d0,0,1,11d,9ef63df5) at
> > sched_switch+0x180
> >   100136 (0xc2e1b960)  sched_switch(c2e1b960,0,1,11d,a7a83071) at
> > sched_switch+0x180
> >   100148 (0xc2e42c80)  sched_switch(c2e42c80,0,1,11d,7a8a9915) at
> > sched_switch+0x180
> >   100139 (0xc2e1be10)  sched_switch(c2e1be10,0,1,11d,7c3a76d3) at
> > sched_switch+0x180
> >   100169 (0xc2e46e10)  sched_switch(c2e46e10,0,1,11d,d6efc515) at
> > sched_switch+0x180
> >   100237 (0xc3199af0)  sched_switch(c3199af0,0,1,11d,e07672c9) at
> > sched_switch+0x180
> >   100234 (0xc3199640)  sched_switch(c3199640,0,1,11d,e7fafd81) at
> > sched_switch+0x180
> >   100235 (0xc31997d0)  sched_switch(c31997d0,0,1,11d,dac1a0b1) at
> > sched_switch+0x180
> >   100170 (0xc2fb6000)  sched_switch(c2fb6000,0,1,11d,f5857545) at
> > sched_switch+0x180
> >   100114 (0xc2bbd640)  sched_switch(c2bbd640,0,1,11d,da2f9ac9) at
> > sched_switch+0x180
> >   100219 (0xc3057e10)  sched_switch(c3057e10,0,1,11d,5ef6bfe9) at
> > sched_switch+0x180
> >   100245 (0xc319a7d0)  sched_switch(c319a7d0,0,1,11d,5286fd) at
> > sched_switch+0x180
> >   100215 (0xc30577d0)  sched_switch(c30577d0,0,1,11d,8ac81441) at
> > sched_switch+0x180
> >   100178 (0xc2fb6c80)  sched_switch(c2fb6c80,0,1,11d,8b611a95) at
> > sched_switch+0x180
> >   100066 (0xc27e5960)  sched_switch(c27e5960,0,1,11d,f8ff04b1) at
> > sched_switch+0x180
> >   100246 (0xc319a960)  sched_switch(c319a960,0,1,11d,303b85b5) at
> > sched_switch+0x180
> >   100177 (0xc2fb6af0)  sched_switch(c2fb6af0,0,1,11d,cacbc04f) at
> > sched_switch+0x180
> >   100065 (0xc27e57d0)  sched_switch(c27e57d0,0,1,11d,5d2bec9) at
> > sched_switch+0x180
> >   100200 (0xc3011000)  sched_switch(c3011000,0,1,11d,c3c865) at
> > sched_switch+0x180
> >   100106 (0xc2bbb960)  sched_switch(c2bbb960,0,1,11d,dcf621a5) at
> > sched_switch+0x180
> >   100105 (0xc2bbb7d0)  sched_switch(c2bbb7d0,0,1,11d,60cf4d39) at
> > sched_switch+0x180
> >   100069 (0xc27e5e10)  sched_switch(c27e5e10,0,1,11d,49383521) at
> > sched_switch+0x180
> >   100107 (0xc2bbbaf0)  sched_switch(c2bbbaf0,0,1,11d,3c6bcab5) at
> > sched_switch+0x180
> >   100110 (0xc2bbd000)  sched_switch(c2bbd000,0,1,11d,2d219d87) at
> > sched_switch+0x180
> >   100067 (0xc27e5af0)  sched_switch(c27e5af0,0,1,11d,e45ff411) at
> > sched_switch+0x180
> >   100080 (0xc2b65000)  sched_switch(c2b65000,0,1,11d,53ec71a7) at
> > sched_switch+0x180
> >   100113 (0xc2bbd4b0)  sched_switch(c2bbd4b0,0,1,11d,e062c4db) at
> > sched_switch+0x180
> >   100089 (0xc2b65e10)  sched_switch(c2b65e10,0,1,11d,63e38f69) at
> > sched_switch+0x180
> >   100096 (0xc2bb8960)  sched_switch(c2bb8960,0,1,11d,c20f2dbf) at
> > sched_switch+0x180
> >   100093 (0xc2bb84b0)  sched_switch(c2bb84b0,0,1,11d,f0cf29c5) at
> > sched_switch+0x180
> >   100073 (0xc2b644b0)  sched_switch(c2b644b0,0,1,11d,5406e9ef) at
> > sched_switch+0x180
> >   100072 (0xc2b64320)  sched_switch(c2b64320,0,1,11d,78cee013) at
> > sched_switch+0x180
> >   100071 (0xc2b64190)  sched_switch(c2b64190,0,1,11d,9790d9b7) at
> > sched_switch+0x180
> >   100039 (0xc27c3e10)  sched_switch(c27c3e10,0,1,11d,96ad7513) at
> > sched_switch+0x180
> >   100038 (0xc27c3c80)  sched_switch(c27c3c80,0,1,11d,ca744551) at
> > sched_switch+0x180
> >   100037 (0xc27c3af0)  sched_switch(c27c3af0,0,1,11d,a3d309c5) at
> > sched_switch+0x180
> >   100036 (0xc27c3960)  sched_switch(c27c3960,0,1,11d,b23a2533) at
> > sched_switch+0x180
> >   100064 (0xc27e5640)  sched_switch(c27e5640,0,1,11d,9a1f276b) at
> > sched_switch+0x180
> >   100063 (0xc27e54b0)  sched_switch(c27e54b0,0,1,11d,a08b2329) at
> > sched_switch+0x180
> >   100062 (0xc27e5320)  sched_switch(c27e5320,0,1,11d,334e4c53) at
> > sched_switch+0x180
> >   100061 (0xc27e5190)  sched_switch(c27e5190,0,1,11d,278c08d3) at
> > sched_switch+0x180
> >   100060 (0xc27e5000)  sched_switch(c27e5000,0,1,11d,7bb99f0b) at
> > sched_switch+0x180
> >   100059 (0xc27e2e10)  sched_switch(c27e2e10,0,1,11d,ebf94e5b) at
> > sched_switch+0x180
> >   100058 (0xc27e2c80)  sched_switch(c27e2c80,0,1,11d,dfe1de45) at
> > sched_switch+0x180
> >   100057 (0xc27e2af0)  sched_switch(c27e2af0,0,1,11d,d8b10679) at
> > sched_switch+0x180
> >   100056 (0xc27e2960)  fork_trampoline() at fork_trampoline
> >   100055 (0xc27e27d0)  sched_switch(c27e27d0,0,1,11d,7805ac1b) at
> > sched_switch+0x180
> >   100025 (0xc27687d0)  fork_trampoline() at fork_trampoline
> >   100024 (0xc2768640)  sched_switch(c2768640,0,1,11d,99382d45) at
> > sched_switch+0x180
> >   100023 (0xc27684b0)  fork_trampoline() at fork_trampoline
> >   100049 (0xc27dfe10)  fork_trampoline() at fork_trampoline
> >   100048 (0xc27dfc80)  sched_switch(c27dfc80,0,1,11d,f4d93a93) at
> > sched_switch+0x180
> >   100047 (0xc27dfaf0)  sched_switch(c27dfaf0,0,1,11d,6602877b) at
> > sched_switch+0x180
> >   100046 (0xc27df960)  sched_switch(c27df960,0,1,11d,303d9b09) at
> > sched_switch+0x180
> >   100045 (0xc27df7d0)  sched_switch(c27df7d0,0,1,11d,d1480c21) at
> > sched_switch+0x180
> >   100044 (0xc27df640) 
> > kdb_enter(c06497b7,0,c064f89d,e8955b18,c27df640) at kdb_enter+0x30
> >   100043 (0xc27df4b0)  fork_trampoline() at fork_trampoline
> >   100042 (0xc27df320)  sched_switch(c27df320,0,1,11d,8f05e591) at
> > sched_switch+0x180
> >   100041 (0xc27df190)  fork_trampoline() at fork_trampoline
> >   100040 (0xc27df000)  fork_trampoline() at fork_trampoline
> >   100012 (0xc2764320)  sched_switch(c2764320,0,1,11d,ad12d591) at
> > sched_switch+0x180
> >   100011 (0xc2764190)  fork_trampoline() at fork_trampoline
> >   100035 (0xc27c37d0)  fork_trampoline() at fork_trampoline
> >   100034 (0xc27c3640)  fork_trampoline() at fork_trampoline
> >   100033 (0xc27c34b0)  fork_trampoline() at fork_trampoline
> >   100032 (0xc27c3320)  fork_trampoline() at fork_trampoline
> >   100031 (0xc27c3190)  fork_trampoline() at fork_trampoline
> >   100030 (0xc27c3000)  fork_trampoline() at fork_trampoline
> >   100029 (0xc2768e10)  sched_switch(c2768e10,0,1,11d,4e7277ab) at
> > sched_switch+0x180
> >   100028 (0xc2768c80)  fork_trampoline() at fork_trampoline
> >   100027 (0xc2768af0)  fork_trampoline() at fork_trampoline
> >   100026 (0xc2768960)  fork_trampoline() at fork_trampoline
> >   100000 (0xc275f000)  fork_trampoline() at fork_trampoline
> >   100022 (0xc2768320)  sched_switch(c2768320,0,1,11d,ad732327) at
> > sched_switch+0x180
> >   100021 (0xc2768190)  fork_trampoline() at fork_trampoline
> >   100020 (0xc2768000)  fork_trampoline() at fork_trampoline
> >   100019 (0xc2764e10)  fork_trampoline() at fork_trampoline
> >   100018 (0xc2764c80)  sched_switch(c2764c80,0,1,11d,8991dd45) at
> > sched_switch+0x180
> >   100017 (0xc2764af0)  fork_trampoline() at fork_trampoline
> >   100016 (0xc2764960)  fork_trampoline() at fork_trampoline
> >   100015 (0xc27647d0)  fork_trampoline() at fork_trampoline
> >   100014 (0xc2764640)  fork_trampoline() at fork_trampoline
> >   100013 (0xc27644b0)  fork_trampoline() at fork_trampoline
> >   100010 (0xc2764000)  fork_trampoline() at fork_trampoline
> >   100009 (0xc275fe10)  fork_trampoline() at fork_trampoline
> >   100008 (0xc275fc80)  fork_trampoline() at fork_trampoline
> >   100007 (0xc275faf0)  fork_trampoline() at fork_trampoline
> >   100006 (0xc275f960)  fork_trampoline() at fork_trampoline
> >   100005 (0xc275f7d0)  sched_switch(c275f7d0,0,1,11d,ee0be6c5) at
> > sched_switch+0x180
> >   100004 (0xc275f640)  sched_switch(c275f640,0,1,117,5f9ce7f9) at
> > sched_switch+0x180
> >   100003 (0xc275f4b0)  sched_switch(c275f4b0,0,1,117,b7b18581) at
> > sched_switch+0x180
> >   100002 (0xc275f320)  sched_switch(c275f320,0,1,11d,c07088eb) at
> > sched_switch+0x180
> >   100001 (0xc275f190)  sched_switch(c275f190,0,1,11d,f5b6edc5) at
> > sched_switch+0x180
> >        0 (0xc0697020)  sched_switch(c0697020,0,1,11d,704cd037) at
> > sched_switch+0x180
> > db>show map
> > Task map 0xc0504cf0: pmap=0x4c70424, nentries=-1869574000, version=2072190603
> > db> show page
> > cnt.v_free_count: 23492
> > cnt.v_cache_count: 13141
> > cnt.v_inactive_count: 124090
> > cnt.v_active_count: 180777
> > cnt.v_wire_count: 43501
> > cnt.v_free_reserved: 583
> > cnt.v_free_min: 2507
> > cnt.v_free_target: 10611
> > cnt.v_cache_min: 10611
> > cnt.v_inactive_target: 15916
> > db> show cbstat
> > tot = 107460 (active = 0, free = 107460 (reserved = 102060, slush = 5400))
> > db> show rtc
> > 04/09/26 09:22:38, A = 29, B = 42, C = d0
> > db> show intrcnt
> > irq1: atkbd0            4
> > irq4: sio0              273
> > irq6: fdc0              10
> > irq8: rtc               4186630
> > irq13: npx0             1
> > irq14: ata0             46
> > irq18: fxp0             14933714
> > irq23: asr0             100627
> > irq31: acpi0            24214
> > irq0: clk               3270470
> > db> show pcpu
> > cpuid        = 0
> > curthread    = 0xc27df640: pid 46 "swi1: net"
> > curpcb       = 0xe8955da0
> > fpcurthread  = none
> > idlethread   = 0xc275f640: pid 12 "idle: cpu0"
> > APIC ID      = 3
> > currentldt   = 0x28
> > spin locks held:
> > db>
> > 
> > 
> >>INVARIANTS requires the option 'INVARIANT_SUPPORT'.
> >>
> >>--
> >>John Baldwin <jhb@FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
> >>"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org
> >>_______________________________________________
> >>freebsd-current@freebsd.org mailing list
> >>http://lists.freebsd.org/mailman/listinfo/freebsd-current
> >>To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
> >>
> > 
> > 
> > 
> > 
> 



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1040928232554.15557D-100000>