From owner-freebsd-bugs@FreeBSD.ORG Fri Feb 14 10:10:03 2014 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D03AD639 for ; Fri, 14 Feb 2014 10:10:03 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9AD1215E4 for ; Fri, 14 Feb 2014 10:10:03 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id s1EAA38m038444 for ; Fri, 14 Feb 2014 10:10:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s1EAA33k038442; Fri, 14 Feb 2014 10:10:03 GMT (envelope-from gnats) Resent-Date: Fri, 14 Feb 2014 10:10:03 GMT Resent-Message-Id: <201402141010.s1EAA33k038442@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Anton Shterenlikht Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15955630 for ; Fri, 14 Feb 2014 10:09:50 +0000 (UTC) Received: from eu1sys200aog117.obsmtp.com (eu1sys200aog117.obsmtp.com [207.126.144.143]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id B75B915DD for ; Fri, 14 Feb 2014 10:09:48 +0000 (UTC) Received: from mail-wi0-f175.google.com ([209.85.212.175]) (using TLSv1) by eu1sys200aob117.postini.com ([207.126.147.11]) with SMTP ID DSNKUv3rZVELXBHVohBiH7tsgvBWvyVNI70M@postini.com; Fri, 14 Feb 2014 10:09:48 UTC Received: by mail-wi0-f175.google.com with SMTP id hm4so298268wib.2 for ; Fri, 14 Feb 2014 02:09:41 -0800 (PST) Received: from mech-cluster241.men.bris.ac.uk (mech-cluster241.men.bris.ac.uk. [137.222.187.241]) by mx.google.com with ESMTPSA id dk9sm11478974wjb.4.2014.02.14.02.09.39 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 14 Feb 2014 02:09:39 -0800 (PST) Received: from mech-cluster241.men.bris.ac.uk (localhost [127.0.0.1]) by mech-cluster241.men.bris.ac.uk (8.14.7/8.14.6) with ESMTP id s1EA9bnK071888 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 14 Feb 2014 10:09:37 GMT (envelope-from mexas@mech-cluster241.men.bris.ac.uk) Received: (from mexas@localhost) by mech-cluster241.men.bris.ac.uk (8.14.7/8.14.6/Submit) id s1EA9bHn071887; Fri, 14 Feb 2014 10:09:37 GMT (envelope-from mexas) Message-Id: <201402141009.s1EA9bHn071887@mech-cluster241.men.bris.ac.uk> Date: Fri, 14 Feb 2014 02:09:39 -0800 (PST) From: Anton Shterenlikht Sender: Anton Shterenlikht To: FreeBSD-gnats-submit@freebsd.org X-Send-Pr-Version: 3.114 Subject: kern/186760: sparc64 11.0-CURRENT #9 r260689 panic: Bad link elm 0xfffff8017ef106d8 prev->next != elm X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: Anton Shterenlikht List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Feb 2014 10:10:03 -0000 >Number: 186760 >Category: kern >Synopsis: sparc64 11.0-CURRENT #9 r260689 panic: Bad link elm 0xfffff8017ef106d8 prev->next != elm >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Feb 14 10:10:02 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Anton Shterenlikht >Release: FreeBSD 11.0-CURRENT sparc64 >Organization: University of Bristol >Environment: System: FreeBSD mech-as222.men.bris.ac.uk 11.0-CURRENT FreeBSD 11.0-CURRENT #9 r260689: Thu Jan 16 09:04:52 GMT 2014 root@mech-as222.men.bris.ac.uk:/usr/obj/usr/src /sys/DALET sparc64 >Description: with textdump configured I got these files: # cat info.last Dump header from device /dev/ada1b Architecture: sparc64 Architecture Version: 1 Dump Length: 62976B (0 MB) Blocksize: 512 Dumptime: Thu Feb 13 13:56:08 2014 Hostname: mech-as222.men.bris.ac.uk Magic: FreeBSD Text Dump Version String: FreeBSD 11.0-CURRENT #9 r260689: Thu Jan 16 09:04:52 GMT 2014 root@mech-as222.men.bris.ac.uk:/usr/obj/usr/src/sys/DALET Panic String: Bad link elm 0xfffff8017ef106d8 prev->next != elm Dump Parity: 1422462080 Bounds: 0 Dump Status: good # # cat panic.txt Bad link elm 0xfffff8017ef106d8 prev->next != elm # cat version.txt FreeBSD 11.0-CURRENT #9 r260689: Thu Jan 16 09:04:52 GMT 2014 root@mech-as222.men.bris.ac.uk:/usr/obj/usr/src/sys/DALET # cat config.txt options CONFIG_AUTOGENERATED ident DALET machine sparc64 cpu SUN4U makeoptions MODULES_OVERRIDE=geom/geom_part geom/geom_label makeoptions DEBUG=-g options USB_DEBUG options MALLOC_DEBUG_MAXZONES=8 options WITNESS_SKIPSPIN options WITNESS options INVARIANT_SUPPORT options INVARIANTS options DEADLKRES options GDB options DDB options KDB_TRACE options KDB options INCLUDE_CONFIG_FILE options MAC options CAPABILITIES options CAPABILITY_MODE options AUDIT options HWPMC_HOOKS options PRINTF_BUFR_SIZE=128 options _KPOSIX_PRIORITY_SCHEDULING options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options GEOM_LABEL options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options NFS_ROOT options NFSLOCKD options NFSD options NFSCL options MD_ROOT options QUOTA options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options INET6 options INET options PREEMPTION options SCHED_ULE options TMPFS options NULLFS options FDESCFS options NEW_PCIB options SUNKBD_EMULATE_ATKBD options GEOM_PART_VTOC8 options GEOM_PART_BSD device mem device uart_ns8250 device uart_sab82532 device uart_z8530 device ebus device isa device pci device sbus device central device fhc device ahci device ata device scbus device da device cd device pass device auxio device eeprom device mk48txx device rtc device mc146818 device epic device sbbc device puc device scc device uart device miibus device bge device loop device random device ether device vlan device tun device md device gif device faith device firmware device bpf device uhci device ohci device ehci device usb device umass # cat msgbuf.txt KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2014 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 11.0-CURRENT #9 r260689: Thu Jan 16 09:04:52 GMT 2014 root@mech-as222.men.bris.ac.uk:/usr/obj/usr/src/sys/DALET sparc64 gcc version 4.2.1 20070831 patched [FreeBSD] WARNING: WITNESS option enabled, expect reduced performance. real memory = 5368709120 (5120 MB) avail memory = 5242200064 (4999 MB) cpu0: Sun Microsystems UltraSparc-IIIi Processor (1503.00 MHz CPU) random: initialized nexus0: pcib0: mem 0x4000f600000-0x4000f60afff,0x4000f410000-0x4000f41701f,0x7fe00000000-0x7fe000000ff,0x4000f780000-0x4000f78ffff irq 1970,1968,1969,1972,1953 on nexus0 pcib0: Tomatillo, version 4, IGN 0x1e, bus A, PCI mode, 33MHz pcib0: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries pci0: on pcib0 pcib1: mem 0x4000ff00000-0x4000ff0afff,0x4000fc10000-0x4000fc1701f,0x7f600000000-0x7f6000000ff,0x4000ff80000-0x4000ff8ffff irq 2035,2032,2033,2036,2019 on nexus0 pcib1: Tomatillo, version 4, IGN 0x1f, bus B, PCI mode, 66MHz pcib1: DVMA map: 0xc0000000 to 0xdfffffff 65536 entries pci1: on pcib1 nexus0: mem 0x40000000000-0x40000000007 type memory-controller (no driver attached) isab0: at device 7.0 on pci0 pcib0: could not route pin 1 for device 7.0 isa0: could not map ISA interrupt 1 for node 0xf0097238: parallel isa0: on isab0 pci0: at device 6.0 (no driver attached) pci0: at device 8.0 (no driver attached) ohci0: mem 0x1000000-0x1000fff at device 10.0 on pci0 usbus0 on ohci0 ohci1: mem 0x2000000-0x2000fff at device 11.0 on pci0 usbus1 on ohci1 atapci0: port 0xa00-0xa07,0xa18-0xa1b,0xa10-0xa17,0xa08-0xa0b,0xa20-0xa2f at device 13.0 on pci0 atapci0: using PIO transfers above 137GB as workaround for 48bit DMA access bug, expect reduced performance ata2: at channel 0 on atapci0 ata3: at channel 1 on atapci0 jbusppm0: mem 0x4000f000000-0x4000f000007,0x4000f410050-0x4000f41005f on nexus0 bge0: mem 0x200000-0x20ffff at device 2.0 on pci1 bge0: CHIP ID 0x00001002; ASIC REV 0x01; CHIP REV 0x10; PCI on PCI-X 66 MHz; 64bit miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:03:ba:d9:93:0f nexus0: mem 0x4000fc64000-0x4000fc6400f type i2c (no driver attached) rtc0: at port 0x70-0x71 on isa0 uart0: <16550 or compatible> at port 0x3f8-0x3ff irq 44 on isa0 uart0: console (9600,n,8,1) uart1: <16550 or compatible> at port 0x2e8-0x2ef irq 44 on isa0 WITNESS: unable to allocate a new witness object Timecounter "tick" frequency 1503000000 Hz quality 1000 Timecounter "stick" frequency 12000000 Hz quality 1000 Event timer "tick" frequency 1503000000 Hz quality 1000 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered ada0 at ata2 bus 0 scbus0 target 0 lun 0 cd0 at ata3 bus 0 scbus1 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present ada0: ATA-6 device ada0: Serial Number 3JVBAPMQ ada0: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada0: 76319MB (156301488 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad0 ada1 at ata2 bus 0 scbus0 target 1 lun 0 ada1: ATA-6 device ada1: Serial Number 4JT0RMG0 ada1: 100.000MB/s transfers (UDMA5, PIO 8192bytes) ada1: 114473MB (234441648 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad1 random: unblocking device. WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ada0a [rw]... <118>Setting hostuuid: 0400d9da-567d-11e2-ab7c-0003bad9930f. <118>Setting hostid: 0xdbb9d474. <118>Starting ddb. <118>Entropy harvesting: interrupts ethernet point_to_point swi. <118>Starting file system checks: <118>/dev/ada0a: FILE SYSTEM CLEAN; SKIPPING CHECKS <118>/dev/ada0a: clean, 16638059 free (131987 frags, 2063259 blocks, 0.7% fragmentation) <118>Mounting local file systems:. <118>Writing entropy file:. <118>Setting hostname: mech-as222.men.bris.ac.uk. <118>Starting Network: lo0 bge0. <118>lo0: flags=8049 metric 0 mtu 16384 <118> options=600003 <118> inet6 ::1 prefixlen 128 <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 <118> inet 127.0.0.1 netmask 0xff000000 <118> nd6 options=21 <118>bge0: flags=8843 metric 0 mtu 1500 <118> options=8009b <118> ether 00:03:ba:d9:93:0f <118> inet 137.222.187.222 netmask 0xffffff00 broadcast 137.222.187.255 <118> inet6 fe80::203:baff:fed9:930f%bge0 prefixlen 64 scopeid 0x1 <118> nd6 options=29 <118> media: Ethernet autoselect (none) <118> status: no carrier <118>Starting devd. <118>Starting pflogd: <118>add net default: gateway 137.222.187.250 <118>add net fe80::: gateway ::1 <118>add net ff02::: gateway ::1 <118>add net ::ffff:0.0.0.0: gateway ::1 <118>add net ::0.0.0.0: gateway ::1 <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib <118>Creating and/or trimming log files. <118>Starting syslogd. <118>No core dumps found. <118>Starting casperd. <118>Clearing /tmp (X related). <118>Updating motd:. <118>Mounting late file systems:. <118>Starting ntpd. <118>Starting bsdstats. <118>Posting monthly OS statistics to rpt.bsdstats.org <118>Performing sanity check on sshd configuration. <118>Starting sshd. <118>Starting sendmail_submit. <118>Starting sendmail_msp_queue. <118>Starting cron. <118>Starting inetd. <118>Starting background file system checks in 60 seconds. <118> <118>Thu Feb 13 13:20:02 GMT 2014 <118>Feb 13 13:23:30 mech-as222 su: mexas to root on /dev/pts/0 panic: Bad link elm 0xfffff8017ef106d8 prev->next != elm KDB: stack backtrace: vpanic() at vpanic+0xfc panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done KDB: enter: panic # cat ddb.txt db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks db:1:locks> show alllocks db:1:alllocks> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid = 0 dynamic pcpu = 0xfffff7ff40d07c80 curthread = 0xfffff8010aad2490: pid 29728 "sh" curpcb = 0xe7ae1980 fpcurthread = none idlethread = 0xfffff80002077b60: tid 100003 "idle" spin locks held: db:0:kdb.enter.panic> bt Tracing pid 29728 tid 100100 td 0xfffff8010aad2490 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done db:0:kdb.enter.panic> ps pid ppid pgrp uid state wmesg wchan cmd 29728 29707 825 0 RLJ CPU 0 sh 29707 27535 825 0 SJ piperd 0xfffff8000267dce0 sh 29660 12031 825 0 S nanslp 0xc071ba80 sleep 27535 27530 825 0 SJ wait 0xfffff8010aacee10 gmake 27530 27529 825 0 SJ wait 0xfffff8010aacf2c0 sh 27529 27528 825 0 SJ wait 0xfffff8010aace4b0 gmake 27528 26281 825 0 SJ wait 0xfffff800029632c0 gmake 26281 26280 825 0 SJ wait 0xfffff8000295c4b0 sh 26280 26275 825 0 SJ wait 0xfffff8004fd0ae10 gmake 26275 26274 825 0 SJ wait 0xfffff80002963770 sh 26274 26273 825 0 SJ wait 0xfffff8004fd0a4b0 gmake 26273 26255 825 0 SJ wait 0xfffff8004fd0a960 sh 26255 26253 825 0 SJ wait 0xfffff8010a52c960 make 26253 12590 825 0 S wait 0xfffff800028df2c0 sh 21797 811 21797 0 S+ select 0xfffff8000262a340 top 15378 819 819 80 S kqread 0xfffff8003d637100 nginx 12678 12590 825 0 S piperd 0xfffff8020dc82e70 tee 12590 1414 825 0 S select 0xfffff8000262af40 sh 12031 1414 825 0 S wait 0xfffff8004fd0b2c0 sh 1414 825 825 0 S select 0xfffff8000262a740 sh 830 823 830 25 Ss piperd 0xfffff8000267da78 sendmail 825 823 825 0 Ss wait 0xfffff800028dee10 sh 823 751 751 0 S piperd 0xfffff8000267d340 cron 819 1 819 0 Ss pause 0xfffff8000295ca08 nginx 811 810 811 0 S+ wait 0xfffff8000295d2c0 sh 810 807 810 1001 S+ wait 0xfffff8000295d770 su 807 806 807 1001 Ss+ pause 0xfffff800026bea08 tcsh 806 804 804 1001 S select 0xfffff800021bdf40 sshd 804 741 804 0 Ss select 0xfffff800021c00c0 sshd 799 1 1 0 S ttydcd 0xfffff800021dcce8 getty 798 1 798 0 Ss+ ttyin 0xfffff800021dd0a8 getty 776 1 776 0 Ss select 0xfffff8000262c440 inetd 751 1 751 0 Ss nanslp 0xc071ba80 cron 747 1 747 25 Ss pause 0xfffff800028de558 sendmail 744 1 744 0 Ss select 0xfffff8000262c540 sendmail 741 1 741 0 Ss select 0xfffff8000262c5c0 sshd 669 1 669 0 Ss select 0xfffff800021bdac0 ntpd 609 1 609 0 Ss select 0xfffff800021c02c0 casperd 608 1 17 0 S+ select 0xfffff800021c07c0 casperd 584 1 584 0 Ss select 0xfffff800021bdc40 syslogd 461 1 461 0 Ss select 0xfffff800021bdb40 devd 126 1 126 0 Ss pause 0xfffff800026be558 adjkerntz 16 0 0 0 DL sdflush 0xc08e02c8 [softdepflush] 9 0 0 0 DL vlruwt 0xfffff80002624000 [vnlru] 8 0 0 0 DL syncer 0xc08d8178 [syncer] 7 0 0 0 DL psleep 0xc08d7c80 [bufdaemon] 6 0 0 0 RL [pagezero] 5 0 0 0 DL psleep 0xc08e191c [vmdaemon] 4 0 0 0 DL psleep 0xc08e196c [pagedaemon] 3 0 0 0 DL waiting_ 0xc08dac70 [sctp_iterator] 15 0 0 0 DL (threaded) [usb] 100028 D - 0x100165460 [usbus1] 100027 D - 0x100165408 [usbus1] 100026 D - 0x1001653b0 [usbus1] 100025 D - 0x100165358 [usbus1] 100023 D - 0x100161460 [usbus0] 100022 D - 0x100161408 [usbus0] 100021 D - 0x1001613b0 [usbus0] 100020 D - 0x100161358 [usbus0] 2 0 0 0 DL (threaded) [cam] 100035 D ccb_scan 0xc06f93c0 [scanner] 100012 D - 0xc06f9580 [doneq0] 14 0 0 0 DL - 0xc06fcfc0 [rand_harvestq] 13 0 0 0 DL (threaded) [geom] 100009 D - 0xc07167b0 [g_down] 100008 D - 0xc07167a8 [g_up] 100007 D - 0xc0716798 [g_event] 12 0 0 0 WL (threaded) [intr] 100031 I [swi0: uart uart] 100030 I [vec2012: bge0] 100029 I [vec1944: atapci0] 100024 I [vec1958: ohci1] 100019 I [vec1959: ohci0] 100018 I [swi6: task queue] 100017 I [swi6: Giant taskq] 100015 I [swi5: fast taskq] 100006 I [swi3: vm] 100005 I [swi4: clock] 100004 I [swi1: netisr 0] 11 0 0 0 RL [idle] 1 0 1 0 SLs wait 0xfffff800020744b0 [init] 10 0 0 0 DL audit_wo 0xc08df698 [audit] 0 0 0 0 DLs (threaded) [kernel] 100034 D - 0xfffff80002088d00 [CAM taskq] 100032 D - 0xc071a768 [deadlkres] 100016 D - 0xfffff80002088700 [thread taskq] 100014 D - 0xfffff80002088a00 [kqueue taskq] 100013 D - 0xfffff80002088b00 [ffs_trim taskq] 100010 D - 0xfffff80002089500 [firmware taskq] 100000 D swapin 0xc07169a0 [swapper] db:0:kdb.enter.panic> alltrace Tracing command sh pid 29728 tid 100100 td 0xfffff8010aad2490 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 29707 tid 100083 td 0xfffff80002966db0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 pipe_read() at pipe_read+0x484 dofileread() at dofileread+0x74 kern_readv() at kern_readv+0x58 sys_read() at sys_read+0x74 syscall() at syscall+0x304 -- syscall (3, FreeBSD ELF64, sys_read) %o7=0x10abfc -- userland() at 0x40719768 user trace: trap %o7=0x10abfc pc 0x40719768, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sleep pid 29660 tid 100092 td 0xfffff8004fd1f6d0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x8 _sleep() at _sleep+0x474 kern_nanosleep() at kern_nanosleep+0x15c sys_nanosleep() at sys_nanosleep+0x50 syscall() at syscall+0x304 -- syscall (240, FreeBSD ELF64, sys_nanosleep) %o7=0x100b04 -- userland() at 0x40474668 user trace: trap %o7=0x100b04 pc 0x40474668, sp 0x7fdffffe101 pc 0x4e473d3100504b47, sp 0x636d702000504b47 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x636d702000504a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command gmake pid 27535 tid 100095 td 0xfffff8010aad3b60 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x110380 -- userland() at 0x405ad668 user trace: trap %o7=0x110380 pc 0x405ad668, sp 0x7fdffffc091 pc 0x61, sp 0x41024000 done Tracing command sh pid 27530 tid 100094 td 0xfffff8010aad4000 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40719668 user trace: trap %o7=0x10ef58 pc 0x40719668, sp 0x7fdffffbe11 pc 0x4, sp 0x4107ac84 pc 0x5a5a5a5a5a5a5a5a, sp 0x5a5a5a5a5a5a5a5a kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x5a5a5a5a5a5a6a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command gmake pid 27529 tid 100097 td 0xfffff8010aad3240 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x110380 -- userland() at 0x405ad668 user trace: trap %o7=0x110380 pc 0x405ad668, sp 0x7fdffffc661 pc 0x225800, sp 0x225800 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x226a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command gmake pid 27528 tid 100082 td 0xfffff80002967240 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x110380 -- userland() at 0x405ad668 user trace: trap %o7=0x110380 pc 0x405ad668, sp 0x7fdffffc671 done Tracing command sh pid 26281 tid 100073 td 0xfffff80002964920 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40719668 user trace: trap %o7=0x10ef58 pc 0x40719668, sp 0x7fdffffc891 pc 0, sp 0x41185468 pc 0x726573735f6f7074, sp 0xfec18173757070 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xfec18173756a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command gmake pid 26280 tid 100077 td 0xfffff8004fd1e920 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x110380 -- userland() at 0x405ad668 user trace: trap %o7=0x110380 pc 0x405ad668, sp 0x7fdffffc491 done Tracing command sh pid 26275 tid 100081 td 0xfffff800029676d0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40719668 user trace: trap %o7=0x10ef58 pc 0x40719668, sp 0x7fdffffc911 pc 0x11d190, sp 0x7fdffffc9d1 pc 0x106278, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command gmake pid 26274 tid 100079 td 0xfffff8004fd1e000 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x110380 -- userland() at 0x405ad668 user trace: trap %o7=0x110380 pc 0x405ad668, sp 0x7fdffffc571 pc 0x7fdffffce98, sp 0x1056c8 pc 0xc827000896a938a0, sp 0xfe03000415ac1062 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xfe03000415ac0a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 26273 tid 100078 td 0xfffff8004fd1e490 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40719668 user trace: trap %o7=0x10ef58 pc 0x40719668, sp 0x7fdffffcf71 pc 0x46, sp 0x348 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xa2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command make pid 26255 tid 100084 td 0xfffff80002966920 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x1027e8 -- userland() at 0x1524c8 user trace: trap %o7=0x1027e8 pc 0x1524c8, sp 0x7fdffffc681 pc 0x7fdffffcfa8, sp 0x1056c8 pc 0xc827000896a938a0, sp 0xfe03000415ac1062 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xfe03000415ac0a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 26253 tid 100055 td 0xfffff800028e3b60 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40718b28 user trace: trap %o7=0x10ef58 pc 0x40718b28, sp 0x7fdffffb2e1 done Tracing command top pid 21797 tid 100088 td 0xfffff8010a534920 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x8 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x390 seltdwait() at seltdwait+0x84 kern_select() at kern_select+0x6f8 sys_select() at sys_select+0x44 syscall() at syscall+0x304 -- syscall (93, FreeBSD ELF64, sys_select) %o7=0x1098c0 -- userland() at 0x40837ca8 user trace: trap %o7=0x1098c0 pc 0x40837ca8, sp 0x7fdffffe0f1 pc 0x6c74696e2d6d656d, sp 0x2d666e6f2d627569 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x2d666e6f2d626a2e %o7=0xc05d9dc0 -- copyin() at copyin+0x8c db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command nginx pid 15378 tid 100089 td 0xfffff8010a534490 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x8 _sleep() at _sleep+0x474 kern_kevent() at kern_kevent+0x3f0 sys_kevent() at sys_kevent+0x178 syscall() at syscall+0x304 -- syscall (363, FreeBSD ELF64, sys_kevent) %o7=0x1245d0 -- userland() at 0x40979128 user trace: trap %o7=0x1245d0 pc 0x40979128, sp 0x7fdffffdb91 pc 0x676c736c2f676c63, sp 0x2f2e2e2f7372632f kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x2f2e2e2f73726a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command tee pid 12678 tid 100044 td 0xfffff8000215c920 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 pipe_read() at pipe_read+0x484 dofileread() at dofileread+0x74 kern_readv() at kern_readv+0x58 sys_read() at sys_read+0x74 syscall() at syscall+0x304 -- syscall (3, FreeBSD ELF64, sys_read) %o7=0x100c8c -- userland() at 0x40476c28 user trace: trap %o7=0x100c8c pc 0x40476c28, sp 0x7fdffffe0b1 pc 0x32202d7069706520, sp 0x464c4147533d2d4f kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x464c4147533d2a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 12590 tid 100067 td 0xfffff80002966490 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x8 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x390 seltdwait() at seltdwait+0x84 kern_select() at kern_select+0x6f8 sys_select() at sys_select+0x44 syscall() at syscall+0x304 -- syscall (93, FreeBSD ELF64, sys_select) %o7=0x112fac -- userland() at 0x40717ca8 user trace: trap %o7=0x112fac pc 0x40717ca8, sp 0x7fdffffb641 pc 0x4085c770, sp 0xd3 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xa2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 12031 tid 100076 td 0xfffff8004fd1edb0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40718b28 user trace: trap %o7=0x10ef58 pc 0x40718b28, sp 0x7fdffffd031 pc 0x4033e480, sp 0x878 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xa2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 1414 tid 100075 td 0xfffff80002964000 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_timedwait_sig() at sleepq_timedwait_sig+0x8 _cv_timedwait_sig_sbt() at _cv_timedwait_sig_sbt+0x390 seltdwait() at seltdwait+0x84 kern_select() at kern_select+0x6f8 sys_select() at sys_select+0x44 syscall() at syscall+0x304 -- syscall (93, FreeBSD ELF64, sys_select) %o7=0x112fac -- userland() at 0x40717ca8 user trace: trap %o7=0x112fac pc 0x40717ca8, sp 0x7fdffffd281 done Tracing command sendmail pid 830 tid 100074 td 0xfffff80002964490 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 pipe_read() at pipe_read+0x484 dofileread() at dofileread+0x74 kern_readv() at kern_readv+0x58 sys_read() at sys_read+0x74 syscall() at syscall+0x304 -- syscall (3, FreeBSD ELF64, sys_read) %o7=0x19cd34 -- userland() at 0x40ccec28 user trace: trap %o7=0x19cd34 pc 0x40ccec28, sp 0x7fdffffb791 pc 0x20, sp 0xd0 kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0xa2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 825 tid 100056 td 0xfffff800028e36d0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40718b28 user trace: trap %o7=0x10ef58 pc 0x40718b28, sp 0x7fdffffdd21 pc 0x6f63616c2f736861, sp 0x523d2f7573722f6c kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x523d2f7573722a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command cron pid 823 tid 100065 td 0xfffff800026c5b60 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 pipe_read() at pipe_read+0x484 dofileread() at dofileread+0x74 kern_readv() at kern_readv+0x58 sys_read() at sys_read+0x74 syscall() at syscall+0x304 -- syscall (3, FreeBSD ELF64, sys_read) %o7=0x40607f30 -- userland() at 0x406a4c28 user trace: trap %o7=0x40607f30 pc 0x406a4c28, sp 0x7fdffffd661 pc 0x7fdffffec6e, sp 0x7fdffffec3d done Tracing command nginx pid 819 tid 100072 td 0xfffff80002964db0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_sigsuspend() at kern_sigsuspend+0xc4 sys_sigsuspend() at sys_sigsuspend+0x28 syscall() at syscall+0x304 -- syscall (341, FreeBSD ELF64, sys_sigsuspend) %o7=0x122750 -- userland() at 0x409796a8 user trace: trap %o7=0x122750 pc 0x409796a8, sp 0x7fdffffdfa1 pc 0x6f63616c2f62696e, sp 0x453d2f7573722f6c kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x453d2f7573722a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command sh pid 811 tid 100070 td 0xfffff800029656d0 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x10ef58 -- userland() at 0x40718b28 user trace: trap %o7=0x10ef58 pc 0x40718b28, sp 0x7fdffffdce1 pc 0x6f002e2e2f2e2e2f, sp 0x756e726f6c6c2e6c kdb_reenter() at kdb_reenter+0x5c trap() at trap+0x2fc -- fast data access mmu miss tar=0x756e726f6c6c2a2e %o7=0xc05d9dc0 -- copyin() at copyin+0xac db_trace_thread() at db_trace_thread+0x18 db_stack_trace_all() at db_stack_trace_all+0xa4 db_command() at db_command+0x48c db_command_script() at db_command_script+0x84 db_script_exec() at db_script_exec+0x1e0 db_script_kdbenter() at db_script_kdbenter+0x70 db_trap() at db_trap+0x1a4 kdb_trap() at kdb_trap+0xfc trap() at trap+0x358 -- breakpoint %o7=0xc03160a0 -- kdb_enter() at kdb_enter+0x80 vpanic() at vpanic+0x140 panic() at panic+0x20 vm_page_dequeue() at vm_page_dequeue+0xf8 vm_fault_hold() at vm_fault_hold+0x518 vm_fault() at vm_fault+0x88 trap_pfault() at trap_pfault+0x1ac trap() at trap+0xdc -- data access protection tar=0x4026aa2e sfar=0x4026a0c8 sfsr=0x800005 %o7=0x4062f258 -- userland() at 0x4062f268 user trace: trap %o7=0x4062f258 pc 0x4062f268, sp 0x7fdffffb8b1 pc 0x112398, sp 0x7fdffffb991 pc 0x110064, sp 0x7fdffffba51 pc 0x1063a8, sp 0x7fdffffbb11 pc 0x107b58, sp 0x7fdffffbc21 pc 0x10aaec, sp 0x7fdffffbd41 pc 0x10c314, sp 0x7fdffffbf51 pc 0x106860, sp 0x7fdffffc011 pc 0x106360, sp 0x7fdffffc1d1 pc 0x10604c, sp 0x7fdffffc2e1 pc 0x107160, sp 0x7fdffffc3f1 pc 0x106360, sp 0x7fdffffc5b1 pc 0x10604c, sp 0x7fdffffc6c1 pc 0x107160, sp 0x7fdffffc7d1 pc 0x106360, sp 0x7fdffffc991 pc 0x106284, sp 0x7fdffffcaa1 pc 0x10604c, sp 0x7fdffffcbb1 pc 0x107160, sp 0x7fdffffccc1 pc 0x106360, sp 0x7fdffffce81 pc 0x111c6c, sp 0x7fdffffcf91 pc 0x112118, sp 0x7fdffffd071 pc 0x102bc4, sp 0x7fdffffd171 pc 0x402258f4, sp 0x7fdffffd231 done Tracing command su pid 810 tid 100069 td 0xfffff80002965b60 mi_switch() at mi_switch+0x2d8 sleepq_switch() at sleepq_switch+0x22c sleepq_catch_signals() at sleepq_catch_signals+0x134 sleepq_wait_sig() at sleepq_wait_sig+0x8 _sleep() at _sleep+0x4a8 kern_wait6() at kern_wait6+0x7b4 kern_wait() at kern_wait+0x80 sys_wait4() at sys_wait4+0x20 syscall() at syscall+0x304 -- syscall (7, FreeBSD ELF64, sys_wait4) %o7=0x1025a8 -- userland() at 0x407beb28 user trace: trap %o7=0x1025a8 pc 0x407beb28, sp 0x7fdffffdc01 pc 0xGDB: no debug ports present >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: