From owner-freebsd-net@freebsd.org Fri Jul 15 04:38:21 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 721E4B970AA; Fri, 15 Jul 2016 04:38:21 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E48D1ADF; Fri, 15 Jul 2016 04:38:21 +0000 (UTC) (envelope-from mmacy@nextbsd.org) Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1468557492950338.6732714114978; Thu, 14 Jul 2016 21:38:12 -0700 (PDT) Date: Thu, 14 Jul 2016 21:38:12 -0700 From: Matthew Macy To: "Larry Rosenman" Cc: =?UTF-8?Q?=22Edward_Tomasz_Napiera=C5=82a=22?= , "" , "" , "freebsd-net@freebsd.org" Message-ID: <155ecd901bd.10421acf2569401.8259879667024383313@nextbsd.org> In-Reply-To: <90e04710a003b900f9d16d853636f767@thebighonker.lerctr.org> References: <20160706231707.GA2352@borg.lerctr.org> <9f51ccaf5e8f15f269fa865a3adfdc7f@thebighonker.lerctr.org> <20160707132814.GA77070@brick> <90e04710a003b900f9d16d853636f767@thebighonker.lerctr.org> Subject: Re: refcnt 0 on LLE at boot.... MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Priority: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jul 2016 04:38:21 -0000 ---- On Thu, 07 Jul 2016 06:36:19 -0700 Larry Rosenman wr= ote ----=20 > Thanks for that. I've added myself to the cc list, and a comment about= =20 > having 2 vmcore's. >=20 This was introduced by 302350. It broke the return value of callout_{stop= ,drain}. returning 1 even if the callout system did not hold a reference. T= hat in turn broke the following code in lltable_free: =09LIST_FOREACH_SAFE(lle, &dchain, lle_chain, next) { =09=09if (callout_stop(&lle->lle_timer) > 0) =09=09=09LLE_REMREF(lle); =09=09llentry_free(lle); =09} >=20 > On 2016-07-07 08:28, Edward Tomasz Napiera=C5=82a wrote: > > FWIW, I'm seeing this too. I've filed a PR: > >=20 > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210884 > >=20 > > On 0707T0813, Larry Rosenman wrote: > >> and now it's been up for 13+ hours. I do have both VMCORE's from the= =20 > >> 2 > >> crashes. > >>=20 > >>=20 > >>=20 > >> On 2016-07-06 18:22, Larry Rosenman wrote: > >> > Got a similar crash a few minutes later. > >> > > >> > > >> > On 2016-07-06 18:17, Larry Rosenman wrote: > >> >> First boot, and I got the following panic. 2nd boot ran just fine= . > >> >> > >> >> > >> >> borg.lerctr.org dumped core - see /var/crash/vmcore.0 > >> >> > >> >> Wed Jul 6 18:13:34 CDT 2016 > >> >> > >> >> FreeBSD borg.lerctr.org 11.0-ALPHA6 FreeBSD 11.0-ALPHA6 #5 r302379= : > >> >> Wed Jul 6 16:59:11 CDT 2016 > >> >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > >> >> > >> >> panic: bogus refcnt 0 on lle 0xfffff800aa941200 > >> >> > >> >> GNU gdb 6.1.1 [FreeBSD] > >> >> Copyright 2004 Free Software Foundation, Inc. > >> >> GDB is free software, covered by the GNU General Public License, a= nd > >> >> you are > >> >> welcome to change it and/or distribute copies of it under certain > >> >> conditions. > >> >> Type "show copying" to see the conditions. > >> >> There is absolutely no warranty for GDB. Type "show warranty" for > >> >> details. > >> >> This GDB was configured as "amd64-marcel-freebsd"... > >> >> > >> >> Unread portion of the kernel message buffer: > >> >> Copyright (c) 1992-2016 The FreeBSD Project. > >> >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993= , > >> >> 1994 > >> >> The Regents of the University of California. All rights reserv= ed. > >> >> FreeBSD is a registered trademark of The FreeBSD Foundation. > >> >> FreeBSD 11.0-ALPHA6 #5 r302379: Wed Jul 6 16:59:11 CDT 2016 > >> >> root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > >> >> FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based= on > >> >> LLVM 3.8.0) > >> >> can't re-use a leaf (ixl_rx_miss_bufs)! > >> >> MEMGUARD DEBUGGING ALLOCATOR INITIALIZED: > >> >> MEMGUARD map base: 0xfffffe0000400000 > >> >> MEMGUARD map size: 128604256 KBytes > >> >> VT(vga): resolution 640x480 > >> >> CPU: Intel(R) Xeon(R) CPU E5410 @ 2.33GHz (2327.55-MHz > >> >> K8-class CPU) > >> >> Origin=3D"GenuineIntel" Id=3D0x10676 Family=3D0x6 Model=3D0x1= 7 > >> >> Stepping=3D6 > >> >> > >> >> Features=3D0xbfebfbff > >> >> > >> >> Features2=3D0xce3bd > >> >> AMD Features=3D0x20100800 > >> >> AMD Features2=3D0x1 > >> >> VT-x: HLT,PAUSE > >> >> TSC: P-state invariant, performance statistics > >> >> real memory =3D 68719476736 (65536 MB) > >> >> avail memory =3D 65382842368 (62353 MB) > >> >> Event timer "LAPIC" quality 400 > >> >> ACPI APIC Table: > >> >> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs > >> >> FreeBSD/SMP: 2 package(s) x 4 core(s) > >> >> random: unblocking device. > >> >> ioapic0 irqs 0-23 on motherboard > >> >> ioapic1 irqs 24-47 on motherboard > >> >> random: entropy device external interface > >> >> netmap: loaded module > >> >> module_register_init: MOD_LOAD (vesa, 0xffffffff80f2cb40, 0) error= 19 > >> >> kbd1 at kbdmux0 > >> >> vtvga0: on motherboard > >> >> cryptosoft0: on motherboard > >> >> acpi0: on motherboard > >> >> acpi0: Power Button (fixed) > >> >> unknown: I/O range not supported > >> >> cpu0: on acpi0 > >> >> cpu1: on acpi0 > >> >> cpu2: on acpi0 > >> >> cpu3: on acpi0 > >> >> cpu4: on acpi0 > >> >> cpu5: on acpi0 > >> >> cpu6: on acpi0 > >> >> cpu7: on acpi0 > >> >> hpet0: iomem 0xfed00000-0xfed003ff ir= q > >> >> 0,8 on acpi0 > >> >> Timecounter "HPET" frequency 14318180 Hz quality 950 > >> >> Event timer "HPET" frequency 14318180 Hz quality 350 > >> >> Event timer "HPET1" frequency 14318180 Hz quality 340 > >> >> Event timer "HPET2" frequency 14318180 Hz quality 340 > >> >> atrtc0: port 0x70-0x71 on acpi0 > >> >> Event timer "RTC" frequency 32768 Hz quality 0 > >> >> attimer0: port 0x40-0x43,0x50-0x53 on acpi0 > >> >> Timecounter "i8254" frequency 1193182 Hz quality 0 > >> >> Event timer "i8254" frequency 1193182 Hz quality 100 > >> >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > >> >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on a= cpi0 > >> >> pcib0: port 0xcf8-0xcff on acpi0 > >> >> pci0: on pcib0 > >> >> pcib1: at device 2.0 on pci0 > >> >> pci1: on pcib1 > >> >> pcib2: irq 16 at device 0.0 on pci1 > >> >> pci2: on pcib2 > >> >> pcib3: irq 16 at device 0.0 on pci2 > >> >> pci3: on pcib3 > >> >> pcib4: at device 0.0 on pci3 > >> >> pci4: on pcib4 > >> >> pcib5: at device 0.2 on pci3 > >> >> pci5: on pcib5 > >> >> pcib6: irq 18 at device 2.0 on pci2 > >> >> pci6: on pcib6 > >> >> em0: port 0x2000-0x= 201f > >> >> mem 0xd9220000-0xd923ffff,0xd9200000-0xd921ffff irq 18 at device 0= .0 > >> >> on pci6 > >> >> em0: Using an MSI interrupt > >> >> em0: Ethernet address: 00:30:48:f2:29:9c > >> >> em0: netmap queues/slots: TX 1/1024, RX 1/1024 > >> >> em1: port 0x2020-0x= 203f > >> >> mem 0xd9260000-0xd927ffff,0xd9240000-0xd925ffff irq 19 at device 0= .1 > >> >> on pci6 > >> >> em1: Using an MSI interrupt > >> >> em1: Ethernet address: 00:30:48:f2:29:9d > >> >> em1: netmap queues/slots: TX 1/1024, RX 1/1024 > >> >> pcib7: at device 0.3 on pci1 > >> >> pci7: on pcib7 > >> >> pcib8: at device 4.0 on pci0 > >> >> pci8: on pcib8 > >> >> vgapci0: port 0x3000-0x307f mem > >> >> 0xd8000000-0xd8ffffff,0xc0000000-0xc7ffffff,0xc8000000-0xc9ffffff = irq > >> >> 36 at device 0.0 on pci8 > >> >> hdac0: mem 0xd9000000-0xd9003fff irq= 37 > >> >> at device 0.1 on pci8 > >> >> pcib9: at device 6.0 on pci0 > >> >> pci9: on pcib9 > >> >> pcib10: irq 17 at device 28.0 on pci0 > >> >> pcib10: [GIANT-LOCKED] > >> >> pci10: on pcib10 > >> >> pcib11: irq 32 at device 0.0 on pci10 > >> >> pci11: on pcib11 > >> >> pcm0: port 0x4080-0x409f,0x4000-0x407f = irq > >> >> 32 at device 0.0 on pci11 > >> >> pcm0: system configuration > >> >> SubVendorID: 0x1412, SubDeviceID: 0x2403 > >> >> XIN2 Clock Source: 24.576MHz(96kHz*256) > >> >> MPU-401 UART(s) #: not implemented > >> >> ADC #: 1 and SPDIF receiver connected > >> >> DAC #: 4 > >> >> Multi-track converter type: AC'97(SDATA_OUT:packed) > >> >> S/PDIF(IN/OUT): 1/1 ID# 0x00 > >> >> GPIO(mask/dir/state): 0xff/0xff/0xff > >> >> uhci0: port > >> >> 0x1800-0x181f irq 17 at device 29.0 on pci0 > >> >> usbus0 on uhci0 > >> >> uhci1: port > >> >> 0x1820-0x183f irq 19 at device 29.1 on pci0 > >> >> usbus1 on uhci1 > >> >> uhci2: port > >> >> 0x1840-0x185f irq 18 at device 29.2 on pci0 > >> >> usbus2 on uhci2 > >> >> ehci0: mem 0xd9600400-0xd96007f= f > >> >> irq 17 at device 29.7 on pci0 > >> >> usbus3: EHCI version 1.0 > >> >> usbus3 on ehci0 > >> >> pcib12: at device 30.0 on pci0 > >> >> pci12: on pcib12 > >> >> vgapci1: port 0x5000-0x50ff mem > >> >> 0xd0000000-0xd7ffffff,0xd9300000-0xd930ffff irq 18 at device 1.0 o= n > >> >> pci12 > >> >> vgapci1: Boot video device > >> >> isab0: at device 31.0 on pci0 > >> >> isa0: on isab0 > >> >> atapci0: port > >> >> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x1860-0x186f at device 31.1 o= n > >> >> pci0 > >> >> ata0: at channel 0 on atapci0 > >> >> ahci0: port > >> >> 0x18a0-0x18a7,0x1874-0x1877,0x1878-0x187f,0x1870-0x1873,0x1880-0x1= 89f > >> >> mem 0xd9600800-0xd9600bff irq 19 at device 31.2 on pci0 > >> >> ahci0: AHCI v1.10 with 6 3Gbps ports, Port Multiplier supported > >> >> ahcich0: at channel 0 on ahci0 > >> >> ahcich1: at channel 1 on ahci0 > >> >> ahcich2: at channel 2 on ahci0 > >> >> ahcich3: at channel 3 on ahci0 > >> >> ahcich4: at channel 4 on ahci0 > >> >> ahcich5: at channel 5 on ahci0 > >> >> ichsmb0: port > >> >> 0x1100-0x111f irq 19 at device 31.3 on pci0 > >> >> smbus0: on ichsmb0 > >> >> acpi_button0: on acpi0 > >> >> ipmi0: port 0xca2-0xca3 on acpi0 > >> >> ipmi0: KCS mode found at io 0xca2 on acpi > >> >> atkbdc0: port 0x60,0x64 irq 1 on acp= i0 > >> >> atkbd0: irq 1 on atkbdc0 > >> >> kbd0 at atkbd0 > >> >> atkbd0: [GIANT-LOCKED] > >> >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on > >> >> acpi0 > >> >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > >> >> fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2= on > >> >> acpi0 > >> >> fd0: <1440-KB 3.5" drive> on fdc0 drive 0 > >> >> ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on > >> >> acpi0 > >> >> ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode > >> >> ppc0: FIFO with 16/16/9 bytes threshold > >> >> ppbus0: on ppc0 > >> >> lpt0: on ppbus0 > >> >> lpt0: Interrupt-driven port > >> >> ppi0: on ppbus0 > >> >> ichwd0: on isa0 > >> >> orm0: at iomem 0xc0000-0xcafff on isa0 > >> >> coretemp0: on cpu0 > >> >> est0: on cpu0 > >> >> coretemp1: on cpu1 > >> >> est1: on cpu1 > >> >> coretemp2: on cpu2 > >> >> est2: on cpu2 > >> >> coretemp3: on cpu3 > >> >> est3: on cpu3 > >> >> coretemp4: on cpu4 > >> >> est4: on cpu4 > >> >> coretemp5: on cpu5 > >> >> est5: on cpu5 > >> >> coretemp6: on cpu6 > >> >> est6: on cpu6 > >> >> coretemp7: on cpu7 > >> >> est7: on cpu7 > >> >> usbus0: 12Mbps Full Speed USB v1.0 > >> >> fuse-freebsd: version 0.4.4, FUSE ABI 7.8 > >> >> ZFS filesystem version: 5 > >> >> ZFS storage pool version: features support (5000) > >> >> Timecounters tick every 1.000 msec > >> >> nvme cam probe device init > >> >> hdacc0: at cad 0 on hdac0 > >> >> hdaa0: at nid 1 on hdacc0 > >> >> pcm1: at nid 4 on hdaa0 > >> >> pcm2: at nid 5 on hdaa0 > >> >> ipmi0: IPMI device rev. 1, firmware rev. 1.64, version 2.0 > >> >> ipmi0: Number of channels 8 > >> >> ipmi0: Attached watchdog > >> >> usbus1: 12Mbps Full Speed USB v1.0 > >> >> usbus2: 12Mbps Full Speed USB v1.0 > >> >> ugen0.1: at usbus0 > >> >> uhub0: on > >> >> usbus0 > >> >> ugen1.1: at usbus1 > >> >> uhub1: on > >> >> usbus1 > >> >> ugen2.1: at usbus2 > >> >> uhub2: on > >> >> usbus2 > >> >> usbus3: 480Mbps High Speed USB v2.0 > >> >> ugen3.1: at usbus3 > >> >> uhub3: on > >> >> usbus3 > >> >> ata0: DMA limited to UDMA33, controller found non-ATA66 cable > >> >> uhub0: 2 ports with 2 removable, self powered > >> >> uhub1: 2 ports with 2 removable, self powered > >> >> uhub2: 2 ports with 2 removable, self powered > >> >> ada0 at ahcich0 bus 0 scbus1 target 0 lun 0 > >> >> ada0: ACS-2 ATA SATA 3.x device > >> >> cd0 at ata0 bus 0 scbus0 target 0 lun 0 > >> >> cd0: Removable CD-ROM SCSI device > >> >> cd0: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 65534bytes) > >> >> cd0: Attempt to query device size failed: NOT READY, Medium not > >> >> present > >> >> ada0: Serial Number Z3032MSL > >> >> ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada0: 3815447MB (7814037168 512 byte sectors) > >> >> ada1 at ahcich1 bus 0 scbus2 target 0 lun 0 > >> >> ada1: ATA8-ACS SATA 3.x device > >> >> ada1: Serial Number 5YD6FPLG > >> >> ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada1: 1907729MB (3907029168 512 byte sectors) > >> >> ada1: quirks=3D0x1<4K> > >> >> ada2 at ahcich2 bus 0 scbus3 target 0 lun 0 > >> >> ada2: ATA8-ACS SATA 3.x device > >> >> ada2: Serial Number 5YDA3PC5 > >> >> ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada2: 1907729MB (3907029168 512 byte sectors) > >> >> ada2: quirks=3D0x1<4K> > >> >> ada3 at ahcich3 bus 0 scbus4 target 0 lun 0 > >> >> ada3: ATA8-ACS SATA 3.x device > >> >> ada3: Serial Number Y4ODK8VHFSBA > >> >> ada3: 300.000MB/s transfers (SATA 2.x, UDMA5, PIO 8192bytes) > >> >> ada3: 3815447MB (7814037168 512 byte sectors) > >> >> ada4 at ahcich4 bus 0 scbus5 target 0 lun 0 > >> >> ada4: ACS-2 ATA SATA 3.x device > >> >> ada4: Serial Number S30144ZV > >> >> ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada4: 3815447MB (7814037168 512 byte sectors) > >> >> ada4: quirks=3D0x1<4K> > >> >> ada5 at ahcich5 bus 0 scbus6 target 0 lun 0 > >> >> ada5: ATA8-ACS SATA 3.x device > >> >> ada5: Serial Number 5YD5RBS8 > >> >> ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > >> >> ada5: 1907729MB (3907029168 512 byte sectors) > >> >> ada5: quirks=3D0x1<4K> > >> >> SMP: AP CPU #6 Launched! > >> >> SMP: AP CPU #4 Launched! > >> >> SMP: AP CPU #7 Launched! > >> >> SMP: AP CPU #2 Launched! > >> >> SMP: AP CPU #5 Launched! > >> >> SMP: AP CPU #1 Launched! > >> >> SMP: AP CPU #3 Launched! > >> >> Timecounter "TSC-low" frequency 1163772834 Hz quality 1000 > >> >> hwpmc: SOFT/16/64/0x67 TSC/1/64/0x20 > >> >> IAP/2/40/0x3ff > >> >> IAF/3/40/0x67 > >> >> Trying to mount root from zfs:zroot/ROOT/oldroot []... > >> >> Root mount waiting for: usbus3 > >> >> uhub3: 6 ports with 6 removable, self powered > >> >> Root mount waiting for: usbus3 > >> >> Root mount waiting for: usbus3 > >> >> ugen3.2: at usbus3 > >> >> ukbd0: on > >> >> usbus3 > >> >> kbd2 at ukbd0 > >> >> ugen0.2: at usbus0 > >> >> <118>Setting hostuuid: 53d19f64-d663-a017-8922-003048f2299c. > >> >> <118>Setting hostid: 0xbf916f25. > >> >> <118>Starting file system checks: > >> >> <118>Mounting local filesystems:. > >> >> <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/l= ib > >> >> /usr/local/lib/compat /usr/local/lib/gcc48 /usr/local/lib/mysql > >> >> /usr/local/lib/perl5/5.20/mach/CORE /usr/local/lib/qt4 > >> >> /usr/local/lib/virtualbox /usr/local/llvm36/lib /usr/local/llvm37/= lib > >> >> <118>32-bit compatibility ldconfig path: /usr/lib32 > >> >> /usr/local/lib32/compat > >> >> <118>/etc/rc: WARNING: $kldxref_enable is not set properly - see > >> >> rc.conf(5). > >> >> <118>Setting hostname: borg.lerctr.org. > >> >> <118>Setting up harvesting: > >> >> [UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,NET_ETHER,NET_TUN,MOUSE,KEYB= OARD,ATTACH,CACHED > >> >> <118>Feeding entropy: . > >> >> <5>lagg0: link state changed to DOWN > >> >> <118>Created clone interfaces: lagg0. > >> >> <118>Starting Network: lo0 em0 em1 lagg0. > >> >> <118>lo0: flags=3D8049 metric 0 mtu= 16384 > >> >> <118> options=3D600003 > >> >> <118> inet6 ::1 prefixlen 128 > >> >> <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 > >> >> <118> inet 127.0.0.1 netmask 0xff000000 > >> >> <118> nd6 options=3D21 > >> >> <118> groups: lo > >> >> <118>em0: flags=3D8843 met= ric 0 > >> >> mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> nd6 options=3D29 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118>em1: flags=3D8843 met= ric 0 > >> >> mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> nd6 options=3D29 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118>lagg0: flags=3D8843 m= etric > >> >> 0 mtu 1500 > >> >> <118> options=3D4219b > >> >> <118> ether 00:30:48:f2:29:9c > >> >> <118> inet 192.168.200.4 netmask 0xfffffc00 broadcast 192.168.2= 03.255 > >> >> <118> inet6 fe80::230:48ff:fef2:299c%lagg0 prefixlen 64 scopeid= 0x4 > >> >> <118> nd6 options=3D23 > >> >> <118> media: Ethernet autoselect > >> >> <118> status: no carrier > >> >> <118> groups: lagg > >> >> <118> laggproto loadbalance lagghash l2,l3,l4 > >> >> <118> laggport: em0 flags=3D0<> > >> >> <118> laggport: em1 flags=3D0<> > >> >> <118>Starting devd. > >> >> uhid0: on > >> >> usbus3 > >> >> <118>add host 127.0.0.1: gateway lo0 fib 0: route already in table > >> >> <118>add net default: gateway 192.168.200.11 > >> >> <118>add host ::1: gateway lo0 fib 0: route already in table > >> >> <118>add net fe80::: gateway ::1 > >> >> <118>add net ff02::: gateway ::1 > >> >> <118>add net ::ffff:0.0.0.0: gateway ::1 > >> >> <118>add net ::0.0.0.0: gateway ::1 > >> >> <118>Starting local_unbound. > >> >> <5>em0: link state changed to UP > >> >> <5>lagg0: link state changed to UP > >> >> <118>Waiting for nameserver to start... good > >> >> <118>Starting rtsold. > >> >> <5>em1: link state changed to UP > >> >> panic: bogus refcnt 0 on lle 0xfffff800aa941200 > >> >> cpuid =3D 4 > >> >> KDB: stack backtrace: > >> >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > >> >> 0xfffffe2dfe57b1a0 > >> >> vpanic() at vpanic+0x182/frame 0xfffffe2dfe57b220 > >> >> kassert_panic() at kassert_panic+0x126/frame 0xfffffe2dfe57b290 > >> >> llentry_free() at llentry_free+0x136/frame 0xfffffe2dfe57b2c0 > >> >> nd6_cache_lladdr() at nd6_cache_lladdr+0x2ff/frame 0xfffffe2dfe57b= 390 > >> >> nd6_ra_input() at nd6_ra_input+0x1347/frame 0xfffffe2dfe57b670 > >> >> icmp6_input() at icmp6_input+0xa93/frame 0xfffffe2dfe57b810 > >> >> ip6_input() at ip6_input+0xc70/frame 0xfffffe2dfe57b8f0 > >> >> netisr_dispatch_src() at netisr_dispatch_src+0x80/frame > >> >> 0xfffffe2dfe57b950 > >> >> ether_demux() at ether_demux+0x14a/frame 0xfffffe2dfe57b980 > >> >> ether_nh_input() at ether_nh_input+0x340/frame 0xfffffe2dfe57b9c0 > >> >> netisr_dispatch_src() at netisr_dispatch_src+0x80/frame > >> >> 0xfffffe2dfe57ba20 > >> >> ether_input() at ether_input+0x62/frame 0xfffffe2dfe57ba50 > >> >> if_input() at if_input+0xa/frame 0xfffffe2dfe57ba60 > >> >> em_rxeof() at em_rxeof+0x28c/frame 0xfffffe2dfe57bae0 > >> >> em_handle_que() at em_handle_que+0x40/frame 0xfffffe2dfe57bb20 > >> >> taskqueue_run_locked() at taskqueue_run_locked+0x13c/frame > >> >> 0xfffffe2dfe57bb80 > >> >> taskqueue_thread_loop() at taskqueue_thread_loop+0x88/frame > >> >> 0xfffffe2dfe57bbb0 > >> >> fork_exit() at fork_exit+0x84/frame 0xfffffe2dfe57bbf0 > >> >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe2dfe57bbf0 > >> >> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > >> >> Uptime: 22s > >> >> Dumping 3120 out of 64465 > >> >> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > >> >> > >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/zfs.ko > >> >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols > >> >> from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/opensolaris.ko > >> >> Reading symbols from /boot/kernel/linux.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/linux.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/linux.ko > >> >> Reading symbols from /boot/kernel/linux_common.ko...Reading symbol= s > >> >> from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/linux_common.ko > >> >> Reading symbols from /boot/kernel/if_lagg.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/if_lagg.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/if_lagg.ko > >> >> Reading symbols from /boot/kernel/snd_envy24ht.ko...Reading symbol= s > >> >> from /usr/lib/debug//boot/kernel/snd_envy24ht.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/snd_envy24ht.ko > >> >> Reading symbols from /boot/kernel/snd_spicds.ko...Reading symbols = from > >> >> /usr/lib/debug//boot/kernel/snd_spicds.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/snd_spicds.ko > >> >> Reading symbols from /boot/kernel/coretemp.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/coretemp.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/coretemp.ko > >> >> Reading symbols from /boot/kernel/fuse.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/fuse.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fuse.ko > >> >> Reading symbols from /boot/kernel/ichsmb.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ichsmb.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ichsmb.ko > >> >> Reading symbols from /boot/kernel/smbus.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/smbus.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/smbus.ko > >> >> Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ichwd.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ichwd.ko > >> >> Reading symbols from /boot/kernel/cpuctl.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/cpuctl.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/cpuctl.ko > >> >> Reading symbols from /boot/kernel/cryptodev.ko...Reading symbols f= rom > >> >> /usr/lib/debug//boot/kernel/cryptodev.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/cryptodev.ko > >> >> Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols f= rom > >> >> /usr/lib/debug//boot/kernel/dtraceall.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtraceall.ko > >> >> Reading symbols from /boot/kernel/profile.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/profile.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/profile.ko > >> >> Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/dtrace.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtrace.ko > >> >> Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading > >> >> symbols from > >> >> /usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/systrace_freebsd32.ko > >> >> Reading symbols from /boot/kernel/systrace.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/systrace.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/systrace.ko > >> >> Reading symbols from /boot/kernel/sdt.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/sdt.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/sdt.ko > >> >> Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/fasttrap.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fasttrap.ko > >> >> Reading symbols from /boot/kernel/fbt.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/fbt.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/fbt.ko > >> >> Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/dtnfscl.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtnfscl.ko > >> >> Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols fr= om > >> >> /usr/lib/debug//boot/kernel/dtmalloc.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/dtmalloc.ko > >> >> Reading symbols from /boot/kernel/ipmi.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/ipmi.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ipmi.ko > >> >> Reading symbols from /boot/kernel/ipmi_linux.ko...Reading symbols = from > >> >> /usr/lib/debug//boot/kernel/ipmi_linux.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/ipmi_linux.ko > >> >> Reading symbols from /boot/kernel/hwpmc.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/hwpmc.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/hwpmc.ko > >> >> Reading symbols from /boot/kernel/filemon.ko...Reading symbols fro= m > >> >> /usr/lib/debug//boot/kernel/filemon.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/filemon.ko > >> >> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from > >> >> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. > >> >> done. > >> >> Loaded symbols for /boot/kernel/uhid.ko > >> >> #0 doadump (textdump=3D1) at pcpu.h:221 > >> >> 221 pcpu.h: No such file or directory. > >> >> in pcpu.h > >> >> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 > >> >> #1 0xffffffff80a48945 in kern_reboot (howto=3D) > >> >> at /usr/src/sys/kern/kern_shutdown.c:366 > >> >> #2 0xffffffff80a48f1b in vpanic (fmt=3D, > >> >> ap=3D) at /usr/src/sys/kern/kern_shutdown= .c:759 > >> >> #3 0xffffffff80a48d56 in kassert_panic (fmt=3D) > >> >> at /usr/src/sys/kern/kern_shutdown.c:649 > >> >> #4 0xffffffff80b351d6 in llentry_free (lle=3D) > >> >> at /usr/src/sys/net/if_llatbl.c:432 > >> >> #5 0xffffffff80c5c90f in nd6_cache_lladdr (ifp=3D0xfffff80068d658= 00, > >> >> from=3D, lladdr=3D, > >> >> lladdrlen=3D, type=3D134, code=3D0) > >> >> at /usr/src/sys/netinet6/nd6.c:1972 > >> >> #6 0xffffffff80c62f57 in nd6_ra_input (m=3D, > >> >> off=3D, icmp6len=3D) > >> >> at /usr/src/sys/netinet6/nd6_rtr.c:437 > >> >> #7 0xffffffff80c3ab03 in icmp6_input (mp=3D, > >> >> offp=3D, proto=3D) > >> >> at /usr/src/sys/netinet6/icmp6.c:798 > >> >> #8 0xffffffff80c50e90 in ip6_input (m=3DCannot access memory at a= ddress > >> >> 0x0 > >> >> ) > >> >> at /usr/src/sys/netinet6/ip6_input.c:921 > >> >> #9 0xffffffff80b47470 in netisr_dispatch_src (proto=3D6, source= =3D0, > >> >> m=3D0xfffff80068dfc600) at /usr/src/sys/net/netisr.c:1121 > >> >> #10 0xffffffff80b3209a in ether_demux (ifp=3D= , > >> >> m=3D0x0) > >> >> at /usr/src/sys/net/if_ethersubr.c:850 > >> >> #11 0xffffffff80b32e90 in ether_nh_input (m=3D) > >> >> at /usr/src/sys/net/if_ethersubr.c:639 > >> >> #12 0xffffffff80b47470 in netisr_dispatch_src (proto=3D5, source= =3D0, > >> >> m=3D0xfffff80068dfc600) at /usr/src/sys/net/netisr.c:1121 > >> >> #13 0xffffffff80b32402 in ether_input (ifp=3D= , > >> >> m=3D0x0) > >> >> at /usr/src/sys/net/if_ethersubr.c:759 > >> >> #14 0xffffffff80b2ee8a in if_input (ifp=3D0x0, sendmp=3D0x0) > >> >> at /usr/src/sys/net/if.c:3956 > >> >> #15 0xffffffff80523a6c in em_rxeof (count=3D99) > >> >> at /usr/src/sys/dev/e1000/if_em.c:4872 > >> >> #16 0xffffffff805250b0 in em_handle_que (context=3D0xfffffe1eaa699= 000, > >> >> pending=3D) at > >> >> /usr/src/sys/dev/e1000/if_em.c:1598 > >> >> #17 0xffffffff80a9a23c in taskqueue_run_locked (queue=3D >> >> out>) > >> >> at /usr/src/sys/kern/subr_taskqueue.c:465 > >> >> #18 0xffffffff80a9ad38 in taskqueue_thread_loop (arg=3D >> >> out>) > >> >> at /usr/src/sys/kern/subr_taskqueue.c:719 > >> >> #19 0xffffffff80a0ba54 in fork_exit ( > >> >> callout=3D0xffffffff80a9acb0 , > >> >> arg=3D0xfffffe1eaa69b730, frame=3D0xfffffe2dfe57bc00) > >> >> at /usr/src/sys/kern/kern_fork.c:1038 > >> >> #20 0xffffffff80e943fe in fork_trampoline () > >> >> at /usr/src/sys/amd64/amd64/exception.S:611 > >> >> #21 0x0000000000000000 in ?? () > >> >> Current language: auto; currently minimal > >> >> (kgdb) > >> >> > >> >> vmcore IS available. > >>=20 > >> -- > >> Larry Rosenman http://www.lerctr.org/~ler > >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > >> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > >> _______________________________________________ > >> freebsd-current@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-current > >> To unsubscribe, send any mail to=20 > >> "freebsd-current-unsubscribe@freebsd.org" >=20 > --=20 > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g"