From owner-freebsd-current@FreeBSD.ORG Mon May 10 22:08:55 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 44EDA106566B; Mon, 10 May 2010 22:08:55 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from bunrab.catwhisker.org (adsl-63-193-123-122.dsl.snfc21.pacbell.net [63.193.123.122]) by mx1.freebsd.org (Postfix) with ESMTP id 9EEE18FC15; Mon, 10 May 2010 22:08:54 +0000 (UTC) Received: from bunrab.catwhisker.org (localhost [127.0.0.1]) by bunrab.catwhisker.org (8.13.3/8.13.3) with ESMTP id o4AM8skw096011; Mon, 10 May 2010 15:08:54 -0700 (PDT) (envelope-from david@bunrab.catwhisker.org) Received: (from david@localhost) by bunrab.catwhisker.org (8.13.3/8.13.3/Submit) id o4AM8sJ3096010; Mon, 10 May 2010 15:08:54 -0700 (PDT) (envelope-from david) Date: Mon, 10 May 2010 15:08:54 -0700 From: David Wolfskill To: "K. Macy" Message-ID: <20100510220854.GK49209@bunrab.catwhisker.org> Mail-Followup-To: David Wolfskill , "K. Macy" , current@freebsd.org References: <20100510140722.GD49209@bunrab.catwhisker.org> <20100510182243.GV56080@hoeg.nl> <20100510182625.GF49209@bunrab.catwhisker.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="KBQi/TxAuyHKotC8" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.1i Cc: current@freebsd.org Subject: Re: Panic @r207844; current process: flowcleaner "Fatal trap 12: page fault while in kernel mode" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 May 2010 22:08:55 -0000 --KBQi/TxAuyHKotC8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, May 10, 2010 at 02:32:14PM -0700, K. Macy wrote: > Could you please try with 207902? > ... First, thanks for the response. OK; I grabbed r207902 & applied it (via "patch -p1"), then rebuilt the kernel & rebooted; here's the panic now: 3 Select option, [Enter] for default 3 3 or [Space] to pause timer 9 3 @DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDY GDB: no debug ports present KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-CURRENT #155 r207844M: Mon May 10 14:55:41 PDT 2010 root@freebeast.catwhisker.org:/usr/obj/usr/src/sys/GENERIC i386 WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 3.60GHz (3610.95-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf41 Family =3D f Model =3D 4 Stepp= ing =3D 1 Features=3D0xbfebfbff Features2=3D0x659d AMD Features=3D0x20100000 TSC: P-state invariant real memory =3D 2147483648 (2048 MB) avail memory =3D 2086129664 (1989 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 2 package(s) x 1 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.1 (no driver attached) pci0: at device 1.0 (no driver attached) pcib1: irq 16 at device 2.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 aac0: mem 0xdc000000-0xdfffffff irq 24 at device = 1.0 on pci2 aac0: Enable Raw I/O aac0: New comm. interface enabled aac0: [ITHREAD] aac0: Adaptec 2200S, aac driver 2.1.9-1 aacp0: on aac0 aacp1: on aac0 pcib3: at device 0.2 on pci1 pci3: on pcib3 em0: port 0x2000-0x203f= mem 0xd8200000-0xd821ffff irq 54 at device 2.0 on pci3 em0: [FILTER] em0: Ethernet address: 00:30:48:2d:32:6a em1: port 0x2040-0x207f= mem 0xd8220000-0xd823ffff irq 55 at device 2.1 on pci3 em1: [FILTER] em1: Ethernet address: 00:30:48:2d:32:6b pcib4: irq 16 at device 4.0 on pci0 pci4: on pcib4 pcib5: irq 16 at device 6.0 on pci0 pci5: on pcib5 uhci0: port 0x1400-0x141f irq 1= 6 at device 29.0 on pci0 uhci0: [ITHREAD] usbus0: on uhci0 uhci1: port 0x1420-0x143f irq 1= 9 at device 29.1 on pci0 uhci1: [ITHREAD] usbus1: on uhci1 uhci2: port 0x1440-0x145f irq 1= 8 at device 29.2 on pci0 uhci2: [ITHREAD] usbus2: on uhci2 uhci3: port 0x1460-0x147f irq 1= 6 at device 29.3 on pci0 uhci3: [ITHREAD] usbus3: on uhci3 ehci0: mem 0xd8001000-0xd80013f= f irq 23 at device 29.7 on pci0 ehci0: [ITHREAD] usbus4: EHCI version 1.0 usbus4: on ehci0 pcib6: at device 30.0 on pci0 pci6: on pcib6 vgapci0: port 0x3000-0x30ff mem 0xd9000000-0xd9fff= fff,0xd8300000-0xd8300fff irq 17 at device 1.0 on pci6 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177= ,0x376,0x14a0-0x14af at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atrtc0: port 0x70-0x77 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart0: console (9600,n,8,1) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xc9000-0x= c9fff,0xca000-0xcafff,0xcb000-0xcf7ff pnpid ORM0000 on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: parallel port not found. est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 122d00000e24 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 122d00000e24 device_attach: est1 attach returned 6 p4tcc1: on cpu1 Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 480Mbps High Speed USB v2.0 ata1: DMA limited to UDMA33, controller found non-ATA66 cable ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 acd0: DVDROM at ata1-slave UDMA33=20 aacd0: on aac0 aacd0: 34970MB (71619584 sectors) aacd1: on aac0 aacd1: 69974MB (143307008 sectors) uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 8 ports with 8 removable, self powered ses0 at aacp0 bus 0 scbus0 target 6 lun 0 ses0: Fixed Uninstalled SCSI-2 device=20 ses0: 3.300MB/s transfers ses0: SAF-TE Compliant Device pass0 at aacp0 bus 0 scbus0 target 0 lun 0 pass0: Fixed Uninstalled SCSI-3 device=20 pass0: 3.300MB/s transfers pass1 at aacp0 bus 0 scbus0 target 1 lun 0 pass1: Fixed Uninstalled SCSI-3 device=20 pass1: 3.300MB/s transfers pass2 at aacp0 bus 0 scbus0 target 2 lun 0 pass2: Fixed Uninstalled SCSI-3 device=20 pass2: 3.300MB/s transfers pass3 at aacp0 bus 0 scbus0 target 3 lun 0 pass3: Fixed Uninstalled SCSI-3 device=20 pass3: 3.300MB/s transfers SMP: AP CPU #1 Launched! WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/aacd0s4a Setting hostuuid: 80f1e964-dc63-0010-89d5-0030482d326a. Setting hostid: 0xc74551dd. Entropy harvesting: interrupts ethernet point_to_point kickstart. Starting file system checks: /dev/aacd0s4a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4a: clean, 422048 free (1216 frags, 52604 blocks, 0.2% fragmenta= tion) /dev/aacd1s1e: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd1s1e: clean, 4681662 free (261046 frags, 552577 blocks, 2.0% fragm= entation) /dev/aacd0s1a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s1a: clean, 424969 free (1913 frags, 52882 blocks, 0.3% fragmenta= tion) /dev/aacd0s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s1d: clean, 1031838 free (4190 frags, 128456 blocks, 0.2% fragmen= tation) /dev/aacd1s1d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd1s1d: clean, 11717829 free (197893 frags, 1439992 blocks, 1.2% fra= gmentation) /dev/aacd0s2a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s2a: clean, 596256 free (1488 frags, 74346 blocks, 0.2% fragmenta= tion) /dev/aacd0s2d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s2d: clean, 1108521 free (23105 frags, 135677 blocks, 1.3% fragme= ntation) /dev/aacd0s3a: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s3a: clean, 578034 free (1146 frags, 72111 blocks, 0.2% fragmenta= tion) /dev/aacd0s3d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s3d: clean, 1070844 free (39380 frags, 128933 blocks, 2.2% fragme= ntation) /dev/aacd0s4d: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4d: clean, 1075950 free (60166 frags, 126973 blocks, 3.4% fragme= ntation) /dev/aacd0s4f: FILE SYSTEM CLEAN; SKIPPING CHECKS /dev/aacd0s4f: clean, 2106304 free (344 frags, 263245 blocks, 0.0% fragment= ation) Mounting local file systems:. Setting hostname: freebeast.catwhisker.org. lock order reversal: 1st 0xc683e7c8 ufs (ufs) @ /usr/src/sys/kern/vfs_lookup.c:502 2nd 0xd94fe700 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:11363 3rd 0xc6871168 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2091 KDB: stack backtrace: db_trace_self_wrapper(c0cc72a5,ebdf2300,c08e9f65,c08da2db,c0cca298,...) at = db_trace_self_wrapper+0x26 kdb_backtrace(c08da2db,c0cca298,c553d030,c5540568,ebdf235c,...) at kdb_back= trace+0x29 _witness_debugger(c0cca298,c6871168,c0cbc5bd,c5540568,c0cd13b1,...) at _wit= ness_debugger+0x25 witness_checkorder(c6871168,9,c0cd13b1,82b,0,...) at witness_checkorder+0x8= 39 __lockmgr_args(c6871168,80100,c6871188,0,0,...) at __lockmgr_args+0x7f9 ffs_lock(ebdf2480,c08e9d0b,c0cd0856,80100,c6871110,...) at ffs_lock+0x8a VOP_LOCK1_APV(c0dd4ee0,ebdf2480,c63cecd4,c0def920,c6871110,...) at VOP_LOCK= 1_APV+0xb5 _vn_lock(c6871110,80100,c0cd13b1,82b,4,...) at _vn_lock+0x5e vget(c6871110,80100,c63cec30,50,0,...) at vget+0xb9 vfs_hash_get(c6857ca8,78c22,80000,c63cec30,ebdf25d0,...) at vfs_hash_get+0x= e6 ffs_vgetf(c6857ca8,78c22,80000,ebdf25d0,1,...) at ffs_vgetf+0x49 softdep_sync_metadata(c683e770,0,c0ced5f8,144,0,...) at softdep_sync_metada= ta+0xc92 ffs_syncvnode(c683e770,1,c63cec30,ebdf2690,246,...) at ffs_syncvnode+0x3e2 ffs_truncate(c683e770,200,0,880,c5588480,...) at ffs_truncate+0x862 ufs_direnter(c683e770,c6871110,ebdf2944,ebdf2bd4,0,...) at ufs_direnter+0x8= d4 ufs_makeinode(ebdf2bd4,0,ebdf2b30,ebdf2a8c,c0c032e5,...) at ufs_makeinode+0= x557 ufs_create(ebdf2b30,ebdf2b48,0,0,ebdf2ba8,...) at ufs_create+0x30 VOP_CREATE_APV(c0dd4ee0,ebdf2b30,ebdf2bd4,ebdf2ac8,0,...) at VOP_CREATE_APV= +0xa5 vn_open_cred(ebdf2ba8,ebdf2c5c,1a4,0,c5588480,...) at vn_open_cred+0x215 vn_open(ebdf2ba8,ebdf2c5c,1a4,c637ee70,0,...) at vn_open+0x3b kern_openat(c63cec30,ffffff9c,804c5e8,0,602,...) at kern_openat+0x125 kern_open(c63cec30,804c5e8,0,601,21b6,...) at kern_open+0x35 open(c63cec30,ebdf2cf8,c0cffdfb,c0caa6d4,c63c9d48,...) at open+0x30 syscall(ebdf2d38) at syscall+0x220 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip =3D 0x281744b3, esp =3D 0xbfbfec4= c, ebp =3D 0xbfbfecb8 --- Starting Network: lo0 em0 em1. lo0: flags=3D8049 metric 0 mtu 16384 options=3D3 inet 127.0.0.1 netmask 0xff000000=20 inet6 ::1 prefixlen 128=20 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3=20 nd6 options=3D21 em0: flags=3D8843 metric 0 mtu 1500 options=3D389b ether 00:30:48:2d:32:6a inet 172.16.8.10 netmask 0xffffff00 broadcast 172.16.8.255 inet6 fe80::230:48ff:fe2d:326a%em0 prefixlen 64 tentative scopeid 0= x1=20 nd6 options=3D29 media: Ethernet autoselect status: no carrier em1: flags=3D8802 metric 0 mtu 1500 options=3D389b ether 00:30:48:2d:32:6b media: Ethernet autoselect status: no carrier Starting devd. Starting Network: em1. em1: flags=3D8802 metric 0 mtu 1500 options=3D389b ether 00:30:48:2d:32:6b media: Ethernet autoselect status: no carrier add net default: gateway 172.16.8.1 add net ::ffff:0.0.0.0: gateway ::1 add net ::0.0.0.0: gateway ::1 add net fe80::: gateway ::1 add net ff02::: gateway ::1 ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/= lib/compat a.out ldconfig path: /usr/lib/aout /usr/lKernel page fault with the followi= ng non-sleepable locks held: exclusive sleep mutex mld_mtx (mld_mtx) r =3D 0 (0xc0f96468) locked @ /usr/= src/sys/netinet6/mld6.c:1352 exclusive sleep mutex in6_multi_mtx (in6_multi_mtx) r =3D 0 (0xc0f95408) lo= cked @ /usr/src/sys/netinet6/mld6.c:1351 KDB: stack backtrace: db_trace_self_wrapper(c0cc72a5,c5213864,c08e9f65,c0ce3e07,547,...) at db_tr= ace_self_wrapper+0x26 kdb_backtrace(c0ce3e07,547,ffffffff,c0f65a1c,c521389c,...) at kdb_backtrace= +0x29 _witness_debugger(c0cc982b,c52138b0,4,1,0,...) at _witness_debugger+0x25 witness_warn(5,0,c0cffec8,c55810a4,c557f7f8,...) at witness_warn+0x1fd trap(c521393c) at trap+0x19e calltrap() at calltrap+0x6 --- trap 0xc, eip =3D 0xc0a4d340, esp =3D 0xc521397c, ebp =3D 0xc5213b14 --- ip6_output(c5a4fc00,c0f964a0,0,1,c5213b44,...) at ip6_output+0x840 mld_dispatch_packet(0,0,c5213c1c,c0a509a7,c56fca2c,...) at mld_dispatch_pac= ket+0x30d mld_dispatch_queue(c56fca2c,0,c0ce3e07,591,c0cbb984,...) at mld_dispatch_qu= eue+0x38 mld_fasttimo(c5213c48,c0904cc9,c5213c48,c0893aa4,c0e24f80,...) at mld_fastt= imo+0x747 icmp6_fasttimo(c5213c48,c0893aa4,c0e24f80,4,c0cc24d6,...) at icmp6_fasttimo= +0x8 pffasttimo(0,0,c0cc5113,189,c0e24fb8,...) at pffasttimo+0x29 softclock(c0e24f80,c5213cc8,c0893aa4,c0e28940,c5588838,...) at softclock+0x= 24a intr_event_execute_handlers(c557f7f8,c5588800,c0cbf33f,533,c5588870,...) at= intr_event_execute_handlers+0x125 ithread_loop(c557e120,c5213d38,c0cbf0b8,343,c557f7f8,...) at ithread_loop+0= x9f fork_exit(c087c780,c557e120,c5213d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip =3D 0, esp =3D 0xc5213d70, ebp =3D 0 --- Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x3c fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0xc0a4d340 stack pointer =3D 0x28:0xc521397c frame pointer =3D 0x28:0xc5213b14 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 12 (swi4: clock) [ thread pid 12 tid 100007 ] Stopped at ip6_output+0x840: movl 0x3c(%eax),%eax db> bt Tracing pid 12 tid 100007 td 0xc5581000 ip6_output(c5a4fc00,c0f964a0,0,1,c5213b44,...) at ip6_output+0x840 mld_dispatch_packet(0,0,c5213c1c,c0a509a7,c56fca2c,...) at mld_dispatch_pac= ket+0x30d mld_dispatch_queue(c56fca2c,0,c0ce3e07,591,c0cbb984,...) at mld_dispatch_qu= eue+0x38 mld_fasttimo(c5213c48,c0904cc9,c5213c48,c0893aa4,c0e24f80,...) at mld_fastt= imo+0x747 icmp6_fasttimo(c5213c48,c0893aa4,c0e24f80,4,c0cc24d6,...) at icmp6_fasttimo= +0x8 pffasttimo(0,0,c0cc5113,189,c0e24fb8,...) at pffasttimo+0x29 softclock(c0e24f80,c5213cc8,c0893aa4,c0e28940,c5588838,...) at softclock+0x= 24a intr_event_execute_handlers(c557f7f8,c5588800,c0cbf33f,533,c5588870,...) at= intr_event_execute_handlers+0x125 ithread_loop(c557e120,c5213d38,c0cbf0b8,343,c557f7f8,...) at ithread_loop+0= x9f fork_exit(c087c780,c557e120,c5213d38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip =3D 0, esp =3D 0xc5213d70, ebp =3D 0 --- db>=20 Again, I can poke at it a bit if additional information might be useful. On a positive note, when I powered it off earlier, I had (only) disabled wol_mcast, so I was able to power it back up remotely by telling my firewall machine to send magic packets to it. :-} Peace, david --=20 David H. Wolfskill david@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --KBQi/TxAuyHKotC8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iEYEARECAAYFAkvog/UACgkQmprOCmdXAD0GQQCdERNo/Mmb6+odE7b+jBFuLIxZ BbUAniAf9AI5qKJNX8+EOqUybfCUX8UP =VQZQ -----END PGP SIGNATURE----- --KBQi/TxAuyHKotC8--