From owner-freebsd-hackers@FreeBSD.ORG Tue May 26 03:33:34 2009 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0514F1065673 for ; Tue, 26 May 2009 03:33:34 +0000 (UTC) (envelope-from hiyorin@gmail.com) Received: from mail-pz0-f105.google.com (mail-pz0-f105.google.com [209.85.222.105]) by mx1.freebsd.org (Postfix) with ESMTP id CB4408FC17 for ; Tue, 26 May 2009 03:33:33 +0000 (UTC) (envelope-from hiyorin@gmail.com) Received: by pzk3 with SMTP id 3so3061310pzk.3 for ; Mon, 25 May 2009 20:33:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from :user-agent:mime-version:to:subject:content-type :content-transfer-encoding; bh=9xwijdRBIk3d8uBjLrh1VJe44b1hq1ZFHHEDXaiVPtc=; b=NsWHPx6kmipChyxkga2yrnN7VImSPM1KAQNXolPkj2nfA7dKcFYcdZGTLJX1hCxoJu lWi9Sek8ZYFYVFipod+ROqSJWliRL/C+RM/58h265YM1OZjG+giO5uPJt6j4Nj3ZY5zJ lpek/n66eC0ISDqPRVI9+9Z9oOYtxYg1dlQnE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; b=chX1bkua6J+fmzLBI5ixnrGsmE9IAsIAjeARKwtwWZ1wbKwcfG2S37TLTPecixak9i Xhqnr4SScyUl1SNSYMDm0vr/jaUHzID9ZC+gTYEbYMLto0ZdAXJkqMUwDK9gVSZQd9w2 WY8cjTwtgii/4lg7110P2yBAjlEDyYju8UxXQ= Received: by 10.114.134.1 with SMTP id h1mr16409470wad.47.1243306883879; Mon, 25 May 2009 20:01:23 -0700 (PDT) Received: from ?10.130.10.181? ([202.82.159.125]) by mx.google.com with ESMTPS id b39sm12201091rvf.1.2009.05.25.20.01.22 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 25 May 2009 20:01:23 -0700 (PDT) Message-ID: <4A1B5B80.1070207@gmail.com> Date: Tue, 26 May 2009 11:01:20 +0800 From: "C. C. Tang" User-Agent: Thunderbird 2.0.0.21 (Windows/20090302) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Random(?) Fatal trap 12 in Freebsd 7.2 release X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 May 2009 03:33:34 -0000 Sorry for my bad english. I am having a FreeBSD with Atom processor. It was running for 3 months without any panic when it is 7.1 release. After I upgraded it to 7.2 release weeks ago, it seems fine but it starts to have panic several days ago. The machine will sometimes panic without a heavy load(at least it is not in its peak load but usually it will panic at the time everybody is using it.) I would be grateful if anyone can help me to figure out what is the real problem... uname -a: FreeBSD xxxx 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC 2009 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386 dmesg: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC 2009 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1618.46-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Stepping = 2 Features=0xbfe9fbff Features2=0x40e31d> AMD Features=0x20100000 AMD Features2=0x1 Cores per package: 2 Logical CPUs per core: 2 real memory = 2137391104 (2038 MB) avail memory = 2081775616 (1985 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP/HT): APIC ID: 3 ioapic0: Changing APIC ID to 2 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 vgapci0: port 0x30e0-0x30e7 mem 0x90300000-0x9037ffff,0x80000000-0x8fffffff,0x90380000-0x903bffff irq 16 at device 2.0 on pci0 agp0: on vgapci0 agp0: detected 7932k stolen memory agp0: aperture size is 256M pci0: at device 27.0 (no driver attached) pcib1: at device 28.0 on pci0 pci1: on pcib1 re0: port 0x2000-0x20ff mem 0x90200000-0x90200fff,0x90000000-0x9000ffff irq 16 at device 0.0 on pci1 re0: Using 1 MSI messages re0: Chip rev. 0x3c000000 re0: MAC rev. 0x00400000 miibus0: on re0 rgephy0: PHY 1 on miibus0 rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re0: Ethernet address: 00:1c:c0:a8:f3:73 re0: [FILTER] pcib2: at device 28.2 on pci0 pci2: on pcib2 pcib3: at device 28.3 on pci0 pci3: on pcib3 uhci0: port 0x3080-0x309f irq 23 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x3060-0x307f irq 19 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x3040-0x305f irq 18 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x3020-0x303f irq 16 at device 29.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0x903c4000-0x903c43ff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered pcib4: at device 30.0 on pci0 pci4: on pcib4 re1: port 0x1000-0x10ff mem 0x90100000-0x901000ff irq 21 at device 0.0 on pci4 re1: Chip rev. 0x04000000 re1: MAC rev. 0x00000000 miibus1: on re1 rgephy1: PHY 1 on miibus1 rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto re1: Ethernet address: 00:16:01:5c:4a:0f re1: [FILTER] isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30b0-0x30bf irq 18 at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] atapci1: port 0x30c8-0x30cf,0x30ec-0x30ef,0x30c0-0x30c7,0x30e8-0x30eb,0x30a0-0x30af irq 19 at device 31.2 on pci0 atapci1: [ITHREAD] ata2: on atapci1 ata2: [ITHREAD] ata3: on atapci1 ata3: [ITHREAD] pci0: at device 31.3 (no driver attached) sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] cpu0: on acpi0 p4tcc0: on cpu0 cpu1: on acpi0 p4tcc1: on cpu1 cpu2: on acpi0 p4tcc2: on cpu2 cpu3: on acpi0 p4tcc3: on cpu3 pmtimer0 on isa0 orm0: at iomem 0xcb000-0xcbfff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [ITHREAD] plip0: on ppbus0 plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec ad4: 476940MB at ata2-master SATA150 SMP: AP CPU #1 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529. GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34. GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18. GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76. GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a. Trying to mount root from ufs:/dev/ad4s1a GEOM_LABEL: Label ufsid/497231ebaad35c34 removed. GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34. GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed. GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76. GEOM_LABEL: Label ufsid/497231eb16db295a removed. GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a. GEOM_LABEL: Label ufsid/497231edae4da529 removed. GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529. GEOM_LABEL: Label ufsid/497231f823755a18 removed. GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18. GEOM_LABEL: Label ufsid/497231ebaad35c34 removed. GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed. GEOM_LABEL: Label ufsid/497231eb16db295a removed. GEOM_LABEL: Label ufsid/497231edae4da529 removed. GEOM_LABEL: Label ufsid/497231f823755a18 removed. tap0: Ethernet address: 00:bd:9c:3f:00:00 re0: link state changed to DOWN re0: link state changed to UP The following is the info obtained by running crashinfo: ### instance 1 (May 21 about 23:00 local time) Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 02 fault virtual address = 0x19 fault code = supervisor write, page not present instruction pointer = 0x20:0xc085af6c stack pointer = 0x28:0xe5a84bfc frame pointer = 0x28:0xe5a84c18 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 46 (syncer) trap number = 12 panic: page fault cpuid = 2 Uptime: 11d9h43m49s Physical memory: 2025 MB Dumping 294 MB: 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xe5a84bbc, eva=25) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xe5a84bbc, usermode=0, eva=25) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xe5a84bbc) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc085af6c in __mnt_vnode_next (mvp=0xe5a84c48, mp=0xc5874b40) at /usr/src/sys/kern/vfs_mount.c:2033 #8 0xc0864202 in vfs_msync (mp=0xc5874b40, flags=2) at /usr/src/sys/kern/vfs_subr.c:3135 #9 0xc08644ba in sync_fsync (ap=0xe5a84cd4) at /usr/src/sys/kern/vfs_subr.c:3389 #10 0xc0af8e72 in VOP_FSYNC_APV (vop=0xc0c598a0, a=0xe5a84cd4) at vnode_if.c:1007 #11 0xc0864cc8 in sched_sync () at vnode_if.h:538 #12 0xc07bd059 in fork_exit (callout=0xc08645c0 , arg=0x0, frame=0xe5a84d38) at /usr/src/sys/kern/kern_fork.c:810 #13 0xc0ac9270 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 (kgdb) ### instance 2 (May 22 about 20:30 local time) Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0xf3e83561 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0880d65 stack pointer = 0x28:0xc53f7be0 frame pointer = 0x28:0xc53f7c00 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 19 (swi5: +) trap number = 12 panic: page fault cpuid = 0 Uptime: 21h4m54s Physical memory: 2025 MB Dumping 291 MB: 276 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xc53f7ba0, eva=4092081505) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xc53f7ba0, usermode=0, eva=4092081505) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xc53f7ba0) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc0880d65 in ether_input (ifp=0xc5601800, m=0xc5a2c200) at /usr/src/sys/net/if_ethersubr.c:545 #8 0xc06c4348 in re_rxeof (sc=0xc561b000) at /usr/src/sys/dev/re/if_re.c:1982 #9 0xc06c5dce in re_int_task (arg=0xc561b000, npending=1) at /usr/src/sys/dev/re/if_re.c:2189 #10 0xc0817835 in taskqueue_run (queue=0xc5498400) at /usr/src/sys/kern/subr_taskqueue.c:282 #11 0xc0817973 in taskqueue_fast_run (dummy=0x0) at /usr/src/sys/kern/subr_taskqueue.c:460 #12 0xc07c050b in ithread_loop (arg=0xc55830b0) at /usr/src/sys/kern/kern_intr.c:1088 #13 0xc07bd059 in fork_exit (callout=0xc07c0350 , arg=0xc55830b0, frame=0xc53f7d38) at /usr/src/sys/kern/kern_fork.c:810 #14 0xc0ac9270 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:264 (kgdb) ### instance 3 (May 22 about 20:40 local time) Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x200 fault code = supervisor read, page not present instruction pointer = 0x20:0x200 stack pointer = 0x28:0xc53ed8f0 frame pointer = 0x28:0xc53ed910 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 4 (g_down) trap number = 12 (bt is not available since the core dump seems lost. This panic occured immediately after rebooted from instance 2..) ### instance 4 (May 25 about 22:40 local time) Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x3b57860 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0a1cfb6 stack pointer = 0x28:0xe7d00a94 frame pointer = 0x28:0xe7d00ab0 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 62673 (httpd) trap number = 12 panic: page fault cpuid = 1 Uptime: 3d1h54m6s Physical memory: 2025 MB Dumping 275 MB: 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:196 196 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:196 #1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418 #2 0xc07e2879 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0xc0ae3ebc in trap_fatal (frame=0xe7d00a54, eva=62224480) at /usr/src/sys/i386/i386/trap.c:939 #4 0xc0ae4140 in trap_pfault (frame=0xe7d00a54, usermode=0, eva=62224480) at /usr/src/sys/i386/i386/trap.c:852 #5 0xc0ae4aec in trap (frame=0xe7d00a54) at /usr/src/sys/i386/i386/trap.c:530 #6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #7 0xc0a1cfb6 in vm_map_entry_splay (addr=685764608, root=0xc60ac044) at /usr/src/sys/vm/vm_map.c:726 #8 0xc0a1d367 in vm_map_lookup_entry (map=0xc5d0fa24, address=685764608, entry=0xe7d00bf8) at /usr/src/sys/vm/vm_map.c:904 #9 0xc0a1e76a in vm_map_lookup (var_map=0xe7d00bf4, vaddr=685764608, fault_typea=Variable "fault_typea" is not available. ) at /usr/src/sys/vm/vm_map.c:3136 #10 0xc0a17f03 in vm_fault (map=0xc5d0fa24, vaddr=685764608, fault_type=2 '\002', fault_flags=8) at /usr/src/sys/vm/vm_fault.c:233 #11 0xc0ae402b in trap_pfault (frame=0xe7d00d38, usermode=1, eva=685767244) at /usr/src/sys/i386/i386/trap.c:829 #12 0xc0ae4977 in trap (frame=0xe7d00d38) at /usr/src/sys/i386/i386/trap.c:397 #13 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159 #14 0x2809997e in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb)