Date: Tue, 26 May 2009 11:01:20 +0800 From: "C. C. Tang" <hiyorin@gmail.com> To: freebsd-hackers@freebsd.org Subject: Random(?) Fatal trap 12 in Freebsd 7.2 release Message-ID: <4A1B5B80.1070207@gmail.com>
next in thread | raw e-mail | index | archive | help
Sorry for my bad english.
I am having a FreeBSD with Atom processor.
It was running for 3 months without any panic when it is 7.1 release.
After I upgraded it to 7.2 release weeks ago, it seems fine but it
starts to have panic several days ago.
The machine will sometimes panic without a heavy load(at least it is not
in its peak load but usually it will panic at the time everybody is
using it.)
I would be grateful if anyone can help me to figure out what is the real
problem...
uname -a:
FreeBSD xxxx 7.2-RELEASE FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC
2009 root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
dmesg:
Copyright (c) 1992-2009 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.2-RELEASE #0: Fri May 1 08:49:13 UTC 2009
root@walker.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1618.46-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x106c2 Stepping = 2
Features=0xbfe9fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,<b22>>
AMD Features=0x20100000<NX,LM>
AMD Features2=0x1<LAHF>
Cores per package: 2
Logical CPUs per core: 2
real memory = 2137391104 (2038 MB)
avail memory = 2081775616 (1985 MB)
ACPI APIC Table: <INTEL D945GLF2>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
cpu0 (BSP): APIC ID: 0
cpu1 (AP/HT): APIC ID: 1
cpu2 (AP): APIC ID: 2
cpu3 (AP/HT): APIC ID: 3
ioapic0: Changing APIC ID to 2
ioapic0 <Version 2.0> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <INTEL D945GLF2> on motherboard
acpi0: [ITHREAD]
acpi0: Power Button (fixed)
Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
acpi_hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on
acpi0
Timecounter "HPET" frequency 14318180 Hz quality 900
acpi_button0: <Sleep Button> on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
vgapci0: <VGA-compatible display> port 0x30e0-0x30e7 mem
0x90300000-0x9037ffff,0x80000000-0x8fffffff,0x90380000-0x903bffff irq 16
at device 2.0 on pci0
agp0: <Intel 82945G (945G GMCH) SVGA controller> on vgapci0
agp0: detected 7932k stolen memory
agp0: aperture size is 256M
pci0: <multimedia, HDA> at device 27.0 (no driver attached)
pcib1: <ACPI PCI-PCI bridge> at device 28.0 on pci0
pci1: <ACPI PCI bus> on pcib1
re0: <RealTek 8168/8168B/8168C/8168CP/8168D/8111B/8111C/8111CP PCIe
Gigabit Ethernet> port 0x2000-0x20ff mem
0x90200000-0x90200fff,0x90000000-0x9000ffff irq 16 at device 0.0 on pci1
re0: Using 1 MSI messages
re0: Chip rev. 0x3c000000
re0: MAC rev. 0x00400000
miibus0: <MII bus> on re0
rgephy0: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus0
rgephy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
re0: Ethernet address: 00:1c:c0:a8:f3:73
re0: [FILTER]
pcib2: <ACPI PCI-PCI bridge> at device 28.2 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> at device 28.3 on pci0
pci3: <ACPI PCI bus> on pcib3
uhci0: <UHCI (generic) USB controller> port 0x3080-0x309f irq 23 at
device 29.0 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <UHCI (generic) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
uhci1: <UHCI (generic) USB controller> port 0x3060-0x307f irq 19 at
device 29.1 on pci0
uhci1: [GIANT-LOCKED]
uhci1: [ITHREAD]
usb1: <UHCI (generic) USB controller> on uhci1
usb1: USB revision 1.0
uhub1: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb1
uhub1: 2 ports with 2 removable, self powered
uhci2: <UHCI (generic) USB controller> port 0x3040-0x305f irq 18 at
device 29.2 on pci0
uhci2: [GIANT-LOCKED]
uhci2: [ITHREAD]
usb2: <UHCI (generic) USB controller> on uhci2
usb2: USB revision 1.0
uhub2: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb2
uhub2: 2 ports with 2 removable, self powered
uhci3: <UHCI (generic) USB controller> port 0x3020-0x303f irq 16 at
device 29.3 on pci0
uhci3: [GIANT-LOCKED]
uhci3: [ITHREAD]
usb3: <UHCI (generic) USB controller> on uhci3
usb3: USB revision 1.0
uhub3: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb3
uhub3: 2 ports with 2 removable, self powered
ehci0: <Intel 82801GB/R (ICH7) USB 2.0 controller> mem
0x903c4000-0x903c43ff irq 23 at device 29.7 on pci0
ehci0: [GIANT-LOCKED]
ehci0: [ITHREAD]
usb4: EHCI version 1.0
usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3
usb4: <Intel 82801GB/R (ICH7) USB 2.0 controller> on ehci0
usb4: USB revision 2.0
uhub4: <Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1> on usb4
uhub4: 8 ports with 8 removable, self powered
pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib4
re1: <RealTek 8169/8169S/8169SB(L)/8110S/8110SB(L) Gigabit Ethernet>
port 0x1000-0x10ff mem 0x90100000-0x901000ff irq 21 at device 0.0 on pci4
re1: Chip rev. 0x04000000
re1: MAC rev. 0x00000000
miibus1: <MII bus> on re1
rgephy1: <RTL8169S/8110S/8211B media interface> PHY 1 on miibus1
rgephy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
1000baseT-FDX, auto
re1: Ethernet address: 00:16:01:5c:4a:0f
re1: [FILTER]
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel ICH7 UDMA100 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30b0-0x30bf irq 18 at device 31.1
on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
atapci1: <Intel ICH7 SATA300 controller> port
0x30c8-0x30cf,0x30ec-0x30ef,0x30c0-0x30c7,0x30e8-0x30eb,0x30a0-0x30af
irq 19 at device 31.2 on pci0
atapci1: [ITHREAD]
ata2: <ATA channel 0> on atapci1
ata2: [ITHREAD]
ata3: <ATA channel 1> on atapci1
ata3: [ITHREAD]
pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on
acpi0
sio0: type 16550A
sio0: [FILTER]
cpu0: <ACPI CPU> on acpi0
p4tcc0: <CPU Frequency Thermal Control> on cpu0
cpu1: <ACPI CPU> on acpi0
p4tcc1: <CPU Frequency Thermal Control> on cpu1
cpu2: <ACPI CPU> on acpi0
p4tcc2: <CPU Frequency Thermal Control> on cpu2
cpu3: <ACPI CPU> on acpi0
p4tcc3: <CPU Frequency Thermal Control> on cpu3
pmtimer0 on isa0
orm0: <ISA Option ROM> at iomem 0xcb000-0xcbfff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
ppbus0: [ITHREAD]
plip0: <PLIP network interface> on ppbus0
plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio1: configured irq 3 not in bitmap of probed irqs 0
sio1: port may not be enabled
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
Timecounters tick every 1.000 msec
ad4: 476940MB <Seagate ST3500320NS SN06> at ata2-master SATA150
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529.
GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34.
GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18.
GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76.
GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a.
Trying to mount root from ufs:/dev/ad4s1a
GEOM_LABEL: Label ufsid/497231ebaad35c34 removed.
GEOM_LABEL: Label for provider ad4s1a is ufsid/497231ebaad35c34.
GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed.
GEOM_LABEL: Label for provider ad4s1e is ufsid/497231eb4cb04b76.
GEOM_LABEL: Label ufsid/497231eb16db295a removed.
GEOM_LABEL: Label for provider ad4s1f is ufsid/497231eb16db295a.
GEOM_LABEL: Label ufsid/497231edae4da529 removed.
GEOM_LABEL: Label for provider ad4s2 is ufsid/497231edae4da529.
GEOM_LABEL: Label ufsid/497231f823755a18 removed.
GEOM_LABEL: Label for provider ad4s1d is ufsid/497231f823755a18.
GEOM_LABEL: Label ufsid/497231ebaad35c34 removed.
GEOM_LABEL: Label ufsid/497231eb4cb04b76 removed.
GEOM_LABEL: Label ufsid/497231eb16db295a removed.
GEOM_LABEL: Label ufsid/497231edae4da529 removed.
GEOM_LABEL: Label ufsid/497231f823755a18 removed.
tap0: Ethernet address: 00:bd:9c:3f:00:00
re0: link state changed to DOWN
re0: link state changed to UP
The following is the info obtained by running crashinfo:
### instance 1 (May 21 about 23:00 local time)
Fatal trap 12: page fault while in kernel mode
cpuid = 2; apic id = 02
fault virtual address = 0x19
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc085af6c
stack pointer = 0x28:0xe5a84bfc
frame pointer = 0x28:0xe5a84c18
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 46 (syncer)
trap number = 12
panic: page fault
cpuid = 2
Uptime: 11d9h43m49s
Physical memory: 2025 MB
Dumping 294 MB: 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55
39 23 7
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:196
#1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc07e2879 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3 0xc0ae3ebc in trap_fatal (frame=0xe5a84bbc, eva=25)
at /usr/src/sys/i386/i386/trap.c:939
#4 0xc0ae4140 in trap_pfault (frame=0xe5a84bbc, usermode=0, eva=25)
at /usr/src/sys/i386/i386/trap.c:852
#5 0xc0ae4aec in trap (frame=0xe5a84bbc) at
/usr/src/sys/i386/i386/trap.c:530
#6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7 0xc085af6c in __mnt_vnode_next (mvp=0xe5a84c48, mp=0xc5874b40)
at /usr/src/sys/kern/vfs_mount.c:2033
#8 0xc0864202 in vfs_msync (mp=0xc5874b40, flags=2)
at /usr/src/sys/kern/vfs_subr.c:3135
#9 0xc08644ba in sync_fsync (ap=0xe5a84cd4)
at /usr/src/sys/kern/vfs_subr.c:3389
#10 0xc0af8e72 in VOP_FSYNC_APV (vop=0xc0c598a0, a=0xe5a84cd4)
at vnode_if.c:1007
#11 0xc0864cc8 in sched_sync () at vnode_if.h:538
#12 0xc07bd059 in fork_exit (callout=0xc08645c0 <sched_sync>, arg=0x0,
frame=0xe5a84d38) at /usr/src/sys/kern/kern_fork.c:810
#13 0xc0ac9270 in fork_trampoline () at
/usr/src/sys/i386/i386/exception.s:264
(kgdb)
### instance 2 (May 22 about 20:30 local time)
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0xf3e83561
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc0880d65
stack pointer = 0x28:0xc53f7be0
frame pointer = 0x28:0xc53f7c00
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 19 (swi5: +)
trap number = 12
panic: page fault
cpuid = 0
Uptime: 21h4m54s
Physical memory: 2025 MB
Dumping 291 MB: 276 260 244 228 212 196 180 164 148 132 116 100 84 68 52
36 20 4
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:196
#1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc07e2879 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3 0xc0ae3ebc in trap_fatal (frame=0xc53f7ba0, eva=4092081505)
at /usr/src/sys/i386/i386/trap.c:939
#4 0xc0ae4140 in trap_pfault (frame=0xc53f7ba0, usermode=0, eva=4092081505)
at /usr/src/sys/i386/i386/trap.c:852
#5 0xc0ae4aec in trap (frame=0xc53f7ba0) at
/usr/src/sys/i386/i386/trap.c:530
#6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7 0xc0880d65 in ether_input (ifp=0xc5601800, m=0xc5a2c200)
at /usr/src/sys/net/if_ethersubr.c:545
#8 0xc06c4348 in re_rxeof (sc=0xc561b000) at
/usr/src/sys/dev/re/if_re.c:1982
#9 0xc06c5dce in re_int_task (arg=0xc561b000, npending=1)
at /usr/src/sys/dev/re/if_re.c:2189
#10 0xc0817835 in taskqueue_run (queue=0xc5498400)
at /usr/src/sys/kern/subr_taskqueue.c:282
#11 0xc0817973 in taskqueue_fast_run (dummy=0x0)
at /usr/src/sys/kern/subr_taskqueue.c:460
#12 0xc07c050b in ithread_loop (arg=0xc55830b0)
at /usr/src/sys/kern/kern_intr.c:1088
#13 0xc07bd059 in fork_exit (callout=0xc07c0350 <ithread_loop>,
arg=0xc55830b0, frame=0xc53f7d38) at /usr/src/sys/kern/kern_fork.c:810
#14 0xc0ac9270 in fork_trampoline () at
/usr/src/sys/i386/i386/exception.s:264
(kgdb)
### instance 3 (May 22 about 20:40 local time)
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x200
fault code = supervisor read, page not present
instruction pointer = 0x20:0x200
stack pointer = 0x28:0xc53ed8f0
frame pointer = 0x28:0xc53ed910
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 4 (g_down)
trap number = 12
(bt is not available since the core dump seems lost. This panic occured
immediately after rebooted from instance 2..)
### instance 4 (May 25 about 22:40 local time)
Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address = 0x3b57860
fault code = supervisor read, page not present
instruction pointer = 0x20:0xc0a1cfb6
stack pointer = 0x28:0xe7d00a94
frame pointer = 0x28:0xe7d00ab0
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 62673 (httpd)
trap number = 12
panic: page fault
cpuid = 1
Uptime: 3d1h54m6s
Physical memory: 2025 MB
Dumping 275 MB: 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4
#0 doadump () at pcpu.h:196
196 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:196
#1 0xc07e25a7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2 0xc07e2879 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3 0xc0ae3ebc in trap_fatal (frame=0xe7d00a54, eva=62224480)
at /usr/src/sys/i386/i386/trap.c:939
#4 0xc0ae4140 in trap_pfault (frame=0xe7d00a54, usermode=0, eva=62224480)
at /usr/src/sys/i386/i386/trap.c:852
#5 0xc0ae4aec in trap (frame=0xe7d00a54) at
/usr/src/sys/i386/i386/trap.c:530
#6 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7 0xc0a1cfb6 in vm_map_entry_splay (addr=685764608, root=0xc60ac044)
at /usr/src/sys/vm/vm_map.c:726
#8 0xc0a1d367 in vm_map_lookup_entry (map=0xc5d0fa24, address=685764608,
entry=0xe7d00bf8) at /usr/src/sys/vm/vm_map.c:904
#9 0xc0a1e76a in vm_map_lookup (var_map=0xe7d00bf4, vaddr=685764608,
fault_typea=Variable "fault_typea" is not available.
) at /usr/src/sys/vm/vm_map.c:3136
#10 0xc0a17f03 in vm_fault (map=0xc5d0fa24, vaddr=685764608,
fault_type=2 '\002', fault_flags=8) at /usr/src/sys/vm/vm_fault.c:233
#11 0xc0ae402b in trap_pfault (frame=0xe7d00d38, usermode=1, eva=685767244)
at /usr/src/sys/i386/i386/trap.c:829
#12 0xc0ae4977 in trap (frame=0xe7d00d38) at
/usr/src/sys/i386/i386/trap.c:397
#13 0xc0ac91fb in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#14 0x2809997e in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4A1B5B80.1070207>