Date: Fri, 21 Sep 2007 16:11:52 -0700 From: "Wil Hatfield" <freebsd@hyperconx.com> To: <freebsd-questions@freebsd.org> Subject: Kernel Panics in 6.1 and 6.2 always Exim 4 Message-ID: <NGBBKBOMKLFOJCCDHPEOCENEHDAB.freebsd@hyperconx.com> In-Reply-To: <1181757311.1161.42.camel@soundwave.pgh.priv.collaborativefusion.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Well after a year we still haven't tracked down the kernel panic problems that are occuring on both our 6.1 and 6.2 machines for those we have had time to upgrade. It occurs on 6.1-RC, 6.1-RELEASE 6.1-STABLE, 6.2, you name it. We are noticing that all of the dumps are during Exim 4.6x runtime. I am suspicious of PR-97095 but would like others insights into the possibility. #### References http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2006-06/msg00011.html http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2006-06/msg00621.html http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2006-06/msg00636.html Here is my ditty: #### uname -a FreeBSD machine1.ourdomain.net 6.1-RC FreeBSD 6.1-RC #1: Tue Apr 11 23:19:28 PDT 2006 root@machine1.ourdomain.net:/usr/obj/usr/src/sys/CUSTOM-KERNEL i386 #### dmesg Fatal trap 12: page fault while in kernel mode fault virtual address = 0x5c fault code = supervisor read, page not present instruction pointer = 0x20:0xc06ab64c stack pointer = 0x28:0xf7ab1b10 frame pointer = 0x28:0xf7ab1b2c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 17074 (exim) trap number = 12 panic: page fault Uptime: 4d5h40m36s Dumping 1015 MB (2 chunks) chunk 0: 1MB (160 pages) ... ok chunk 1: 1015MB (259840 pages) 1000 984 968 952 936 920 904 888 872 856 840 824 808 792 776 760 744 728 712 696 680 664 648 632 616 600 584 568 552 536 520 504 488 472 456 440 424 408 392 376 360 344 328 312 296 280 264 248 232 216 200 184 168 152 136 120 104 88 72 56 40 24 8Copyright (c) 1992-2006 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 6.1-RC #1: Tue Apr 11 23:19:28 PDT 2006 root@machine1.ourdomain.net:/usr/obj/usr/src/sys/CUSTOM-KERNEL Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Pentium(R) 4 CPU 2.00GHz (2000.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf24 Stepping = 4 Features=0x3febfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,M CA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM> real memory = 1065353216 (1016 MB) avail memory = 1032658944 (984 MB) MPTable: <OEM00000 PROD00000000> ioapic0: Assuming intbase of 0 ioapic0 <Version 2.0> irqs 0-23 on motherboard kbd1 at kbdmux0 npx0: [FAST] npx0: <math processor> on motherboard npx0: INT 16 interface cpu0 on motherboard pcib0: <MPTable Host-PCI bridge> pcibus 0 on motherboard pci0: <PCI bus> on pcib0 agp0: <Intel 82845G (845G GMCH) SVGA controller> mem 0xd0000000-0xd7ffffff,0xdc100000-0xdc17ffff irq 16 at device 2.0 on pci0 agp0: detected 8060k stolen memory agp0: aperture size is 128M pci0: <serial bus, USB> at device 29.0 (no driver attached) pci0: <serial bus, USB> at device 29.1 (no driver attached) pci0: <serial bus, USB> at device 29.2 (no driver attached) pci0: <serial bus, USB> at device 29.7 (no driver attached) pcib1: <MPTable PCI-PCI bridge> at device 30.0 on pci0 pci1: <PCI bus> on pcib1 fxp0: <Intel 82801DB (ICH4) Pro/100 Ethernet> port 0xa000-0xa03f mem 0xdc000000-0xdc000fff irq 20 at device 8.0 on pci1 miibus0: <MII bus> on fxp0 inphy0: <i82562ET 10/100 media interface> on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:10:dc:52:d4:3d isab0: <PCI-ISA bridge> at device 31.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <Intel ICH4 UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xcc00-0xcc0f mem 0xdc181000-0xdc1813ff irq 16 at device 31.1 on pci0 ata0: <ATA channel 0> on atapci0 ata1: <ATA channel 1> on atapci0 pci0: <serial bus, SMBus> at device 31.3 (no driver attached) pci0: <multimedia, audio> at device 31.5 (no driver attached) pmtimer0 on isa0 atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FAST] ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/16 bytes threshold ppbus0: <Parallel port bus> on ppc0 plip0: <PLIP network interface> on ppbus0 lpt0: <Printer> on ppbus0 lpt0: Interrupt-driven port ppi0: <Parallel I/O> on ppbus0 sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: <PNP0303> can't assign resources (port) unknown: <PNP0c01> can't assign resources (memory) unknown: <PNP0c02> can't assign resources (port) unknown: <PNP0501> can't assign resources (port) unknown: <PNP0700> can't assign resources (port) unknown: <PNP0401> can't assign resources (port) unknown: <PNP0501> can't assign resources (port) Timecounter "TSC" frequency 2000033536 Hz quality 800 Timecounters tick every 1.000 msec IP Filter: v4.1.8 initialized. Default = block all, Logging = enabled ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to deny, logging unlimited ad0: 76319MB <WDC WD800BB-00CAA1 17.07W17> at ata0-master UDMA100 acd0: CDROM <HL-DT-ST CD-ROM GCR-8480B/1.02> at ata1-master UDMA33 Trying to mount root from ufs:/dev/ad0s1a WARNING: / was not properly dismounted g_vfs_done():md0[WRITE(offset=23527424, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=23805952, length=32768)]error = 28 g_vfs_done():md0[WRITE(offset=23887872, length=32768)]error = 28 g_vfs_done():md0[WRITE(offset=23937024, length=81920)]error = 28 g_vfs_done():md0[WRITE(offset=24035328, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=24166400, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=27623424, length=16384)]error = 28 g_vfs_done():md0[WRITE(offset=27787264, length=16384)]error = 28 g_vfs_done():md0[WRITE(offset=27934720, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=28180480, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=28311552, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=28459008, length=98304)]error = 28 g_vfs_done():md0[WRITE(offset=28573696, length=16384)]error = 28 g_vfs_done():md0[WRITE(offset=28983296, length=65536)]error = 28 g_vfs_done():md0[WRITE(offset=33243136, length=114688)]error = 28 g_vfs_done():md0[WRITE(offset=33374208, length=16384)]error = 28 g_vfs_done():md0[WRITE(offset=33669120, length=32768)]error = 28 g_vfs_done():md0[WRITE(offset=33832960, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=34226176, length=65536)]error = 28 g_vfs_done():md0[WRITE(offset=34308096, length=131072)]error = 28 g_vfs_done():md0[WRITE(offset=34439168, length=98304)]error = 28 g_vfs_done():md0[WRITE(offset=40517632, length=131072)]error = 28 #### BACKTRACE #### # kgdb kernel.debug /var/crash/vmcore.1 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0x5c fault code = supervisor read, page not present instruction pointer = 0x20:0xc06ab64c stack pointer = 0x28:0xf7ab1b10 frame pointer = 0x28:0xf7ab1b2c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 17074 (exim) trap number = 12 panic: page fault Uptime: 4d5h40m36s Dumping 1015 MB (2 chunks) chunk 0: 1MB (160 pages) ... ok chunk 1: 1015MB (259840 pages) 1000 984 968 952 936 920 904 888 872 856 840 824 808 792 776 760 744 728 712 696 680 664 648 632 616 600 584 568 552 536 520 504 488 472 456 440 424 408 392 376 360 344 328 312 296 280 264 248 232 216 200 184 168 152 136 120 104 88 72 56 40 24 8 #0 doadump () at pcpu.h:165 165 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc0617755 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402 #2 0xc06179ec in panic (fmt=0xc0828e2d "%s") at /usr/src/sys/kern/kern_shutdown.c:558 #3 0xc07e4f24 in trap_fatal (frame=0xf7ab1ad0, eva=92) at /usr/src/sys/i386/i386/trap.c:836 #4 0xc07e4c8b in trap_pfault (frame=0xf7ab1ad0, usermode=0, eva=92) at /usr/src/sys/i386/i386/trap.c:744 #5 0xc07e48c9 in trap (frame= {tf_fs = -1064173560, tf_es = -811597784, tf_ds = -139788248, tf_edi = 0, tf_esi = 0, tf_ebp = -139781332, tf_isp = -139781380, tf_ebx = -139780988, tf_edx = -818015104, tf_ecx = 4, tf_eax = 1, tf_trapno = 12, tf_err = 0, tf_eip = -1066748340, tf_cs = 32, tf_eflags = 66199, tf_esp = -818015104, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:434 #6 0xc07d3efa in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc06ab64c in ip_ctloutput (so=0x1, sopt=0xf7ab1c84) at /usr/src/sys/netinet/ip_output.c:1337 #8 0xc06bb10b in tcp_ctloutput (so=0xcfedc000, sopt=0xf7ab1c84) at /usr/src/sys/netinet/tcp_usrreq.c:1038 #9 0xc0651683 in sogetopt (so=0xcfedc000, sopt=0xf7ab1c84) at /usr/src/sys/kern/uipc_socket.c:1773 #10 0xc0656539 in kern_getsockopt (td=0xcf3e1480, s=5, level=1, name=1, val=0xcf3e1480, valseg=4, valsize=0xf7ab1cd4) at /usr/src/sys/kern/uipc_syscalls.c:1434 #11 0xc065644c in getsockopt (td=0xcf3e1480, uap=0xf7ab1d04) at /usr/src/sys/kern/uipc_syscalls.c:1382 #12 0xc07e523b in syscall (frame= {tf_fs = 137429051, tf_es = 59, tf_ds = -1078001605, tf_edi = 0, tf_esi = 0, tf_ebp = -1077942184, tf_isp = -139780764, tf_ebx = 0, tf_edx = -1077942256, tf_ecx = 0, tf_eax = 118, tf_trapno = 12, tf_err = 2, tf_eip = 135800687, tf_cs = 51, tf_eflags = 582, tf_esp = -1077942356, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:981 #13 0xc07d3f4f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) I have other dumps and backtrace from other machines with same problem if anyone requires a comparison. Any help looking into this would be GREATLY appreciated. -Wil
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?NGBBKBOMKLFOJCCDHPEOCENEHDAB.freebsd>