From owner-freebsd-stable@FreeBSD.ORG Mon Jul 28 10:18:47 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 380EA1065677; Mon, 28 Jul 2008 10:18:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (skuns.zoral.com.ua [91.193.166.194]) by mx1.freebsd.org (Postfix) with ESMTP id 890198FC20; Mon, 28 Jul 2008 10:18:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id m6SAIfkm044647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 28 Jul 2008 13:18:41 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2) with ESMTP id m6SAIesV040006; Mon, 28 Jul 2008 13:18:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.2/8.14.2/Submit) id m6SAIecA040005; Mon, 28 Jul 2008 13:18:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 28 Jul 2008 13:18:40 +0300 From: Kostik Belousov To: Michael Toth Message-ID: <20080728101840.GK97161@deviant.kiev.zoral.com.ua> References: <488CACD9.7060002@queldor.net> <488CBB02.1020105@FreeBSD.org> <488CBBAC.7040507@queldor.net> <488CC13F.1020204@FreeBSD.org> <20080727190742.GF97161@deviant.kiev.zoral.com.ua> <488CD9AB.8040401@queldor.net> <20080727204339.GG97161@deviant.kiev.zoral.com.ua> <488CFCC3.2030504@queldor.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="D85EWoRcDXv1uhH1" Content-Disposition: inline In-Reply-To: <488CFCC3.2030504@queldor.net> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: ClamAV version 0.93.3, clamav-milter version 0.93.3 on skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-4.3 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: Kris Kennaway , freebsd-stable@freebsd.org Subject: Re: 7.0 Crashing X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2008 10:18:47 -0000 --D85EWoRcDXv1uhH1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 27, 2008 at 06:54:59PM -0400, Michael Toth wrote: >=20 >=20 >=20 > Kostik Belousov wrote: > >On Sun, Jul 27, 2008 at 04:25:15PM -0400, Michael toth wrote: > > =20 > >>Fatal trap 12: page fault while in kernel mode > >>cpuid =3D 4; apic id =3D 04 > >>fault virtual address =3D 0x188 > >>fault code =3D supervisor read, page not present > >>instruction pointer =3D 0x20:0xc0775284 > >>stack pointer =3D 0x28:0xe7d6bad0 > >>frame pointer =3D 0x28:0xe7d6bae8 > >>code segment =3D base 0x0, limit 0xfffff, type 0x1b > >> =3D DPL 0, pres 1, def32 1, gran 1 > >>processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > >>current process =3D 4838 (egrep) > >>trap number =3D 12 > >>panic: page fault > >>cpuid =3D 4 > >>Uptime: 1h2m48s > >>Physical memory: 2035 MB > >>Dumping 87 MB: 72 56 40 24 8 > >> > >>Reading symbols from /boot/kernel/acpi.ko...Reading symbols from=20 > >>/boot/kernel/acpi.ko.symbols...done. > >>done. > >>Loaded symbols for /boot/kernel/acpi.ko > >>#0 doadump () at pcpu.h:195 > >>195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); > >>(kgdb) backtrace > >>#0 doadump () at pcpu.h:195 > >>#1 0xc0782597 in boot (howto=3D260) at=20 > >>/usr/src/sys/kern/kern_shutdown.c:418 > >>#2 0xc0782859 in panic (fmt=3DVariable "fmt" is not available. > >>) at /usr/src/sys/kern/kern_shutdown.c:572 > >>#3 0xc0a8b39c in trap_fatal (frame=3D0xe7d6ba90, eva=3D392) at=20 > >>/usr/src/sys/i386/i386/trap.c:899 > >>#4 0xc0a8b620 in trap_pfault (frame=3D0xe7d6ba90, usermode=3D0, eva=3D= 392) at=20 > >>/usr/src/sys/i386/i386/trap.c:812 > >>#5 0xc0a8bfcc in trap (frame=3D0xe7d6ba90) at=20 > >>/usr/src/sys/i386/i386/trap.c:490 > >>#6 0xc0a71bdb in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > >>#7 0xc0775284 in _mtx_lock_sleep (m=3D0xc600d174, tid=3D3318745216, op= ts=3D0,=20 > >>file=3D0x0, line=3D0) at /usr/src/sys/kern/kern_mutex.c:339 > >>#8 0xc09a93d7 in vm_fault (map=3D0xc56b5570, vaddr=3D671809536,=20 > >>fault_type=3D2 '\002', fault_flags=3D8) at /usr/src/sys/vm/vm_fault.c:2= 93 > >>#9 0xc0a8b50b in trap_pfault (frame=3D0xe7d6bd38, usermode=3D1,=20 > >>eva=3D671813488) at /usr/src/sys/i386/i386/trap.c:789 > >>#10 0xc0a8be57 in trap (frame=3D0xe7d6bd38) at=20 > >>/usr/src/sys/i386/i386/trap.c:357 > >>#11 0xc0a71bdb in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > >>#12 0x2806e607 in ?? () > >>Previous frame inner to this frame (corrupt stack?) > >>(kgdb) up > >>#1 0xc0782597 in boot (howto=3D260) at=20 > >>/usr/src/sys/kern/kern_shutdown.c:418 > >>418 doadump(); > >>(kgdb) up > >>#2 0xc0782859 in panic (fmt=3DVariable "fmt" is not available. > >>) at /usr/src/sys/kern/kern_shutdown.c:572 > >>572 boot(bootopt); > >>(kgdb) up > >>#3 0xc0a8b39c in trap_fatal (frame=3D0xe7d6ba90, eva=3D392) at=20 > >>/usr/src/sys/i386/i386/trap.c:899 > >>899 panic("%s", trap_msg[type]); > >>(kgdb) up > >>#4 0xc0a8b620 in trap_pfault (frame=3D0xe7d6ba90, usermode=3D0, eva=3D= 392) at=20 > >>/usr/src/sys/i386/i386/trap.c:812 > >>812 trap_fatal(frame, eva); > >>(kgdb) up > >>#5 0xc0a8bfcc in trap (frame=3D0xe7d6ba90) at=20 > >>/usr/src/sys/i386/i386/trap.c:490 > >>490 (void) trap_pfault(frame, FALSE, eva); > >>(kgdb) up > >>#6 0xc0a71bdb in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > >>139 call trap > >>Current language: auto; currently asm > >>(kgdb) up > >>#7 0xc0775284 in _mtx_lock_sleep (m=3D0xc600d174, tid=3D3318745216, op= ts=3D0,=20 > >>file=3D0x0, line=3D0) at /usr/src/sys/kern/kern_mutex.c:339 > >>339 owner =3D (struct thread *)(v &=20 > >>~MTX_FLAGMASK); > >>Current language: auto; currently c > >>(kgdb) up > >>#8 0xc09a93d7 in vm_fault (map=3D0xc56b5570, vaddr=3D671809536,=20 > >>fault_type=3D2 '\002', fault_flags=3D8) at /usr/src/sys/vm/vm_fault.c:2= 93 > >>293 VM_OBJECT_LOCK(fs.first_object); > >>(kgdb) p fs > >>$1 =3D {m =3D 0x0, object =3D 0x12, pindex =3D 13878757899709627520, fi= rst_m =3D=20 > >>0xc5f0a8b8, first_object =3D 0xc600d174, first_pindex =3D 0, map =3D=20 > >>0xc56b5570, entry =3D 0xc59fc7f8, lookup_still_valid =3D 2, vp =3D 0xc5= 5c5220} > >>(kgdb) p fs.first_object > >>$2 =3D 0xc600d174 > >>(kgdb) > >> =20 > > > >Please, show the output of > > p/x *(fs.first_object) > > > >BTW, you have said that you got a lot of the panics. Are backtraces > >the same for all of them ? > > =20 > Here is the p/x *(fs.first_object) .. > and it appears that vmcore.6 is different (vmcore.6 is new from a few= =20 > hours ago) >=20 > So does this point to a hardware issue? >=20 > Thanks >=20 > # kgdb kernel.debug /var/crash/vmcore.5 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain=20 > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd"... >=20 > Unread portion of the kernel message buffer: >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 4; apic id =3D 04 > fault virtual address =3D 0x188 > fault code =3D supervisor read, page not present > instruction pointer =3D 0x20:0xc0775284 > stack pointer =3D 0x28:0xe7d6bad0 > frame pointer =3D 0x28:0xe7d6bae8 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, def32 1, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 4838 (egrep) > trap number =3D 12 > panic: page fault > cpuid =3D 4 > Uptime: 1h2m48s > Physical memory: 2035 MB > Dumping 87 MB: 72 56 40 24 8 >=20 > Reading symbols from /boot/kernel/acpi.ko...Reading symbols from=20 > /boot/kernel/acpi.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/acpi.ko > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); > (kgdb) up > #1 0xc0782597 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :418 > 418 doadump(); > (kgdb) up > #2 0xc0782859 in panic (fmt=3DVariable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:572 > 572 boot(bootopt); > (kgdb) up > #3 0xc0a8b39c in trap_fatal (frame=3D0xe7d6ba90, eva=3D392) at=20 > /usr/src/sys/i386/i386/trap.c:899 > 899 panic("%s", trap_msg[type]); > (kgdb) up > #4 0xc0a8b620 in trap_pfault (frame=3D0xe7d6ba90, usermode=3D0, eva=3D39= 2) at=20 > /usr/src/sys/i386/i386/trap.c:812 > 812 trap_fatal(frame, eva); > (kgdb) up > #5 0xc0a8bfcc in trap (frame=3D0xe7d6ba90) at=20 > /usr/src/sys/i386/i386/trap.c:490 > 490 (void) trap_pfault(frame, FALSE, eva); > (kgdb) up > #6 0xc0a71bdb in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > 139 call trap > Current language: auto; currently asm > (kgdb) up > #7 0xc0775284 in _mtx_lock_sleep (m=3D0xc600d174, tid=3D3318745216, opts= =3D0,=20 > file=3D0x0, line=3D0) at /usr/src/sys/kern/kern_mutex.c:339 > 339 owner =3D (struct thread *)(v &=20 > ~MTX_FLAGMASK); > Current language: auto; currently c > (kgdb) up > #8 0xc09a93d7 in vm_fault (map=3D0xc56b5570, vaddr=3D671809536,=20 > fault_type=3D2 '\002', fault_flags=3D8) at /usr/src/sys/vm/vm_fault.c:293 > 293 VM_OBJECT_LOCK(fs.first_object); > (kgdb) p/x *(fs.first_object) > $1 =3D {mtx =3D {lock_object =3D {lo_name =3D 0x0, lo_type =3D 0x0, lo_fl= ags =3D=20 > 0x0, lo_witness_data =3D {lod_list =3D {stqe_next =3D 0x0}, lod_witness = =3D=20 > 0x0}}, mtx_lock =3D 0x0, mtx_recurse =3D 0x0}, object_list =3D {tqe_next = =3D 0x0, > tqe_prev =3D 0xc5f3a300}, shadow_head =3D {lh_first =3D 0x0}, shadow_l= ist=20 > =3D {le_next =3D 0xc5f3a2e8, le_prev =3D 0xc55c39d0}, memq =3D {tqh_first= =3D 0x0,=20 > tqh_last =3D 0xc600d1a0}, root =3D 0x0, size =3D 0x1, generation =3D 0x1, > ref_count =3D 0x1, shadow_count =3D 0x0, type =3D 0x0, flags =3D 0x2000,= =20 > pg_color =3D 0x0, paging_in_progress =3D 0x0, resident_page_count =3D 0x0= ,=20 > backing_object =3D 0xc55c39b0, backing_object_offset =3D 0xf000,=20 > pager_object_list =3D { > tqe_next =3D 0x0, tqe_prev =3D 0x0}, cache =3D 0x0, handle =3D 0x0, un= _pager=20 > =3D {vnp =3D {vnp_size =3D 0x0}, devp =3D {devp_pglist =3D {tqh_first =3D= 0x0,=20 > tqh_last =3D 0x0}}, swp =3D {swp_bcount =3D 0x0}}} > (kgdb) >=20 >=20 >=20 >=20 > # kgdb kernel.debug /var/crash/vmcore.6 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain=20 > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd"... >=20 > Unread portion of the kernel message buffer: > TPTE at 0xbfefeffc IS ZERO @ VA bfbff000 > panic: bad pte > cpuid =3D 2 > Uptime: 4h12m47s > Physical memory: 2035 MB > Dumping 121 MB: 106 90 74 58 42 26 10 >=20 > Reading symbols from /boot/kernel/acpi.ko...Reading symbols from=20 > /boot/kernel/acpi.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/acpi.ko > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); > (kgdb) bt > #0 doadump () at pcpu.h:195 > #1 0xc0782597 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :418 > #2 0xc0782859 in panic (fmt=3DVariable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:572 > #3 0xc0a86f26 in pmap_remove_pages (pmap=3D0xc5527c54) at=20 > /usr/src/sys/i386/i386/pmap.c:3093 > #4 0xc09b294c in vmspace_exit (td=3D0xc5fb9aa0) at=20 > /usr/src/sys/vm/vm_map.c:404 > #5 0xc075e780 in exit1 (td=3D0xc5fb9aa0, rv=3D0) at=20 > /usr/src/sys/kern/kern_exit.c:294 > #6 0xc075fadd in sys_exit (td=3DCould not find the frame base for "sys_e= xit". > ) at /usr/src/sys/kern/kern_exit.c:98 > #7 0xc0a8b975 in syscall (frame=3D0xe7e6cd38) at=20 > /usr/src/sys/i386/i386/trap.c:1035 > #8 0xc0a71c40 in Xint0x80_syscall () at=20 > /usr/src/sys/i386/i386/exception.s:196 > #9 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) Both panics you shown are caused by zeroing kernel memory in what seems to be random locations. This might be caused by the bug, but I would suggest first making the thorough test of the hardware. --D85EWoRcDXv1uhH1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iEYEARECAAYFAkiNnP8ACgkQC3+MBN1Mb4jtsQCffO6VzX00I/g2Li+31l8hrnMt Yy4AoNz8Bk30rl/kVfplcixBCEw/+a5u =R4DW -----END PGP SIGNATURE----- --D85EWoRcDXv1uhH1--