From owner-freebsd-stable@FreeBSD.ORG Mon Apr 23 15:57:29 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 635B016A403; Mon, 23 Apr 2007 15:57:29 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 4F8EB13C455; Mon, 23 Apr 2007 15:57:29 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 3A1A01A4DBD; Mon, 23 Apr 2007 08:57:51 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 58EFD51342; Mon, 23 Apr 2007 11:57:28 -0400 (EDT) Date: Mon, 23 Apr 2007 11:57:28 -0400 From: Kris Kennaway To: Tom Judge Message-ID: <20070423155728.GC1006@xor.obsecurity.org> References: <462CA594.5000904@tomjudge.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="WhfpMioaduB5tiZL" Content-Disposition: inline In-Reply-To: <462CA594.5000904@tomjudge.com> User-Agent: Mutt/1.4.2.2i Cc: bms@FreeBSD.org, freebsd-stable@freebsd.org Subject: Re: 6.2-STABLE (i386) Repeating crash (supervisor read, page not present) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2007 15:57:29 -0000 --WhfpMioaduB5tiZL Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Apr 23, 2007 at 01:24:52PM +0100, Tom Judge wrote: > Hi, >=20 > Recently I have noticed that one of our Dell PE1950's has been crashing= =20 > a lot with the following reason "supervisor read, page not present". >=20 > The system runs 6.2 Release under i386. >=20 > I have attached 2 back traces, and I still have both cores if any more=20 > information is required. Any light that can be shed on this problem=20 > would be greatly appreciated. >=20 > Tom >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >=20 > uname -a > FreeBSD narthex.mintel.co.uk 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Mon=20 > Apr 2 20:13:11 BST 2007 =20 > root@bob.mintel.co.uk:/usr/obj/usr/src/sys/PE1950 i386 >=20 >=20 > ## Core 1 >=20 > root@narthex '13:14:47' '/home/london/tj' > > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.1 > [GDB will not be able to debug user-mode threads:=20 > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain=20 > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd". >=20 > Unread portion of the kernel message buffer: >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 0; apic id =3D 00 > fault virtual address =3D 0x100005c > fault code =3D supervisor read, page not present > instruction pointer =3D 0x20:0xc05df61f > stack pointer =3D 0x28:0xe4f63c30 > frame pointer =3D 0x28:0xe4f63c90 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, def32 1, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 12 (swi1: net) > trap number =3D 12 > panic: page fault > cpuid =3D 0 > Uptime: 1h25m33s > Dumping 2047 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919= =20 > 1903 1887 > <7>arp_rtrequest: bad gateway 172.31.1.1 (!AF_LINK) > <7>arp_rtrequest: bad gateway 172.31.0.1 (!AF_LINK) You might be hitting a bug in an obscure code path because of the above errors. I'm CC'ing someone who might be able to help. Kris > 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663=20 > 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439=20 > 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215=20 > 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975= =20 > 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687= =20 > 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399= =20 > 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111= =20 > 95 79 63 47 31 15 >=20 > #0 doadump () at pcpu.h:165 > 165 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc05622ba in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :409 > #2 0xc05625e1 in panic (fmt=3D0xc06e2578 "%s") at=20 > /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc06b4580 in trap_fatal (frame=3D0xe4f63bf0, eva=3D16777308) at=20 > /usr/src/sys/i386/i386/trap.c:837 > #4 0xc06b42bf in trap_pfault (frame=3D0xe4f63bf0, usermode=3D0,=20 > eva=3D16777308) at /usr/src/sys/i386/i386/trap.c:745 > #5 0xc06b3f19 in trap (frame=3D > {tf_fs =3D -1067581432, tf_es =3D -965803992, tf_ds =3D -964624344,= =20 > tf_edi =3D -957112288, tf_esi =3D -965676032, tf_ebp =3D -453624688, tf_i= sp =3D=20 > -453624804, tf_ebx =3D 16777216, tf_edx =3D -968955648, tf_ecx =3D 4, tf_= eax =3D=20 > 0, tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1067583969, tf_cs =3D 32,= =20 > tf_eflags =3D 66118, tf_esp =3D 3, tf_ss =3D 0}) at=20 > /usr/src/sys/i386/i386/trap.c:435 > #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > #7 0xc05df61f in in_arpinput (m=3D0xc68ba200) at=20 > /usr/src/sys/netinet/if_ether.c:636 > #8 0xc05df4ea in arpintr (m=3D0xc68ba200) at=20 > /usr/src/sys/netinet/if_ether.c:551 > #9 0xc05d861b in netisr_processqueue (ni=3D0xc076b078) at=20 > /usr/src/sys/net/netisr.c:236 > #10 0xc05d881a in swi_net (dummy=3D0x0) at /usr/src/sys/net/netisr.c:349 > #11 0xc054cc49 in ithread_execute_handlers (p=3D0xc63ed860, ie=3D0xc643bb= 80)=20 > at /usr/src/sys/kern/kern_intr.c:682 > #12 0xc054cd59 in ithread_loop (arg=3D0xc63bb870) at=20 > /usr/src/sys/kern/kern_intr.c:765 > #13 0xc054b9fd in fork_exit (callout=3D0xc054cd04 ,=20 > arg=3D0xc63bb870, frame=3D0xe4f63d38) at /usr/src/sys/kern/kern_fork.c:821 > #14 0xc06a09bc in fork_trampoline () at=20 > /usr/src/sys/i386/i386/exception.s:208 > (kgdb) exit > Undefined command: "exit". Try "help". > (kgdb) quit >=20 >=20 > ## Core 2 > root@narthex '13:15:32' '/home/london/tj' > > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.0 > [GDB will not be able to debug user-mode threads:=20 > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you = are > welcome to change it and/or distribute copies of it under certain=20 > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for detail= s. > This GDB was configured as "i386-marcel-freebsd". >=20 > Unread portion of the kernel message buffer: >=20 >=20 > Fatal trap 12: page fault while in kernel mode > cpuid =3D 1; apic id =3D 01 > fault virtual address =3D 0xb1 > fault code =3D supervisor read, page not present > instruction pointer =3D 0x20:0xc047ed16 > stack pointer =3D 0x28:0xecd9c9e0 > frame pointer =3D 0x28:0xecd9c9e0 > code segment =3D base 0x0, limit 0xfffff, type 0x1b > =3D DPL 0, pres 1, def32 1, gran 1 > processor eflags =3D interrupt enabled, resume, IOPL =3D 0 > current process =3D 22409 (pfctl) > trap number =3D 12 > panic: page fault > cpuid =3D 1 > Uptime: 3d1h41m58s > Dumping 2047 MB (2 chunks) > chunk 0: 1MB (159 pages) ... ok > chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919= =20 > 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695=20 > 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471=20 > 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247=20 > 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023=20 > 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735= =20 > 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447= =20 > 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159= =20 > 143 127 111 95 79 63 47 31 15 >=20 > #0 doadump () at pcpu.h:165 > 165 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:165 > #1 0xc05622ba in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c= :409 > #2 0xc05625e1 in panic (fmt=3D0xc06e2578 "%s") at=20 > /usr/src/sys/kern/kern_shutdown.c:565 > #3 0xc06b4580 in trap_fatal (frame=3D0xecd9c9a0, eva=3D177) at=20 > /usr/src/sys/i386/i386/trap.c:837 > #4 0xc06b42bf in trap_pfault (frame=3D0xecd9c9a0, usermode=3D0, eva=3D17= 7) at=20 > /usr/src/sys/i386/i386/trap.c:745 > #5 0xc06b3f19 in trap (frame=3D > {tf_fs =3D 8, tf_es =3D -321322968, tf_ds =3D -1067909080, tf_edi = =3D=20 > -965816192, tf_esi =3D -257, tf_ebp =3D -321271328, tf_isp =3D -321271348= ,=20 > tf_ebx =3D -957803852, tf_edx =3D 1, tf_ecx =3D -957803852, tf_eax =3D 0,= =20 > tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1069028074, tf_cs =3D 32, tf_= eflags=20 > =3D 66050, tf_esp =3D -321271304, tf_ss =3D -1069021108}) at=20 > /usr/src/sys/i386/i386/trap.c:435 > #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 > #7 0xc047ed16 in pfi_ifhead_RB_NEXT (elm=3D0x1) at=20 > /usr/src/sys/contrib/pf/net/pf_if.c:120 > #8 0xc048084c in pfi_clear_flags (name=3D0xc66ed080 "", flags=3D-257) at= =20 > /usr/src/sys/contrib/pf/net/pf_if.c:1004 > #9 0xc0485629 in pfioctl (dev=3D0xc66d6100, cmd=3D3223602266,=20 > addr=3D0xc66ed080 "", flags=3D3, td=3D0x0) at=20 > /usr/src/sys/contrib/pf/net/pf_ioctl.c:3191 > #10 0xc050fac3 in devfs_ioctl_f (fp=3D0xc6c72120, com=3D3223602266,=20 > data=3D0xc66ed080, cred=3D0xc6cab500, td=3D0xc6bcc900) at=20 > /usr/src/sys/fs/devfs/devfs_vnops.c:479 > #11 0xc0586249 in ioctl (td=3D0xc6bcc900, uap=3D0xecd9cd04) at file.h:264 > #12 0xc06b48c7 in syscall (frame=3D > {tf_fs =3D 59, tf_es =3D 59, tf_ds =3D 59, tf_edi =3D -1077944364, t= f_esi=20 > =3D 2, tf_ebp =3D -1077944344, tf_isp =3D -321270428, tf_ebx =3D 0, tf_ed= x =3D 0,=20 > tf_ecx =3D 0, tf_eax =3D 54, tf_trapno =3D 12, tf_err =3D 2, tf_eip =3D 6= 72674543,=20 > tf_cs =3D 51, tf_eflags =3D 582, tf_esp =3D -1077944420, tf_ss =3D 59}) > at /usr/src/sys/i386/i386/trap.c:983 > #13 0xc06a09af in Xint0x80_syscall () at=20 > /usr/src/sys/i386/i386/exception.s:200 > #14 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) supervisor read, page not presentQuit > (kgdb) >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >=20 --WhfpMioaduB5tiZL Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGLNdnWry0BWjoQKURAnUhAJ9MSwWjiktPFOecObfeFExB/d+ljwCgxOVe G4EItvPRt/3mXX29PMNTpDY= =Qatj -----END PGP SIGNATURE----- --WhfpMioaduB5tiZL--