Date: Mon, 23 Apr 2007 11:57:28 -0400 From: Kris Kennaway <kris@obsecurity.org> To: Tom Judge <tom@tomjudge.com> Cc: bms@FreeBSD.org, freebsd-stable@freebsd.org Subject: Re: 6.2-STABLE (i386) Repeating crash (supervisor read, page not present) Message-ID: <20070423155728.GC1006@xor.obsecurity.org> In-Reply-To: <462CA594.5000904@tomjudge.com> References: <462CA594.5000904@tomjudge.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--WhfpMioaduB5tiZL
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Apr 23, 2007 at 01:24:52PM +0100, Tom Judge wrote:
> Hi,
>=20
> Recently I have noticed that one of our Dell PE1950's has been crashing=
=20
> a lot with the following reason "supervisor read, page not present".
>=20
> The system runs 6.2 Release under i386.
>=20
> I have attached 2 back traces, and I still have both cores if any more=20
> information is required. Any light that can be shed on this problem=20
> would be greatly appreciated.
>=20
> Tom
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> uname -a
> FreeBSD narthex.mintel.co.uk 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Mon=20
> Apr 2 20:13:11 BST 2007 =20
> root@bob.mintel.co.uk:/usr/obj/usr/src/sys/PE1950 i386
>=20
>=20
> ## Core 1
>=20
> root@narthex '13:14:47' '/home/london/tj'
> > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.1
> [GDB will not be able to debug user-mode threads:=20
> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you =
are
> welcome to change it and/or distribute copies of it under certain=20
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for detail=
s.
> This GDB was configured as "i386-marcel-freebsd".
>=20
> Unread portion of the kernel message buffer:
>=20
>=20
> Fatal trap 12: page fault while in kernel mode
> cpuid =3D 0; apic id =3D 00
> fault virtual address =3D 0x100005c
> fault code =3D supervisor read, page not present
> instruction pointer =3D 0x20:0xc05df61f
> stack pointer =3D 0x28:0xe4f63c30
> frame pointer =3D 0x28:0xe4f63c90
> code segment =3D base 0x0, limit 0xfffff, type 0x1b
> =3D DPL 0, pres 1, def32 1, gran 1
> processor eflags =3D interrupt enabled, resume, IOPL =3D 0
> current process =3D 12 (swi1: net)
> trap number =3D 12
> panic: page fault
> cpuid =3D 0
> Uptime: 1h25m33s
> Dumping 2047 MB (2 chunks)
> chunk 0: 1MB (159 pages) ... ok
> chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919=
=20
> 1903 1887
> <7>arp_rtrequest: bad gateway 172.31.1.1 (!AF_LINK)
> <7>arp_rtrequest: bad gateway 172.31.0.1 (!AF_LINK)
You might be hitting a bug in an obscure code path because of the
above errors. I'm CC'ing someone who might be able to help.
Kris
> 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695 1679 1663=20
> 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471 1455 1439=20
> 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247 1231 1215=20
> 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023 1007 991 975=
=20
> 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735 719 703 687=
=20
> 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447 431 415 399=
=20
> 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159 143 127 111=
=20
> 95 79 63 47 31 15
>=20
> #0 doadump () at pcpu.h:165
> 165 pcpu.h: No such file or directory.
> in pcpu.h
> (kgdb) bt
> #0 doadump () at pcpu.h:165
> #1 0xc05622ba in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c=
:409
> #2 0xc05625e1 in panic (fmt=3D0xc06e2578 "%s") at=20
> /usr/src/sys/kern/kern_shutdown.c:565
> #3 0xc06b4580 in trap_fatal (frame=3D0xe4f63bf0, eva=3D16777308) at=20
> /usr/src/sys/i386/i386/trap.c:837
> #4 0xc06b42bf in trap_pfault (frame=3D0xe4f63bf0, usermode=3D0,=20
> eva=3D16777308) at /usr/src/sys/i386/i386/trap.c:745
> #5 0xc06b3f19 in trap (frame=3D
> {tf_fs =3D -1067581432, tf_es =3D -965803992, tf_ds =3D -964624344,=
=20
> tf_edi =3D -957112288, tf_esi =3D -965676032, tf_ebp =3D -453624688, tf_i=
sp =3D=20
> -453624804, tf_ebx =3D 16777216, tf_edx =3D -968955648, tf_ecx =3D 4, tf_=
eax =3D=20
> 0, tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1067583969, tf_cs =3D 32,=
=20
> tf_eflags =3D 66118, tf_esp =3D 3, tf_ss =3D 0}) at=20
> /usr/src/sys/i386/i386/trap.c:435
> #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
> #7 0xc05df61f in in_arpinput (m=3D0xc68ba200) at=20
> /usr/src/sys/netinet/if_ether.c:636
> #8 0xc05df4ea in arpintr (m=3D0xc68ba200) at=20
> /usr/src/sys/netinet/if_ether.c:551
> #9 0xc05d861b in netisr_processqueue (ni=3D0xc076b078) at=20
> /usr/src/sys/net/netisr.c:236
> #10 0xc05d881a in swi_net (dummy=3D0x0) at /usr/src/sys/net/netisr.c:349
> #11 0xc054cc49 in ithread_execute_handlers (p=3D0xc63ed860, ie=3D0xc643bb=
80)=20
> at /usr/src/sys/kern/kern_intr.c:682
> #12 0xc054cd59 in ithread_loop (arg=3D0xc63bb870) at=20
> /usr/src/sys/kern/kern_intr.c:765
> #13 0xc054b9fd in fork_exit (callout=3D0xc054cd04 <ithread_loop>,=20
> arg=3D0xc63bb870, frame=3D0xe4f63d38) at /usr/src/sys/kern/kern_fork.c:821
> #14 0xc06a09bc in fork_trampoline () at=20
> /usr/src/sys/i386/i386/exception.s:208
> (kgdb) exit
> Undefined command: "exit". Try "help".
> (kgdb) quit
>=20
>=20
> ## Core 2
> root@narthex '13:15:32' '/home/london/tj'
> > $ kgdb /usr/obj/usr/src/sys/PE1950/kernel.debug /var/crash/vmcore.0
> [GDB will not be able to debug user-mode threads:=20
> /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
> GNU gdb 6.1.1 [FreeBSD]
> Copyright 2004 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you =
are
> welcome to change it and/or distribute copies of it under certain=20
> conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB. Type "show warranty" for detail=
s.
> This GDB was configured as "i386-marcel-freebsd".
>=20
> Unread portion of the kernel message buffer:
>=20
>=20
> Fatal trap 12: page fault while in kernel mode
> cpuid =3D 1; apic id =3D 01
> fault virtual address =3D 0xb1
> fault code =3D supervisor read, page not present
> instruction pointer =3D 0x20:0xc047ed16
> stack pointer =3D 0x28:0xecd9c9e0
> frame pointer =3D 0x28:0xecd9c9e0
> code segment =3D base 0x0, limit 0xfffff, type 0x1b
> =3D DPL 0, pres 1, def32 1, gran 1
> processor eflags =3D interrupt enabled, resume, IOPL =3D 0
> current process =3D 22409 (pfctl)
> trap number =3D 12
> panic: page fault
> cpuid =3D 1
> Uptime: 3d1h41m58s
> Dumping 2047 MB (2 chunks)
> chunk 0: 1MB (159 pages) ... ok
> chunk 1: 2047MB (523944 pages) 2031 2015 1999 1983 1967 1951 1935 1919=
=20
> 1903 1887 1871 1855 1839 1823 1807 1791 1775 1759 1743 1727 1711 1695=20
> 1679 1663 1647 1631 1615 1599 1583 1567 1551 1535 1519 1503 1487 1471=20
> 1455 1439 1423 1407 1391 1375 1359 1343 1327 1311 1295 1279 1263 1247=20
> 1231 1215 1199 1183 1167 1151 1135 1119 1103 1087 1071 1055 1039 1023=20
> 1007 991 975 959 943 927 911 895 879 863 847 831 815 799 783 767 751 735=
=20
> 719 703 687 671 655 639 623 607 591 575 559 543 527 511 495 479 463 447=
=20
> 431 415 399 383 367 351 335 319 303 287 271 255 239 223 207 191 175 159=
=20
> 143 127 111 95 79 63 47 31 15
>=20
> #0 doadump () at pcpu.h:165
> 165 pcpu.h: No such file or directory.
> in pcpu.h
> (kgdb) bt
> #0 doadump () at pcpu.h:165
> #1 0xc05622ba in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c=
:409
> #2 0xc05625e1 in panic (fmt=3D0xc06e2578 "%s") at=20
> /usr/src/sys/kern/kern_shutdown.c:565
> #3 0xc06b4580 in trap_fatal (frame=3D0xecd9c9a0, eva=3D177) at=20
> /usr/src/sys/i386/i386/trap.c:837
> #4 0xc06b42bf in trap_pfault (frame=3D0xecd9c9a0, usermode=3D0, eva=3D17=
7) at=20
> /usr/src/sys/i386/i386/trap.c:745
> #5 0xc06b3f19 in trap (frame=3D
> {tf_fs =3D 8, tf_es =3D -321322968, tf_ds =3D -1067909080, tf_edi =
=3D=20
> -965816192, tf_esi =3D -257, tf_ebp =3D -321271328, tf_isp =3D -321271348=
,=20
> tf_ebx =3D -957803852, tf_edx =3D 1, tf_ecx =3D -957803852, tf_eax =3D 0,=
=20
> tf_trapno =3D 12, tf_err =3D 0, tf_eip =3D -1069028074, tf_cs =3D 32, tf_=
eflags=20
> =3D 66050, tf_esp =3D -321271304, tf_ss =3D -1069021108}) at=20
> /usr/src/sys/i386/i386/trap.c:435
> #6 0xc06a095a in calltrap () at /usr/src/sys/i386/i386/exception.s:139
> #7 0xc047ed16 in pfi_ifhead_RB_NEXT (elm=3D0x1) at=20
> /usr/src/sys/contrib/pf/net/pf_if.c:120
> #8 0xc048084c in pfi_clear_flags (name=3D0xc66ed080 "", flags=3D-257) at=
=20
> /usr/src/sys/contrib/pf/net/pf_if.c:1004
> #9 0xc0485629 in pfioctl (dev=3D0xc66d6100, cmd=3D3223602266,=20
> addr=3D0xc66ed080 "", flags=3D3, td=3D0x0) at=20
> /usr/src/sys/contrib/pf/net/pf_ioctl.c:3191
> #10 0xc050fac3 in devfs_ioctl_f (fp=3D0xc6c72120, com=3D3223602266,=20
> data=3D0xc66ed080, cred=3D0xc6cab500, td=3D0xc6bcc900) at=20
> /usr/src/sys/fs/devfs/devfs_vnops.c:479
> #11 0xc0586249 in ioctl (td=3D0xc6bcc900, uap=3D0xecd9cd04) at file.h:264
> #12 0xc06b48c7 in syscall (frame=3D
> {tf_fs =3D 59, tf_es =3D 59, tf_ds =3D 59, tf_edi =3D -1077944364, t=
f_esi=20
> =3D 2, tf_ebp =3D -1077944344, tf_isp =3D -321270428, tf_ebx =3D 0, tf_ed=
x =3D 0,=20
> tf_ecx =3D 0, tf_eax =3D 54, tf_trapno =3D 12, tf_err =3D 2, tf_eip =3D 6=
72674543,=20
> tf_cs =3D 51, tf_eflags =3D 582, tf_esp =3D -1077944420, tf_ss =3D 59})
> at /usr/src/sys/i386/i386/trap.c:983
> #13 0xc06a09af in Xint0x80_syscall () at=20
> /usr/src/sys/i386/i386/exception.s:200
> #14 0x00000033 in ?? ()
> Previous frame inner to this frame (corrupt stack?)
> (kgdb) supervisor read, page not presentQuit
> (kgdb)
>=20
> _______________________________________________
> freebsd-stable@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
>=20
--WhfpMioaduB5tiZL
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (FreeBSD)
iD8DBQFGLNdnWry0BWjoQKURAnUhAJ9MSwWjiktPFOecObfeFExB/d+ljwCgxOVe
G4EItvPRt/3mXX29PMNTpDY=
=Qatj
-----END PGP SIGNATURE-----
--WhfpMioaduB5tiZL--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070423155728.GC1006>