From owner-freebsd-bugs@FreeBSD.ORG Mon Oct 20 15:20:02 2008 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07C9510656C9 for ; Mon, 20 Oct 2008 15:20:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id D843E8FC20 for ; Mon, 20 Oct 2008 15:20:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id m9KFK1eE007054 for ; Mon, 20 Oct 2008 15:20:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id m9KFK1sl007053; Mon, 20 Oct 2008 15:20:01 GMT (envelope-from gnats) Resent-Date: Mon, 20 Oct 2008 15:20:01 GMT Resent-Message-Id: <200810201520.m9KFK1sl007053@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, "Mark Atkinson" Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D3A01065670 for ; Mon, 20 Oct 2008 15:16:52 +0000 (UTC) (envelope-from root@moby.pdsea.f5net.com) Received: from mail.f5.com (mail.f5.com [65.197.145.96]) by mx1.freebsd.org (Postfix) with ESMTP id D9BE68FC18 for ; Mon, 20 Oct 2008 15:16:51 +0000 (UTC) (envelope-from root@moby.pdsea.f5net.com) Received: from moby.pdsea.f5net.com ([192.168.10.5]) by mail.f5.com with Microsoft SMTPSVC(6.0.3790.1830); Mon, 20 Oct 2008 08:16:51 -0700 Received: by moby.pdsea.f5net.com (Postfix, from userid 0) id CF619164833; Mon, 20 Oct 2008 08:16:50 -0700 (PDT) Message-Id: <20081020151650.CF619164833@moby.pdsea.f5net.com> Date: Mon, 20 Oct 2008 08:16:50 -0700 From: "Mark Atkinson" To: X-Send-Pr-Version: 3.113 Cc: Subject: kern/128247: [panic] Fatal Trap 12 in ip6_forward (/usr/src/sys/netinet6/ip6_forward.c:420) X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Mark Atkinson List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Oct 2008 15:20:02 -0000 >Number: 128247 >Category: kern >Synopsis: [panic] Fatal Trap 12 in ip6_forward = >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Oct 20 15:20:01 UTC 2008 >Closed-Date: >Last-Modified: >Originator: Mark Atkinson >Release: FreeBSD 8.0-CURRENT i386 >Organization: F5 networks >Environment: System: FreeBSD marka-k8we 8.0-CURRENT FreeBSD 8.0-CURRENT #22: Fri Oct = 17 15:18:26 PDT 2008 root@marka-k8we:/usr/obj/usr/src/sys/K8WE i386 CPU: AMD Opteron(tm) Processor 252 (2612.05-MHz 686-class CPU) Origin =3D "AuthenticAMD" Id =3D 0x20f51 Stepping =3D 1 Features=3D0x78bfbff = FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C= LFLUSH,MMX,FXSR,SSE,SSE2 Features2=3D0x1 SSE3=20 AMD Features=3D0xe2500800 SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!=20 AMD Features2=3D0x1 LAHF=20 real memory =3D 2146435072 (2047 MB) avail memory =3D 2087993344 (1991 MB) ACPI APIC Table: PTLTD APIC =20 FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 $ diff -b -u /usr/src/sys/i386/conf/GENERIC /usr/src/sys/i386/conf/K8WE --- /usr/src/sys/i386/conf/GENERIC 2008-10-17 14:33:05.000000000 = -0700 +++ /usr/src/sys/i386/conf/K8WE 2008-10-17 14:38:52.000000000 -0700 @@ -329,3 +329,18 @@ device fwip # IP over FireWire (RFC 2734,3146) device dcons # Dumb console driver device dcons_crom # Configuration ROM for dcons + +#K8WE options +options IPSEC +options TCP_SIGNATURE #include support for RFC 2385 +device crypto +device cryptodev + +device pf +device pflog + +device vlan $ cat /etc/pf.conf |grep -v "^#" ext_if=3D"external" # replace with actual external interface name = i.e., dc0 int_if=3D"internal" # replace with actual internal interface name = i.e., dc1 adm_if=3D"admin" scrub in all random-id no-df nat on $adm_if from $int_if:network to any -> $(adm_if) nat on $adm_if from $ext_if:network to any -> $(adm_if) pass in all pass out all >Description: On this test system, there is a repeatable panic during large file=20 transfers, when the box is forwarding packets over ipv6 [root@marka-k8we K8WE]$ kgdb ./kernel.debug /var/crash/vmcore.4 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain = conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for = details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0x38 fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0xc09af288 stack pointer =3D 0x28:0xe5c26a1c frame pointer =3D 0x28:0xe5c26b0c code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 0 (nfe1 taskq) Physical memory: 2030 MB Dumping 84 MB: 69 53 37 21 5 #0 doadump () at pcpu.h:221 221 __asm __volatile("movl %%fs:0,%0" : "=3Dr" (td)); (kgdb) l *0xc09af288 0xc09af288 is in ip6_forward (/usr/src/sys/netinet6/ip6_forward.c:420). 415 * address). We use a local copy of ip6_src, since = in6_setscope() 416 * will possibly modify its first argument. 417 * [draft-ietf-ipngwg-icmp-v3-04.txt, Section 3.1] 418 */ 419 src_in6 =3D ip6->ip6_src; 420 if (in6_setscope(&src_in6, rt->rt_ifp, &outzone)) { 421 /* XXX: this should not happen */ 422 V_ip6stat.ip6s_cantforward++; 423 V_ip6stat.ip6s_badscope++; 424 m_freem(m); (kgdb) where #0 doadump () at pcpu.h:221 #1 0xc04f34f9 in db_fncall (dummy1=3D-1067542808, dummy2=3D0, = dummy3=3D-440244288, dummy4=3D0xe5c267b0 "@#=D6=C0=DFx=C3=C0=D8") at = /usr/src/sys/ddb/db_command.c:548 #2 0xc04f38f1 in db_command (last_cmdp=3D0xc0d58bdc, cmd_table=3D0x0, = dopager=3D1) at /usr/src/sys/ddb/db_command.c:445 #3 0xc04f3a4a in db_command_loop () at = /usr/src/sys/ddb/db_command.c:498 #4 0xc04f58cc in db_trap (type=3D12, code=3D0) at = /usr/src/sys/ddb/db_main.c:229 #5 0xc0861755 in kdb_trap (type=3D12, code=3D0, tf=3D0xe5c269dc) at /usr/src/sys/kern/subr_kdb.c:534 #6 0xc0b7f8df in trap_fatal (frame=3D0xe5c269dc, eva=3D56) at /usr/src/sys/i386/i386/trap.c:934 #7 0xc0b7fb11 in trap_pfault (frame=3D0xe5c269dc, usermode=3D0, = eva=3D56) at /usr/src/sys/i386/i386/trap.c:856 #8 0xc0b80505 in trap (frame=3D0xe5c269dc) at = /usr/src/sys/i386/i386/trap.c:536 #9 0xc0b648bb in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #10 0xc09af288 in ip6_forward (m=3D0xc5ed8300, srcrt=3D0) at /usr/src/sys/netinet6/ip6_forward.c:420 #11 0xc09b1831 in ip6_input (m=3D0xc5ed8300) at /usr/src/sys/netinet6/ip6_input.c:722 #12 0xc08e1032 in netisr_dispatch (num=3D27, m=3D0xc5ed8300) at /usr/src/sys/net/netisr.c:178 #13 0xc08d8e01 in ether_demux (ifp=3D0xc574b400, m=3D0xc5ed8300) at /usr/src/sys/net/if_ethersubr.c:845 #14 0xc08d926f in ether_input (ifp=3D0xc574b400, m=3D0xc5ed8300) at /usr/src/sys/net/if_ethersubr.c:702 #15 0xc0b542fa in nfe_int_task (arg=3D0xc574c000, pending=3D1) at /usr/src/sys/dev/nfe/if_nfe.c:2116 #16 0xc086bfcb in taskqueue_run (queue=3D0xc575b080) at /usr/src/sys/kern/subr_taskqueue.c:282 #17 0xc086c128 in taskqueue_thread_loop (arg=3D0xc574c130) at /usr/src/sys/kern/subr_taskqueue.c:403 #18 0xc0811818 in fork_exit (callout=3D0xc086c0c0 = , arg=3D0xc574c130, frame=3D0xe5c26d38) at = /usr/src/sys/kern/kern_fork.c:810 #19 0xc0b64930 in fork_trampoline () at = /usr/src/sys/i386/i386/exception.s:270 (kgdb) frame 10 #10 0xc09af288 in ip6_forward (m=3D0xc5ed8300, srcrt=3D0) at /usr/src/sys/netinet6/ip6_forward.c:420 420 if (in6_setscope(&src_in6, rt->rt_ifp, &outzone)) { (kgdb) p rt $3 =3D (struct rtentry *) 0x0 >How-To-Repeat: ftp a large file over ipv6 through the box >Fix: I'm working on the thought that this might because of the vague=20 nat pf rule, so I've changed it to be specific to ipv4 via inet and specific inet4 addresses ranges right now. Currently unknown if this will work around it, however. ------_=_NextPart_001_01C932C6.E09BF380 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable [panic] Fatal Trap 12 in ip6_forward = (/usr/src/sys/netinet6/ip6_forward.c:420)

>Submitter-Id:  current-users
>Originator:    Mark Atkinson
>Organization:  F5 networks
>Confidential:  no
>Synopsis:      [panic] Fatal Trap 12 in = ip6_forward (/usr/src/sys/netinet6/ip6_forward.c:420)
>Severity:      serious
>Priority:      medium
>Category:      kern
>Class:         sw-bug
>Release:       FreeBSD 8.0-CURRENT = i386
>Environment:
System: FreeBSD marka-k8we 8.0-CURRENT FreeBSD 8.0-CURRENT #22: Fri Oct = 17 15:18:26 PDT 2008     = root@marka-k8we:/usr/obj/usr/src/sys/K8WE  i386

CPU: AMD Opteron(tm) Processor 252 (2612.05-MHz 686-class CPU)
Origin =3D "AuthenticAMD"  Id =3D 0x20f51  Stepping = =3D 1
Features=3D0x78bfbff = FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,C= LFLUSH,MMX,FXSR,SSE,SSE2
Features2=3D0x1 SSE3
AMD Features=3D0xe2500800 SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!
AMD Features2=3D0x1 LAHF
real memory  =3D 2146435072 (2047 MB)
avail memory =3D 2087993344 (1991 MB)
ACPI APIC Table:  = PTLTD          = APIC  
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0 (BSP): APIC ID:  0
cpu1 (AP): APIC ID:  1

$ diff -b -u  /usr/src/sys/i386/conf/GENERIC = /usr/src/sys/i386/conf/K8WE
--- /usr/src/sys/i386/conf/GENERIC      = 2008-10-17 14:33:05.000000000 -0700
+++ /usr/src/sys/i386/conf/K8WE 2008-10-17 14:38:52.000000000 -0700
@@ -329,3 +329,18 @@
 device         = fwip            # = IP over FireWire (RFC 2734,3146)
 device         = dcons           # Dumb = console driver
 device         = dcons_crom      # Configuration ROM for = dcons
+
+#K8WE options
+options        IPSEC
+options        = TCP_SIGNATURE          = #include support for RFC 2385
+device         crypto
+device         cryptodev
+
+device         pf
+device         pflog
+
+device         vlan

$ cat /etc/pf.conf |grep -v "^#"

ext_if=3D"external"       # = replace with actual external interface name i.e., dc0
int_if=3D"internal"       # = replace with actual internal interface name i.e., dc1
adm_if=3D"admin"

scrub in all random-id no-df

nat on $adm_if from $int_if:network to any -> $(adm_if)
nat on $adm_if from $ext_if:network to any -> $(adm_if)

pass in all
pass out all

>Description:

On this test system, there is a repeatable panic during large file
transfers, when the box is forwarding packets over ipv6

[root@marka-k8we K8WE]$ kgdb ./kernel.debug /var/crash/vmcore.4
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you = are
welcome to change it and/or distribute copies of it under certain = conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show = warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid =3D 0; apic id =3D 00
fault virtual address   =3D 0x38
fault = code           &nb= sp;  =3D supervisor read, page not present
instruction pointer     =3D 0x20:0xc09af288
stack = pointer           =3D = 0x28:0xe5c26a1c
frame = pointer           =3D = 0x28:0xe5c26b0c
code = segment           = =3D base 0x0, limit 0xfffff, type 0x1b
            &= nbsp;           =3D = DPL 0, pres 1, def32 1, gran 1
processor eflags        =3D interrupt = enabled, resume, IOPL =3D 0
current process         =3D 0 = (nfe1 taskq)
Physical memory: 2030 MB
Dumping 84 MB: 69 53 37 21 5

#0  doadump () at pcpu.h:221
221           &nbs= p; __asm __volatile("movl %%fs:0,%0" : "=3Dr" = (td));
(kgdb) l *0xc09af288
0xc09af288 is in ip6_forward = (/usr/src/sys/netinet6/ip6_forward.c:420).
415           &nbs= p;  * address).  We use a local copy of ip6_src, since = in6_setscope()
416           &nbs= p;  * will possibly modify its first argument.
417           &nbs= p;  * [draft-ietf-ipngwg-icmp-v3-04.txt, Section 3.1]
418           &nbs= p;  */
419           &nbs= p; src_in6 =3D ip6->ip6_src;
420           &nbs= p; if (in6_setscope(&src_in6, rt->rt_ifp, &outzone)) {
421           &nbs= p;         /* XXX: this should = not happen */
422           &nbs= p;         = V_ip6stat.ip6s_cantforward++;
423           &nbs= p;         = V_ip6stat.ip6s_badscope++;
424           &nbs= p;         m_freem(m);


(kgdb) where
#0  doadump () at pcpu.h:221
#1  0xc04f34f9 in db_fncall (dummy1=3D-1067542808, dummy2=3D0, = dummy3=3D-440244288,
    dummy4=3D0xe5c267b0 = "@#=D6=C0=DFx=C3=C0=D8") at = /usr/src/sys/ddb/db_command.c:548
#2  0xc04f38f1 in db_command (last_cmdp=3D0xc0d58bdc, = cmd_table=3D0x0, dopager=3D1)
        at = /usr/src/sys/ddb/db_command.c:445
#3  0xc04f3a4a in db_command_loop () at = /usr/src/sys/ddb/db_command.c:498
#4  0xc04f58cc in db_trap (type=3D12, code=3D0) at = /usr/src/sys/ddb/db_main.c:229
#5  0xc0861755 in kdb_trap (type=3D12, code=3D0, = tf=3D0xe5c269dc)
        at = /usr/src/sys/kern/subr_kdb.c:534
#6  0xc0b7f8df in trap_fatal (frame=3D0xe5c269dc, eva=3D56)
        at = /usr/src/sys/i386/i386/trap.c:934
#7  0xc0b7fb11 in trap_pfault (frame=3D0xe5c269dc, usermode=3D0, = eva=3D56)
        at = /usr/src/sys/i386/i386/trap.c:856
#8  0xc0b80505 in trap (frame=3D0xe5c269dc) at = /usr/src/sys/i386/i386/trap.c:536
#9  0xc0b648bb in calltrap () at = /usr/src/sys/i386/i386/exception.s:165
#10 0xc09af288 in ip6_forward (m=3D0xc5ed8300, srcrt=3D0)
        at = /usr/src/sys/netinet6/ip6_forward.c:420
#11 0xc09b1831 in ip6_input (m=3D0xc5ed8300)
        at = /usr/src/sys/netinet6/ip6_input.c:722
#12 0xc08e1032 in netisr_dispatch (num=3D27, m=3D0xc5ed8300)
        at = /usr/src/sys/net/netisr.c:178
#13 0xc08d8e01 in ether_demux (ifp=3D0xc574b400, m=3D0xc5ed8300)
        at = /usr/src/sys/net/if_ethersubr.c:845
#14 0xc08d926f in ether_input (ifp=3D0xc574b400, m=3D0xc5ed8300)
        at = /usr/src/sys/net/if_ethersubr.c:702
#15 0xc0b542fa in nfe_int_task (arg=3D0xc574c000, pending=3D1)
        at = /usr/src/sys/dev/nfe/if_nfe.c:2116
#16 0xc086bfcb in taskqueue_run (queue=3D0xc575b080)
        at = /usr/src/sys/kern/subr_taskqueue.c:282
#17 0xc086c128 in taskqueue_thread_loop (arg=3D0xc574c130)
        at = /usr/src/sys/kern/subr_taskqueue.c:403
#18 0xc0811818 in fork_exit (callout=3D0xc086c0c0 = <taskqueue_thread_loop>,
        arg=3D0xc574c130, = frame=3D0xe5c26d38) at /usr/src/sys/kern/kern_fork.c:810
#19 0xc0b64930 in fork_trampoline () at = /usr/src/sys/i386/i386/exception.s:270

(kgdb) frame 10
#10 0xc09af288 in ip6_forward (m=3D0xc5ed8300, srcrt=3D0)
    at /usr/src/sys/netinet6/ip6_forward.c:420
420           &nbs= p; if (in6_setscope(&src_in6, rt->rt_ifp, &outzone)) {
(kgdb) p rt
$3 =3D (struct rtentry *) 0x0

>How-To-Repeat:

ftp a large file over ipv6 through the box


>Fix:

I'm working on the thought that this might because of the vague
nat pf rule, so I've changed it to be specific to ipv4 via inet and
specific inet4 addresses ranges right now.  Currently unknown if = this
will work around it, however.

------_=_NextPart_001_01C932C6.E09BF380-- >Release-Note: >Audit-Trail: >Unformatted: This is a multi-part message in MIME format. ------_=_NextPart_001_01C932C6.E09BF380 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable (/usr/src/sys/netinet6/ip6_forward.c:420)