From owner-freebsd-net@FreeBSD.ORG Fri Mar 2 08:00:34 2012 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BA9801065670 for ; Fri, 2 Mar 2012 08:00:34 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A85868FC14 for ; Fri, 2 Mar 2012 08:00:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q2280YS1003551 for ; Fri, 2 Mar 2012 08:00:34 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q2280YWZ003550; Fri, 2 Mar 2012 08:00:34 GMT (envelope-from gnats) Date: Fri, 2 Mar 2012 08:00:34 GMT Message-Id: <201203020800.q2280YWZ003550@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: "Eugene M. Zheganin" Cc: Subject: Re: kern/164400: [ipsec] immediate crash after the start of ipsec processing X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: "Eugene M. Zheganin" List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2012 08:00:34 -0000 The following reply was made to PR kern/164400; it has been noted by GNATS. From: "Eugene M. Zheganin" To: bug-followup@FreeBSD.org, eugene@zhegan.in Cc: Subject: Re: kern/164400: [ipsec] immediate crash after the start of ipsec processing Date: Fri, 02 Mar 2012 13:52:09 +0600 sorry for the delay. the thing is, 9.0 with pf enabled is almost useless when the kernel is build with WITNESS/WITNESS_KDB - it keeps witnessing to KDB in infinite cycle after entering multiuser. I took the dump with the debug kernel. Here's the stuff: Fatal trap 12: page fault while in kernel mode fault virtual address = 0x60 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0965a55 stack pointer = 0x28:0xccf145a0 frame pointer = 0x28:0xccf145c8 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2010 (ping) trap number = 12 panic: page fault KDB: stack backtrace: db_trace_self_wrapper(c0b1382c,70797420,78302065,a6231,bfbfe518,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0b0fa75,c0bf5900,c0aaaeaa,ccf1444c,ccf1444c,...) at kdb_backtrace+0x2a panic(c0aaaeaa,c0b5fa29,c28af770,1,1,...) at panic+0xaf trap_fatal(97d8,4fca2d23,4,ccf144b0,60,...) at trap_fatal+0x2f3 trap_pfault(ccf144e2,6f20,40000,ccf144f4,c28a8000,...) at trap_pfault+0xac trap(ccf14560) at trap+0x495 calltrap() at calltrap+ <3>stray irq7 0x6 --- trap 0xc, eip = 0xc0965a55, esp = 0xccf145a0, ebp = 0xccf145c8 --- ipsec_process_done(c286fc00,c28f3880,3a3,4,c0b0e147,...) at ipsec_process_done+0x195 esp_output_cb(c28fa000,91214f13,c0bc1220,ccf14630,c098c8a4,...) at esp_output_cb+0x1aa crypto_done(c28fa000,c286fcd4,ccf146c8,8,ccf147ec,...) at crypto_done+0xb7 swcr_process(c2053680,c28fa000,0,2,c2255740,...) at swcr_process+0x12ce crypto_invoke(1 <3>stray irq7 01,0,c0bc1220,c28f8d80,c2255740,...) at crypto_invoke+0x141 crypto_dispatch(c28fa000,c0b3ef45,371,ccf148a7,c28f6300,...) at crypto_dispatch+0x64 esp_output(c286fc00,c28f3880,0,14,9,...) at esp_output+0x5a6 ipsec4_process_packet(c286fc00,c28f3880,1,0,0,...) at ipsec4_process_packet+0x29f ip_ipsec_output(ccf149b0,0,ccf14a00,ccf149b8,201,...) at ip_ipsec_ <3>stray irq7 output+0x1e0 ip_output(c286fc00,0,c2818220,1,0,...) at ip_output+0x810 gre_output(c20f7400,c286fc00,ccf14ab4,ccf14aa4,c280d9d8,...) at gre_output+0x469 ip_output(c286fc00,0,0,20,0,...) at ip_output+0xaa6 rip_output(c286fc00,c2686b60,3e0310ac,ccf14b7c,c07f5d8d,...) at rip_output+0x2ff rip_send(c2686b60,0,c286fc00,c284c320,0,...) at rip_send+0x76 sosend_generic(c2686b60,c284c320,ccf14bd4,0,0,...) at sosend_generic+0x50d sosend(c2686b60,c284c320,ccf14bd4,0,0,...) at sosend+0x3f kern_sendit(c28af5c0,3,ccf14c48,0,0,...) at kern_sendit+0x1d4 sendit(0,c284c320,10,ccf14c64,1,...) at sendit+0xb1 sys_sendto(c28af5c0,ccf14cec,c,c,246,...) at sys_sendto+0x48 syscall(ccf14d28) at syscall+0x284 Xint0x80_syscall() at Xint0x80_syscall+0x21 --- syscall (133, FreeBSD ELF32, sys_sendto), eip = 0x2818b237, esp = 0xbfbee70c, ebp = 0xbfbee748 --- Uptime: 10m29s Physical memory: 243 MB Dumping 42 MB: 27 11 No symbol "stopped_cpus" in current context. No symbol "stoppcbs" in current context. Reading symbols from /boot/kernel/if_gre.ko...done. Loaded symbols for /boot/kernel/if_gre.ko #0 doadump (textdump=1) at pcpu.h:244 244 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump (textdump=1) at pcpu.h:244 #1 0xc078cf75 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:442 #2 0xc078c8ee in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:607 #3 0xc0a6cf53 in trap_fatal (frame=0xccf14560, eva=96) at /usr/src/sys/i386/i386/trap.c:975 #4 0xc0a6d00c in trap_pfault (frame=0xccf14560, usermode=0, eva=96) at /usr/src/sys/i386/i386/trap.c:839 #5 0xc0a6dc45 in trap (frame=0xccf14560) at /usr/src/sys/i386/i386/trap.c:558 #6 0xc0a58e1c in calltrap () at /usr/src/sys/i386/i386/exception.s:168 #7 0xc0965a55 in ipsec_process_done (m=0xc286fc00, isr=0xc28f3880) at /usr/src/sys/netipsec/ipsec_output.c:170 #8 0xc0974dba in esp_output_cb (crp=0xc28fa000) at /usr/src/sys/netipsec/xform_esp.c:1007 #9 0xc09848f7 in crypto_done (crp=0xc28fa000) at /usr/src/sys/opencrypto/crypto.c:1156 #10 0xc098777e in swcr_process (dev=0xc2053680, crp=0xc28fa000, hint=0) at /usr/src/sys/opencrypto/cryptosoft.c:1054 #11 0xc0985991 in crypto_invoke (cap=0xc2053680, crp=0xc28fa000, hint=0) at cryptodev_if.h:53 #12 0xc0985db4 in crypto_dispatch (crp=0xc28fa000) at /usr/src/sys/opencrypto/crypto.c:806 #13 0xc0975416 in esp_output (m=0xc286fc00, isr=0xc28f3880, mp=0x0, skip=20, protoff=9) at /usr/src/sys/netipsec/xform_esp.c:907 #14 0xc0965e2f in ipsec4_process_packet (m=0xc286fc00, isr=0xc28f3880, flags=1, tunalready=0) at /usr/src/sys/netipsec/ipsec_output.c:580 #15 0xc08b09d0 in ip_ipsec_output (m=0xccf149b0, inp=0x0, flags=0xccf14a00, error=0xccf149b8) at /usr/src/sys/netinet/ip_ipsec.c:353 #16 0xc08b2680 in ip_output (m=0xc286fc00, opt=0x0, ro=0xc2818220, flags=1, imo=0x0, inp=0x0) at /usr/src/sys/netinet/ip_output.c:480 #17 0xc28e1259 in gre_output () from /boot/kernel/if_gre.ko #18 0xc08b2916 in ip_output (m=0xc286fc00, opt=0x0, ro=0xccf14aa4, flags=Variable "flags" is not available. ) at /usr/src/sys/netinet/ip_output.c:631 #19 0xc08b468f in rip_output (m=0xc286fc00, so=0xc2686b60, dst=1040388268) at /usr/src/sys/netinet/raw_ip.c:517 #20 0xc08b4776 in rip_send (so=0xc2686b60, flags=0, m=0xc286fc00, nam=0xc284c320, control=0x0, td=0xc28af5c0) at /usr/src/sys/netinet/raw_ip.c:994 #21 0xc07f5d8d in sosend_generic (so=0xc2686b60, addr=0xc284c320, uio=0xccf14bd4, top=0xc286fc00, control=0x0, flags=0, td=0xc28af5c0) at /usr/src/sys/kern/uipc_socket.c:1303 #22 0xc07f159f in sosend (so=0xc2686b60, addr=0xc284c320, uio=0xccf14bd4, top=0x0, control=0x0, flags=0, td=0xc28af5c0) at /usr/src/sys/kern/uipc_socket.c:1347 #23 0xc07fbb44 in kern_sendit (td=0xc28af5c0, s=3, mp=0xccf14c48, flags=0, control=0x0, segflg=UIO_USERSPACE) at /usr/src/sys/kern/uipc_syscalls.c:810 #24 0xc07fbd51 in sendit (td=0xc28af5c0, s=3, mp=0xccf14c48, flags=0) at /usr/src/sys/kern/uipc_syscalls.c:738 #25 0xc07fbe68 in sys_sendto (td=0xc28af5c0, uap=0xccf14cec) at /usr/src/sys/kern/uipc_syscalls.c:862 #26 0xc0a6d414 in syscall (frame=0xccf14d28) at subr_syscall.c:131 #27 0xc0a58e81 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:266 #28 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?)