From owner-freebsd-stable@FreeBSD.ORG Fri Jan 22 19:55:48 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4DDEF10656A8 for ; Fri, 22 Jan 2010 19:55:48 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id EB24D8FC25 for ; Fri, 22 Jan 2010 19:55:47 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-045-145.pools.arcor-ip.net [88.66.45.145]) by mrelayeu.kundenserver.de (node=mreu1) with ESMTP (Nemesis) id 0MXCaX-1NMiIB1tRN-00WKGx; Fri, 22 Jan 2010 20:55:46 +0100 Received: (qmail 91371 invoked from network); 22 Jan 2010 19:55:46 -0000 Received: from f8x64.laiers.local (192.168.4.188) by mx.laiers.local with SMTP; 22 Jan 2010 19:55:46 -0000 From: Max Laier Organization: FreeBSD To: John Baldwin Date: Fri, 22 Jan 2010 20:55:45 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-RELEASE-p2; KDE/4.3.4; amd64; ; ) References: <4B58280C.50602@smeets.im> <201001221818.20409.max@love2party.net> <201001221349.19810.jhb@freebsd.org> In-Reply-To: <201001221349.19810.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201001222055.45980.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1/Iwmy3qcgcq0ldKb3nriEo3vXNyYhYOyMPFYD Wf6eWbh/w32KItgPRqR2u+RWzV0SAT80Ya+jA3OzdNui4uQYvb 8uGdrsGt/YIzrRCGOLTtw== Cc: Bjoern Zeeb , Luigi Rizzo , Florian Smeets , freebsd-stable@freebsd.org Subject: Re: 7.2-STABLE page fault with kernel from 12.01.2010 / crashinfo available X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 19:55:48 -0000 On Friday 22 January 2010 19:49:19 John Baldwin wrote: > On Friday 22 January 2010 12:18:20 pm Max Laier wrote: > > On Friday 22 January 2010 15:20:13 John Baldwin wrote: > > > On Friday 22 January 2010 3:08:45 am Florian Smeets wrote: > > > > ... > > > > > > If it really is IPsec traffic then there are no rewrite rules only 10 > > > > pf pass rules on the enc0 interface and a "scrub in all" rule. > > > > > > > > Perhaps it matters that i have these set: > > > > > > > > net.enc.out.ipsec_bpf_mask=0x00000001 > > > > net.enc.out.ipsec_filter_mask=0x00000001 > > > > net.enc.in.ipsec_bpf_mask=0x00000002 > > > > net.enc.in.ipsec_filter_mask=0x00000002 > > > > > > > > so that i can filter the "encapsulated" traffic. > > > > > > I have no idea, I've cc'd mlaier@ (pf) and bz@ (ipsec) to see if they > > > have any ideas. > > > > pf could be the culprit if it were present in the trace, but I don't see > > any sign of it: > > > > On Thursday 21 January 2010 11:10:20 Florian Smeets wrote: > > > #7 0xc0572e48 in m_copydata (m=0x0, off=0, len=40, cp=0xc23cced8 > > > "\203??b??\237\f)h?M\220\224?\023?\205K(e??s?\"???k?oQ?~\223\020g\030") > > > at /usr/src/sys/kern/uipc_mbuf.c:815 > > > #8 0xc05f8b28 in ip_forward (m=0xc23dc900, srcrt=0) at > > > /usr/src/sys/netinet/ip_input.c:1307 > > > #9 0xc05fa30c in ip_input (m=0xc23dc900) at > > > /usr/src/sys/netinet/ip_input.c:609 > > > #10 0xc05c83d5 in netisr_dispatch (num=2, m=0xc23dc900) at > > > /usr/src/sys/net/netisr.c:185 > > > #11 0xc05bf581 in ether_demux (ifp=0xc20a4800, m=0xc23dc900) at > > > /usr/src/sys/net/if_ethersubr.c:834 > > > #12 0xc05bf973 in ether_input (ifp=0xc20a4800, m=0xc23dc900) at > > > /usr/src/sys/net/if_ethersubr.c:692 > > > #13 0xc04b8749 in sis_rxeof (sc=0xc2093800) at > > > /usr/src/sys/dev/sis/if_sis.c:1476 > > > #14 0xc04b8973 in sis_intr (arg=0xc2093800) at > > > /usr/src/sys/dev/sis/if_sis.c:1667 > > > #15 0xc050344b in ithread_loop (arg=0xc20ab410) at > > > /usr/src/sys/kern/kern_intr.c:1126 > > > #16 0xc04ffe36 in fork_exit (callout=0xc05032a0 , > > > arg=0xc20ab410, frame=0xc1f15d38) at /usr/src/sys/kern/kern_fork.c:811 > > > #17 0xc06d9180 in fork_trampoline () at > > > /usr/src/sys/i386/i386/exception.s:271 > > > > pf does change the byte order in the pfil hook, but changes it back on > > return to the stack either when returning from the hook or when calling > > back into the stack. There have been some issues where we missed returns > > to the stack that would result in this situation, but since pf is not in > > the trace, this is clearly not the case here. > > That isn't necessarily the case. ip_input() invokes the PFIL hooks which > then return after possibly modifying the packet. The (possibly modified) > packet is then passed to ip_forward() from ip_input(). If the PFIL hook > modified the packet and returned ip_len in network byte order then it would > cause this breakage without showing up in the stack trace. What I meant to say was: if we return from the pfil hook we either report error (and/or consume the mbuf) or switch back to network byte order: http://fxr.watson.org/fxr/source/contrib/pf/net/pf_ioctl.c?v=FREEBSD72#L3655 While I can't completely rule out that there is a double flip happening in some obscure path through pf, I very much doubt this is what is going on (or there would be more reports and it would happen straight away, not only after passing some data). A quick search through the sources also didn't turn up any red flags. All byte order operations inside pf are either temporary or performed on a properly copied packet that is send back through the stack (icmp error, tcp packet, ...). Depending on how easily this can be reproduced, my money is on modifying a shared mbuf (possibly inside enc(4)). Regards, -- Max