From owner-freebsd-stable@FreeBSD.ORG Fri Aug 14 22:38:52 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD75A1065672 for ; Fri, 14 Aug 2009 22:38:52 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.171]) by mx1.freebsd.org (Postfix) with ESMTP id 4B1128FC43 for ; Fri, 14 Aug 2009 22:38:52 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-011-250.pools.arcor-ip.net [88.66.11.250]) by mrelayeu.kundenserver.de (node=mreu0) with ESMTP (Nemesis) id 0MKuxg-1Mc5QA3wuu-000RRY; Sat, 15 Aug 2009 00:38:51 +0200 Received: (qmail 17430 invoked from network); 14 Aug 2009 22:38:50 -0000 Received: from kvm.laiers.local (HELO kvm.localnet) (192.168.4.200) by mx.laiers.local with SMTP; 14 Aug 2009 22:38:50 -0000 From: Max Laier Organization: FreeBSD To: freebsd-stable@freebsd.org Date: Sat, 15 Aug 2009 00:38:49 +0200 User-Agent: KMail/1.12.0 (Linux/2.6.30-ARCH; KDE/4.3.0; x86_64; ; ) References: <20090812191609.GA60973@server.vk2pj.dyndns.org> In-Reply-To: <20090812191609.GA60973@server.vk2pj.dyndns.org> MIME-Version: 1.0 Content-Type: Multipart/Mixed; boundary="Boundary-00=_6dehKweeb3P+Few" Message-Id: <200908150038.50281.max@love2party.net> X-Provags-ID: V01U2FsdGVkX198Xn1h2nOL/BIDJDf7oKDXAvHUB7xRN3tuJJK ifSyGQOx6Ll5xeR+/gb+Z3ePJGIGxs2++7iokMHwzr0R0vd2A9 Jg/+DUxfNefhWYHwf0nUQ== Cc: Peter Jeremy Subject: Re: Panic due to junk pointer in pf(4) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Aug 2009 22:38:53 -0000 --Boundary-00=_6dehKweeb3P+Few Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable On Wednesday 12 August 2009 21:16:09 Peter Jeremy wrote: > My firewall (7.2p3/i386) recently panic'd: > Fatal trap 12: page fault while in kernel mode > fault virtual address =3D 0x1065e > fault code =3D supervisor read, page not present > ... > I have a crashdump that shows: > #6 0xc06c9c1b in calltrap () at /usr/src/sys/i386/i386/exception.s:159 > #7 0xc044ecd0 in pf_state_tree_lan_ext_RB_REMOVE_COLOR (head=3D0xc2a256a= 8, > parent=3D0xc442c6a0, elm=3D0xc40aa8e0) at > /usr/src/sys/contrib/pf/net/pf.c:391 #8 0xc044ef79 in > pf_state_tree_lan_ext_RB_REMOVE (head=3D0xc2a256a8, elm=3D0xc404a11c) at > /usr/src/sys/contrib/pf/net/pf.c:391 > #9 0xc045383e in pf_unlink_state (cur=3D0xc404a11c) > at /usr/src/sys/contrib/pf/net/pf.c:1158 > #10 0xc0456b6e in pf_purge_expired_states (maxcheck=3D119) > at /usr/src/sys/contrib/pf/net/pf.c:1242 > #11 0xc04570f9 in pf_purge_thread (v=3D0x0) > at /usr/src/sys/contrib/pf/net/pf.c:998 > #12 0xc0535781 in fork_exit (callout=3D0xc0456f50 , arg= =3D0x0, > frame=3D0xd2d4cd38) at /usr/src/sys/kern/kern_fork.c:810 > #13 0xc06c9c90 in fork_trampoline () at > /usr/src/sys/i386/i386/exception.s:264 > > Working up, 'parent' in pf_state_tree_lan_ext_RB_REMOVE_COLOR() has > a garbage u.s.entry_lan_ext: > (kgdb) p parent->u > $3 =3D {s =3D {entry_lan_ext =3D {rbe_left =3D 0x10602, rbe_right =3D 0x5= 0000, > rbe_parent =3D 0xc40aa8e0, rbe_color =3D -1002258432}, entry_ext_gw= y =3D { > rbe_left =3D 0xc3c42238, rbe_right =3D 0x1, rbe_parent =3D 0x0, > rbe_color =3D 0}, entry_id =3D {rbe_left =3D 0xc3c54470, rbe_right = =3D 0x0, > rbe_parent =3D 0x0, rbe_color =3D 0}, entry_list =3D {tqe_next =3D > 0xc41f9e6c, tqe_prev =3D 0x0}, kif =3D 0xc442c58c}, > ifname =3D "\002\006\001\000\000\000\005\000=E0=A8\n=C4\000=C0B=C4"} > > Does anyone have any suggestions on where to look next? You could try the attached patch that I just set to re@ for inclusion. The= re=20 is an obvious error in how I handle the pf_consistency_lock in the purge=20 thread that might well be the culprit for the error you are seeing. I assu= me=20 you can't trigger the panic at will, though. In any case I'd be interested= in=20 your feedback, thanks. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --Boundary-00=_6dehKweeb3P+Few Content-Type: text/x-patch; charset="UTF-8"; name="pfpurge_lock.diff" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="pfpurge_lock.diff" Index: sys/contrib/pf/net/pfvar.h =================================================================== --- sys/contrib/pf/net/pfvar.h (revision 196216) +++ sys/contrib/pf/net/pfvar.h (working copy) @@ -1593,8 +1593,13 @@ extern struct pool pf_state_pl, pf_altq_pl, pf_p extern struct pool pf_state_scrub_pl; #endif extern void pf_purge_thread(void *); +#ifdef __FreeBSD__ +extern int pf_purge_expired_src_nodes(int); +extern int pf_purge_expired_states(u_int32_t, int); +#else extern void pf_purge_expired_src_nodes(int); extern void pf_purge_expired_states(u_int32_t); +#endif extern void pf_unlink_state(struct pf_state *); extern void pf_free_state(struct pf_state *); extern int pf_insert_state(struct pfi_kif *, Index: sys/contrib/pf/net/pf.c =================================================================== --- sys/contrib/pf/net/pf.c (revision 196216) +++ sys/contrib/pf/net/pf.c (working copy) @@ -971,6 +971,9 @@ void pf_purge_thread(void *v) { int nloops = 0, s; +#ifdef __FreeBSD__ + int locked; +#endif for (;;) { tsleep(pf_purge_thread, PWAIT, "pftm", 1 * hz); @@ -978,14 +981,19 @@ pf_purge_thread(void *v) #ifdef __FreeBSD__ sx_slock(&pf_consistency_lock); PF_LOCK(); + locked = 0; if (pf_end_threads) { - pf_purge_expired_states(pf_status.states); + PF_UNLOCK(); + sx_sunlock(&pf_consistency_lock); + sx_xlock(&pf_consistency_lock); + PF_LOCK(); + pf_purge_expired_states(pf_status.states, 1); pf_purge_expired_fragments(); - pf_purge_expired_src_nodes(0); + pf_purge_expired_src_nodes(1); pf_end_threads++; - sx_sunlock(&pf_consistency_lock); + sx_xunlock(&pf_consistency_lock); PF_UNLOCK(); wakeup(pf_purge_thread); kproc_exit(0); @@ -994,20 +1002,44 @@ pf_purge_thread(void *v) s = splsoftnet(); /* process a fraction of the state table every second */ +#ifdef __FreeBSD__ + if(!pf_purge_expired_states(1 + (pf_status.states + / pf_default_rule.timeout[PFTM_INTERVAL]), 0)) { + PF_UNLOCK(); + sx_sunlock(&pf_consistency_lock); + sx_xlock(&pf_consistency_lock); + PF_LOCK(); + locked = 1; + + pf_purge_expired_states(1 + (pf_status.states + / pf_default_rule.timeout[PFTM_INTERVAL]), 1); + } +#else pf_purge_expired_states(1 + (pf_status.states / pf_default_rule.timeout[PFTM_INTERVAL])); +#endif /* purge other expired types every PFTM_INTERVAL seconds */ if (++nloops >= pf_default_rule.timeout[PFTM_INTERVAL]) { pf_purge_expired_fragments(); - pf_purge_expired_src_nodes(0); + if (!pf_purge_expired_src_nodes(locked)) { + PF_UNLOCK(); + sx_sunlock(&pf_consistency_lock); + sx_xlock(&pf_consistency_lock); + PF_LOCK(); + locked = 1; + pf_purge_expired_src_nodes(1); + } nloops = 0; } splx(s); #ifdef __FreeBSD__ PF_UNLOCK(); - sx_sunlock(&pf_consistency_lock); + if (locked) + sx_xunlock(&pf_consistency_lock); + else + sx_sunlock(&pf_consistency_lock); #endif } } @@ -1056,8 +1088,13 @@ pf_state_expires(const struct pf_state *state) return (state->expire + timeout); } +#ifdef __FreeBSD__ +int +pf_purge_expired_src_nodes(int waslocked) +#else void pf_purge_expired_src_nodes(int waslocked) +#endif { struct pf_src_node *cur, *next; int locked = waslocked; @@ -1068,12 +1105,8 @@ pf_purge_expired_src_nodes(int waslocked) if (cur->states <= 0 && cur->expire <= time_second) { if (! locked) { #ifdef __FreeBSD__ - if (!sx_try_upgrade(&pf_consistency_lock)) { - PF_UNLOCK(); - sx_sunlock(&pf_consistency_lock); - sx_xlock(&pf_consistency_lock); - PF_LOCK(); - } + if (!sx_try_upgrade(&pf_consistency_lock)) + return (0); #else rw_enter_write(&pf_consistency_lock); #endif @@ -1100,6 +1133,10 @@ pf_purge_expired_src_nodes(int waslocked) #else rw_exit_write(&pf_consistency_lock); #endif + +#ifdef __FreeBSD__ + return (1); +#endif } void @@ -1202,12 +1239,21 @@ pf_free_state(struct pf_state *cur) pf_status.states--; } +#ifdef __FreeBSD__ +int +pf_purge_expired_states(u_int32_t maxcheck, int waslocked) +#else void pf_purge_expired_states(u_int32_t maxcheck) +#endif { static struct pf_state *cur = NULL; struct pf_state *next; +#ifdef __FreeBSD__ + int locked = waslocked; +#else int locked = 0; +#endif while (maxcheck--) { /* wrap to start of list when we hit the end */ @@ -1224,12 +1270,8 @@ pf_purge_expired_states(u_int32_t maxcheck) /* free unlinked state */ if (! locked) { #ifdef __FreeBSD__ - if (!sx_try_upgrade(&pf_consistency_lock)) { - PF_UNLOCK(); - sx_sunlock(&pf_consistency_lock); - sx_xlock(&pf_consistency_lock); - PF_LOCK(); - } + if (!sx_try_upgrade(&pf_consistency_lock)) + return (0); #else rw_enter_write(&pf_consistency_lock); #endif @@ -1241,12 +1283,8 @@ pf_purge_expired_states(u_int32_t maxcheck) pf_unlink_state(cur); if (! locked) { #ifdef __FreeBSD__ - if (!sx_try_upgrade(&pf_consistency_lock)) { - PF_UNLOCK(); - sx_sunlock(&pf_consistency_lock); - sx_xlock(&pf_consistency_lock); - PF_LOCK(); - } + if (!sx_try_upgrade(&pf_consistency_lock)) + return (0); #else rw_enter_write(&pf_consistency_lock); #endif @@ -1257,10 +1295,13 @@ pf_purge_expired_states(u_int32_t maxcheck) cur = next; } - if (locked) #ifdef __FreeBSD__ + if (!waslocked && locked) sx_downgrade(&pf_consistency_lock); + + return (1); #else + if (locked) rw_exit_write(&pf_consistency_lock); #endif } --Boundary-00=_6dehKweeb3P+Few--