From owner-freebsd-pf@freebsd.org Mon Aug 13 15:59:20 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 697E01076512 for ; Mon, 13 Aug 2018 15:59:20 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 137547FB59; Mon, 13 Aug 2018 15:59:20 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 99E3423400; Mon, 13 Aug 2018 15:59:19 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8rgnodwri04zzlnkb79.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:dd66:2162:6071:50b5]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 4E0B752882; Mon, 13 Aug 2018 17:59:17 +0200 (CEST) From: "Kristof Provost" To: "Kajetan Staszkiewicz" Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Mon, 13 Aug 2018 17:59:15 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: In-Reply-To: <2313127.kTuY2QdDqf@energia> References: <8680316.SccKl5VnxN@energia> <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> <2313127.kTuY2QdDqf@energia> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 15:59:20 -0000 This is an OpenPGP/MIME signed message (RFC 3156 and 4880). --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 13 Aug 2018, at 17:06, Kajetan Staszkiewicz wrote: > On Monday, 13 August 2018 15:22:33 CEST Kristof Provost wrote: >> rules (and associated tables) won=E2=80=99t just go away while there=E2= =80=99s still >> state, > > This is mostly what I wanted to ask about in this message. How is it en= sured > that table and counters are gone only after everybody stops using them?= What > if I delete a table, then change ruleset, but there is still active con= nection > keeping a state? I really had hard time finding how this is guarded in = source. > pf keeps rules around until there are no more states left referencing the= rule. Look at pf_commit_rules(): The old rules are unlinked rather than removed= =2E They=E2=80=99re kept on the V_pf_unlinked rules list. Every so often pf runs through all states (in pf_purge_thread()) to mark = their associated rules as still referenced. Only rules which are not refe= renced by any state are removed. This means that while there=E2=80=99s still a state which was created by = the rule (and can thus put packets towards its table), the rule will exis= t. Once the state goes away it=E2=80=99ll still take one full iteration thro= ugh all states before the rule can be freed. Hence my statement that it=E2=80=99s highly unlikely (pretty much impossi= ble) for us to run into a situation where the rule no longer exists. >> I don=E2=80=99t want to re-take the rules lock for this, so my current= >> thinking is that the best approach would be to already get rid of the >> potential memory leak by just always allocating the pfrke_counters whe= n >> the table is created (i.e. when the rule is first set). That might was= te >> a little memory if we didn=E2=80=99t need it, but it should simplify t= hings a >> bit. > >> We can resolve the counting issue by using the counter_u64_*() functio= ns >> for them. We should be able to get away with not locking this. > > Sure, I can use counter(9). The question, as always with my patches, is= what > can go to FreeBSD and what won't go. > > My current goal is to modify round-robin pf target to always point to t= able > entry with least amount of states. > > As I see it for now: > 1. Modify pfrke_counters to be always allocated. > 2. Rewrite pfrke_counters to use counter(9). > 3. Provide state counter in pfrke_counters. > 4. Modify round-robin target. > > 1. and 2. make a good PR. I'm not sure about 3. Do you want patches for= least- > connections target too? I want to just replace existing round-robin but= if > there is any chance of getting it into kernel code, I could make it wor= k as > new target in pf.conf. > Do you have a bit more information about your use case? What are you tryi= ng to accomplish with this change? > There are some more issues I found around pf_map_addr. Some of them I > mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092= =2E Some > more came out while working on this least-states loadbalancing. I will = group > them into something meaningful and make another PR for them. > Yeah, that bug is still on my todo list somewhere, but things are extreme= ly hectic at the moment, and I can=E2=80=99t make any promises about when= I=E2=80=99ll have time for it. Regards, Kristof --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQFDBAEBCAAtFiEEhvumznUbtMyaDlFyb8TccfteSkYFAltxqtMPHGtwQGZyZWVi c2Qub3JnAAoJEG/E3HH7XkpGlJsH/iDZumm2TZ038vQgWpi3Z93fd7L/evWip09/ N4e1T4eVCUAPqY/fTBs7EbJTaxkCFSwuexkLGWULeO6Q19zJ0ck34ufOzw8DGgAN uJNkzN4j6+ny3mkYHsdAZ4e0JE3wJYwQFZeQRTWu4SQq0J0myX+1Sztjiv/Uh2Tq JcmemUnVdIOwhUZ7u7YOdB3DNjFs7gUqCZPvo0Wgs51CW/PwJmmA2dpIsxJc1TwB sJtI+9A3T7b9306hO8DMUP/t+5J6g2P+tA60KszvT75sC6vEcroFd4SvrEoftFeG OIfDk9ZLbyeFYZP7Q3Yup4EkByo9hBiP7vA7WCShrQpZve34K7I= =gp1j -----END PGP SIGNATURE----- --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_=--