From owner-freebsd-pf@freebsd.org Sun Aug 12 21:00:49 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3ACFF107C005 for ; Sun, 12 Aug 2018 21:00:49 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CF74A792EA for ; Sun, 12 Aug 2018 21:00:48 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 9491E107BFFF; Sun, 12 Aug 2018 21:00:48 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 838D2107BFFE for ; Sun, 12 Aug 2018 21:00:48 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 28D98792E7 for ; Sun, 12 Aug 2018 21:00:48 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 87C7D22D02 for ; Sun, 12 Aug 2018 21:00:47 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7CL0l5V014850 for ; Sun, 12 Aug 2018 21:00:47 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7CL0lwd014845 for pf@FreeBSD.org; Sun, 12 Aug 2018 21:00:47 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201808122100.w7CL0lwd014845@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: pf@FreeBSD.org Subject: Problem reports for pf@FreeBSD.org that need special attention Date: Sun, 12 Aug 2018 21:00:47 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 21:00:49 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 203735 | Transparent interception of ipv6 with squid and p 1 problems total for which you should take action. From owner-freebsd-pf@freebsd.org Sun Aug 12 22:09:39 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B80C105523A for ; Sun, 12 Aug 2018 22:09:39 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x52a.google.com (mail-ed1-x52a.google.com [IPv6:2a00:1450:4864:20::52a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0101C7C3CD for ; Sun, 12 Aug 2018 22:09:38 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x52a.google.com with SMTP id b10-v6so7315617eds.4 for ; Sun, 12 Aug 2018 15:09:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:organization:user-agent :mime-version; bh=/rSAE7deDWS3SWSS2S+V5Kz22XGomSmlXS3XQstvyJM=; b=FwTlODk9t2yIIq/GQ+7/AMO05hsWIPuI1JTWdrmZmwfyvhRPdToHR64ApQ5mGllT4u ake0Vba7GzJXcF6ppNZ7qnG5D+xFO9el+0pA1dt11jRasnNj+MtIb5Z6qdeCRdZk4Eim /7E4Mw1fwawcD7VvAoWKagZcN368rZJ3fUyZXuyVAHpax57NwH9w3aX92ZP5701JqzDX YYM68ChLgXQK4pPYosZh6x5HNxGFvoQoOvnh/kDy8722t3whYW0S8eVd95FTCt7YAydF rhD+oPypMMwQoA8xPhkMnRyhcFsGKPHkEErpcxcFfUhcoUAoAfk4rxdfi+G+/b4sI6r9 zLZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:organization :user-agent:mime-version; bh=/rSAE7deDWS3SWSS2S+V5Kz22XGomSmlXS3XQstvyJM=; b=j6P9R5hNNErFaePxUeV4esvEPc/ySiEkVUHmrVfYv+i+FR2SUjY7IHpTwS5aHSDi89 w2c3vDOmLlRxWDT85TJmbk/84lZR5S3phc6Kre5i9Is6VBIrc7pemGjD4Iqvtp3xqk9K OhVKB6tZWQn7asofC5dBrBELELs1xf4EiZS+B6hDsUxb44lJ1R4zDr2BzhaUkcfR1jOl a0RfqkaLAjDKLTiOYLrTzkU/MyGR6o4Ma90Ldmy6QbE9wy/acPuUoo7Ym4GxMV9bvJEa 6sV87HKKb1m0NfQ9lQ+TNabpbxk7gsNYQJ3FQ7ChMApq+NLwlCvzRyWOryZp3aEi0S6h QapQ== X-Gm-Message-State: AOUpUlGztBM4Muz4Is5W04Eu36vrFVEGQQ7IX3VwX4qhlrst4T9qUPNe wyrjspxZRZPpZacKI29DMg1j/IFSj7Q= X-Google-Smtp-Source: AA+uWPyvHVnGgklCsy7xoB9OiCWg4s3ynr78SOCWgxZBrfHC0UcxU5Uuj+8oSE8QgDIPVRZvM3izCw== X-Received: by 2002:a50:f390:: with SMTP id g16-v6mr19536908edm.226.1534111777246; Sun, 12 Aug 2018 15:09:37 -0700 (PDT) Received: from energia.localnet ([2a02:8108:50bf:d514::5]) by smtp.gmail.com with ESMTPSA id e2-v6sm6322880edn.11.2018.08.12.15.09.35 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 12 Aug 2018 15:09:36 -0700 (PDT) From: Kajetan Staszkiewicz To: freebsd-pf@freebsd.org Subject: pf tables locking Date: Mon, 13 Aug 2018 00:09:32 +0200 Message-ID: <8680316.SccKl5VnxN@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart3483859.od8LnuUY3l"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 22:09:39 -0000 --nextPart3483859.od8LnuUY3l Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Hello group, Can anybody help me iwth pf_table.c and all operations on tables, especially pfr_update_stats? I'm working on implementing stats for redirection targets, that is for nat or route-to. I'm going through the code and I've found out that many table-related function are guarded by lock on pf ruleset. But that is not true for pfr_update_stats. This function is called from pf_test only after PF_RULES_RUNLOCK(). -- | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart3483859.od8LnuUY3l Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3CwHAAKCRDjtFCvbXs6 FG4xAJ4kp++Es3MkSdq4CIYNtrHPQ94x9wCgvlwztRD0C8SydrcgzJt2SlxKgto= =AR0+ -----END PGP SIGNATURE----- --nextPart3483859.od8LnuUY3l-- From owner-freebsd-pf@freebsd.org Mon Aug 13 01:53:08 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A01510610C6 for ; Mon, 13 Aug 2018 01:53:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id ABD4A83A3C for ; Mon, 13 Aug 2018 01:53:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6D58610610C3; Mon, 13 Aug 2018 01:53:07 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5C16110610C2 for ; Mon, 13 Aug 2018 01:53:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F2AF483A39 for ; Mon, 13 Aug 2018 01:53:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 4CCCF25736 for ; Mon, 13 Aug 2018 01:53:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7D1r6kZ093675 for ; Mon, 13 Aug 2018 01:53:06 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7D1r68R093674 for pf@FreeBSD.org; Mon, 13 Aug 2018 01:53:06 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229241] pfctl -f /etc/pf.conf blocks loopback interface Date: Mon, 13 Aug 2018 01:53:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 01:53:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229241 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |pf@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Mon Aug 13 13:03:05 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A9F91071FD6 for ; Mon, 13 Aug 2018 13:03:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AC6837841F for ; Mon, 13 Aug 2018 13:03:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 7169C1071FCE; Mon, 13 Aug 2018 13:03:04 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6004E1071FCD for ; Mon, 13 Aug 2018 13:03:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 024197841E for ; Mon, 13 Aug 2018 13:03:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 38704B4E4 for ; Mon, 13 Aug 2018 13:03:03 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7DD33ix022845 for ; Mon, 13 Aug 2018 13:03:03 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7DD33iU022844 for pf@FreeBSD.org; Mon, 13 Aug 2018 13:03:03 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229241] pfctl -f /etc/pf.conf blocks loopback interface Date: Mon, 13 Aug 2018 13:03:01 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: see_also Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 13:03:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229241 Kristof Provost changed: What |Removed |Added ---------------------------------------------------------------------------- See Also| |https://bugs.freebsd.org/bu | |gzilla/show_bug.cgi?id=3D2= 305 | |88 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Mon Aug 13 13:03:59 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C5FA107205A for ; Mon, 13 Aug 2018 13:03:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8C59B784A6 for ; Mon, 13 Aug 2018 13:03:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4E3901072054; Mon, 13 Aug 2018 13:03:58 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D0FB1072053 for ; Mon, 13 Aug 2018 13:03:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D326B784A2 for ; Mon, 13 Aug 2018 13:03:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 2D614B4E9 for ; Mon, 13 Aug 2018 13:03:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7DD3vWI024130 for ; Mon, 13 Aug 2018 13:03:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7DD3vBh024128 for pf@FreeBSD.org; Mon, 13 Aug 2018 13:03:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229241] pfctl -f /etc/pf.conf blocks loopback interface Date: Mon, 13 Aug 2018 13:03:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: kp@freebsd.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 13:03:59 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229241 Kristof Provost changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress Assignee|pf@FreeBSD.org |kp@freebsd.org --- Comment #13 from Kristof Provost --- First partial fix went in on r337643. I forgot to mark it as such, but it'll get MFCd next week. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Mon Aug 13 13:22:43 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C42210729A5 for ; Mon, 13 Aug 2018 13:22:43 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23952794DF; Mon, 13 Aug 2018 13:22:43 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id C7651224DD; Mon, 13 Aug 2018 13:22:42 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8rgnodwri04zzlnkb79.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:dd66:2162:6071:50b5]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 32C4F523F8; Mon, 13 Aug 2018 15:22:41 +0200 (CEST) From: "Kristof Provost" To: "Kajetan Staszkiewicz" Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Mon, 13 Aug 2018 15:22:33 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> In-Reply-To: <8680316.SccKl5VnxN@energia> References: <8680316.SccKl5VnxN@energia> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 13:22:43 -0000 On 13 Aug 2018, at 0:09, Kajetan Staszkiewicz wrote: > Hello group, > > Can anybody help me iwth pf_table.c and all operations on tables, > especially > pfr_update_stats? I'm working on implementing stats for redirection > targets, > that is for nat or route-to. > > I'm going through the code and I've found out that many table-related > function > are guarded by lock on pf ruleset. But that is not true for > pfr_update_stats. > This function is called from pf_test only after PF_RULES_RUNLOCK(). > I think you’re right, this does look wrong. It’s very unlikely that this will actually lead to a crash, because rules (and associated tables) won’t just go away while there’s still state, but we could theoretically lose memory (in the pfrke_counters allocation), and miscount. I don’t want to re-take the rules lock for this, so my current thinking is that the best approach would be to already get rid of the potential memory leak by just always allocating the pfrke_counters when the table is created (i.e. when the rule is first set). That might waste a little memory if we didn’t need it, but it should simplify things a bit. We can resolve the counting issue by using the counter_u64_*() functions for them. We should be able to get away with not locking this. Regards, Kristof From owner-freebsd-pf@freebsd.org Mon Aug 13 15:06:52 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF5B51075279 for ; Mon, 13 Aug 2018 15:06:51 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x544.google.com (mail-ed1-x544.google.com [IPv6:2a00:1450:4864:20::544]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A8577D959 for ; Mon, 13 Aug 2018 15:06:51 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x544.google.com with SMTP id s24-v6so8426720edr.8 for ; Mon, 13 Aug 2018 08:06:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=jM/Vo7z6SqJwDuoP8S6zF/nomT2qPu3zVJ5GAkg4K1g=; b=BUhieC/L7aOQfY2dI9+i60OS70bHf3c+CDtZj7fPTeREA1ZnQmgGQaZiPOfMOp6BPC f8AHU5Fq5YHndPKTn19q+PcYCrZgsTlEeDVg1Ft0GfIWz8lOaVV8OwASId2bbuAc9mev MlFtTcrS9icIyh7SeXu0kOs2LAkty1TNcAzAJw/K2Me1BD6fIgNiLCNQUddloONLdOZZ LdhtlxQyme4lirB1qtXkyx6LT3QIvJoy4QjRPp4jFIDFJrhFCpHc7muyPdrjM4rRdDN1 uIIr0vEjaDBENRTQY1z3O2rCyVci9kO4YoiPIswRTTvDFT9+u6nOLNpF5cxkaH45blq5 PsFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=jM/Vo7z6SqJwDuoP8S6zF/nomT2qPu3zVJ5GAkg4K1g=; b=jFgB0/Blw8dd1MaArz/dcSPRRknmx1hs1TmWLHZH13Hxj+GwYiv3kUsEztCLiasLgs Qq7aDMpA2bMaeXGA3k4I+aiLW3esRpyLkyjmbEJf8jWP5pybDyiI2JvM84lUmXRkeJVd CNMFcJWHff2SPiMiF3Rrg2xB+lrHSu7KSHYDDWHN+kbGvPBphKgpOvVOczMI+czxOi/P SpsiiVWDhXHZKKPGyhXrywnKhePZ8nsiOm0ai2S9ZP6i/BzlIzJBaax4NZj/ZdoPniiv I6keYJwbKK1/Bs1b+mAs7iojnUdhU3klAtdab3hnmmbnRL6fAUTebUjAwb06PyPZQxCr toig== X-Gm-Message-State: AOUpUlHK8ZNha0SX7jfWM2TBSalk8DPvw+6Gn+SeLfwQKKb+i8ztsDX6 1lGHvDj6xHdshx7tHskwLUp0vCFp1F8= X-Google-Smtp-Source: AA+uWPzp0dcBI3Axb5bbMEx/0YbUjyvS72qakyxr4LSUxoNKsNTDnE7NpO41pt+oRp0lev7XsYR28w== X-Received: by 2002:a50:a1c6:: with SMTP id 64-v6mr22042656edk.309.1534172810227; Mon, 13 Aug 2018 08:06:50 -0700 (PDT) Received: from energia.localnet ([212.48.107.10]) by smtp.gmail.com with ESMTPSA id c21-v6sm13074434eda.21.2018.08.13.08.06.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 08:06:49 -0700 (PDT) From: Kajetan Staszkiewicz To: Kristof Provost Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Mon, 13 Aug 2018 17:06:45 +0200 Message-ID: <2313127.kTuY2QdDqf@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> References: <8680316.SccKl5VnxN@energia> <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2812603.zyBN6blRsM"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 15:06:52 -0000 --nextPart2812603.zyBN6blRsM Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Monday, 13 August 2018 15:22:33 CEST Kristof Provost wrote: > > I'm going through the code and I've found out that many table-related > > function > > are guarded by lock on pf ruleset. But that is not true for > > pfr_update_stats. > > This function is called from pf_test only after PF_RULES_RUNLOCK(). >=20 > I think you=E2=80=99re right, this does look wrong. >=20 > It=E2=80=99s very unlikely that this will actually lead to a crash, becau= se I don't like the word "unlikely". With my traffic and frequent ruleset and= =20 carp changes I'm catching all the fanciest locking bugs as it seems. > rules (and associated tables) won=E2=80=99t just go away while there=E2= =80=99s still > state, This is mostly what I wanted to ask about in this message. How is it ensure= d=20 that table and counters are gone only after everybody stops using them? Wha= t=20 if I delete a table, then change ruleset, but there is still active connect= ion=20 keeping a state? I really had hard time finding how this is guarded in sour= ce. > but we could theoretically lose memory (in the pfrke_counters > allocation), and miscount. Pre-allocating counters seems a good idea, it will simplify some other code. > I don=E2=80=99t want to re-take the rules lock for this, so my current > thinking is that the best approach would be to already get rid of the > potential memory leak by just always allocating the pfrke_counters when > the table is created (i.e. when the rule is first set). That might waste > a little memory if we didn=E2=80=99t need it, but it should simplify thin= gs a > bit. =20 > We can resolve the counting issue by using the counter_u64_*() functions > for them. We should be able to get away with not locking this. Sure, I can use counter(9). The question, as always with my patches, is wha= t=20 can go to FreeBSD and what won't go. My current goal is to modify round-robin pf target to always point to table= =20 entry with least amount of states. As I see it for now: 1. Modify pfrke_counters to be always allocated. 2. Rewrite pfrke_counters to use counter(9). 3. Provide state counter in pfrke_counters. 4. Modify round-robin target. 1. and 2. make a good PR. I'm not sure about 3. Do you want patches for lea= st- connections target too? I want to just replace existing round-robin but if= =20 there is any chance of getting it into kernel code, I could make it work as= =20 new target in pf.conf. Point 3. is the puzzle for me. For now I just call pfr_update_stats (modifi= ed=20 to handle state counter) in pf_create_state and pf_unlink_state. But again = =2D=20 how do I know if the table (I added a pointer in struct pf_state) is still= =20 allocated in memory? There are some more issues I found around pf_map_addr. Some of them I=20 mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092. So= me=20 more came out while working on this least-states loadbalancing. I will grou= p=20 them into something meaningful and make another PR for them. =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart2812603.zyBN6blRsM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3GehQAKCRDjtFCvbXs6 FPiMAKCWbU5HlmpRZdlci0l3fXFYW6Ic+ACeNjCElC40Fw7z5NKxpqZjplZKDZg= =yHP4 -----END PGP SIGNATURE----- --nextPart2812603.zyBN6blRsM-- From owner-freebsd-pf@freebsd.org Mon Aug 13 15:59:20 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 697E01076512 for ; Mon, 13 Aug 2018 15:59:20 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 137547FB59; Mon, 13 Aug 2018 15:59:20 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id 99E3423400; Mon, 13 Aug 2018 15:59:19 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [10.0.2.164] (ptr-8rgnodwri04zzlnkb79.18120a2.ip6.access.telenet.be [IPv6:2a02:1811:240b:b802:dd66:2162:6071:50b5]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 4E0B752882; Mon, 13 Aug 2018 17:59:17 +0200 (CEST) From: "Kristof Provost" To: "Kajetan Staszkiewicz" Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Mon, 13 Aug 2018 17:59:15 +0200 X-Mailer: MailMate (2.0BETAr6116) Message-ID: In-Reply-To: <2313127.kTuY2QdDqf@energia> References: <8680316.SccKl5VnxN@energia> <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> <2313127.kTuY2QdDqf@energia> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_="; micalg=pgp-sha256; protocol="application/pgp-signature" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 15:59:20 -0000 This is an OpenPGP/MIME signed message (RFC 3156 and 4880). --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 13 Aug 2018, at 17:06, Kajetan Staszkiewicz wrote: > On Monday, 13 August 2018 15:22:33 CEST Kristof Provost wrote: >> rules (and associated tables) won=E2=80=99t just go away while there=E2= =80=99s still >> state, > > This is mostly what I wanted to ask about in this message. How is it en= sured > that table and counters are gone only after everybody stops using them?= What > if I delete a table, then change ruleset, but there is still active con= nection > keeping a state? I really had hard time finding how this is guarded in = source. > pf keeps rules around until there are no more states left referencing the= rule. Look at pf_commit_rules(): The old rules are unlinked rather than removed= =2E They=E2=80=99re kept on the V_pf_unlinked rules list. Every so often pf runs through all states (in pf_purge_thread()) to mark = their associated rules as still referenced. Only rules which are not refe= renced by any state are removed. This means that while there=E2=80=99s still a state which was created by = the rule (and can thus put packets towards its table), the rule will exis= t. Once the state goes away it=E2=80=99ll still take one full iteration thro= ugh all states before the rule can be freed. Hence my statement that it=E2=80=99s highly unlikely (pretty much impossi= ble) for us to run into a situation where the rule no longer exists. >> I don=E2=80=99t want to re-take the rules lock for this, so my current= >> thinking is that the best approach would be to already get rid of the >> potential memory leak by just always allocating the pfrke_counters whe= n >> the table is created (i.e. when the rule is first set). That might was= te >> a little memory if we didn=E2=80=99t need it, but it should simplify t= hings a >> bit. > >> We can resolve the counting issue by using the counter_u64_*() functio= ns >> for them. We should be able to get away with not locking this. > > Sure, I can use counter(9). The question, as always with my patches, is= what > can go to FreeBSD and what won't go. > > My current goal is to modify round-robin pf target to always point to t= able > entry with least amount of states. > > As I see it for now: > 1. Modify pfrke_counters to be always allocated. > 2. Rewrite pfrke_counters to use counter(9). > 3. Provide state counter in pfrke_counters. > 4. Modify round-robin target. > > 1. and 2. make a good PR. I'm not sure about 3. Do you want patches for= least- > connections target too? I want to just replace existing round-robin but= if > there is any chance of getting it into kernel code, I could make it wor= k as > new target in pf.conf. > Do you have a bit more information about your use case? What are you tryi= ng to accomplish with this change? > There are some more issues I found around pf_map_addr. Some of them I > mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092= =2E Some > more came out while working on this least-states loadbalancing. I will = group > them into something meaningful and make another PR for them. > Yeah, that bug is still on my todo list somewhere, but things are extreme= ly hectic at the moment, and I can=E2=80=99t make any promises about when= I=E2=80=99ll have time for it. Regards, Kristof --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQFDBAEBCAAtFiEEhvumznUbtMyaDlFyb8TccfteSkYFAltxqtMPHGtwQGZyZWVi c2Qub3JnAAoJEG/E3HH7XkpGlJsH/iDZumm2TZ038vQgWpi3Z93fd7L/evWip09/ N4e1T4eVCUAPqY/fTBs7EbJTaxkCFSwuexkLGWULeO6Q19zJ0ck34ufOzw8DGgAN uJNkzN4j6+ny3mkYHsdAZ4e0JE3wJYwQFZeQRTWu4SQq0J0myX+1Sztjiv/Uh2Tq JcmemUnVdIOwhUZ7u7YOdB3DNjFs7gUqCZPvo0Wgs51CW/PwJmmA2dpIsxJc1TwB sJtI+9A3T7b9306hO8DMUP/t+5J6g2P+tA60KszvT75sC6vEcroFd4SvrEoftFeG OIfDk9ZLbyeFYZP7Q3Yup4EkByo9hBiP7vA7WCShrQpZve34K7I= =gp1j -----END PGP SIGNATURE----- --=_MailMate_8D9200E3-CEB2-4CC1-937A-BACD3CB90E9F_=-- From owner-freebsd-pf@freebsd.org Mon Aug 13 23:32:26 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DAD081055E07 for ; Mon, 13 Aug 2018 23:32:25 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E0F571EF1 for ; Mon, 13 Aug 2018 23:32:25 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x541.google.com with SMTP id r4-v6so9164842edp.9 for ; Mon, 13 Aug 2018 16:32:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=w+3bHCX5z0x8UDn9p8Pcsl45s9bLbDZsPFxKoF9pd9c=; b=sD9DY61kYhFcf6JU5jjCPTl1wjf1Qknw16qfhE13uDqJ6F031T5bNnH8SSWuPmwoMR Y64oAREiEeT/6ENxVOWTkgYPxpT3427jjefK/pfCPNXktb9SXYET1sCzolAXFidavTOr FCGQGDwHfuQZjgIWGImpa+tYbIRr6QIcbf/+R4xWcpiSWNsZesScVxHPkDK0ht4A1mjy lrvJlU/FDSmwMGrdgMumulw9ZU+lZOoiFTwngUAMru1HMz80D30Jar0SHAk73nd83uj9 RBi405rE87yVsuLKytPq4oAC2NX5FwMh0mgfm7GyzgoMg7BGjRdCqMhnMh5jN8LzwYRy MrXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=w+3bHCX5z0x8UDn9p8Pcsl45s9bLbDZsPFxKoF9pd9c=; b=Pqn5QxLMN6K3kqqt5Vk6twQvludy7X5VNZikH8JCWMN4u7OEu0tvntSSXGJPTd4Yqq Lu77JkTEleiV3CoWS8PaAiFYc60SSf+AYHlbhJFUptL9AwnXtPOu/5ICU7J+GSMLdg3F 4MkRK78Miyt3TJCjOSSqT3c3QXRVN4Tv29M7Nn7JprI6hVQDbineIjQKDNb5Y2AcX8sh SbSDJ344oHMMXq27oC7MSqtntASfOeReWsarbWhHIncXGPtVa6jZpB2LVLVkNhlrlPa/ cXpO008AOqJsnJITDmul90nuS9TqqXqmHQ3DhhDFfpqT7+pFri3q7NbJMDYDfS4HoOav gojA== X-Gm-Message-State: AOUpUlEiGZF86MyfN4APWA+Pxz33vGayv0qnUT3e2zq/UQff/yEIs1HX KMbLFDYRJbOq7vu6ujOn0bp+ZSV3llI= X-Google-Smtp-Source: AA+uWPyRxYpxi+AEZ2ucPnWqc86YTvrB6JsKy5GY5ahLr4NopjHDu5/PM/rgwUQ5DpBeSZNPgY3HIg== X-Received: by 2002:a50:8c06:: with SMTP id p6-v6mr24310644edp.282.1534203144192; Mon, 13 Aug 2018 16:32:24 -0700 (PDT) Received: from energia.localnet ([2a02:8108:50bf:d514::5]) by smtp.gmail.com with ESMTPSA id c21-v6sm14117607eda.21.2018.08.13.16.32.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Aug 2018 16:32:22 -0700 (PDT) From: Kajetan Staszkiewicz To: Kristof Provost Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Tue, 14 Aug 2018 01:32:17 +0200 Message-ID: <1546233.jncNNXsBuh@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: References: <8680316.SccKl5VnxN@energia> <2313127.kTuY2QdDqf@energia> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart10585032.jW1J6F8Yqn"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2018 23:32:26 -0000 --nextPart10585032.jW1J6F8Yqn Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Monday, 13 August 2018 17:59:15 CEST Kristof Provost wrote: > pf keeps rules around until there are no more states left referencing the > rule. Look at pf_commit_rules(): The old rules are unlinked rather than > removed. They=E2=80=99re kept on the V_pf_unlinked rules list. Every so o= ften pf > runs through all states (in pf_purge_thread()) to mark their associated > rules as still referenced. Only rules which are not referenced by any sta= te > are removed. >=20 > This means that while there=E2=80=99s still a state which was created by = the rule > (and can thus put packets towards its table), the rule will exist. Once t= he > state goes away it=E2=80=99ll still take one full iteration through all s= tates > before the rule can be freed. Hence my statement that it=E2=80=99s highly= unlikely > (pretty much impossible) for us to run into a situation where the rule no > longer exists. OK, now it makes sense. > >> I don=E2=80=99t want to re-take the rules lock for this, so my current > >> thinking is that the best approach would be to already get rid of the > >> potential memory leak by just always allocating the pfrke_counters when > >> the table is created (i.e. when the rule is first set). That might was= te > >> a little memory if we didn=E2=80=99t need it, but it should simplify t= hings a > >> bit. > >>=20 > >> We can resolve the counting issue by using the counter_u64_*() functio= ns > >> for them. We should be able to get away with not locking this. How about this? https://github.com/innogames/freebsd/commit/ d44a0d9487285fac8ed1d7372cc99cca83f616e6 > Do you have a bit more information about your use case? What are you tryi= ng > to accomplish with this change? I have a loadbalancer which uses pf and route-to targets. After a server is= =20 added to a pool, I want this server to immediately take over much traffic.= =20 With round-robin the server receives new clients rather slowly. If kernel=20 could measure amount of states per table entry, I could send new clients to= =20 this new server until it serves as many clients as other servers. > > There are some more issues I found around pf_map_addr. Some of them I > > mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092. > > Some > > more came out while working on this least-states loadbalancing. I will > > group them into something meaningful and make another PR for them. >=20 > Yeah, that bug is still on my todo list somewhere, but things are extreme= ly > hectic at the moment, and I can=E2=80=99t make any promises about when I= =E2=80=99ll have > time for it. I thought that was rather on my todo :) If you can agree on patch sent in this message (I would still make a PR and= =20 submit the patch there, just for documentation), I will re-work my other=20 patches and show you what I came up with. I had working code for counting=20 states per table entry, I only lack the modified round-robin selection itse= lf. =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart10585032.jW1J6F8Yqn Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3IVAQAKCRDjtFCvbXs6 FEJtAJ40MRDrNLR4WN9gc9CX4B4on1dmjwCgudhTlMok6Oubi4U8/LPKDmzNFEg= =Y4em -----END PGP SIGNATURE----- --nextPart10585032.jW1J6F8Yqn-- From owner-freebsd-pf@freebsd.org Tue Aug 14 15:15:53 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5EE49107B3BE for ; Tue, 14 Aug 2018 15:15:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A8FD74212; Tue, 14 Aug 2018 15:15:53 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) (Authenticated sender: kp) by smtp.freebsd.org (Postfix) with ESMTPSA id B1C85C239; Tue, 14 Aug 2018 15:15:52 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [192.168.14.247] (unknown [62.49.66.12]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id 550B15656D; Tue, 14 Aug 2018 17:15:50 +0200 (CEST) From: "Kristof Provost" To: "Kajetan Staszkiewicz" Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Tue, 14 Aug 2018 16:15:48 +0100 X-Mailer: MailMate (2.0BETAr6116) Message-ID: <69D19AE4-2F17-4DBC-AF62-A2489049FC9C@FreeBSD.org> In-Reply-To: <1546233.jncNNXsBuh@energia> References: <8680316.SccKl5VnxN@energia> <2313127.kTuY2QdDqf@energia> <1546233.jncNNXsBuh@energia> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; markup=markdown Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 15:15:53 -0000 On 14 Aug 2018, at 0:32, Kajetan Staszkiewicz wrote: > On Monday, 13 August 2018 17:59:15 CEST Kristof Provost wrote: > How about this? > > https://github.com/innogames/freebsd/commit/ > d44a0d9487285fac8ed1d7372cc99cca83f616e6 > That looks good to me. There’s a few minor issues, things like inconsistent indentation and overly long lines, but that’s about the only criticism I have. >> Do you have a bit more information about your use case? What are you >> trying >> to accomplish with this change? > > I have a loadbalancer which uses pf and route-to targets. After a > server is > added to a pool, I want this server to immediately take over much > traffic. > With round-robin the server receives new clients rather slowly. If > kernel > could measure amount of states per table entry, I could send new > clients to > this new server until it serves as many clients as other servers. > I see. I’m not quite sure yet if that’s a feature we want to import or not, but at least your ‘support’ patches should probably go in. The above one certainly. >>> There are some more issues I found around pf_map_addr. Some of them >>> I >>> mentioned in >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229092. >>> Some >>> more came out while working on this least-states loadbalancing. I >>> will >>> group them into something meaningful and make another PR for them. >> >> Yeah, that bug is still on my todo list somewhere, but things are >> extremely >> hectic at the moment, and I can’t make any promises about when >> I’ll have >> time for it. > > I thought that was rather on my todo :) > I’m not going to stop you. I love it when other people do the work ;) Regards, Kristof From owner-freebsd-pf@freebsd.org Tue Aug 14 16:35:25 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0A2A7107D9BA for ; Tue, 14 Aug 2018 16:35:25 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 80D76790DE for ; Tue, 14 Aug 2018 16:35:24 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x542.google.com with SMTP id f23-v6so10427020edr.11 for ; Tue, 14 Aug 2018 09:35:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=r/Jhs7OH5mceuwNBOd2aS6JinlrM8MYzirO/urIz0fE=; b=vrB9iqdzXSVfmCierAqM+6rB4A/ISFVkvAkG56YvauAHpQzG0bMtSRydYrJuyeNB77 u8U3/X4Xhw0NHa0cRCIriZenO03SJGjfKQ2aKu9Ski/GZMNSEFRY9yd9NwxaxVVajP/C 1svAiZBRYwbme25o2uWq5jsSHv6k6mxM7webhT19WSVUyzlIOPVPRumlq9Gb/cTyRgzh PCSVMjRmtM+DIKZXm9fyTLzV0GKsdVQl6v+gJjOCax4VEGBgdZAANAKI5W91I2WMbnKn kP3HnxFgUuM1lXCTIe8Cwaqd/hTHVxnHAJdWeIZZZorf96Rg0Y+6ec4KS6Odw4ukLFh6 A5tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=r/Jhs7OH5mceuwNBOd2aS6JinlrM8MYzirO/urIz0fE=; b=O8MynTTKHmBDOlmhX3CKy2NDzSroaZNAxA+JBHxZtiXjpY//Vdf3cYe4GWmjpUHQix VQwXUpbgY+LDeheHpt3xDSD47l1RaMeYLVlZw55cBkqZiha23P+qFZ2gMgqzK+Tv84YQ Wq8HfrGjzu/s6D5unJWJjo6w/N+FtVAzXhf+Xt1Cmgw/xlsE4YHPELa/xVsAhkovurRY EV2YiXKs5wCU8sZ4IfnhhYam1L8fMOwrSV7rJDNjhPveAth5/I8yXc8j6bEyRa6URWyZ JvxndCZs7eIFq4PQY6V/hizveREOWEnXAsL/S1VkyuqGxWjk5utwZ6eXVx+o3ILyN55j R2vw== X-Gm-Message-State: AOUpUlExsaTn60lUoCxLEQ1PXs/JnQ6O9nRizs6uVC14rCIHhRaa31Li sO6ISTBe6Pj29tQAcCAzJw5BTZVvhOg= X-Google-Smtp-Source: AA+uWPyRIRUus7KuuPPZt7nC/05Tra/RqBqOjLXVu2MqvmIe3Yq3YD9uYkETwic+CiZoFoYxj7YB2A== X-Received: by 2002:a50:aa43:: with SMTP id p3-v6mr28598014edc.233.1534264523255; Tue, 14 Aug 2018 09:35:23 -0700 (PDT) Received: from energia.localnet ([212.48.107.10]) by smtp.gmail.com with ESMTPSA id a15-v6sm22330205edd.47.2018.08.14.09.35.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Aug 2018 09:35:22 -0700 (PDT) From: Kajetan Staszkiewicz To: Kristof Provost Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Tue, 14 Aug 2018 18:35:16 +0200 Message-ID: <13826523.m2ultlLLsi@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: <69D19AE4-2F17-4DBC-AF62-A2489049FC9C@FreeBSD.org> References: <8680316.SccKl5VnxN@energia> <1546233.jncNNXsBuh@energia> <69D19AE4-2F17-4DBC-AF62-A2489049FC9C@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1543202.jXq12AkZmL"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 16:35:25 -0000 --nextPart1543202.jXq12AkZmL Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Tuesday, 14 August 2018 16:15:48 CEST Kristof Provost wrote: > On 14 Aug 2018, at 0:32, Kajetan Staszkiewicz wrote: > > On Monday, 13 August 2018 17:59:15 CEST Kristof Provost wrote: > > How about this? > >=20 > > https://github.com/innogames/freebsd/commit/ > > d44a0d9487285fac8ed1d7372cc99cca83f616e6 >=20 > That looks good to me. > There=E2=80=99s a few minor issues, things like inconsistent indentation = and > overly long lines, but that=E2=80=99s about the only criticism I have. I fixed some issues with unallocated counters and submitted bug 230619. > I see. I=E2=80=99m not quite sure yet if that=E2=80=99s a feature we want= to import > or not, > but at least your =E2=80=98support=E2=80=99 patches should probably go in= =2E The above > one certainly. There are some more things which require changes before I can do least- connections balancing. If you have a moment, please have a look at https://github.com/innogames/ freebsd/commits/iglb/11.2/GetOnWithIt_2 , maybe some of those things can ge= t=20 imported anyway, like full support for counters of states. > >> Yeah, that bug is still on my todo list somewhere, but things are > >> extremely > >> hectic at the moment, and I can=E2=80=99t make any promises about when > >> I=E2=80=99ll have > >> time for it. > >=20 > > I thought that was rather on my todo :) >=20 > I=E2=80=99m not going to stop you. I love it when other people do the wor= k ;) Since I have you here, let me explain the issues I see with pf_map_addr(). = =46or=20 round-robin target a list of interface,table pairs can be specified. This l= ist=20 is iterated and within each table addresses are iterated too. There is no=20 locking around it "because performance is assumed more important than round- robin precision" according to comment in code. Yet I believe there are way more serious issues possible with the current=20 approach. Interface is in fact picked up outside of pf_map_addr(). Another= =20 thread could have already moved the rpool->counter to another table for whi= ch=20 the interface is not valid anymore. I came up with this: https://github.com/innogames/freebsd/commit/ 61ffb96a4dc948a0b06204ff39210c0578f77f08 although without locking this is=20 still not really a solution. It only moves interface selection to inside of= =20 pf_map_addr() Another one is https://github.com/innogames/freebsd/commit/ 8fe6cd2d820052d2166afbaa311f34318a41db48 which stores table used for=20 loadbalancing in state and src_node. Then the table can be used for state=20 counting. The 2 patches above are also included in the first link I gave above. =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart1543202.jXq12AkZmL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3MExAAKCRDjtFCvbXs6 FGV3AJ47XVjgEPfb2BtwpORCuNfQVuG+eACg1jPfpc6+llVR/vyBdA6RgusU/YM= =v6SD -----END PGP SIGNATURE----- --nextPart1543202.jXq12AkZmL-- From owner-freebsd-pf@freebsd.org Tue Aug 14 22:44:54 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8200910686E0 for ; Tue, 14 Aug 2018 22:44:54 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 183D88C808; Tue, 14 Aug 2018 22:44:54 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by mail-it0-x236.google.com with SMTP id h23-v6so22426139ita.5; Tue, 14 Aug 2018 15:44:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=LnyBrHzGcSqbztYVg/rmEfTXlk457OQ84sc71rJhBRg=; b=qQU4aDs5KNs9e3kUYWeTvZV67+Pt3pg0ptXUhAvlaUiRK1or3250R/0Kv1R5uiQ7Jj /kQY5G+vDjEgET5OQkBeGnsDk9OTwNrTHKzG4zGpnS3G4pF5LEfdKusP0DKumRR8l5+o DYwjHorGRCYlSLZOXXMAfZ86wzejfCGeaEqadwf5bmkCWGrGHbC57zWUJWvvxMqObi41 T0zWNoaRnf2KESqiGvFZFuRvJVKMyQPHZm9XsKfHvjKYHrm0OKu9dX8jMoYikljwgN2P fFCj0AH2RjcQy+uIvBCGXoOaJG6eePqK12Vxu0OXT0hWvwYrh8mDKu0pniyyZpy4PxuB XWwQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=LnyBrHzGcSqbztYVg/rmEfTXlk457OQ84sc71rJhBRg=; b=g67AFYoHfYmq97ZclkBclJsm+mhxc2mU5c1qZK4jbyUNpPWo06rd30dy+lrb7dxMOi bHrg7zf4rRsHUc37PZ+HGu/gJCQ1iJdkN5MXI9+pyIP96qjwiUhVbk6qjVDlETjLUuMM mdJKPVXtjMuEOsm3jb5Q1NYYhjIflEWt2Gr3Gu1sLURyYIMMcpq2+QDO+dGuZFnLGK64 G3smxVzEcEkOgDpayJwiANpf9KoynIatxpiIRMhrQS1FyRhoR+tDd0JChU9eHsoy3WVv 3YAU/cspRsX9i1LyXieFAvN2ERlyVRW8nohotfyyAfKTC3F63XyC2810o9KYF0TJwm7Q WiZw== X-Gm-Message-State: AOUpUlEWZkIZg4gQZY+qXr9e1JyUmSkXPw6JuADaTQhfs94x9o+aMDxz 33/byEvwfOgdiugkwRg9WadcBGYGLYjw6awZ6EvSpg== X-Google-Smtp-Source: AA+uWPx/cZqHFhCYy2Y3VPHgqsy8CloVR0zaofHw0Zwl+3czijAH2dVI/qzl0DmGHMYh8AkQ0s1mh+owh4EvmMhgGtg= X-Received: by 2002:a24:988b:: with SMTP id n133-v6mr16247708itd.10.1534286693423; Tue, 14 Aug 2018 15:44:53 -0700 (PDT) MIME-Version: 1.0 Sender: ermal.luci@gmail.com Received: by 2002:a6b:3ad7:0:0:0:0:0 with HTTP; Tue, 14 Aug 2018 15:44:52 -0700 (PDT) In-Reply-To: <13826523.m2ultlLLsi@energia> References: <8680316.SccKl5VnxN@energia> <1546233.jncNNXsBuh@energia> <69D19AE4-2F17-4DBC-AF62-A2489049FC9C@FreeBSD.org> <13826523.m2ultlLLsi@energia> From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= Date: Tue, 14 Aug 2018 15:44:52 -0700 X-Google-Sender-Auth: AvbI-L7IiFgzmeY1Fb7GsyxbXVU Message-ID: Subject: Re: pf tables locking To: Kajetan Staszkiewicz Cc: Kristof Provost , "freebsd-pf@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 22:44:54 -0000 (sorry for the top post) If you really want to spend time on it, the best option is to pull out the pool concept used by the rules/nat... and manage it outside of the rules/states but in its own module referenced by the former ones. This would allow extensibility and propper reasoning about it. On Tue, Aug 14, 2018 at 9:35 AM, Kajetan Staszkiewicz wrote: > On Tuesday, 14 August 2018 16:15:48 CEST Kristof Provost wrote: > > On 14 Aug 2018, at 0:32, Kajetan Staszkiewicz wrote: > > > On Monday, 13 August 2018 17:59:15 CEST Kristof Provost wrote: > > > How about this? > > > > > > https://github.com/innogames/freebsd/commit/ > > > d44a0d9487285fac8ed1d7372cc99cca83f616e6 > > > > That looks good to me. > > There=E2=80=99s a few minor issues, things like inconsistent indentatio= n and > > overly long lines, but that=E2=80=99s about the only criticism I have. > > I fixed some issues with unallocated counters and submitted bug 230619. > > > I see. I=E2=80=99m not quite sure yet if that=E2=80=99s a feature we wa= nt to import > > or not, > > but at least your =E2=80=98support=E2=80=99 patches should probably go = in. The above > > one certainly. > > There are some more things which require changes before I can do least- > connections balancing. > > If you have a moment, please have a look at https://github.com/innogames/ > freebsd/commits/iglb/11.2/GetOnWithIt_2 , maybe some of those things can > get > imported anyway, like full support for counters of states. > > > >> Yeah, that bug is still on my todo list somewhere, but things are > > >> extremely > > >> hectic at the moment, and I can=E2=80=99t make any promises about wh= en > > >> I=E2=80=99ll have > > >> time for it. > > > > > > I thought that was rather on my todo :) > > > > I=E2=80=99m not going to stop you. I love it when other people do the w= ork ;) > > Since I have you here, let me explain the issues I see with pf_map_addr()= . > For > round-robin target a list of interface,table pairs can be specified. This > list > is iterated and within each table addresses are iterated too. There is no > locking around it "because performance is assumed more important than > round- > robin precision" according to comment in code. > > Yet I believe there are way more serious issues possible with the current > approach. Interface is in fact picked up outside of pf_map_addr(). Anothe= r > thread could have already moved the rpool->counter to another table for > which > the interface is not valid anymore. > > I came up with this: https://github.com/innogames/freebsd/commit/ > 61ffb96a4dc948a0b06204ff39210c0578f77f08 although without locking this is > still not really a solution. It only moves interface selection to inside > of > pf_map_addr() > > Another one is https://github.com/innogames/freebsd/commit/ > 8fe6cd2d820052d2166afbaa311f34318a41db48 which stores table used for > loadbalancing in state and src_node. Then the table can be used for state > counting. > > The 2 patches above are also included in the first link I gave above. > > -- > | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | > | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | > | Vegeta | www: http://vegeta.tuxpowered.net | > `------------------------^---------------------------------------' > > -- > Ermal > From owner-freebsd-pf@freebsd.org Tue Aug 14 23:54:52 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 650A2106A408 for ; Tue, 14 Aug 2018 23:54:52 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x544.google.com (mail-ed1-x544.google.com [IPv6:2a00:1450:4864:20::544]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E6BF38E91D for ; Tue, 14 Aug 2018 23:54:51 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x544.google.com with SMTP id k15-v6so11037774edr.3 for ; Tue, 14 Aug 2018 16:54:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=4QAvYZyLN3WWnDTYtltlcNL9YIbKjIH4/fg6hMAiX7s=; b=lsjZJkbPwGHceBVUQifGiEI8JMaiuGKd6DmUHUyWSmcjSrWrPh7Kb8sf1jNu2mW3ih niBkZjA4WcqJXAtsyPbGEO7LUWzsc5K00MBSxL+4sqkNeDKqtYMmNtOVEQCHaqJ3vwd2 PJCiqdUAzFbivxY5Nxy43qOlxwKEItfn4Egv5kICGcxw0BXmwLY+S2WGCzg6jEFLSLaP +anOOdS4z3frhDvn11FdLhnsDPV7PLYhBF+cgHV1hqC9t4hp7od+gDJOCarKbJkD9OHN 2YVQh4q9ZZuXmxB9FtRYgM1470u+9jyZGcEuv3tM5hPMAndM8eCg3KMsYYZDsX79lIHd R5/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=4QAvYZyLN3WWnDTYtltlcNL9YIbKjIH4/fg6hMAiX7s=; b=qJTtr3C1EdkkggbKzCf84Wv5vXY49tnmbm7lk3ovEIOCcvV/c/Sj+PWPt4viYTSNbs 9hGMt5J/PmFv3upYCzYIn0uPO++vb2/XKhLAdmzWUTJgM0uSeQTK/UIQqxmUbY623RuU N2nbJ+y1jUwbuWHemQXW1kBZ35px0rXflx/JyuPMwFUD5XCKp4SPXNr8QdK6iSV9DFD4 XXGrgZZq1JE+22zqDvANVbhQMIpRybu7v8RuVDoetlT0QF2lXKarlHQG9lJrgX312dKI wWElCS7ygZPkoGBJgC2vib8yeTrgNLdgmRisG3VH6EY5mkU8T+2MSc8sCDp6ZR9Cv2Do 5kag== X-Gm-Message-State: AOUpUlGZ/bbA5+IW1f7AUWA8WOjMOt37Ci6cz1JxD+zz0kzZnid+W0vO VpuYWlN/VPh7f1q7aMrSnhhRqg== X-Google-Smtp-Source: AA+uWPxQVdxTBFbP/9aV532ydOAlfkmi7v5WQK3C4qgoKNM5wIPpZi2sGgl49prCdUkeoF57V7Vuaw== X-Received: by 2002:a50:9704:: with SMTP id c4-v6mr29485410edb.246.1534290890777; Tue, 14 Aug 2018 16:54:50 -0700 (PDT) Received: from energia.localnet ([2a02:8108:50bf:d514::5]) by smtp.gmail.com with ESMTPSA id b58-v6sm20187621ede.37.2018.08.14.16.54.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Aug 2018 16:54:49 -0700 (PDT) From: Kajetan Staszkiewicz To: Ermal =?ISO-8859-1?Q?Lu=E7i?= Cc: Kristof Provost , "freebsd-pf@freebsd.org" Subject: Re: pf tables locking Date: Wed, 15 Aug 2018 01:54:43 +0200 Message-ID: <6021147.AAtAggGk6h@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: References: <8680316.SccKl5VnxN@energia> <13826523.m2ultlLLsi@energia> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2036494.pfWuGqApS5"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Aug 2018 23:54:52 -0000 --nextPart2036494.pfWuGqApS5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="iso-8859-1" On Tuesday, 14 August 2018 15:44:52 CEST Ermal Lu=E7i wrote: > If you really want to spend time on it, the best option is to pull out the > pool concept used by the rules/nat... and manage it outside of the > rules/states but in its own module referenced by the former ones. Do you mean as separate kernel module? Or totally outside of kernel? I was= =20 considering doing this outside of kernel by providing a weighted round-robi= n=20 algorithm but that would still require most of the patches as for doing it= =20 within kernel, in order to get counters working for redirection tables and= =20 state counter per table element, which both are missing in kernel now. > This would allow extensibility and propper reasoning about it. It might be the late hour but I really don't see how it would be extensible= =2E=20 Please be more specific. =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart2036494.pfWuGqApS5 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3NrwwAKCRDjtFCvbXs6 FE1dAJ979AM5qro0P+tx/f1WbBTnKJIXVQCgmaCW6/OG3hfWoxKzoIVEWHlZXgA= =3czq -----END PGP SIGNATURE----- --nextPart2036494.pfWuGqApS5-- From owner-freebsd-pf@freebsd.org Wed Aug 15 00:00:15 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C3EB106A93D for ; Wed, 15 Aug 2018 00:00:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BE77B8EB94 for ; Wed, 15 Aug 2018 00:00:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 82986106A936; Wed, 15 Aug 2018 00:00:14 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71638106A934 for ; Wed, 15 Aug 2018 00:00:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FACF8EB88 for ; Wed, 15 Aug 2018 00:00:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6D6221DC92 for ; Wed, 15 Aug 2018 00:00:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7F00D0k002676 for ; Wed, 15 Aug 2018 00:00:13 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7F00Drl002668 for pf@FreeBSD.org; Wed, 15 Aug 2018 00:00:13 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230619] pf: tables use non SMP-friendly counters Date: Wed, 15 Aug 2018 00:00:13 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 00:00:15 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230619 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch Assignee|bugs@FreeBSD.org |pf@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Wed Aug 15 09:49:26 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8E28107BBB4 for ; Wed, 15 Aug 2018 09:49:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 564B9829D8 for ; Wed, 15 Aug 2018 09:49:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 17F4E107BBB0; Wed, 15 Aug 2018 09:49:26 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06A51107BBAE for ; Wed, 15 Aug 2018 09:49:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9CD64829D5 for ; Wed, 15 Aug 2018 09:49:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id EB28622EB6 for ; Wed, 15 Aug 2018 09:49:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7F9nO8T034460 for ; Wed, 15 Aug 2018 09:49:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7F9nORk034459 for pf@FreeBSD.org; Wed, 15 Aug 2018 09:49:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230619] pf: tables use non SMP-friendly counters Date: Wed, 15 Aug 2018 09:49:24 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 09:49:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230619 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ae@FreeBSD.org --- Comment #1 from Andrey V. Elsukov --- It seems you don't check the result of counter allocation, that with M_NOWA= IT can fail. And then you are doing an access to such pointers. I'm not famili= ar with PF, but what happens if you try to limit UMA zone used for these count= ers and try to create enough number of entries? I suspect it will just panic. A= lso, PCPU counters are very expensive memory consumers, on modern machines with = tens CPU cores, they require a lot of memory. And tables usually used to keep la= rge number of entries, at least for ipfw. Is it really needed feature for PF for such cost? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Wed Aug 15 10:06:31 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 527AF107C5AF for ; Wed, 15 Aug 2018 10:06:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E4135834BE for ; Wed, 15 Aug 2018 10:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A5E48107C5AE; Wed, 15 Aug 2018 10:06:30 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94B96107C5AD for ; Wed, 15 Aug 2018 10:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 36679834B9 for ; Wed, 15 Aug 2018 10:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6B1362319C for ; Wed, 15 Aug 2018 10:06:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7FA6Tl5093013 for ; Wed, 15 Aug 2018 10:06:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7FA6TTe093012 for pf@FreeBSD.org; Wed, 15 Aug 2018 10:06:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230619] pf: tables use non SMP-friendly counters Date: Wed, 15 Aug 2018 10:06:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 10:06:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230619 Kristof Provost changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |kp@freebsd.org --- Comment #2 from Kristof Provost --- It's a tradeoff. pfr_update_stats() is currently called without any relevant locks held, so there's a risk of both a memory leak and incorrect counting. Using PCPU counters (and always allocating them) mitigates this. One alternative would be to take the rules lock, which is usually used to protect tables, but we'd have to take it for writing, to ensure no other threads are updating the counters at the same time, which I would expect to= be devastating for throughput. We might be able to get away with a per-table (but there are throughput concerns for that too), or even per pfr_kentry lock, but the locking struct= ure of pf is already complex, and I'm not immediately clear on how it would interact with the rest of the locking. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Wed Aug 15 10:20:51 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 451CC107C9F1 for ; Wed, 15 Aug 2018 10:20:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D66D283A16 for ; Wed, 15 Aug 2018 10:20:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 9B624107C9F0; Wed, 15 Aug 2018 10:20:50 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8A133107C9EF for ; Wed, 15 Aug 2018 10:20:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C2FB83A10 for ; Wed, 15 Aug 2018 10:20:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 606802330C for ; Wed, 15 Aug 2018 10:20:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7FAKnvT017689 for ; Wed, 15 Aug 2018 10:20:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7FAKnOU017688 for pf@FreeBSD.org; Wed, 15 Aug 2018 10:20:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230619] pf: tables use non SMP-friendly counters Date: Wed, 15 Aug 2018 10:20:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 10:20:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230619 --- Comment #3 from Kajetan Staszkiewicz --- Andrey, you are right about allocation. I will change it to M_WAITOK just as other counters in pf are done. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Wed Aug 15 10:52:40 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78E21107EA6C for ; Wed, 15 Aug 2018 10:52:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 371148582D for ; Wed, 15 Aug 2018 10:52:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id EFCBD107EA6B; Wed, 15 Aug 2018 10:52:39 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE898107EA69 for ; Wed, 15 Aug 2018 10:52:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 808518582A for ; Wed, 15 Aug 2018 10:52:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id CEB9F2389D for ; Wed, 15 Aug 2018 10:52:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7FAqckH089867 for ; Wed, 15 Aug 2018 10:52:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7FAqcYV089863 for pf@FreeBSD.org; Wed, 15 Aug 2018 10:52:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230619] pf: tables use non SMP-friendly counters Date: Wed, 15 Aug 2018 10:52:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.isobsolete attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 10:52:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230619 Kajetan Staszkiewicz changed: What |Removed |Added ---------------------------------------------------------------------------- Attachment #196197|0 |1 is obsolete| | --- Comment #4 from Kajetan Staszkiewicz --- Created attachment 196214 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D196214&action= =3Dedit Use counter(9) in pf tables. Updated version of patch using M_WAITOK. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Wed Aug 15 11:13:16 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA818107FAD8 for ; Wed, 15 Aug 2018 11:13:16 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3259486534 for ; Wed, 15 Aug 2018 11:13:16 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x536.google.com with SMTP id o8-v6so589113edt.13 for ; Wed, 15 Aug 2018 04:13:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=WP/iJ3wKEkwMxmpaQBkRxA1+qs4VPZC2+LKs61Z5f/g=; b=coZbTDHYo76C9pWLUGpvw+xbv/dGafktMRxLCV3GE1ceEIKzjXtZo4+Q5ip37MZ/hF s+Sh0AKj1ndv58fC/9KmmAZXTRaR8YkcGHhqJ+E1Yw2EEKv6YMbcWConTVwQldZhZNX3 GazqvXWgpSDwTeSWQmaalYENnN9VKKnRD8dStBEW/Tm2UonxZDSWAW4x/kTU7RMoJ2vn rAKnhQ9xKmCXuh9AJuwAT01uY4Z6/mMibqLHzUH4bcfDvUh0Ha5QooOAiAy2AYgGzaj4 oekxTToD51HA0//mX/jop0JN/GjjJ9cTBVQHhT/zH4eBhGHD19RgHVWrFcLGmrjQ7KRS wmTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=WP/iJ3wKEkwMxmpaQBkRxA1+qs4VPZC2+LKs61Z5f/g=; b=sg+Moj59VC7Frsgx+Ssx/dP67/JopFag1SnPfWnbvtJm6jVKfVMgaU75Mm+NMt9myM O/hMElP5p83uwx5mbTsySkjz1u1ei0dVmz4O01fBaC8586byYlQ+Qni7D5W9yVksbJ8p BPUF/HPzVNWKkMlabuFUA9lTwjwzJyQiY+U3/vFzT9CAHRnuRg28gMpsLU/1FwVz9MKR 192Y5vPnfY592WO43qV8r7zr5n8yq5deC3ydiOtUs0Seq7M6wVoW900cWrKiN6sp1Xde NQhqWI5Z4wrE1UQNbLxD3XFpGYtiMDKjZJ95I0ak6UIQxYByadVICjedYnuOY5yr61rP 3qBw== X-Gm-Message-State: AOUpUlHpckhA9iLX3XIP2AH1vnxl0MAQW7lXatpsOMzpbzZZGwKfkLNH rkOPzRBRwStLC5czmlDGh/1+0A== X-Google-Smtp-Source: AA+uWPzku3J1gjB7Rb+He282THC1Pas51QQkpMvrwNlDDQGRospDla4tqYz+chdUGEXjypJIj7LNZA== X-Received: by 2002:a50:ce19:: with SMTP id y25-v6mr31237150edi.207.1534331594690; Wed, 15 Aug 2018 04:13:14 -0700 (PDT) Received: from energia.localnet ([212.48.107.10]) by smtp.gmail.com with ESMTPSA id s12-v6sm8538902edq.20.2018.08.15.04.13.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 Aug 2018 04:13:13 -0700 (PDT) From: Kajetan Staszkiewicz To: Kristof Provost Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Wed, 15 Aug 2018 13:13:08 +0200 Message-ID: <1963876.mpGq17E7dF@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: <1546233.jncNNXsBuh@energia> References: <8680316.SccKl5VnxN@energia> <1546233.jncNNXsBuh@energia> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2773872.m3asgv598W"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Aug 2018 11:13:17 -0000 --nextPart2773872.m3asgv598W Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Tuesday, 14 August 2018 01:32:17 CEST Kajetan Staszkiewicz wrote: > > > There are some more issues I found around pf_map_addr. Some of them I > > > mentioned in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D2290= 92. > > > Some > > > more came out while working on this least-states loadbalancing. I will > > > group them into something meaningful and make another PR for them. > >=20 > > Yeah, that bug is still on my todo list somewhere, but things are > > extremely > > hectic at the moment, and I can=E2=80=99t make any promises about when = I=E2=80=99ll have > > time for it. >=20 > I thought that was rather on my todo :) =2E.. mostly because I though of other issues found in pf_map_addr I took the liberty of opening another bug report just for those: 230640. I= =20 think that should be addressed first because 229092 can be really correctly= =20 fixed. =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart2773872.m3asgv598W Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3QKxAAKCRDjtFCvbXs6 FH6HAKCDMQIv2KzVAMmFezuOnS621l0QtQCfajEzWO6kg061Bwz0OvSAh6s8ILs= =/tVL -----END PGP SIGNATURE----- --nextPart2773872.m3asgv598W-- From owner-freebsd-pf@freebsd.org Thu Aug 16 17:37:06 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 980811070029 for ; Thu, 16 Aug 2018 17:37:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 266CE8F2E4 for ; Thu, 16 Aug 2018 17:37:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DFB39107001D; Thu, 16 Aug 2018 17:37:05 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE616107001C for ; Thu, 16 Aug 2018 17:37:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FBD28F2E1 for ; Thu, 16 Aug 2018 17:37:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BE43B13CEB for ; Thu, 16 Aug 2018 17:37:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7GHb4lG080602 for ; Thu, 16 Aug 2018 17:37:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7GHb4Ko080601 for pf@FreeBSD.org; Thu, 16 Aug 2018 17:37:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 230640] pf_map_addr operates on unlocked src_nodes and pools Date: Thu, 16 Aug 2018 17:37:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2018 17:37:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230640 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |patch Assignee|bugs@FreeBSD.org |pf@FreeBSD.org CC| |net@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Fri Aug 17 16:26:32 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD2D3107481C for ; Fri, 17 Aug 2018 16:26:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4A62A870FB for ; Fri, 17 Aug 2018 16:26:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0F33E1074818; Fri, 17 Aug 2018 16:26:32 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F21B01074817 for ; Fri, 17 Aug 2018 16:26:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 922E5870F8 for ; Fri, 17 Aug 2018 16:26:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BFBDA1FCE7 for ; Fri, 17 Aug 2018 16:26:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7HGQUgP023051 for ; Fri, 17 Aug 2018 16:26:30 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7HGQUxP023050 for pf@FreeBSD.org; Fri, 17 Aug 2018 16:26:30 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229092] [pf] [pfsync] States created by route-to rules pfsynced without interface Date: Fri, 17 Aug 2018 16:26:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2018 16:26:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092 --- Comment #7 from Kajetan Staszkiewicz --- Do we consider breaking pfsync protocol compatibility? If we could just mod= ify the protocol to sync redirection interface, there would be no need for reconstrucing it and for identical ruleset with identical table contents. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Sat Aug 18 14:37:49 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABA99106E6AC for ; Sat, 18 Aug 2018 14:37:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 430A093CD1 for ; Sat, 18 Aug 2018 14:37:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 049F3106E6AB; Sat, 18 Aug 2018 14:37:49 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E56FF106E6AA for ; Sat, 18 Aug 2018 14:37:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 69CCC93CCE for ; Sat, 18 Aug 2018 14:37:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B5228B5C7 for ; Sat, 18 Aug 2018 14:37:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7IEblIr076947 for ; Sat, 18 Aug 2018 14:37:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7IEblHx076946 for pf@FreeBSD.org; Sat, 18 Aug 2018 14:37:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229092] [pf] [pfsync] States created by route-to rules pfsynced without interface Date: Sat, 18 Aug 2018 14:37:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kp@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Aug 2018 14:37:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092 --- Comment #8 from Kristof Provost --- (In reply to Kajetan Staszkiewicz from comment #7) I'd be very very hesitant to break compatibility. A common pattern with pfs= ync is that one gateway is upgraded while the other takes over. That'll need to keep working. That said, it might be possible to extend the protocol by using one of the = _pad fields. It'd have to work (minus newly supported/improved cases) when synci= ng with older code, but that might be possible. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-pf@freebsd.org Sat Aug 18 22:16:07 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 531831077821 for ; Sat, 18 Aug 2018 22:16:07 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C2EF679AFB for ; Sat, 18 Aug 2018 22:16:06 +0000 (UTC) (envelope-from vegeta@tuxpowered.net) Received: by mail-ed1-x534.google.com with SMTP id o8-v6so6380601edt.13 for ; Sat, 18 Aug 2018 15:16:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuxpowered-net.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:organization:user-agent :in-reply-to:references:mime-version; bh=Vz3HenC0xYAK/PS6waZX4PmDOmP99iPft8Q+2p+V4Ck=; b=tPAw0RuszcBpf4hyx3D5bpIKu1H4FvaHf9o0GKsDKjfGt/G+SnCirRZeDtCYL19zhv /+eRhHw5yeUy/Ab5b7Gv6hEhG/XPnCtzUQ3bj0nwCRlV2u2qIUS5dAO1JYvMIMeFHP6Z D37qyXtlflJpdlJ64hGyCUFQ7Bf0blmalpxOHFRqdbUIF89t5OT0xfYS/uAJp8eZSBNu noc9N5cvH6/cFcCHtTjaIea5QJTKWqt8Liyt/jDVo007N+ZTYOl/NQR5mTDcOY6AGpoh vchB3GHvPyw/EO+Hr/H3M+6KQHGgmtBXKoQsQI3Rn9jMD1ysZ2qqfVMJLnU/3hWkPTKx 842A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:organization :user-agent:in-reply-to:references:mime-version; bh=Vz3HenC0xYAK/PS6waZX4PmDOmP99iPft8Q+2p+V4Ck=; b=TppzysCMBue0jxs+r/tRe1BbRoVXtDKlBwGv2o0FZZZWpYIQmDql/QvXiMxQFnU6ix sTMRdbB3sdeIrCfAhBTwSSDj8MNkzyChH+o7KgnOHxWeezLBZbMPJrl3/1tI7Jwj2/2+ 7I/5MBG7XQtvaedn9q53ljRnPWTanZ8sPEFnJe727f6ibjURLlkLTqyMKhvyb6lVMMs7 9Q5/mhIlAknf1xS7qZoqIJWrG9kpFWSZo/vDpSkz+lqecF/Mz95qzwOc5w0tGl6hwJFX mu9I8RXbF85H7YzIFus0FcPmFJnvnxc5BHwZ8YqDILX/iSTH+g/eoAMxj2BcAWTGuOaW X6ng== X-Gm-Message-State: AOUpUlHAEctu4G9wMPgSb4VkAjJvK2G/8UZ3D7HYP/2Tj/pKRgCOgr8g uL2hsnhgdhdL5X29kpoWzcfyIlwkf8c= X-Google-Smtp-Source: AA+uWPw+E4QX7ZmSy0rU7IG+59U3ySpgiRhWgGq5z8xoagIVLNACzN2d/iX91/gMyyxGlj4EHlV1kQ== X-Received: by 2002:aa7:d142:: with SMTP id r2-v6mr47916806edo.286.1534630565508; Sat, 18 Aug 2018 15:16:05 -0700 (PDT) Received: from energia.localnet ([2a02:8108:50bf:d514::5]) by smtp.gmail.com with ESMTPSA id p20-v6sm3080092edr.12.2018.08.18.15.16.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 18 Aug 2018 15:16:04 -0700 (PDT) From: Kajetan Staszkiewicz To: Kristof Provost Cc: freebsd-pf@freebsd.org Subject: Re: pf tables locking Date: Sun, 19 Aug 2018 00:15:58 +0200 Message-ID: <1831273.qCtLAga6ZT@energia> Organization: tuxpowered.net User-Agent: KMail/5.2.3 (Linux/4.16.0-16.2-liquorix-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> References: <8680316.SccKl5VnxN@energia> <18F24996-29D6-4792-BCB7-88738F756077@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart5655015.LPuHGhcovh"; micalg="pgp-sha1"; protocol="application/pgp-signature" X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Aug 2018 22:16:07 -0000 --nextPart5655015.LPuHGhcovh Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="UTF-8" On Monday, 13 August 2018 15:22:33 CEST Kristof Provost wrote: > > This function is called from pf_test only after PF_RULES_RUNLOCK(). >=20 > I think you=E2=80=99re right, this does look wrong. >=20 > It=E2=80=99s very unlikely that this will actually lead to a crash, becau= se > rules (and associated tables) won=E2=80=99t just go away while there=E2= =80=99s still > state, but we could theoretically lose memory (in the pfrke_counters > allocation), and miscount. >=20 > I don=E2=80=99t want to re-take the rules lock for this But what about things other than counters and disappearing tables, that is= =20 getting addresses out of pool in pf_map_addr? I understand that rpool can't= =20 change live because it changes only with loading a ruleset. But then there = is=20 pfr_pool_get. This one operates totally unlocked. I proposed a patch lockin= g=20 pools in https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D230640 but now= as I=20 see it locking of each table seems necessary. Why not have granular locking for each pool (or maybe rule) and for each=20 table? =2D-=20 | pozdrawiam / greetings | powered by Debian, FreeBSD and CentOS | | Kajetan Staszkiewicz | jabber,email: vegeta()tuxpowered net | | Vegeta | www: http://vegeta.tuxpowered.net | `------------------------^---------------------------------------' --nextPart5655015.LPuHGhcovh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iF0EABECAB0WIQSOEQZObv2B8mf0JbnjtFCvbXs6FAUCW3iangAKCRDjtFCvbXs6 FPG4AJ4mSh2S9rFxP3NwQlDz1CG9unGiYgCguljhbuVzV9AdKgp3dJDypNo2AvE= =jpec -----END PGP SIGNATURE----- --nextPart5655015.LPuHGhcovh-- From owner-freebsd-pf@freebsd.org Sat Aug 18 22:51:52 2018 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB4CC10784BB for ; Sat, 18 Aug 2018 22:51:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 683217AB87 for ; Sat, 18 Aug 2018 22:51:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2CB51107849F; Sat, 18 Aug 2018 22:51:52 +0000 (UTC) Delivered-To: pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BA4C107849E for ; Sat, 18 Aug 2018 22:51:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1DCA7AB82 for ; Sat, 18 Aug 2018 22:51:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0BA18FD8A for ; Sat, 18 Aug 2018 22:51:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w7IMpoFY096548 for ; Sat, 18 Aug 2018 22:51:50 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w7IMpoDt096547 for pf@FreeBSD.org; Sat, 18 Aug 2018 22:51:50 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: pf@FreeBSD.org Subject: [Bug 229092] [pf] [pfsync] States created by route-to rules pfsynced without interface Date: Sat, 18 Aug 2018 22:51:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vegeta@tuxpowered.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: pf@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Aug 2018 22:51:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D229092 --- Comment #9 from Kajetan Staszkiewicz --- I see only those fields free to be used: struct pfsync_state { u_int8_t __spare[2]; } struct pfsync_state_peer { u_int8_t pad[6]; } None of them is enough to carry char ifname[IFNAMSIZ] information. I thought interfaces maybe have some increasing ID which would fit into those bytes b= ut I can't find such thing. We could add such increasing ID to pfi_kif but that would still be an opportunistic solution, working correctly only if two rou= ters have identical interfaces which were added in the same order. That might in some situations be even harder to achieve than having identical ruleset as required by the patch I proposed. --=20 You are receiving this mail because: You are the assignee for the bug.=