From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 05:59:01 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 08F366542B8; Mon, 5 Jul 2021 05:59:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJFPJ6r35z3rdQ; Mon, 5 Jul 2021 05:59:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CF4862093F; Mon, 5 Jul 2021 05:59:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1655x0pG062813; Mon, 5 Jul 2021 05:59:00 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1655x01s062812; Mon, 5 Jul 2021 05:59:00 GMT (envelope-from git) Date: Mon, 5 Jul 2021 05:59:00 GMT Message-Id: <202107050559.1655x01s062812@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: dc42a0503bc9 - stable/13 - tests/libalias: Bugfix in used variables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: dc42a0503bc97fbf56036458b5616185e0abda37 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 05:59:01 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=dc42a0503bc97fbf56036458b5616185e0abda37 commit dc42a0503bc97fbf56036458b5616185e0abda37 Author: Lutz Donnerhacke AuthorDate: 2021-07-02 21:58:16 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-05 05:58:02 +0000 tests/libalias: Bugfix in used variables Fix two copy and waste errors (referencing the wrong variable). (cherry picked from commit b3dc6f137b8d626c4415a5fa9c9874ac4ca78658) --- tests/sys/netinet/libalias/3_natin.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/sys/netinet/libalias/3_natin.c b/tests/sys/netinet/libalias/3_natin.c index e8e5257e4baa..3bc088ce3da9 100644 --- a/tests/sys/netinet/libalias/3_natin.c +++ b/tests/sys/netinet/libalias/3_natin.c @@ -112,7 +112,7 @@ ATF_TC_BODY(1_portforward, dummy) LibAliasSetAddress(la, masq); /* delete and try again */ - LibAliasRedirectDelete(la, pf2); + LibAliasRedirectDelete(la, pf3); UDP_UNNAT_FAIL(p, u, ext, 0x5678, masq, 0xabcd); /* @@ -133,7 +133,7 @@ ATF_TC_BODY(1_portforward, dummy) LibAliasSetAddress(la, masq); /* delete and try again */ - LibAliasRedirectDelete(la, pf2); + LibAliasRedirectDelete(la, pf4); UDP_UNNAT_FAIL(p, u, ext, 0x5678, masq, 0xabcd); free(p); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 06:01:30 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BDB1D6542DF; Mon, 5 Jul 2021 06:01:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJFSB55Smz3rlW; Mon, 5 Jul 2021 06:01:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 97C2C2047C; Mon, 5 Jul 2021 06:01:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16561UWY075988; Mon, 5 Jul 2021 06:01:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16561UFa075987; Mon, 5 Jul 2021 06:01:30 GMT (envelope-from git) Date: Mon, 5 Jul 2021 06:01:30 GMT Message-Id: <202107050601.16561UFa075987@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 884da892b634 - stable/12 - tests/libalias: Bugfix in used variables MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 884da892b634224bebbeb12deaa0b00442e14c87 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 06:01:30 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=884da892b634224bebbeb12deaa0b00442e14c87 commit 884da892b634224bebbeb12deaa0b00442e14c87 Author: Lutz Donnerhacke AuthorDate: 2021-07-02 21:58:16 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-05 06:01:06 +0000 tests/libalias: Bugfix in used variables Fix two copy and waste errors (referencing the wrong variable). (cherry picked from commit b3dc6f137b8d626c4415a5fa9c9874ac4ca78658) --- tests/sys/netinet/libalias/3_natin.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/sys/netinet/libalias/3_natin.c b/tests/sys/netinet/libalias/3_natin.c index e8e5257e4baa..3bc088ce3da9 100644 --- a/tests/sys/netinet/libalias/3_natin.c +++ b/tests/sys/netinet/libalias/3_natin.c @@ -112,7 +112,7 @@ ATF_TC_BODY(1_portforward, dummy) LibAliasSetAddress(la, masq); /* delete and try again */ - LibAliasRedirectDelete(la, pf2); + LibAliasRedirectDelete(la, pf3); UDP_UNNAT_FAIL(p, u, ext, 0x5678, masq, 0xabcd); /* @@ -133,7 +133,7 @@ ATF_TC_BODY(1_portforward, dummy) LibAliasSetAddress(la, masq); /* delete and try again */ - LibAliasRedirectDelete(la, pf2); + LibAliasRedirectDelete(la, pf4); UDP_UNNAT_FAIL(p, u, ext, 0x5678, masq, 0xabcd); free(p); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:02 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8E866658980; Mon, 5 Jul 2021 09:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSQ3cPFz4k5d; Mon, 5 Jul 2021 09:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 64BD223C02; Mon, 5 Jul 2021 09:47:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l2pc071430; Mon, 5 Jul 2021 09:47:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l2le071429; Mon, 5 Jul 2021 09:47:02 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:02 GMT Message-Id: <202107050947.1659l2le071429@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: a259db87edfe - stable/13 - usr.sbin/pkg: Don't re-define roundup2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a259db87edfe054f51615d7bd08464c575027738 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:02 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=a259db87edfe054f51615d7bd08464c575027738 commit a259db87edfe054f51615d7bd08464c575027738 Author: Alex Richardson AuthorDate: 2021-01-28 17:25:50 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 usr.sbin/pkg: Don't re-define roundup2 The file already includes sys/param.h and should use that definition. I found this while testing D28332. Reviewed By: bapt Differential Revision: https://reviews.freebsd.org/D28331 (cherry picked from commit 9a0a48b12d3d85f0f00b99439e6a8779f5f1abb6) --- usr.sbin/pkg/config.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/usr.sbin/pkg/config.c b/usr.sbin/pkg/config.c index 97983ea1f58a..08e206b93511 100644 --- a/usr.sbin/pkg/config.c +++ b/usr.sbin/pkg/config.c @@ -47,8 +47,6 @@ __FBSDID("$FreeBSD$"); #include "config.h" -#define roundup2(x, y) (((x)+((y)-1))&(~((y)-1))) /* if y is powers of two */ - struct config_value { char *value; STAILQ_ENTRY(config_value) next; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:03 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C268F65886D; Mon, 5 Jul 2021 09:47:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSR4wYtz4kG1; Mon, 5 Jul 2021 09:47:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7D0DE237F5; Mon, 5 Jul 2021 09:47:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l3kA071460; Mon, 5 Jul 2021 09:47:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l3vr071459; Mon, 5 Jul 2021 09:47:03 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:03 GMT Message-Id: <202107050947.1659l3vr071459@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: 0a55a5bf6170 - stable/13 - if_vtnet: Fix pointer-sign and used parameter warnings MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0a55a5bf61707c87b3976eaf203ef4f988c4c396 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:03 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=0a55a5bf61707c87b3976eaf203ef4f988c4c396 commit 0a55a5bf61707c87b3976eaf203ef4f988c4c396 Author: Alex Richardson AuthorDate: 2021-02-22 17:26:21 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 if_vtnet: Fix pointer-sign and used parameter warnings Reviewed By: grehan Differential Revision: https://reviews.freebsd.org/D28726 (cherry picked from commit c1b554c868195ed10ac1eb25fa9ca4a5baae4638) --- sys/dev/virtio/network/if_vtnet.c | 42 ++++++++++++++++++------------------ sys/dev/virtio/network/if_vtnetvar.h | 6 +++--- 2 files changed, 24 insertions(+), 24 deletions(-) diff --git a/sys/dev/virtio/network/if_vtnet.c b/sys/dev/virtio/network/if_vtnet.c index 7aee77839bf3..3650d19fbe0e 100644 --- a/sys/dev/virtio/network/if_vtnet.c +++ b/sys/dev/virtio/network/if_vtnet.c @@ -113,7 +113,7 @@ static void vtnet_free_rx_filters(struct vtnet_softc *); static int vtnet_alloc_virtqueues(struct vtnet_softc *); static int vtnet_alloc_interface(struct vtnet_softc *); static int vtnet_setup_interface(struct vtnet_softc *); -static int vtnet_ioctl_mtu(struct vtnet_softc *, int); +static int vtnet_ioctl_mtu(struct vtnet_softc *, u_int); static int vtnet_ioctl_ifflags(struct vtnet_softc *); static int vtnet_ioctl_multi(struct vtnet_softc *); static int vtnet_ioctl_ifcap(struct vtnet_softc *, struct ifreq *); @@ -206,9 +206,9 @@ static void vtnet_exec_ctrl_cmd(struct vtnet_softc *, void *, static int vtnet_ctrl_mac_cmd(struct vtnet_softc *, uint8_t *); static int vtnet_ctrl_guest_offloads(struct vtnet_softc *, uint64_t); static int vtnet_ctrl_mq_cmd(struct vtnet_softc *, uint16_t); -static int vtnet_ctrl_rx_cmd(struct vtnet_softc *, uint8_t, int); -static int vtnet_set_promisc(struct vtnet_softc *, int); -static int vtnet_set_allmulti(struct vtnet_softc *, int); +static int vtnet_ctrl_rx_cmd(struct vtnet_softc *, uint8_t, bool); +static int vtnet_set_promisc(struct vtnet_softc *, bool); +static int vtnet_set_allmulti(struct vtnet_softc *, bool); static void vtnet_rx_filter(struct vtnet_softc *); static void vtnet_rx_filter_mac(struct vtnet_softc *); static int vtnet_exec_vlan_filter(struct vtnet_softc *, int, uint16_t); @@ -373,7 +373,7 @@ MODULE_DEPEND(vtnet, netmap, 1, 1, 1); VIRTIO_SIMPLE_PNPINFO(vtnet, VIRTIO_ID_NETWORK, "VirtIO Networking Adapter"); static int -vtnet_modevent(module_t mod, int type, void *unused) +vtnet_modevent(module_t mod __unused, int type, void *unused __unused) { int error = 0; static int loaded = 0; @@ -1247,7 +1247,7 @@ vtnet_rx_cluster_size(struct vtnet_softc *sc, int mtu) } static int -vtnet_ioctl_mtu(struct vtnet_softc *sc, int mtu) +vtnet_ioctl_mtu(struct vtnet_softc *sc, u_int mtu) { struct ifnet *ifp; int clustersz; @@ -1808,7 +1808,7 @@ vtnet_rxq_csum_needs_csum(struct vtnet_rxq *rxq, struct mbuf *m, uint16_t etype, static int vtnet_rxq_csum_data_valid(struct vtnet_rxq *rxq, struct mbuf *m, - uint16_t etype, int hoff, struct virtio_net_hdr *hdr) + uint16_t etype, int hoff, struct virtio_net_hdr *hdr __unused) { struct vtnet_softc *sc; int protocol; @@ -1928,7 +1928,7 @@ vtnet_rxq_merged_eof(struct vtnet_rxq *rxq, struct mbuf *m_head, int nbufs) while (--nbufs > 0) { struct mbuf *m; - int len; + uint32_t len; m = virtqueue_dequeue(vq, &len); if (m == NULL) { @@ -2058,7 +2058,7 @@ vtnet_rxq_eof(struct vtnet_rxq *rxq) while (count-- > 0) { struct mbuf *m; - int len, nbufs, adjsz; + uint32_t len, nbufs, adjsz; m = virtqueue_dequeue(vq, &len); if (m == NULL) @@ -2158,7 +2158,7 @@ vtnet_rx_vq_process(struct vtnet_rxq *rxq, int tries) { struct vtnet_softc *sc; struct ifnet *ifp; - int more; + u_int more; #ifdef DEV_NETMAP int nmirq; #endif /* DEV_NETMAP */ @@ -2232,7 +2232,7 @@ vtnet_rx_vq_intr(void *xrxq) } static void -vtnet_rxq_tq_intr(void *xrxq, int pending) +vtnet_rxq_tq_intr(void *xrxq, int pending __unused) { struct vtnet_rxq *rxq; @@ -2749,7 +2749,7 @@ vtnet_txq_mq_start(struct ifnet *ifp, struct mbuf *m) } static void -vtnet_txq_tq_deferred(void *xtxq, int pending) +vtnet_txq_tq_deferred(void *xtxq, int pending __unused) { struct vtnet_softc *sc; struct vtnet_txq *txq; @@ -2784,7 +2784,7 @@ vtnet_txq_start(struct vtnet_txq *txq) } static void -vtnet_txq_tq_intr(void *xtxq, int pending) +vtnet_txq_tq_intr(void *xtxq, int pending __unused) { struct vtnet_softc *sc; struct vtnet_txq *txq; @@ -3619,7 +3619,7 @@ vtnet_ctrl_mq_cmd(struct vtnet_softc *sc, uint16_t npairs) } static int -vtnet_ctrl_rx_cmd(struct vtnet_softc *sc, uint8_t cmd, int on) +vtnet_ctrl_rx_cmd(struct vtnet_softc *sc, uint8_t cmd, bool on) { struct sglist_seg segs[3]; struct sglist sg; @@ -3637,7 +3637,7 @@ vtnet_ctrl_rx_cmd(struct vtnet_softc *sc, uint8_t cmd, int on) s.hdr.class = VIRTIO_NET_CTRL_RX; s.hdr.cmd = cmd; - s.onoff = !!on; + s.onoff = on; s.ack = VIRTIO_NET_ERR; sglist_init(&sg, nitems(segs), segs); @@ -3653,13 +3653,13 @@ vtnet_ctrl_rx_cmd(struct vtnet_softc *sc, uint8_t cmd, int on) } static int -vtnet_set_promisc(struct vtnet_softc *sc, int on) +vtnet_set_promisc(struct vtnet_softc *sc, bool on) { return (vtnet_ctrl_rx_cmd(sc, VIRTIO_NET_CTRL_RX_PROMISC, on)); } static int -vtnet_set_allmulti(struct vtnet_softc *sc, int on) +vtnet_set_allmulti(struct vtnet_softc *sc, bool on) { return (vtnet_ctrl_rx_cmd(sc, VIRTIO_NET_CTRL_RX_ALLMULTI, on)); } @@ -3781,9 +3781,9 @@ vtnet_rx_filter_mac(struct vtnet_softc *sc) if_printf(ifp, "error setting host MAC filter table\n"); out: - if (promisc != 0 && vtnet_set_promisc(sc, 1) != 0) + if (promisc != 0 && vtnet_set_promisc(sc, true) != 0) if_printf(ifp, "cannot enable promiscuous mode\n"); - if (allmulti != 0 && vtnet_set_allmulti(sc, 1) != 0) + if (allmulti != 0 && vtnet_set_allmulti(sc, true) != 0) if_printf(ifp, "cannot enable all-multicast mode\n"); } @@ -3912,7 +3912,7 @@ vtnet_update_speed_duplex(struct vtnet_softc *sc) /* BMV: Ignore duplex. */ speed = virtio_read_dev_config_4(sc->vtnet_dev, offsetof(struct virtio_net_config, speed)); - if (speed != -1) + if (speed != UINT32_MAX) ifp->if_baudrate = IF_Mbps(speed); } @@ -3952,7 +3952,7 @@ vtnet_update_link_status(struct vtnet_softc *sc) } static int -vtnet_ifmedia_upd(struct ifnet *ifp) +vtnet_ifmedia_upd(struct ifnet *ifp __unused) { return (EOPNOTSUPP); } diff --git a/sys/dev/virtio/network/if_vtnetvar.h b/sys/dev/virtio/network/if_vtnetvar.h index d26f7240a67a..55e8a7267499 100644 --- a/sys/dev/virtio/network/if_vtnetvar.h +++ b/sys/dev/virtio/network/if_vtnetvar.h @@ -161,7 +161,7 @@ struct vtnet_softc { #define VTNET_FLAG_FIXUP_NEEDS_CSUM 0x2000 #define VTNET_FLAG_SW_LRO 0x4000 - int vtnet_hdr_size; + u_int vtnet_hdr_size; int vtnet_rx_nmbufs; int vtnet_rx_clustersz; int vtnet_rx_nsegs; @@ -172,7 +172,7 @@ struct vtnet_softc { int vtnet_max_vq_pairs; int vtnet_tx_nsegs; int vtnet_if_flags; - int vtnet_max_mtu; + u_int vtnet_max_mtu; int vtnet_lro_entry_count; int vtnet_lro_mbufq_depth; @@ -189,7 +189,7 @@ struct vtnet_softc { struct mtx vtnet_mtx; char vtnet_mtx_name[16]; - char vtnet_hwaddr[ETHER_ADDR_LEN]; + uint8_t vtnet_hwaddr[ETHER_ADDR_LEN]; }; static bool From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:05 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2506A658636; Mon, 5 Jul 2021 09:47:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSS734gz4k3W; Mon, 5 Jul 2021 09:47:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C7554238D2; Mon, 5 Jul 2021 09:47:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l4CC071485; Mon, 5 Jul 2021 09:47:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l4H9071484; Mon, 5 Jul 2021 09:47:04 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:04 GMT Message-Id: <202107050947.1659l4H9071484@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: ae0219c10fe3 - stable/13 - fwohci: Cast bitfield to uint32_t before passing it to roundup2(). MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ae0219c10fe3a2ea9caa7bc88ddc7d0a7e988efd Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:05 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=ae0219c10fe3a2ea9caa7bc88ddc7d0a7e988efd commit ae0219c10fe3a2ea9caa7bc88ddc7d0a7e988efd Author: John Baldwin AuthorDate: 2021-02-17 21:12:06 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 fwohci: Cast bitfield to uint32_t before passing it to roundup2(). The fallback for __align_up() used by roundup2() uses __typeof__() which doesn't work for bitfields. This fixes the build on GCC which uses the fallback. Reviewed by: arichardson, markj Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D28599 (cherry picked from commit 50a61f8db532676d7dc37410540fff7209b7c072) --- sys/dev/firewire/fwohci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/dev/firewire/fwohci.c b/sys/dev/firewire/fwohci.c index 30eca83900e7..634c78e6afe8 100644 --- a/sys/dev/firewire/fwohci.c +++ b/sys/dev/firewire/fwohci.c @@ -2696,7 +2696,7 @@ fwohci_get_plen(struct fwohci_softc *sc, struct fwohci_dbch *dbch, struct fw_pkt info = &tinfo[fp->mode.common.tcode]; r = info->hdr_len + sizeof(uint32_t); if ((info->flag & FWTI_BLOCK_ASY) != 0) - r += roundup2(fp->mode.wreqb.len, sizeof(uint32_t)); + r += roundup2((uint32_t)fp->mode.wreqb.len, sizeof(uint32_t)); if (r == sizeof(uint32_t)) { /* XXX */ From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:05 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F06B36585AF; Mon, 5 Jul 2021 09:47:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLST676rz4k9M; Mon, 5 Jul 2021 09:47:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B9E0D23BEE; Mon, 5 Jul 2021 09:47:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l5af071509; Mon, 5 Jul 2021 09:47:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l59a071508; Mon, 5 Jul 2021 09:47:05 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:05 GMT Message-Id: <202107050947.1659l59a071508@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: f42db652c3d2 - stable/13 - Expose clang's alignment builtins and use them for roundup2/rounddown2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f42db652c3d2e55ce4bf9d2af4bb20e533f0ab83 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:06 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=f42db652c3d2e55ce4bf9d2af4bb20e533f0ab83 commit f42db652c3d2e55ce4bf9d2af4bb20e533f0ab83 Author: Alex Richardson AuthorDate: 2021-02-03 15:27:17 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 Expose clang's alignment builtins and use them for roundup2/rounddown2 This makes roundup2/rounddown2 type- and const-preserving and allows using it on pointer types without casting to uintptr_t first. Not performing pointer-to-integer conversions also helps the compiler's optimization passes and can therefore result in better code generation. When using it with integer values there should be no change other than the compiler checking that the alignment value is a valid power-of-two. I originally implemented these builtins for CHERI a few years ago and they have been very useful for CheriBSD. However, they are also useful for non-CHERI code so I was able to upstream them for Clang 10.0. Rationale from the clang documentation: Clang provides builtins to support checking and adjusting alignment of pointers and integers. These builtins can be used to avoid relying on implementation-defined behavior of arithmetic on integers derived from pointers. Additionally, these builtins retain type information and, unlike bitwise arithmetic, they can perform semantic checking on the alignment value. There is also a feature request for GCC, so GCC may also support it in the future: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=98641 Reviewed By: brooks, jhb, imp Differential Revision: https://reviews.freebsd.org/D28332 (cherry picked from commit 8fa6abb6f4f64f4f23e2920e2aea7996566851a4) --- sys/sys/cdefs.h | 19 +++++++++++++++++++ sys/sys/param.h | 4 ++-- 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/sys/sys/cdefs.h b/sys/sys/cdefs.h index 3acb165dad05..ff461db9ec2e 100644 --- a/sys/sys/cdefs.h +++ b/sys/sys/cdefs.h @@ -879,4 +879,23 @@ #define __guarded_by(x) __lock_annotate(guarded_by(x)) #define __pt_guarded_by(x) __lock_annotate(pt_guarded_by(x)) +/* Alignment builtins for better type checking and improved code generation. */ +/* Provide fallback versions for other compilers (GCC/Clang < 10): */ +#if !__has_builtin(__builtin_is_aligned) +#define __builtin_is_aligned(x, align) \ + (((__uintptr_t)x & ((align) - 1)) == 0) +#endif +#if !__has_builtin(__builtin_align_up) +#define __builtin_align_up(x, align) \ + ((__typeof__(x))(((__uintptr_t)(x)+((align)-1))&(~((align)-1)))) +#endif +#if !__has_builtin(__builtin_align_down) +#define __builtin_align_down(x, align) \ + ((__typeof__(x))((x)&(~((align)-1)))) +#endif + +#define __align_up(x, y) __builtin_align_up(x, y) +#define __align_down(x, y) __builtin_align_down(x, y) +#define __is_aligned(x, y) __builtin_is_aligned(x, y) + #endif /* !_SYS_CDEFS_H_ */ diff --git a/sys/sys/param.h b/sys/sys/param.h index 6e75123fca77..41db9a1675f7 100644 --- a/sys/sys/param.h +++ b/sys/sys/param.h @@ -305,9 +305,9 @@ #endif #define nitems(x) (sizeof((x)) / sizeof((x)[0])) #define rounddown(x, y) (((x)/(y))*(y)) -#define rounddown2(x, y) ((x)&(~((y)-1))) /* if y is power of two */ +#define rounddown2(x, y) __align_down(x, y) /* if y is power of two */ #define roundup(x, y) ((((x)+((y)-1))/(y))*(y)) /* to any y */ -#define roundup2(x, y) (((x)+((y)-1))&(~((y)-1))) /* if y is powers of two */ +#define roundup2(x, y) __align_up(x, y) /* if y is powers of two */ #define powerof2(x) ((((x)-1)&(x))==0) /* Macros for min/max. */ From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:07 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49C82658879; Mon, 5 Jul 2021 09:47:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSW0ZVmz4k5w; Mon, 5 Jul 2021 09:47:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E3A6323C03; Mon, 5 Jul 2021 09:47:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l6P9071533; Mon, 5 Jul 2021 09:47:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l6Zl071532; Mon, 5 Jul 2021 09:47:06 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:06 GMT Message-Id: <202107050947.1659l6Zl071532@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: c89f104c12e1 - stable/13 - vis(3): avoid out-of-bounds stack buffer reads MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: c89f104c12e196a86dac2aa6d9119ebb8e4aabe4 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:07 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=c89f104c12e196a86dac2aa6d9119ebb8e4aabe4 commit c89f104c12e196a86dac2aa6d9119ebb8e4aabe4 Author: Alex Richardson AuthorDate: 2021-06-16 15:27:13 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 vis(3): avoid out-of-bounds stack buffer reads I found this while running kdump(1) on a CheriBSD system due to a capability length violation when printing the /etc/libmap.conf read() system call: it crashed immediately after printing the first line. Found by: CHERI Reviewed By: jhb MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D30771 (cherry picked from commit 1a2f06d0f2905c9a18340b377cbbe772f2ca6844) --- contrib/libc-vis/vis.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/contrib/libc-vis/vis.c b/contrib/libc-vis/vis.c index 21c07b70619d..c43186a44b51 100644 --- a/contrib/libc-vis/vis.c +++ b/contrib/libc-vis/vis.c @@ -465,7 +465,8 @@ istrsenvisx(char **mbdstp, size_t *dlen, const char *mbsrc, size_t mblength, while (mbslength > 0) { /* Convert one multibyte character to wchar_t. */ if (!cerr) - clen = mbrtowc(src, mbsrc, MB_LEN_MAX, &mbstate); + clen = mbrtowc(src, mbsrc, MIN(mbslength, MB_LEN_MAX), + &mbstate); if (cerr || clen < 0) { /* Conversion error, process as a byte instead. */ *src = (wint_t)(u_char)*mbsrc; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:08 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A632165898E; Mon, 5 Jul 2021 09:47:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSX2FSzz4kGC; Mon, 5 Jul 2021 09:47:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0D597237F6; Mon, 5 Jul 2021 09:47:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l74b071557; Mon, 5 Jul 2021 09:47:07 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l7v1071556; Mon, 5 Jul 2021 09:47:07 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:07 GMT Message-Id: <202107050947.1659l7v1071556@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: 5694f4af7916 - stable/13 - kdump: use VIS_NOLOCALE when printing output MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 5694f4af7916110bc2f71bb150ecad863a46379d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:08 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=5694f4af7916110bc2f71bb150ecad863a46379d commit 5694f4af7916110bc2f71bb150ecad863a46379d Author: Alex Richardson AuthorDate: 2021-06-16 15:28:27 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:11 +0000 kdump: use VIS_NOLOCALE when printing output Attempting to parse the output as UTF-8 exposed an out-of-bounds read in vis(3) (see https://reviews.freebsd.org/D30771). However, I don't think it makes much sense to decode this output at all. Additionally, passing one byte at a time will almost certainly result in incorrect output for multi-byte characters. Reviewed By: brooks MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D30772 (cherry picked from commit 1a3a57d74590bb44e96629aec35897b475bd69fa) --- usr.bin/kdump/kdump.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.bin/kdump/kdump.c b/usr.bin/kdump/kdump.c index 05988f43133e..d863fffea774 100644 --- a/usr.bin/kdump/kdump.c +++ b/usr.bin/kdump/kdump.c @@ -1638,7 +1638,7 @@ visdump(char *dp, int datalen, int screenwidth) printf(" \""); col = 8; for (;datalen > 0; datalen--, dp++) { - vis(visbuf, *dp, VIS_CSTYLE, *(dp+1)); + vis(visbuf, *dp, VIS_CSTYLE | VIS_NOLOCALE, *(dp+1)); cp = visbuf; /* * Keep track of printables and From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:09 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 82F46658951; Mon, 5 Jul 2021 09:47:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSY2r4xz4k7h; Mon, 5 Jul 2021 09:47:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 31814238D3; Mon, 5 Jul 2021 09:47:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659l9Rc071583; Mon, 5 Jul 2021 09:47:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659l9hY071582; Mon, 5 Jul 2021 09:47:09 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:09 GMT Message-Id: <202107050947.1659l9hY071582@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: ed1563aba95b - stable/13 - truss: fix handling of 64-bit arguments/return values for compat32 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ed1563aba95bd76cd316067dee2f5e6ab4f25561 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:09 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=ed1563aba95bd76cd316067dee2f5e6ab4f25561 commit ed1563aba95bd76cd316067dee2f5e6ab4f25561 Author: Alex Richardson AuthorDate: 2021-06-16 15:29:57 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:12 +0000 truss: fix handling of 64-bit arguments/return values for compat32 Deciding whether to combine two values to a 64-bit one should be based on the process ABI, and not dependent on whether truss is compiled for an LP64 ABI. This is a follow-up cleanup for D27625. I found this while looking for uses of the `__LP64__` macro (since using this is wrong for CHERI systems). Test Plan: truss still works. Since I tested on AMD64 and all syscalls in the trace have their quad argument last there was no difference in the output. Should fix output for compat32 on MIPS64 though. Reviewed By: jhb MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D27637 (cherry picked from commit 8ba2e89e987c7ae171c5f099e59d599294d626b7) --- usr.bin/truss/syscalls.c | 40 +++++++++++++++++----------------------- 1 file changed, 17 insertions(+), 23 deletions(-) diff --git a/usr.bin/truss/syscalls.c b/usr.bin/truss/syscalls.c index a8eed9eed355..b8022a3a395b 100644 --- a/usr.bin/truss/syscalls.c +++ b/usr.bin/truss/syscalls.c @@ -1908,32 +1908,29 @@ print_arg(struct syscall_arg *sc, unsigned long *args, register_t *retval, fputs(" ]", fp); break; } -#ifdef __LP64__ - case Quad: - fprintf(fp, "%ld", args[sc->offset]); - break; - case QuadHex: - fprintf(fp, "0x%lx", args[sc->offset]); - break; -#else case Quad: case QuadHex: { - unsigned long long ll; + uint64_t value; + size_t pointer_size = + trussinfo->curthread->proc->abi->pointer_size; + if (pointer_size == 4) { #if _BYTE_ORDER == _LITTLE_ENDIAN - ll = (unsigned long long)args[sc->offset + 1] << 32 | - args[sc->offset]; + value = (uint64_t)args[sc->offset + 1] << 32 | + args[sc->offset]; #else - ll = (unsigned long long)args[sc->offset] << 32 | - args[sc->offset + 1]; + value = (uint64_t)args[sc->offset] << 32 | + args[sc->offset + 1]; #endif + } else { + value = (uint64_t)args[sc->offset]; + } if ((sc->type & ARG_MASK) == Quad) - fprintf(fp, "%lld", ll); + fprintf(fp, "%jd", (intmax_t)value); else - fprintf(fp, "0x%llx", ll); + fprintf(fp, "0x%jx", (intmax_t)value); break; } -#endif case PQuadHex: { uint64_t val; @@ -3015,11 +3012,9 @@ print_syscall_ret(struct trussinfo *trussinfo, int error, register_t *retval) fprintf(trussinfo->outfile, " ERR#%d '%s'\n", sysdecode_freebsd_to_abi_errno(t->proc->abi->abi, error), strerror(error)); - } -#ifndef __LP64__ - else if (sc->decode.ret_type == 2) { + } else if (sc->decode.ret_type == 2 && + t->proc->abi->pointer_size == 4) { off_t off; - #if _BYTE_ORDER == _LITTLE_ENDIAN off = (off_t)retval[1] << 32 | retval[0]; #else @@ -3027,11 +3022,10 @@ print_syscall_ret(struct trussinfo *trussinfo, int error, register_t *retval) #endif fprintf(trussinfo->outfile, " = %jd (0x%jx)\n", (intmax_t)off, (intmax_t)off); - } -#endif - else + } else { fprintf(trussinfo->outfile, " = %jd (0x%jx)\n", (intmax_t)retval[0], (intmax_t)retval[0]); + } } void From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:10 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 888FD658992; Mon, 5 Jul 2021 09:47:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSZ38djz4kRt; Mon, 5 Jul 2021 09:47:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4D8CA23A15; Mon, 5 Jul 2021 09:47:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659lAhb071611; Mon, 5 Jul 2021 09:47:10 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659lAvS071610; Mon, 5 Jul 2021 09:47:10 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:10 GMT Message-Id: <202107050947.1659lAvS071610@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: 6e2b0498aa70 - stable/13 - truss: minor cleanup and pedantic warning fixes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 6e2b0498aa702bace3859273c36bd27d7dff85c3 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:10 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=6e2b0498aa702bace3859273c36bd27d7dff85c3 commit 6e2b0498aa702bace3859273c36bd27d7dff85c3 Author: Alex Richardson AuthorDate: 2021-06-16 15:35:55 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:12 +0000 truss: minor cleanup and pedantic warning fixes Noticed while porting the recent truss compat32 changes to CheriBSD. This also fixes i386 tracing by zero-extending user addresses instead of sign-extending them. Reviewed By: jhb MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D30211 (cherry picked from commit 31dddc6a19042bad303cb3d3aa8f52a2cb22dd77) --- usr.bin/truss/main.c | 5 ++-- usr.bin/truss/syscalls.c | 69 ++++++++++++++++++++++++++++-------------------- 2 files changed, 44 insertions(+), 30 deletions(-) diff --git a/usr.bin/truss/main.c b/usr.bin/truss/main.c index 1a3aae75be88..942f98008071 100644 --- a/usr.bin/truss/main.c +++ b/usr.bin/truss/main.c @@ -55,7 +55,7 @@ __FBSDID("$FreeBSD$"); #include "extern.h" #include "syscall.h" -static void +static __dead2 void usage(void) { fprintf(stderr, "%s\n%s\n", @@ -118,7 +118,8 @@ main(int ac, char **av) fname = optarg; break; case 's': /* Specified string size */ - trussinfo->strsize = strtonum(optarg, 0, INT_MAX, &errstr); + trussinfo->strsize = (int)strtonum(optarg, 0, INT_MAX, + &errstr); if (errstr) errx(1, "maximum string size is %s: %s", errstr, optarg); break; diff --git a/usr.bin/truss/syscalls.c b/usr.bin/truss/syscalls.c index b8022a3a395b..6740ab21ff2c 100644 --- a/usr.bin/truss/syscalls.c +++ b/usr.bin/truss/syscalls.c @@ -48,6 +48,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -863,18 +864,17 @@ lookup(struct xlat *xlat, int val, int base) if (xlat->val == val) return (xlat->str); switch (base) { - case 8: - sprintf(tmp, "0%o", val); - break; - case 16: - sprintf(tmp, "0x%x", val); - break; - case 10: - sprintf(tmp, "%u", val); - break; - default: - errx(1,"Unknown lookup base"); - break; + case 8: + sprintf(tmp, "0%o", val); + break; + case 16: + sprintf(tmp, "0x%x", val); + break; + case 10: + sprintf(tmp, "%u", val); + break; + default: + errx(1, "Unknown lookup base"); } return (tmp); } @@ -1116,12 +1116,12 @@ get_syscall(struct threadinfo *t, u_int number, u_int nargs) * Copy a fixed amount of bytes from the process. */ static int -get_struct(pid_t pid, uintptr_t offset, void *buf, int len) +get_struct(pid_t pid, psaddr_t offset, void *buf, size_t len) { struct ptrace_io_desc iorequest; iorequest.piod_op = PIOD_READ_D; - iorequest.piod_offs = (void *)offset; + iorequest.piod_offs = (void *)(uintptr_t)offset; iorequest.piod_addr = buf; iorequest.piod_len = len; if (ptrace(PT_IO, pid, (caddr_t)&iorequest, 0) < 0) @@ -1137,7 +1137,7 @@ get_struct(pid_t pid, uintptr_t offset, void *buf, int len) * only get that much. */ static char * -get_string(pid_t pid, uintptr_t addr, int max) +get_string(pid_t pid, psaddr_t addr, int max) { struct ptrace_io_desc iorequest; char *buf, *nbuf; @@ -1148,7 +1148,7 @@ get_string(pid_t pid, uintptr_t addr, int max) size = max + 1; else { /* Read up to the end of the current page. */ - size = PAGE_SIZE - ((uintptr_t)addr % PAGE_SIZE); + size = PAGE_SIZE - (addr % PAGE_SIZE); if (size > MAXSIZE) size = MAXSIZE; } @@ -1158,7 +1158,7 @@ get_string(pid_t pid, uintptr_t addr, int max) return (NULL); for (;;) { iorequest.piod_op = PIOD_READ_D; - iorequest.piod_offs = (void *)(addr + offset); + iorequest.piod_offs = (void *)((uintptr_t)addr + offset); iorequest.piod_addr = buf + offset; iorequest.piod_len = size; if (ptrace(PT_IO, pid, (caddr_t)&iorequest, 0) < 0) { @@ -1718,6 +1718,20 @@ print_sysctl(FILE *fp, int *oid, size_t len) fprintf(fp, "%s", name); } +/* + * Convert a 32-bit user-space pointer to psaddr_t. Currently, this + * sign-extends on MIPS and zero-extends on all other architectures. + */ +static psaddr_t +user_ptr32_to_psaddr(int32_t user_pointer) +{ +#if defined(__mips__) + return ((psaddr_t)(intptr_t)user_pointer); +#else + return ((psaddr_t)(uintptr_t)user_pointer); +#endif +} + /* * Converts a syscall argument into a string. Said string is * allocated via malloc(), so needs to be free()'d. sc is @@ -1769,7 +1783,7 @@ print_arg(struct syscall_arg *sc, unsigned long *args, register_t *retval, break; case ShmName: /* Handle special SHM_ANON value. */ - if ((char *)args[sc->offset] == SHM_ANON) { + if ((char *)(uintptr_t)args[sc->offset] == SHM_ANON) { fprintf(fp, "SHM_ANON"); break; } @@ -1829,7 +1843,7 @@ print_arg(struct syscall_arg *sc, unsigned long *args, register_t *retval, case ExecArgs: case ExecEnv: case StringArray: { - uintptr_t addr; + psaddr_t addr; union { int32_t strarray32[PAGE_SIZE / sizeof(int32_t)]; int64_t strarray64[PAGE_SIZE / sizeof(int64_t)]; @@ -1859,7 +1873,7 @@ print_arg(struct syscall_arg *sc, unsigned long *args, register_t *retval, * a partial page. */ addr = args[sc->offset]; - if (addr % pointer_size != 0) { + if (!__is_aligned(addr, pointer_size)) { print_pointer(fp, args[sc->offset]); break; } @@ -1875,20 +1889,19 @@ print_arg(struct syscall_arg *sc, unsigned long *args, register_t *retval, first = 1; i = 0; for (;;) { - uintptr_t straddr; + psaddr_t straddr; if (pointer_size == 4) { - if (u.strarray32[i] == 0) - break; - /* sign-extend 32-bit pointers */ - straddr = (intptr_t)u.strarray32[i]; + straddr = user_ptr32_to_psaddr(u.strarray32[i]); } else if (pointer_size == 8) { - if (u.strarray64[i] == 0) - break; - straddr = (intptr_t)u.strarray64[i]; + straddr = (psaddr_t)u.strarray64[i]; } else { errx(1, "Unsupported pointer size: %zu", pointer_size); } + + /* Stop once we read the first NULL pointer. */ + if (straddr == 0) + break; string = get_string(pid, straddr, 0); fprintf(fp, "%s \"%s\"", first ? "" : ",", string); free(string); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 09:47:11 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D4194658955; Mon, 5 Jul 2021 09:47:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJLSb4NFrz4kGN; Mon, 5 Jul 2021 09:47:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 62C0F23BEF; Mon, 5 Jul 2021 09:47:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1659lBW9071635; Mon, 5 Jul 2021 09:47:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1659lBaj071634; Mon, 5 Jul 2021 09:47:11 GMT (envelope-from git) Date: Mon, 5 Jul 2021 09:47:11 GMT Message-Id: <202107050947.1659lBaj071634@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alex Richardson Subject: git: 58137266b6ea - stable/13 - rtld-elf: Check the return value of obj_enforce_relro() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 58137266b6eab0a117878cb40f58bd693ea2396d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 09:47:12 -0000 The branch stable/13 has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=58137266b6eab0a117878cb40f58bd693ea2396d commit 58137266b6eab0a117878cb40f58bd693ea2396d Author: Alex Richardson AuthorDate: 2021-06-22 08:09:44 +0000 Commit: Alex Richardson CommitDate: 2021-07-05 09:46:12 +0000 rtld-elf: Check the return value of obj_enforce_relro() The mprotect() call was failing on CheriBSD when changing rtld's relro page permissions due to missing CHERI capability permissions on the mprotect() argument but did not report an error since the return value was being ignored. It should never fail on any supported FreeBSD architecture, but checking the return value seems like a good sanity check to me. Reviewed By: kib, imp MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D30820 (cherry picked from commit e3be51b2bc7cb41eb9a238cced2ee650d9bb2d41) --- libexec/rtld-elf/rtld.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/libexec/rtld-elf/rtld.c b/libexec/rtld-elf/rtld.c index 34c42969c779..42abd8391386 100644 --- a/libexec/rtld-elf/rtld.c +++ b/libexec/rtld-elf/rtld.c @@ -2280,7 +2280,8 @@ init_rtld(caddr_t mapbase, Elf_Auxinfo **aux_info) obj_rtld.path = xstrdup(ld_path_rtld); parse_rtld_phdr(&obj_rtld); - obj_enforce_relro(&obj_rtld); + if (obj_enforce_relro(&obj_rtld) == -1) + rtld_die(); r_debug.r_version = R_DEBUG_VERSION; r_debug.r_brk = r_debug_state; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:47 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE4FB65AD28; Mon, 5 Jul 2021 11:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqb4c5kz4sHH; Mon, 5 Jul 2021 11:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 866FC25167; Mon, 5 Jul 2021 11:33:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXljU017953; Mon, 5 Jul 2021 11:33:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXlNf017952; Mon, 5 Jul 2021 11:33:47 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:47 GMT Message-Id: <202107051133.165BXlNf017952@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 3ec1b75a0ddd - stable/13 - pf: add pf_release_staten and use it in pf_unlink_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3ec1b75a0ddd043abc0bf726127e84318de41699 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:47 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=3ec1b75a0ddd043abc0bf726127e84318de41699 commit 3ec1b75a0ddd043abc0bf726127e84318de41699 Author: Mateusz Guzik AuthorDate: 2021-06-28 12:18:43 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:10 +0000 pf: add pf_release_staten and use it in pf_unlink_state Saves one atomic op. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit d09388d013300fc4aa1ee1a6bc24260cd400aff3) --- sys/net/pfvar.h | 11 +++++++++++ sys/netpfil/pf/pf.c | 7 ++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 0d23607a168a..9e6dad5ca2ac 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -1507,6 +1507,17 @@ pf_release_state(struct pf_state *s) return (0); } +static __inline int +pf_release_staten(struct pf_state *s, u_int n) +{ + + if (refcount_releasen(&s->refs, n)) { + pf_free_state(s); + return (1); + } else + return (0); +} + extern struct pf_state *pf_find_state_byid(uint64_t, uint32_t); extern struct pf_state *pf_find_state_all(struct pf_state_key_cmp *, u_int, int *); diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 1b8181b7e1cf..8e0c51238f32 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1732,11 +1732,8 @@ pf_unlink_state(struct pf_state *s, u_int flags) PF_HASHROW_UNLOCK(ih); pf_detach_state(s); - /* pf_state_insert() initialises refs to 2, so we can never release the - * last reference here, only in pf_release_state(). */ - (void)refcount_release(&s->refs); - - return (pf_release_state(s)); + /* pf_state_insert() initialises refs to 2 */ + return (pf_release_staten(s, 2)); } void From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:49 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5AD1665AA99; Mon, 5 Jul 2021 11:33:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqc5lHNz4sVs; Mon, 5 Jul 2021 11:33:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ABD1F25258; Mon, 5 Jul 2021 11:33:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXmrx017983; Mon, 5 Jul 2021 11:33:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXmla017982; Mon, 5 Jul 2021 11:33:48 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:48 GMT Message-Id: <202107051133.165BXmla017982@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 0e3ee7e3a674 - stable/13 - pf: factor out state allocation into pf_alloc_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0e3ee7e3a6744e30f78ed2756f0060367f340179 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:49 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=0e3ee7e3a6744e30f78ed2756f0060367f340179 commit 0e3ee7e3a6744e30f78ed2756f0060367f340179 Author: Mateusz Guzik AuthorDate: 2021-06-28 12:22:31 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:10 +0000 pf: factor out state allocation into pf_alloc_state Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit ccb17a21041e9206b80fa9f64b6ec20233df6403) --- sys/netpfil/pf/pf.c | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 8e0c51238f32..53489f5f0113 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1736,6 +1736,28 @@ pf_unlink_state(struct pf_state *s, u_int flags) return (pf_release_staten(s, 2)); } +static struct pf_state * +pf_alloc_state(int flags) +{ + struct pf_state *s; + + s = uma_zalloc(V_pf_state_z, flags | M_ZERO); + if (__predict_false(s == NULL)) + return (NULL); + + for (int i = 0; i < 2; i++) { + s->bytes[i] = counter_u64_alloc(M_NOWAIT); + s->packets[i] = counter_u64_alloc(M_NOWAIT); + + if (s->bytes[i] == NULL || s->packets[i] == NULL) { + pf_free_state(s); + return (NULL); + } + } + + return (s); +} + void pf_free_state(struct pf_state *cur) { @@ -3730,21 +3752,11 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_SRCLIMIT); goto csfailed; } - s = uma_zalloc(V_pf_state_z, M_NOWAIT | M_ZERO); + s = pf_alloc_state(M_NOWAIT); if (s == NULL) { REASON_SET(&reason, PFRES_MEMORY); goto csfailed; } - for (int i = 0; i < 2; i++) { - s->bytes[i] = counter_u64_alloc(M_NOWAIT); - s->packets[i] = counter_u64_alloc(M_NOWAIT); - - if (s->bytes[i] == NULL || s->packets[i] == NULL) { - pf_free_state(s); - REASON_SET(&reason, PFRES_MEMORY); - goto csfailed; - } - } s->rule.ptr = r; s->nat_rule.ptr = nr; s->anchor.ptr = a; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:50 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 47E9165ABC1; Mon, 5 Jul 2021 11:33:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqd6lsTz4sQW; Mon, 5 Jul 2021 11:33:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CBCAD25168; Mon, 5 Jul 2021 11:33:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXnRF018008; Mon, 5 Jul 2021 11:33:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXnnK018007; Mon, 5 Jul 2021 11:33:49 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:49 GMT Message-Id: <202107051133.165BXnnK018007@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: f75b6fce91b1 - stable/13 - pf: assert that sizeof(struct pf_state) <= 312 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f75b6fce91b12df3494a13075a809a1386bfd9db Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:50 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=f75b6fce91b12df3494a13075a809a1386bfd9db commit f75b6fce91b12df3494a13075a809a1386bfd9db Author: Mateusz Guzik AuthorDate: 2021-06-28 12:49:40 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:11 +0000 pf: assert that sizeof(struct pf_state) <= 312 To prevent accidentally going over a threshold which makes UMA fit only 12 objects per page instead of 13. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit e6dd0e2e8d4f59275fb576a9ccb9d356d36457bb) --- sys/net/pfvar.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 9e6dad5ca2ac..e0eb4e55a46f 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -548,6 +548,11 @@ struct pf_state { u_int8_t sync_updates; u_int8_t _tail[3]; }; + +/* + * Size <= fits 13 objects per page on LP64. Try to not grow the struct beyond that. + */ +_Static_assert(sizeof(struct pf_state) <= 312, "pf_state size crosses 312 bytes"); #endif /* From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:51 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3749965AA9B; Mon, 5 Jul 2021 11:33:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqg0mVkz4sb0; Mon, 5 Jul 2021 11:33:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F2453253AA; Mon, 5 Jul 2021 11:33:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXob8018032; Mon, 5 Jul 2021 11:33:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXoBc018031; Mon, 5 Jul 2021 11:33:50 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:50 GMT Message-Id: <202107051133.165BXoBc018031@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 3fbf56f319f5 - stable/13 - pf: fix error-case leaks in pf_create_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3fbf56f319f5cf01a6ed0a904499e996d7660f5d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:51 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=3fbf56f319f5cf01a6ed0a904499e996d7660f5d commit 3fbf56f319f5cf01a6ed0a904499e996d7660f5d Author: Mateusz Guzik AuthorDate: 2021-06-28 18:29:58 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:11 +0000 pf: fix error-case leaks in pf_create_state The hand-rolled clean up failed to free counters. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 7f025db57c7879bfa43ac74ecd95903439fdd1d0) --- sys/netpfil/pf/pf.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 53489f5f0113..3afb54dbab17 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -3827,7 +3827,7 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_MAPFAILED); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); goto csfailed; } s->rt_kif = r->rpool.cur->kif; @@ -3849,7 +3849,7 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_MEMORY); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } if ((pd->flags & PFDESC_TCP_NORM) && s->src.scrub && @@ -3859,10 +3859,9 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, DPFPRINTF(PF_DEBUG_URGENT, ("pf_normalize_tcp_stateful failed on first " "pkt\n")); - pf_normalize_tcp_cleanup(s); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } } @@ -3886,12 +3885,10 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, if (pf_state_insert(BOUND_IFACE(r, kif), kif, (pd->dir == PF_IN) ? sk : nk, (pd->dir == PF_IN) ? nk : sk, s)) { - if (pd->proto == IPPROTO_TCP) - pf_normalize_tcp_cleanup(s); REASON_SET(&reason, PFRES_STATEINS); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } else *sm = s; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:53 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DC0D065A9A6; Mon, 5 Jul 2021 11:33:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqh3m7Rz4sYP; Mon, 5 Jul 2021 11:33:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2135A2525A; Mon, 5 Jul 2021 11:33:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXqe9018056; Mon, 5 Jul 2021 11:33:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXq7X018055; Mon, 5 Jul 2021 11:33:52 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:52 GMT Message-Id: <202107051133.165BXq7X018055@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 0e69786dae2b - stable/13 - pf: deduplicate V_pf_state_z handling with pfsync MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0e69786dae2bce1496cb3017f679e1e3385a3d0a Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:53 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=0e69786dae2bce1496cb3017f679e1e3385a3d0a commit 0e69786dae2bce1496cb3017f679e1e3385a3d0a Author: Mateusz Guzik AuthorDate: 2021-06-28 18:18:26 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:12 +0000 pf: deduplicate V_pf_state_z handling with pfsync Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 803dfe3da0e30520227a791ba5c2b93da16e1cf5) --- sys/net/pfvar.h | 1 + sys/netpfil/pf/if_pfsync.c | 20 +++----------------- sys/netpfil/pf/pf.c | 2 +- 3 files changed, 5 insertions(+), 18 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index e0eb4e55a46f..12ac791e3c72 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -1492,6 +1492,7 @@ extern int pf_state_insert(struct pfi_kkif *, struct pf_state_key *, struct pf_state_key *, struct pf_state *); +extern struct pf_state *pf_alloc_state(int); extern void pf_free_state(struct pf_state *); static __inline void diff --git a/sys/netpfil/pf/if_pfsync.c b/sys/netpfil/pf/if_pfsync.c index 3514c922c361..16878fe87750 100644 --- a/sys/netpfil/pf/if_pfsync.c +++ b/sys/netpfil/pf/if_pfsync.c @@ -504,16 +504,10 @@ pfsync_state_import(struct pfsync_state *sp, u_int8_t flags) /* * XXXGL: consider M_WAITOK in ioctl path after. */ - if ((st = uma_zalloc(V_pf_state_z, M_NOWAIT | M_ZERO)) == NULL) + st = pf_alloc_state(M_NOWAIT); + if (__predict_false(st == NULL)) goto cleanup; - for (int i = 0; i < 2; i++) { - st->packets[i] = counter_u64_alloc(M_NOWAIT); - st->bytes[i] = counter_u64_alloc(M_NOWAIT); - if (st->packets[i] == NULL || st->bytes[i] == NULL) - goto cleanup; - } - if ((skw = uma_zalloc(V_pf_state_key_z, M_NOWAIT)) == NULL) goto cleanup; @@ -623,15 +617,7 @@ cleanup: cleanup_state: /* pf_state_insert() frees the state keys. */ if (st) { - for (int i = 0; i < 2; i++) { - counter_u64_free(st->packets[i]); - counter_u64_free(st->bytes[i]); - } - if (st->dst.scrub) - uma_zfree(V_pf_state_scrub_z, st->dst.scrub); - if (st->src.scrub) - uma_zfree(V_pf_state_scrub_z, st->src.scrub); - uma_zfree(V_pf_state_z, st); + pf_free_state(st); } return (error); } diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 3afb54dbab17..92e93b62e85f 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1736,7 +1736,7 @@ pf_unlink_state(struct pf_state *s, u_int flags) return (pf_release_staten(s, 2)); } -static struct pf_state * +struct pf_state * pf_alloc_state(int flags) { struct pf_state *s; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:54 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 67E3665AD95; Mon, 5 Jul 2021 11:33:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqj5yBRz4sb9; Mon, 5 Jul 2021 11:33:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3EABE2542C; Mon, 5 Jul 2021 11:33:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXrdS018080; Mon, 5 Jul 2021 11:33:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXrt1018079; Mon, 5 Jul 2021 11:33:53 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:53 GMT Message-Id: <202107051133.165BXrt1018079@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: dfcf1bd2e10e - stable/13 - pf: make sure the dtrace probe has safe access to state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: dfcf1bd2e10e6d2fe32977862bc13f4ff74e3d76 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:54 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=dfcf1bd2e10e6d2fe32977862bc13f4ff74e3d76 commit dfcf1bd2e10e6d2fe32977862bc13f4ff74e3d76 Author: Mateusz Guzik AuthorDate: 2021-06-28 19:32:20 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:13 +0000 pf: make sure the dtrace probe has safe access to state Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit d26ef5c7ac830812f07a02787f25fed5d6f8609e) --- sys/netpfil/pf/pf.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 92e93b62e85f..3846e5335a2a 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -6318,11 +6318,12 @@ done: } break; } - if (s) - PF_STATE_UNLOCK(s); SDT_PROBE4(pf, ip, test, done, action, reason, r, s); + if (s) + PF_STATE_UNLOCK(s); + return (action); } #endif /* INET */ From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1B08265A95E; Mon, 5 Jul 2021 11:33:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqk5lPGz4sDt; Mon, 5 Jul 2021 11:33:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5EF7B2542D; Mon, 5 Jul 2021 11:33:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXsxf018104; Mon, 5 Jul 2021 11:33:54 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXsMM018103; Mon, 5 Jul 2021 11:33:54 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:54 GMT Message-Id: <202107051133.165BXsMM018103@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: aa9d233aadd8 - stable/13 - pf: revert: Use counter(9) for pf_state byte/packet tracking MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: aa9d233aadd840e4d671d352faf5bfc82ef8fbe7 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:55 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=aa9d233aadd840e4d671d352faf5bfc82ef8fbe7 commit aa9d233aadd840e4d671d352faf5bfc82ef8fbe7 Author: Mateusz Guzik AuthorDate: 2021-06-28 18:50:56 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:13 +0000 pf: revert: Use counter(9) for pf_state byte/packet tracking stats are not shared and consequently per-CPU counters only waste memory. No slowdown was measured when passing over 20M pps. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 55cc305dfcad0ad7c4f528fa47f7473927e8223a) --- sys/net/pfvar.h | 4 ++-- sys/netpfil/pf/pf.c | 39 +++++++++------------------------------ sys/netpfil/pf/pf_ioctl.c | 10 ++++------ sys/netpfil/pf/pf_nv.c | 4 ++-- 4 files changed, 17 insertions(+), 40 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 12ac791e3c72..b63f22e84bc0 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -533,8 +533,8 @@ struct pf_state { struct pfi_kkif *rt_kif; struct pf_ksrc_node *src_node; struct pf_ksrc_node *nat_src_node; - counter_u64_t packets[2]; - counter_u64_t bytes[2]; + u_int64_t packets[2]; + u_int64_t bytes[2]; u_int32_t creation; u_int32_t expire; u_int32_t pfsync_time; diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 3846e5335a2a..6ac19c9ad5d3 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1739,23 +1739,8 @@ pf_unlink_state(struct pf_state *s, u_int flags) struct pf_state * pf_alloc_state(int flags) { - struct pf_state *s; - - s = uma_zalloc(V_pf_state_z, flags | M_ZERO); - if (__predict_false(s == NULL)) - return (NULL); - - for (int i = 0; i < 2; i++) { - s->bytes[i] = counter_u64_alloc(M_NOWAIT); - s->packets[i] = counter_u64_alloc(M_NOWAIT); - - if (s->bytes[i] == NULL || s->packets[i] == NULL) { - pf_free_state(s); - return (NULL); - } - } - return (s); + return (uma_zalloc(V_pf_state_z, flags | M_ZERO)); } void @@ -1766,11 +1751,6 @@ pf_free_state(struct pf_state *cur) KASSERT(cur->timeout == PFTM_UNLINKED, ("%s: timeout %u", __func__, cur->timeout)); - for (int i = 0; i < 2; i++) { - counter_u64_free(cur->bytes[i]); - counter_u64_free(cur->packets[i]); - } - pf_normalize_tcp_cleanup(cur); uma_zfree(V_pf_state_z, cur); counter_u64_add(V_pf_status.fcounters[FCNT_STATE_REMOVALS], 1); @@ -4313,9 +4293,8 @@ pf_tcp_track_full(struct pf_state_peer *src, struct pf_state_peer *dst, pf_print_flags(th->th_flags); printf(" seq=%u (%u) ack=%u len=%u ackskew=%d " "pkts=%llu:%llu dir=%s,%s\n", seq, orig_seq, ack, - pd->p_len, ackskew, - (unsigned long long)counter_u64_fetch((*state)->packets[0]), - (unsigned long long)counter_u64_fetch((*state)->packets[1]), + pd->p_len, ackskew, (unsigned long long)(*state)->packets[0], + (unsigned long long)(*state)->packets[1], pd->dir == PF_IN ? "in" : "out", pd->dir == (*state)->direction ? "fwd" : "rev"); } @@ -4370,8 +4349,8 @@ pf_tcp_track_full(struct pf_state_peer *src, struct pf_state_peer *dst, printf(" seq=%u (%u) ack=%u len=%u ackskew=%d " "pkts=%llu:%llu dir=%s,%s\n", seq, orig_seq, ack, pd->p_len, ackskew, - (unsigned long long)counter_u64_fetch((*state)->packets[0]), - (unsigned long long)counter_u64_fetch((*state)->packets[1]), + (unsigned long long)(*state)->packets[0], + (unsigned long long)(*state)->packets[1], pd->dir == PF_IN ? "in" : "out", pd->dir == (*state)->direction ? "fwd" : "rev"); printf("pf: State failure on: %c %c %c %c | %c %c\n", @@ -6276,8 +6255,8 @@ done: pd.tot_len); } dirndx = (dir == s->direction) ? 0 : 1; - counter_u64_add(s->packets[dirndx], 1); - counter_u64_add(s->bytes[dirndx], pd.tot_len); + s->packets[dirndx]++; + s->bytes[dirndx] += pd.tot_len; } tr = r; nr = (s != NULL) ? s->nat_rule.ptr : pd.nat_rule; @@ -6674,8 +6653,8 @@ done: pd.tot_len); } dirndx = (dir == s->direction) ? 0 : 1; - counter_u64_add(s->packets[dirndx], 1); - counter_u64_add(s->bytes[dirndx], pd.tot_len); + s->packets[dirndx]++; + s->bytes[dirndx] += pd.tot_len; } tr = r; nr = (s != NULL) ? s->nat_rule.ptr : pd.nat_rule; diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 657c0e1945d3..49d5b2d216d2 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -4585,12 +4585,10 @@ pfsync_state_export(struct pfsync_state *sp, struct pf_state *st) else sp->nat_rule = htonl(st->nat_rule.ptr->nr); - pf_state_counter_hton(counter_u64_fetch(st->packets[0]), - sp->packets[0]); - pf_state_counter_hton(counter_u64_fetch(st->packets[1]), - sp->packets[1]); - pf_state_counter_hton(counter_u64_fetch(st->bytes[0]), sp->bytes[0]); - pf_state_counter_hton(counter_u64_fetch(st->bytes[1]), sp->bytes[1]); + pf_state_counter_hton(st->packets[0], sp->packets[0]); + pf_state_counter_hton(st->packets[1], sp->packets[1]); + pf_state_counter_hton(st->bytes[0], sp->bytes[0]); + pf_state_counter_hton(st->bytes[1], sp->bytes[1]); } diff --git a/sys/netpfil/pf/pf_nv.c b/sys/netpfil/pf/pf_nv.c index 31943ba69687..553290c88586 100644 --- a/sys/netpfil/pf/pf_nv.c +++ b/sys/netpfil/pf/pf_nv.c @@ -982,9 +982,9 @@ pf_state_to_nvstate(const struct pf_state *s) for (int i = 0; i < 2; i++) { nvlist_append_number_array(nvl, "packets", - counter_u64_fetch(s->packets[i])); + s->packets[i]); nvlist_append_number_array(nvl, "bytes", - counter_u64_fetch(s->bytes[i])); + s->bytes[i]); } nvlist_add_number(nvl, "creatorid", s->creatorid); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D195065A9BD; Mon, 5 Jul 2021 11:33:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNql4sHbz4sWW; Mon, 5 Jul 2021 11:33:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8B0AF2525E; Mon, 5 Jul 2021 11:33:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXt5J018134; Mon, 5 Jul 2021 11:33:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXtAo018133; Mon, 5 Jul 2021 11:33:55 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:55 GMT Message-Id: <202107051133.165BXtAo018133@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 7b78b57f4b6c - stable/13 - mac: cheaper check for ifnet_create_mbuf and ifnet_check_transmit MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 7b78b57f4b6c9fabea6dd20765bc99dba7492f60 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:56 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=7b78b57f4b6c9fabea6dd20765bc99dba7492f60 commit 7b78b57f4b6c9fabea6dd20765bc99dba7492f60 Author: Mateusz Guzik AuthorDate: 2021-06-29 12:56:19 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:14 +0000 mac: cheaper check for ifnet_create_mbuf and ifnet_check_transmit Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit f77697dd9f31df85cd86370888606c81833f7c8a) --- sys/security/mac/mac_framework.c | 6 ++++++ sys/security/mac/mac_framework.h | 34 ++++++++++++++++++++++++++++++++-- sys/security/mac/mac_net.c | 10 ++-------- 3 files changed, 40 insertions(+), 10 deletions(-) diff --git a/sys/security/mac/mac_framework.c b/sys/security/mac/mac_framework.c index f0b4f89db7ca..e773a3840464 100644 --- a/sys/security/mac/mac_framework.c +++ b/sys/security/mac/mac_framework.c @@ -145,6 +145,8 @@ FPFLAG_RARE(vnode_check_access); FPFLAG_RARE(vnode_check_readlink); FPFLAG_RARE(pipe_check_stat); FPFLAG_RARE(pipe_check_poll); +FPFLAG_RARE(ifnet_create_mbuf); +FPFLAG_RARE(ifnet_check_transmit); #undef FPFLAG #undef FPFLAG_RARE @@ -445,6 +447,10 @@ struct mac_policy_fastpath_elem mac_policy_fastpath_array[] = { .flag = &mac_pipe_check_stat_fp_flag }, { .offset = FPO(pipe_check_poll), .flag = &mac_pipe_check_poll_fp_flag }, + { .offset = FPO(ifnet_create_mbuf), + .flag = &mac_ifnet_create_mbuf_fp_flag }, + { .offset = FPO(ifnet_check_transmit), + .flag = &mac_ifnet_check_transmit_fp_flag }, }; static void diff --git a/sys/security/mac/mac_framework.h b/sys/security/mac/mac_framework.h index 481f90a04801..7a46fbedb28d 100644 --- a/sys/security/mac/mac_framework.h +++ b/sys/security/mac/mac_framework.h @@ -143,9 +143,39 @@ void mac_devfs_update(struct mount *mp, struct devfs_dirent *de, void mac_devfs_vnode_associate(struct mount *mp, struct devfs_dirent *de, struct vnode *vp); -int mac_ifnet_check_transmit(struct ifnet *ifp, struct mbuf *m); +int mac_ifnet_check_transmit_impl(struct ifnet *ifp, struct mbuf *m); +#ifdef MAC +extern bool mac_ifnet_check_transmit_fp_flag; +#else +#define mac_ifnet_check_transmit_fp_flag 0 +#endif +#define mac_ifnet_check_transmit_enabled() __predict_false(mac_ifnet_check_transmit_fp_flag) +static inline int +mac_ifnet_check_transmit(struct ifnet *ifp, struct mbuf *m) +{ + + if (mac_ifnet_check_transmit_enabled()) + return (mac_ifnet_check_transmit_impl(ifp, m)); + return (0); +} + void mac_ifnet_create(struct ifnet *ifp); -void mac_ifnet_create_mbuf(struct ifnet *ifp, struct mbuf *m); + +void mac_ifnet_create_mbuf_impl(struct ifnet *ifp, struct mbuf *m); +#ifdef MAC +extern bool mac_ifnet_create_mbuf_fp_flag; +#else +#define mac_ifnet_create_mbuf_fp_flag 0 +#endif +#define mac_ifnet_create_mbuf_enabled() __predict_false(mac_ifnet_create_mbuf_fp_flag) +static inline void +mac_ifnet_create_mbuf(struct ifnet *ifp, struct mbuf *m) +{ + + if (mac_ifnet_create_mbuf_enabled()) + mac_ifnet_create_mbuf_impl(ifp, m); +} + void mac_ifnet_destroy(struct ifnet *); void mac_ifnet_init(struct ifnet *); int mac_ifnet_ioctl_get(struct ucred *cred, struct ifreq *ifr, diff --git a/sys/security/mac/mac_net.c b/sys/security/mac/mac_net.c index 161040edf84f..372619c7b583 100644 --- a/sys/security/mac/mac_net.c +++ b/sys/security/mac/mac_net.c @@ -337,14 +337,11 @@ mac_bpfdesc_create_mbuf(struct bpf_d *d, struct mbuf *m) } void -mac_ifnet_create_mbuf(struct ifnet *ifp, struct mbuf *m) +mac_ifnet_create_mbuf_impl(struct ifnet *ifp, struct mbuf *m) { struct label *label; int locked; - if (mac_policy_count == 0) - return; - label = mac_mbuf_to_label(m); MAC_IFNET_LOCK(ifp, locked); @@ -380,16 +377,13 @@ MAC_CHECK_PROBE_DEFINE2(ifnet_check_transmit, "struct ifnet *", "struct mbuf *"); int -mac_ifnet_check_transmit(struct ifnet *ifp, struct mbuf *m) +mac_ifnet_check_transmit_impl(struct ifnet *ifp, struct mbuf *m) { struct label *label; int error, locked; M_ASSERTPKTHDR(m); - if (mac_policy_count == 0) - return (0); - label = mac_mbuf_to_label(m); MAC_IFNET_LOCK(ifp, locked); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:57 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5274E65A9C5; Mon, 5 Jul 2021 11:33:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqm6HnSz4sYg; Mon, 5 Jul 2021 11:33:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A8B7824FCD; Mon, 5 Jul 2021 11:33:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXui5018158; Mon, 5 Jul 2021 11:33:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXut8018157; Mon, 5 Jul 2021 11:33:56 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:56 GMT Message-Id: <202107051133.165BXut8018157@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: d5fe617b09b2 - stable/13 - pfctl: cache getprotobynumber results MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d5fe617b09b2988676dcc0e323b6a88fd821fd41 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:57 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=d5fe617b09b2988676dcc0e323b6a88fd821fd41 commit d5fe617b09b2988676dcc0e323b6a88fd821fd41 Author: Mateusz Guzik AuthorDate: 2021-07-01 19:25:43 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:14 +0000 pfctl: cache getprotobynumber results As for example pfctl -ss keeps calling it, it saves a lot of overhead from elided parsing of /etc/nsswitch.conf and /etc/protocols. Sample result when running a pre-nvlist binary with nfs root and dumping 7 mln states: before: 24.817u 62.993s 1:28.52 99.1% after: 8.064u 1.117s 0:18.87 48.5% Idea by Jim Thompson Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 858937bea4599d254a97ee6321683f8629604e15) --- sbin/pfctl/parse.y | 8 ++++---- sbin/pfctl/pf_print_state.c | 6 +++--- sbin/pfctl/pfctl.c | 43 +++++++++++++++++++++++++++++++++++++++++++ sbin/pfctl/pfctl.h | 2 ++ sbin/pfctl/pfctl_parser.c | 6 +++--- 5 files changed, 55 insertions(+), 10 deletions(-) diff --git a/sbin/pfctl/parse.y b/sbin/pfctl/parse.y index 4448a8255ce1..acd90e280b53 100644 --- a/sbin/pfctl/parse.y +++ b/sbin/pfctl/parse.y @@ -5017,13 +5017,13 @@ expand_label_port(const char *name, char *label, size_t len, void expand_label_proto(const char *name, char *label, size_t len, u_int8_t proto) { - struct protoent *pe; + const char *protoname; char n[4]; if (strstr(label, name) != NULL) { - pe = getprotobynumber(proto); - if (pe != NULL) - expand_label_str(label, len, name, pe->p_name); + protoname = pfctl_proto2name(proto); + if (protoname != NULL) + expand_label_str(label, len, name, protoname); else { snprintf(n, sizeof(n), "%u", proto); expand_label_str(label, len, name, n); diff --git a/sbin/pfctl/pf_print_state.c b/sbin/pfctl/pf_print_state.c index b1f0079154cf..b66a296d6080 100644 --- a/sbin/pfctl/pf_print_state.c +++ b/sbin/pfctl/pf_print_state.c @@ -211,7 +211,7 @@ print_state(struct pfctl_state *s, int opts) { struct pfctl_state_peer *src, *dst; struct pfctl_state_key *key, *sk, *nk; - struct protoent *p; + const char *protoname; int min, sec; sa_family_t af; uint8_t proto; @@ -243,8 +243,8 @@ print_state(struct pfctl_state *s, int opts) sk->port[1] = nk->port[1]; } printf("%s ", s->ifname); - if ((p = getprotobynumber(proto)) != NULL) - printf("%s ", p->p_name); + if ((protoname = pfctl_proto2name(proto)) != NULL) + printf("%s ", protoname); else printf("%u ", proto); diff --git a/sbin/pfctl/pfctl.c b/sbin/pfctl/pfctl.c index f82d75198d61..14b7f3a01657 100644 --- a/sbin/pfctl/pfctl.c +++ b/sbin/pfctl/pfctl.c @@ -254,6 +254,49 @@ usage(void) exit(1); } +/* + * Cache protocol number to name translations. + * + * Translation is performed a lot e.g., when dumping states and + * getprotobynumber is incredibly expensive. + * + * Note from the getprotobynumber(3) manpage: + * + * These functions use a thread-specific data space; if the data is needed + * for future use, it should be copied before any subsequent calls overwrite + * it. Only the Internet protocols are currently understood. + * + * + * Consequently we only cache the name and strdup it for safety. + * + * At the time of writing this comment the last entry in /etc/protocols is: + * divert 258 DIVERT # Divert pseudo-protocol [non IANA] + */ +const char * +pfctl_proto2name(int proto) +{ + static const char *pfctl_proto_cache[259]; + struct protoent *p; + + if (proto >= nitems(pfctl_proto_cache)) { + p = getprotobynumber(proto); + if (p == NULL) { + return (NULL); + } + return (p->p_name); + } + + if (pfctl_proto_cache[proto] == NULL) { + p = getprotobynumber(proto); + if (p == NULL) { + return (NULL); + } + pfctl_proto_cache[proto] = strdup(p->p_name); + } + + return (pfctl_proto_cache[proto]); +} + int pfctl_enable(int dev, int opts) { diff --git a/sbin/pfctl/pfctl.h b/sbin/pfctl/pfctl.h index f8ff5012e01b..80ef184fa90f 100644 --- a/sbin/pfctl/pfctl.h +++ b/sbin/pfctl/pfctl.h @@ -138,4 +138,6 @@ void pf_remove_if_empty_ruleset(struct pfctl_ruleset *); struct pfctl_ruleset *pf_find_ruleset(const char *); struct pfctl_ruleset *pf_find_or_create_ruleset(const char *); +const char *pfctl_proto2name(int); + #endif /* _PFCTL_H_ */ diff --git a/sbin/pfctl/pfctl_parser.c b/sbin/pfctl/pfctl_parser.c index ce460ab691ca..b4a1cde967bd 100644 --- a/sbin/pfctl/pfctl_parser.c +++ b/sbin/pfctl/pfctl_parser.c @@ -812,10 +812,10 @@ print_rule(struct pfctl_rule *r, const char *anchor_call, int verbose, int numer printf(" inet6"); } if (r->proto) { - struct protoent *p; + const char *protoname; - if ((p = getprotobynumber(r->proto)) != NULL) - printf(" proto %s", p->p_name); + if ((protoname = pfctl_proto2name(r->proto)) != NULL) + printf(" proto %s", protoname); else printf(" proto %u", r->proto); } From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 11:33:58 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5A1D465ABE3; Mon, 5 Jul 2021 11:33:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJNqp0PGnz4sYn; Mon, 5 Jul 2021 11:33:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D53782525F; Mon, 5 Jul 2021 11:33:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165BXvRJ018182; Mon, 5 Jul 2021 11:33:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165BXvYt018181; Mon, 5 Jul 2021 11:33:57 GMT (envelope-from git) Date: Mon, 5 Jul 2021 11:33:57 GMT Message-Id: <202107051133.165BXvYt018181@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: b501c72ad305 - stable/13 - pf: make DIOCGETSTATESNV iterations killable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: b501c72ad3054b6043c23fdcb7f95d7f26abe39c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 11:33:58 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=b501c72ad3054b6043c23fdcb7f95d7f26abe39c commit b501c72ad3054b6043c23fdcb7f95d7f26abe39c Author: Mateusz Guzik AuthorDate: 2021-07-02 07:28:48 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:32:15 +0000 pf: make DIOCGETSTATESNV iterations killable Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 48d5b86364fa8e86318805d0078ccf9066e94ae8) --- sys/netpfil/pf/pf_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 49d5b2d216d2..0097d8b0f6b6 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -5060,6 +5060,11 @@ pf_getstates(struct pfioc_nv *nv) if (s->timeout == PFTM_UNLINKED) continue; + if (SIGPENDING(curthread)) { + PF_HASHROW_UNLOCK(ih); + ERROUT(EINTR); + } + nvls = pf_state_to_nvstate(s); if (nvls == NULL) { PF_HASHROW_UNLOCK(ih); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:01:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 51C0065B9F0; Mon, 5 Jul 2021 12:01:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS31nc8z3Cwb; Mon, 5 Jul 2021 12:01:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 25EBA25B3B; Mon, 5 Jul 2021 12:01:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C1t0l054571; Mon, 5 Jul 2021 12:01:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C1t6V054570; Mon, 5 Jul 2021 12:01:55 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:01:55 GMT Message-Id: <202107051201.165C1t6V054570@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 5372a43bf746 - stable/12 - pf: Add endline to all DPFPRINTF() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 5372a43bf746fec62e8b9312ce96dd703891d187 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:01:55 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=5372a43bf746fec62e8b9312ce96dd703891d187 commit 5372a43bf746fec62e8b9312ce96dd703891d187 Author: Kristof Provost AuthorDate: 2019-11-24 13:53:36 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:37:43 +0000 pf: Add endline to all DPFPRINTF() DPFPRINTF() doesn't automatically add an endline, so be consistent and always add it. (cherry picked from commit 492f3a312a2617899809285cdb6a32baf6836b13) --- sys/netpfil/pf/pf.c | 3 ++- sys/netpfil/pf/pf_norm.c | 29 +++++++++++++++-------------- 2 files changed, 17 insertions(+), 15 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 57bbd355e2fb..544c262a96f0 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -3849,7 +3849,8 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, &s->src, &s->dst, rewrite)) { /* This really shouldn't happen!!! */ DPFPRINTF(PF_DEBUG_URGENT, - ("pf_normalize_tcp_stateful failed on first pkt")); + ("pf_normalize_tcp_stateful failed on first " + "pkt\n")); pf_normalize_tcp_cleanup(s); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); diff --git a/sys/netpfil/pf/pf_norm.c b/sys/netpfil/pf/pf_norm.c index b6e7fd5f1e1c..222e8231ac60 100644 --- a/sys/netpfil/pf/pf_norm.c +++ b/sys/netpfil/pf/pf_norm.c @@ -551,25 +551,25 @@ pf_fillup_fragment(struct pf_fragment_cmp *key, struct pf_frent *frent, /* No empty fragments. */ if (frent->fe_len == 0) { - DPFPRINTF(("bad fragment: len 0")); + DPFPRINTF(("bad fragment: len 0\n")); goto bad_fragment; } /* All fragments are 8 byte aligned. */ if (frent->fe_mff && (frent->fe_len & 0x7)) { - DPFPRINTF(("bad fragment: mff and len %d", frent->fe_len)); + DPFPRINTF(("bad fragment: mff and len %d\n", frent->fe_len)); goto bad_fragment; } /* Respect maximum length, IP_MAXPACKET == IPV6_MAXPACKET. */ if (frent->fe_off + frent->fe_len > IP_MAXPACKET) { - DPFPRINTF(("bad fragment: max packet %d", + DPFPRINTF(("bad fragment: max packet %d\n", frent->fe_off + frent->fe_len)); goto bad_fragment; } DPFPRINTF((key->frc_af == AF_INET ? - "reass frag %d @ %d-%d" : "reass frag %#08x @ %d-%d", + "reass frag %d @ %d-%d\n" : "reass frag %#08x @ %d-%d\n", key->frc_id, frent->fe_off, frent->fe_off + frent->fe_len)); /* Fully buffer all of the fragments in this fragment queue. */ @@ -643,7 +643,7 @@ pf_fillup_fragment(struct pf_fragment_cmp *key, struct pf_frent *frent, precut = prev->fe_off + prev->fe_len - frent->fe_off; if (precut >= frent->fe_len) goto bad_fragment; - DPFPRINTF(("overlap -%d", precut)); + DPFPRINTF(("overlap -%d\n", precut)); m_adj(frent->fe_m, precut); frent->fe_off += precut; frent->fe_len -= precut; @@ -654,7 +654,7 @@ pf_fillup_fragment(struct pf_fragment_cmp *key, struct pf_frent *frent, uint16_t aftercut; aftercut = frent->fe_off + frent->fe_len - after->fe_off; - DPFPRINTF(("adjust overlap %d", aftercut)); + DPFPRINTF(("adjust overlap %d\n", aftercut)); if (aftercut < after->fe_len) { m_adj(after->fe_m, aftercut); old_index = pf_frent_index(after); @@ -693,7 +693,7 @@ pf_fillup_fragment(struct pf_fragment_cmp *key, struct pf_frent *frent, /* If part of the queue gets too long, there is not way to recover. */ if (pf_frent_insert(frag, frent, prev)) { - DPFPRINTF(("fragment queue limit exceeded")); + DPFPRINTF(("fragment queue limit exceeded\n")); goto bad_fragment; } @@ -767,7 +767,7 @@ pf_reassemble(struct mbuf **m0, struct ip *ip, int dir, u_short *reason) m = *m0 = NULL; if (frag->fr_holes) { - DPFPRINTF(("frag %d, holes %d", frag->fr_id, frag->fr_holes)); + DPFPRINTF(("frag %d, holes %d\n", frag->fr_id, frag->fr_holes)); return (PF_PASS); /* drop because *m0 is NULL, no error */ } @@ -798,7 +798,7 @@ pf_reassemble(struct mbuf **m0, struct ip *ip, int dir, u_short *reason) ip->ip_off &= ~(IP_MF|IP_OFFMASK); if (hdrlen + total > IP_MAXPACKET) { - DPFPRINTF(("drop: too big: %d", total)); + DPFPRINTF(("drop: too big: %d\n", total)); ip->ip_len = 0; REASON_SET(reason, PFRES_SHORT); /* PF_DROP requires a valid mbuf *m0 in pf_test() */ @@ -857,7 +857,8 @@ pf_reassemble6(struct mbuf **m0, struct ip6_hdr *ip6, struct ip6_frag *fraghdr, m = *m0 = NULL; if (frag->fr_holes) { - DPFPRINTF(("frag %d, holes %d", frag->fr_id, frag->fr_holes)); + DPFPRINTF(("frag %d, holes %d\n", frag->fr_id, + frag->fr_holes)); PF_FRAG_UNLOCK(); return (PF_PASS); /* Drop because *m0 is NULL, no error. */ } @@ -918,14 +919,14 @@ pf_reassemble6(struct mbuf **m0, struct ip6_hdr *ip6, struct ip6_frag *fraghdr, ip6->ip6_nxt = proto; if (hdrlen - sizeof(struct ip6_hdr) + total > IPV6_MAXPACKET) { - DPFPRINTF(("drop: too big: %d", total)); + DPFPRINTF(("drop: too big: %d\n", total)); ip6->ip6_plen = 0; REASON_SET(reason, PFRES_SHORT); /* PF_DROP requires a valid mbuf *m0 in pf_test6(). */ return (PF_DROP); } - DPFPRINTF(("complete: %p(%d)", m, ntohs(ip6->ip6_plen))); + DPFPRINTF(("complete: %p(%d)\n", m, ntohs(ip6->ip6_plen))); return (PF_PASS); fail: @@ -994,7 +995,7 @@ pf_refragment6(struct ifnet *ifp, struct mbuf **m0, struct m_tag *mtag) action = PF_PASS; } else { /* Drop expects an mbuf to free. */ - DPFPRINTF(("refragment error %d", error)); + DPFPRINTF(("refragment error %d\n", error)); action = PF_DROP; } for (t = m; m; m = t) { @@ -1626,7 +1627,7 @@ pf_normalize_tcp_stateful(struct mbuf *m, int off, struct pf_pdesc *pd, if (got_ts) { /* Huh? Multiple timestamps!? */ if (V_pf_status.debug >= PF_DEBUG_MISC) { - DPFPRINTF(("multiple TS??")); + DPFPRINTF(("multiple TS??\n")); pf_print_state(state); printf("\n"); } From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:01:56 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6EF9865BA7C; Mon, 5 Jul 2021 12:01:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS42mhYz3CkJ; Mon, 5 Jul 2021 12:01:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 47C9025B3C; Mon, 5 Jul 2021 12:01:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C1u5v054595; Mon, 5 Jul 2021 12:01:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C1ulI054594; Mon, 5 Jul 2021 12:01:56 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:01:56 GMT Message-Id: <202107051201.165C1ulI054594@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: f4e50b36e208 - stable/12 - pf: add pf_release_staten and use it in pf_unlink_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: f4e50b36e208807371c568382399e60e2f6ceee8 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:01:56 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=f4e50b36e208807371c568382399e60e2f6ceee8 commit f4e50b36e208807371c568382399e60e2f6ceee8 Author: Mateusz Guzik AuthorDate: 2021-06-28 12:18:43 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:37:50 +0000 pf: add pf_release_staten and use it in pf_unlink_state Saves one atomic op. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit d09388d013300fc4aa1ee1a6bc24260cd400aff3) --- sys/net/pfvar.h | 11 +++++++++++ sys/netpfil/pf/pf.c | 7 ++----- 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 7e16e40ccc45..da7a6731150e 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -1508,6 +1508,17 @@ pf_release_state(struct pf_state *s) return (0); } +static __inline int +pf_release_staten(struct pf_state *s, u_int n) +{ + + if (refcount_releasen(&s->refs, n)) { + pf_free_state(s); + return (1); + } else + return (0); +} + extern struct pf_state *pf_find_state_byid(uint64_t, uint32_t); extern struct pf_state *pf_find_state_all(struct pf_state_key_cmp *, u_int, int *); diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 544c262a96f0..672727293032 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1730,11 +1730,8 @@ pf_unlink_state(struct pf_state *s, u_int flags) PF_HASHROW_UNLOCK(ih); pf_detach_state(s); - /* pf_state_insert() initialises refs to 2, so we can never release the - * last reference here, only in pf_release_state(). */ - (void)refcount_release(&s->refs); - - return (pf_release_state(s)); + /* pf_state_insert() initialises refs to 2 */ + return (pf_release_staten(s, 2)); } void From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:01:57 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D8ADF65B5BE; Mon, 5 Jul 2021 12:01:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS55405z3CxF; Mon, 5 Jul 2021 12:01:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6F96925965; Mon, 5 Jul 2021 12:01:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C1vDJ054619; Mon, 5 Jul 2021 12:01:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C1vpg054618; Mon, 5 Jul 2021 12:01:57 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:01:57 GMT Message-Id: <202107051201.165C1vpg054618@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 47a2d55f00ee - stable/12 - pf: factor out state allocation into pf_alloc_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 47a2d55f00eef32f9cb765eaf094b18ce2da78f0 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:01:58 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=47a2d55f00eef32f9cb765eaf094b18ce2da78f0 commit 47a2d55f00eef32f9cb765eaf094b18ce2da78f0 Author: Mateusz Guzik AuthorDate: 2021-06-28 12:22:31 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:37:53 +0000 pf: factor out state allocation into pf_alloc_state Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit ccb17a21041e9206b80fa9f64b6ec20233df6403) --- sys/netpfil/pf/pf.c | 34 +++++++++++++++++++++++----------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 672727293032..ec27b6ee0ace 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1734,6 +1734,28 @@ pf_unlink_state(struct pf_state *s, u_int flags) return (pf_release_staten(s, 2)); } +static struct pf_state * +pf_alloc_state(int flags) +{ + struct pf_state *s; + + s = uma_zalloc(V_pf_state_z, flags | M_ZERO); + if (__predict_false(s == NULL)) + return (NULL); + + for (int i = 0; i < 2; i++) { + s->bytes[i] = counter_u64_alloc(M_NOWAIT); + s->packets[i] = counter_u64_alloc(M_NOWAIT); + + if (s->bytes[i] == NULL || s->packets[i] == NULL) { + pf_free_state(s); + return (NULL); + } + } + + return (s); +} + void pf_free_state(struct pf_state *cur) { @@ -3731,21 +3753,11 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_SRCLIMIT); goto csfailed; } - s = uma_zalloc(V_pf_state_z, M_NOWAIT | M_ZERO); + s = pf_alloc_state(M_NOWAIT); if (s == NULL) { REASON_SET(&reason, PFRES_MEMORY); goto csfailed; } - for (int i = 0; i < 2; i++) { - s->bytes[i] = counter_u64_alloc(M_NOWAIT); - s->packets[i] = counter_u64_alloc(M_NOWAIT); - - if (s->bytes[i] == NULL || s->packets[i] == NULL) { - pf_free_state(s); - REASON_SET(&reason, PFRES_MEMORY); - goto csfailed; - } - } s->rule.ptr = r; s->nat_rule.ptr = nr; s->anchor.ptr = a; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:01:58 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E036B65BA80; Mon, 5 Jul 2021 12:01:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS64pLQz3Cqk; Mon, 5 Jul 2021 12:01:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8B04D25A18; Mon, 5 Jul 2021 12:01:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C1wh3054646; Mon, 5 Jul 2021 12:01:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C1wp4054644; Mon, 5 Jul 2021 12:01:58 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:01:58 GMT Message-Id: <202107051201.165C1wp4054644@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: e331b1d19f6b - stable/12 - pf: assert that sizeof(struct pf_state) <= 312 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e331b1d19f6bc28b1e64758357f4a408f058a8a6 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:01:59 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=e331b1d19f6bc28b1e64758357f4a408f058a8a6 commit e331b1d19f6bc28b1e64758357f4a408f058a8a6 Author: Mateusz Guzik AuthorDate: 2021-06-28 12:49:40 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:37:56 +0000 pf: assert that sizeof(struct pf_state) <= 312 To prevent accidentally going over a threshold which makes UMA fit only 12 objects per page instead of 13. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit e6dd0e2e8d4f59275fb576a9ccb9d356d36457bb) --- sys/net/pfvar.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index da7a6731150e..7d7e7b20ab6f 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -548,6 +548,11 @@ struct pf_state { u_int8_t sync_updates; u_int8_t _tail[3]; }; + +/* + * Size <= fits 13 objects per page on LP64. Try to not grow the struct beyond that. + */ +_Static_assert(sizeof(struct pf_state) <= 312, "pf_state size crosses 312 bytes"); #endif /* From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:00 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 43A7665BB02; Mon, 5 Jul 2021 12:02:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS76nffz3CxV; Mon, 5 Jul 2021 12:01:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B296225966; Mon, 5 Jul 2021 12:01:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C1xOb054674; Mon, 5 Jul 2021 12:01:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C1xj7054673; Mon, 5 Jul 2021 12:01:59 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:01:59 GMT Message-Id: <202107051201.165C1xj7054673@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: c5c7dd3971bb - stable/12 - pf: fix error-case leaks in pf_create_state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: c5c7dd3971bb88e75ae4e13e7d2df4f935e6b4f1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:00 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=c5c7dd3971bb88e75ae4e13e7d2df4f935e6b4f1 commit c5c7dd3971bb88e75ae4e13e7d2df4f935e6b4f1 Author: Mateusz Guzik AuthorDate: 2021-06-28 18:29:58 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:37:59 +0000 pf: fix error-case leaks in pf_create_state The hand-rolled clean up failed to free counters. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 7f025db57c7879bfa43ac74ecd95903439fdd1d0) --- sys/netpfil/pf/pf.c | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index ec27b6ee0ace..c131e9493ea8 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -3828,7 +3828,7 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_MAPFAILED); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); goto csfailed; } s->rt_kif = r->rpool.cur->kif; @@ -3850,7 +3850,7 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, REASON_SET(&reason, PFRES_MEMORY); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } if ((pd->flags & PFDESC_TCP_NORM) && s->src.scrub && @@ -3860,10 +3860,9 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, DPFPRINTF(PF_DEBUG_URGENT, ("pf_normalize_tcp_stateful failed on first " "pkt\n")); - pf_normalize_tcp_cleanup(s); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } } @@ -3887,12 +3886,10 @@ pf_create_state(struct pf_krule *r, struct pf_krule *nr, struct pf_krule *a, if (pf_state_insert(BOUND_IFACE(r, kif), kif, (pd->dir == PF_IN) ? sk : nk, (pd->dir == PF_IN) ? nk : sk, s)) { - if (pd->proto == IPPROTO_TCP) - pf_normalize_tcp_cleanup(s); REASON_SET(&reason, PFRES_STATEINS); pf_src_tree_remove_state(s); STATE_DEC_COUNTERS(s); - uma_zfree(V_pf_state_z, s); + pf_free_state(s); return (PF_DROP); } else *sm = s; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:01 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5401665BB82; Mon, 5 Jul 2021 12:02:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPS90fWyz3Cfv; Mon, 5 Jul 2021 12:02:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D6146259AC; Mon, 5 Jul 2021 12:02:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C202u054698; Mon, 5 Jul 2021 12:02:00 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C20qF054697; Mon, 5 Jul 2021 12:02:00 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:02:00 GMT Message-Id: <202107051202.165C20qF054697@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: c547f51943d3 - stable/12 - pf: deduplicate V_pf_state_z handling with pfsync MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: c547f51943d35555ddccddbe4237127446d9a5c3 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:01 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=c547f51943d35555ddccddbe4237127446d9a5c3 commit c547f51943d35555ddccddbe4237127446d9a5c3 Author: Mateusz Guzik AuthorDate: 2021-06-28 18:18:26 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:38:02 +0000 pf: deduplicate V_pf_state_z handling with pfsync Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 803dfe3da0e30520227a791ba5c2b93da16e1cf5) --- sys/net/pfvar.h | 1 + sys/netpfil/pf/if_pfsync.c | 20 +++----------------- sys/netpfil/pf/pf.c | 2 +- 3 files changed, 5 insertions(+), 18 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 7d7e7b20ab6f..4f11ff9137d8 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -1493,6 +1493,7 @@ extern int pf_state_insert(struct pfi_kkif *, struct pf_state_key *, struct pf_state_key *, struct pf_state *); +extern struct pf_state *pf_alloc_state(int); extern void pf_free_state(struct pf_state *); static __inline void diff --git a/sys/netpfil/pf/if_pfsync.c b/sys/netpfil/pf/if_pfsync.c index 528195bddec1..fe50b9b270d2 100644 --- a/sys/netpfil/pf/if_pfsync.c +++ b/sys/netpfil/pf/if_pfsync.c @@ -505,16 +505,10 @@ pfsync_state_import(struct pfsync_state *sp, u_int8_t flags) /* * XXXGL: consider M_WAITOK in ioctl path after. */ - if ((st = uma_zalloc(V_pf_state_z, M_NOWAIT | M_ZERO)) == NULL) + st = pf_alloc_state(M_NOWAIT); + if (__predict_false(st == NULL)) goto cleanup; - for (int i = 0; i < 2; i++) { - st->packets[i] = counter_u64_alloc(M_NOWAIT); - st->bytes[i] = counter_u64_alloc(M_NOWAIT); - if (st->packets[i] == NULL || st->bytes[i] == NULL) - goto cleanup; - } - if ((skw = uma_zalloc(V_pf_state_key_z, M_NOWAIT)) == NULL) goto cleanup; @@ -624,15 +618,7 @@ cleanup: cleanup_state: /* pf_state_insert() frees the state keys. */ if (st) { - for (int i = 0; i < 2; i++) { - counter_u64_free(st->packets[i]); - counter_u64_free(st->bytes[i]); - } - if (st->dst.scrub) - uma_zfree(V_pf_state_scrub_z, st->dst.scrub); - if (st->src.scrub) - uma_zfree(V_pf_state_scrub_z, st->src.scrub); - uma_zfree(V_pf_state_z, st); + pf_free_state(st); } return (error); } diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index c131e9493ea8..4f45f5b8ea96 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1734,7 +1734,7 @@ pf_unlink_state(struct pf_state *s, u_int flags) return (pf_release_staten(s, 2)); } -static struct pf_state * +struct pf_state * pf_alloc_state(int flags) { struct pf_state *s; From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:03 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C608565BA88; Mon, 5 Jul 2021 12:02:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPSC2GLKz3D85; Mon, 5 Jul 2021 12:02:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 06834258D7; Mon, 5 Jul 2021 12:02:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C218r054722; Mon, 5 Jul 2021 12:02:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C21cl054721; Mon, 5 Jul 2021 12:02:01 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:02:01 GMT Message-Id: <202107051202.165C21cl054721@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 838edd2c9dc6 - stable/12 - pf: make sure the dtrace probe has safe access to state MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 838edd2c9dc6e7920c8fd9629d3ee6111c1db189 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:04 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=838edd2c9dc6e7920c8fd9629d3ee6111c1db189 commit 838edd2c9dc6e7920c8fd9629d3ee6111c1db189 Author: Mateusz Guzik AuthorDate: 2021-06-28 19:32:20 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:38:05 +0000 pf: make sure the dtrace probe has safe access to state Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit d26ef5c7ac830812f07a02787f25fed5d6f8609e) --- sys/netpfil/pf/pf.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index 4f45f5b8ea96..c76207d1e7f1 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -6405,11 +6405,12 @@ done: } break; } - if (s) - PF_STATE_UNLOCK(s); SDT_PROBE4(pf, ip, test, done, action, reason, r, s); + if (s) + PF_STATE_UNLOCK(s); + return (action); } #endif /* INET */ From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:03 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A005D65B8D8; Mon, 5 Jul 2021 12:02:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPSC2NCwz3D39; Mon, 5 Jul 2021 12:02:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1AB7E25B92; Mon, 5 Jul 2021 12:02:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C22Vj055547; Mon, 5 Jul 2021 12:02:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C22vf055522; Mon, 5 Jul 2021 12:02:02 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:02:02 GMT Message-Id: <202107051202.165C22vf055522@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 3f136d9fea86 - stable/12 - pf: revert: Use counter(9) for pf_state byte/packet tracking MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 3f136d9fea869515900bcda1ef6533f5804d819c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:03 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=3f136d9fea869515900bcda1ef6533f5804d819c commit 3f136d9fea869515900bcda1ef6533f5804d819c Author: Mateusz Guzik AuthorDate: 2021-06-28 18:50:56 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:38:08 +0000 pf: revert: Use counter(9) for pf_state byte/packet tracking stats are not shared and consequently per-CPU counters only waste memory. No slowdown was measured when passing over 20M pps. Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 55cc305dfcad0ad7c4f528fa47f7473927e8223a) --- sys/net/pfvar.h | 4 ++-- sys/netpfil/pf/pf.c | 39 +++++++++------------------------------ sys/netpfil/pf/pf_ioctl.c | 10 ++++------ sys/netpfil/pf/pf_nv.c | 4 ++-- 4 files changed, 17 insertions(+), 40 deletions(-) diff --git a/sys/net/pfvar.h b/sys/net/pfvar.h index 4f11ff9137d8..c11455837072 100644 --- a/sys/net/pfvar.h +++ b/sys/net/pfvar.h @@ -533,8 +533,8 @@ struct pf_state { struct pfi_kkif *rt_kif; struct pf_ksrc_node *src_node; struct pf_ksrc_node *nat_src_node; - counter_u64_t packets[2]; - counter_u64_t bytes[2]; + u_int64_t packets[2]; + u_int64_t bytes[2]; u_int32_t creation; u_int32_t expire; u_int32_t pfsync_time; diff --git a/sys/netpfil/pf/pf.c b/sys/netpfil/pf/pf.c index c76207d1e7f1..6edbd60b8d07 100644 --- a/sys/netpfil/pf/pf.c +++ b/sys/netpfil/pf/pf.c @@ -1737,23 +1737,8 @@ pf_unlink_state(struct pf_state *s, u_int flags) struct pf_state * pf_alloc_state(int flags) { - struct pf_state *s; - - s = uma_zalloc(V_pf_state_z, flags | M_ZERO); - if (__predict_false(s == NULL)) - return (NULL); - - for (int i = 0; i < 2; i++) { - s->bytes[i] = counter_u64_alloc(M_NOWAIT); - s->packets[i] = counter_u64_alloc(M_NOWAIT); - - if (s->bytes[i] == NULL || s->packets[i] == NULL) { - pf_free_state(s); - return (NULL); - } - } - return (s); + return (uma_zalloc(V_pf_state_z, flags | M_ZERO)); } void @@ -1764,11 +1749,6 @@ pf_free_state(struct pf_state *cur) KASSERT(cur->timeout == PFTM_UNLINKED, ("%s: timeout %u", __func__, cur->timeout)); - for (int i = 0; i < 2; i++) { - counter_u64_free(cur->bytes[i]); - counter_u64_free(cur->packets[i]); - } - pf_normalize_tcp_cleanup(cur); uma_zfree(V_pf_state_z, cur); counter_u64_add(V_pf_status.fcounters[FCNT_STATE_REMOVALS], 1); @@ -4317,9 +4297,8 @@ pf_tcp_track_full(struct pf_state_peer *src, struct pf_state_peer *dst, pf_print_flags(th->th_flags); printf(" seq=%u (%u) ack=%u len=%u ackskew=%d " "pkts=%llu:%llu dir=%s,%s\n", seq, orig_seq, ack, - pd->p_len, ackskew, - (unsigned long long)counter_u64_fetch((*state)->packets[0]), - (unsigned long long)counter_u64_fetch((*state)->packets[1]), + pd->p_len, ackskew, (unsigned long long)(*state)->packets[0], + (unsigned long long)(*state)->packets[1], pd->dir == PF_IN ? "in" : "out", pd->dir == (*state)->direction ? "fwd" : "rev"); } @@ -4374,8 +4353,8 @@ pf_tcp_track_full(struct pf_state_peer *src, struct pf_state_peer *dst, printf(" seq=%u (%u) ack=%u len=%u ackskew=%d " "pkts=%llu:%llu dir=%s,%s\n", seq, orig_seq, ack, pd->p_len, ackskew, - (unsigned long long)counter_u64_fetch((*state)->packets[0]), - (unsigned long long)counter_u64_fetch((*state)->packets[1]), + (unsigned long long)(*state)->packets[0], + (unsigned long long)(*state)->packets[1], pd->dir == PF_IN ? "in" : "out", pd->dir == (*state)->direction ? "fwd" : "rev"); printf("pf: State failure on: %c %c %c %c | %c %c\n", @@ -6363,8 +6342,8 @@ done: pd.tot_len); } dirndx = (dir == s->direction) ? 0 : 1; - counter_u64_add(s->packets[dirndx], 1); - counter_u64_add(s->bytes[dirndx], pd.tot_len); + s->packets[dirndx]++; + s->bytes[dirndx] += pd.tot_len; } tr = r; nr = (s != NULL) ? s->nat_rule.ptr : pd.nat_rule; @@ -6764,8 +6743,8 @@ done: pd.tot_len); } dirndx = (dir == s->direction) ? 0 : 1; - counter_u64_add(s->packets[dirndx], 1); - counter_u64_add(s->bytes[dirndx], pd.tot_len); + s->packets[dirndx]++; + s->bytes[dirndx] += pd.tot_len; } tr = r; nr = (s != NULL) ? s->nat_rule.ptr : pd.nat_rule; diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 3d5ae03b5a11..ce7a8d1dc582 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -4652,12 +4652,10 @@ pfsync_state_export(struct pfsync_state *sp, struct pf_state *st) else sp->nat_rule = htonl(st->nat_rule.ptr->nr); - pf_state_counter_hton(counter_u64_fetch(st->packets[0]), - sp->packets[0]); - pf_state_counter_hton(counter_u64_fetch(st->packets[1]), - sp->packets[1]); - pf_state_counter_hton(counter_u64_fetch(st->bytes[0]), sp->bytes[0]); - pf_state_counter_hton(counter_u64_fetch(st->bytes[1]), sp->bytes[1]); + pf_state_counter_hton(st->packets[0], sp->packets[0]); + pf_state_counter_hton(st->packets[1], sp->packets[1]); + pf_state_counter_hton(st->bytes[0], sp->bytes[0]); + pf_state_counter_hton(st->bytes[1], sp->bytes[1]); } diff --git a/sys/netpfil/pf/pf_nv.c b/sys/netpfil/pf/pf_nv.c index 31943ba69687..553290c88586 100644 --- a/sys/netpfil/pf/pf_nv.c +++ b/sys/netpfil/pf/pf_nv.c @@ -982,9 +982,9 @@ pf_state_to_nvstate(const struct pf_state *s) for (int i = 0; i < 2; i++) { nvlist_append_number_array(nvl, "packets", - counter_u64_fetch(s->packets[i])); + s->packets[i]); nvlist_append_number_array(nvl, "bytes", - counter_u64_fetch(s->bytes[i])); + s->bytes[i]); } nvlist_add_number(nvl, "creatorid", s->creatorid); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:05 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5AA2865BA8B; Mon, 5 Jul 2021 12:02:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPSD2pcXz3D0h; Mon, 5 Jul 2021 12:02:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3D75D25B93; Mon, 5 Jul 2021 12:02:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C246A055641; Mon, 5 Jul 2021 12:02:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C24KD055640; Mon, 5 Jul 2021 12:02:04 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:02:04 GMT Message-Id: <202107051202.165C24KD055640@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 98fd94dc8331 - stable/12 - pfctl: cache getprotobynumber results MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 98fd94dc83319db1b6bfc9f741245eefb0af7911 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:05 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=98fd94dc83319db1b6bfc9f741245eefb0af7911 commit 98fd94dc83319db1b6bfc9f741245eefb0af7911 Author: Mateusz Guzik AuthorDate: 2021-07-01 19:25:43 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:38:58 +0000 pfctl: cache getprotobynumber results As for example pfctl -ss keeps calling it, it saves a lot of overhead from elided parsing of /etc/nsswitch.conf and /etc/protocols. Sample result when running a pre-nvlist binary with nfs root and dumping 7 mln states: before: 24.817u 62.993s 1:28.52 99.1% after: 8.064u 1.117s 0:18.87 48.5% Idea by Jim Thompson Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 858937bea4599d254a97ee6321683f8629604e15) --- sbin/pfctl/parse.y | 8 ++++---- sbin/pfctl/pf_print_state.c | 6 +++--- sbin/pfctl/pfctl.c | 43 +++++++++++++++++++++++++++++++++++++++++++ sbin/pfctl/pfctl.h | 2 ++ sbin/pfctl/pfctl_parser.c | 6 +++--- 5 files changed, 55 insertions(+), 10 deletions(-) diff --git a/sbin/pfctl/parse.y b/sbin/pfctl/parse.y index 06d972556195..1083666e00ce 100644 --- a/sbin/pfctl/parse.y +++ b/sbin/pfctl/parse.y @@ -5009,13 +5009,13 @@ expand_label_port(const char *name, char *label, size_t len, void expand_label_proto(const char *name, char *label, size_t len, u_int8_t proto) { - struct protoent *pe; + const char *protoname; char n[4]; if (strstr(label, name) != NULL) { - pe = getprotobynumber(proto); - if (pe != NULL) - expand_label_str(label, len, name, pe->p_name); + protoname = pfctl_proto2name(proto); + if (protoname != NULL) + expand_label_str(label, len, name, protoname); else { snprintf(n, sizeof(n), "%u", proto); expand_label_str(label, len, name, n); diff --git a/sbin/pfctl/pf_print_state.c b/sbin/pfctl/pf_print_state.c index b1f0079154cf..b66a296d6080 100644 --- a/sbin/pfctl/pf_print_state.c +++ b/sbin/pfctl/pf_print_state.c @@ -211,7 +211,7 @@ print_state(struct pfctl_state *s, int opts) { struct pfctl_state_peer *src, *dst; struct pfctl_state_key *key, *sk, *nk; - struct protoent *p; + const char *protoname; int min, sec; sa_family_t af; uint8_t proto; @@ -243,8 +243,8 @@ print_state(struct pfctl_state *s, int opts) sk->port[1] = nk->port[1]; } printf("%s ", s->ifname); - if ((p = getprotobynumber(proto)) != NULL) - printf("%s ", p->p_name); + if ((protoname = pfctl_proto2name(proto)) != NULL) + printf("%s ", protoname); else printf("%u ", proto); diff --git a/sbin/pfctl/pfctl.c b/sbin/pfctl/pfctl.c index f82d75198d61..14b7f3a01657 100644 --- a/sbin/pfctl/pfctl.c +++ b/sbin/pfctl/pfctl.c @@ -254,6 +254,49 @@ usage(void) exit(1); } +/* + * Cache protocol number to name translations. + * + * Translation is performed a lot e.g., when dumping states and + * getprotobynumber is incredibly expensive. + * + * Note from the getprotobynumber(3) manpage: + * + * These functions use a thread-specific data space; if the data is needed + * for future use, it should be copied before any subsequent calls overwrite + * it. Only the Internet protocols are currently understood. + * + * + * Consequently we only cache the name and strdup it for safety. + * + * At the time of writing this comment the last entry in /etc/protocols is: + * divert 258 DIVERT # Divert pseudo-protocol [non IANA] + */ +const char * +pfctl_proto2name(int proto) +{ + static const char *pfctl_proto_cache[259]; + struct protoent *p; + + if (proto >= nitems(pfctl_proto_cache)) { + p = getprotobynumber(proto); + if (p == NULL) { + return (NULL); + } + return (p->p_name); + } + + if (pfctl_proto_cache[proto] == NULL) { + p = getprotobynumber(proto); + if (p == NULL) { + return (NULL); + } + pfctl_proto_cache[proto] = strdup(p->p_name); + } + + return (pfctl_proto_cache[proto]); +} + int pfctl_enable(int dev, int opts) { diff --git a/sbin/pfctl/pfctl.h b/sbin/pfctl/pfctl.h index f8ff5012e01b..80ef184fa90f 100644 --- a/sbin/pfctl/pfctl.h +++ b/sbin/pfctl/pfctl.h @@ -138,4 +138,6 @@ void pf_remove_if_empty_ruleset(struct pfctl_ruleset *); struct pfctl_ruleset *pf_find_ruleset(const char *); struct pfctl_ruleset *pf_find_or_create_ruleset(const char *); +const char *pfctl_proto2name(int); + #endif /* _PFCTL_H_ */ diff --git a/sbin/pfctl/pfctl_parser.c b/sbin/pfctl/pfctl_parser.c index ee0f9f8318a9..c38411a145e7 100644 --- a/sbin/pfctl/pfctl_parser.c +++ b/sbin/pfctl/pfctl_parser.c @@ -812,10 +812,10 @@ print_rule(struct pfctl_rule *r, const char *anchor_call, int verbose, int numer printf(" inet6"); } if (r->proto) { - struct protoent *p; + const char *protoname; - if ((p = getprotobynumber(r->proto)) != NULL) - printf(" proto %s", p->p_name); + if ((protoname = pfctl_proto2name(r->proto)) != NULL) + printf(" proto %s", protoname); else printf(" proto %u", r->proto); } From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:02:06 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0493965BB10; Mon, 5 Jul 2021 12:02:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPSF4mY6z3D0k; Mon, 5 Jul 2021 12:02:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5D01F257BF; Mon, 5 Jul 2021 12:02:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C25XZ055671; Mon, 5 Jul 2021 12:02:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C251t055670; Mon, 5 Jul 2021 12:02:05 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:02:05 GMT Message-Id: <202107051202.165C251t055670@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 98b05b41e8c6 - stable/12 - pf: make DIOCGETSTATESNV iterations killable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 98b05b41e8c69e6c737759ad33fa0117d89aedee Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:02:06 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=98b05b41e8c69e6c737759ad33fa0117d89aedee commit 98b05b41e8c69e6c737759ad33fa0117d89aedee Author: Mateusz Guzik AuthorDate: 2021-07-02 07:28:48 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 11:39:06 +0000 pf: make DIOCGETSTATESNV iterations killable Reviewed by: kp Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 48d5b86364fa8e86318805d0078ccf9066e94ae8) --- sys/netpfil/pf/pf_ioctl.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index ce7a8d1dc582..0ce39db05c15 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -5127,6 +5127,11 @@ pf_getstates(struct pfioc_nv *nv) if (s->timeout == PFTM_UNLINKED) continue; + if (SIGPENDING(curthread)) { + PF_HASHROW_UNLOCK(ih); + ERROUT(EINTR); + } + nvls = pf_state_to_nvstate(s); if (nvls == NULL) { PF_HASHROW_UNLOCK(ih); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:05:10 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1028565C08C; Mon, 5 Jul 2021 12:05:10 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPWn6y03z3DlM; Mon, 5 Jul 2021 12:05:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D6C5825A2C; Mon, 5 Jul 2021 12:05:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C59Eb059030; Mon, 5 Jul 2021 12:05:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C59tl059029; Mon, 5 Jul 2021 12:05:09 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:05:09 GMT Message-Id: <202107051205.165C59tl059029@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: f1c39eb170dd - stable/13 - mbuf: add m_free_raw to be used instead of directly calling uma_zfree MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f1c39eb170ddeca5137edb706d89216f5a1a1d4f Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:05:10 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=f1c39eb170ddeca5137edb706d89216f5a1a1d4f commit f1c39eb170ddeca5137edb706d89216f5a1a1d4f Author: Mateusz Guzik AuthorDate: 2021-06-30 14:15:25 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 12:04:59 +0000 mbuf: add m_free_raw to be used instead of directly calling uma_zfree The intent is to remove all direct zone_mbuf consumers so that ctor/dtor from that zone can be reimplemented as wrappers around uma, avoiding an indirect function call. Reviewed by: kbowling Discussed with: gallatin Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30959 (cherry picked from commit 05462babd424124762375dca186802d7355af566) --- sys/kern/kern_mbuf.c | 10 ++++++++++ sys/sys/mbuf.h | 1 + 2 files changed, 11 insertions(+) diff --git a/sys/kern/kern_mbuf.c b/sys/kern/kern_mbuf.c index a46c576bad90..3b202bf3aab7 100644 --- a/sys/kern/kern_mbuf.c +++ b/sys/kern/kern_mbuf.c @@ -1525,6 +1525,16 @@ m_freem(struct mbuf *mb) mb = m_free(mb); } +/* + * Temporary primitive to allow freeing without going through m_free. + */ +void +m_free_raw(struct mbuf *mb) +{ + + uma_zfree(zone_mbuf, mb); +} + int m_snd_tag_alloc(struct ifnet *ifp, union if_snd_tag_alloc_params *params, struct m_snd_tag **mstp) diff --git a/sys/sys/mbuf.h b/sys/sys/mbuf.h index 729653fa1e55..bb6bf9f62b7a 100644 --- a/sys/sys/mbuf.h +++ b/sys/sys/mbuf.h @@ -827,6 +827,7 @@ void m_extadd(struct mbuf *, char *, u_int, m_ext_free_t, u_int m_fixhdr(struct mbuf *); struct mbuf *m_fragment(struct mbuf *, int, int); void m_freem(struct mbuf *); +void m_free_raw(struct mbuf *); struct mbuf *m_get2(int, int, short, int); struct mbuf *m_getjcl(int, short, int, int); struct mbuf *m_getm2(struct mbuf *, int, int, short, int); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:05:11 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 83A7D65BEF3; Mon, 5 Jul 2021 12:05:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPWq1dKKz3Dc2; Mon, 5 Jul 2021 12:05:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 15A2B256ED; Mon, 5 Jul 2021 12:05:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C5Acj059054; Mon, 5 Jul 2021 12:05:10 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C5A7o059053; Mon, 5 Jul 2021 12:05:10 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:05:10 GMT Message-Id: <202107051205.165C5A7o059053@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: eae6de040612 - stable/13 - iflib: switch bare zone_mbuf use to m_free_raw MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: eae6de040612845aa2f0582d1f84914ed8aa1a4a Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:05:11 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=eae6de040612845aa2f0582d1f84914ed8aa1a4a commit eae6de040612845aa2f0582d1f84914ed8aa1a4a Author: Mateusz Guzik AuthorDate: 2021-06-30 14:17:29 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 12:05:00 +0000 iflib: switch bare zone_mbuf use to m_free_raw Reviewed by: kbowling Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30961 (cherry picked from commit bad5f0b6c2944453db2d70a982e8abf203f21669) --- sys/net/iflib.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/net/iflib.c b/sys/net/iflib.c index 788cc51822a1..a71c48e772e5 100644 --- a/sys/net/iflib.c +++ b/sys/net/iflib.c @@ -2219,7 +2219,7 @@ iflib_fl_bufs_free(iflib_fl_t fl) *sd_cl = NULL; if (*sd_m != NULL) { m_init(*sd_m, M_NOWAIT, MT_DATA, 0); - uma_zfree(zone_mbuf, *sd_m); + m_free_raw(*sd_m); *sd_m = NULL; } } else { From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:05:12 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CACC865C04A; Mon, 5 Jul 2021 12:05:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJPWr3Qnbz3DhQ; Mon, 5 Jul 2021 12:05:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 334AA2596E; Mon, 5 Jul 2021 12:05:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165C5CtV059078; Mon, 5 Jul 2021 12:05:12 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165C5CQa059077; Mon, 5 Jul 2021 12:05:12 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:05:12 GMT Message-Id: <202107051205.165C5CQa059077@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 8413d2a78665 - stable/13 - ktls: switch bare zone_mbuf use to m_free_raw MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 8413d2a786655005140c75060b7cba321079a5dd Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:05:13 -0000 The branch stable/13 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=8413d2a786655005140c75060b7cba321079a5dd commit 8413d2a786655005140c75060b7cba321079a5dd Author: Mateusz Guzik AuthorDate: 2021-06-30 13:02:45 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 12:05:01 +0000 ktls: switch bare zone_mbuf use to m_free_raw Reviewed by: gallatin Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30955 (cherry picked from commit 904a08f3427c8ae42b667c1f5dc3e441b48a8e84) --- sys/kern/uipc_ktls.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/kern/uipc_ktls.c b/sys/kern/uipc_ktls.c index a648d37cd60f..0f5cc7c1b28f 100644 --- a/sys/kern/uipc_ktls.c +++ b/sys/kern/uipc_ktls.c @@ -2140,7 +2140,7 @@ ktls_work_thread(void *ctx) STAILQ_FOREACH_SAFE(m, &local_m_head, m_epg_stailq, n) { if (m->m_epg_flags & EPG_FLAG_2FREE) { ktls_free(m->m_epg_tls); - uma_zfree(zone_mbuf, m); + m_free_raw(m); } else { ktls_encrypt(m); counter_u64_add(ktls_cnt_tx_queued, -1); From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 12:48:56 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A23B365C833; Mon, 5 Jul 2021 12:48:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJQVJ4FXDz3Jsl; Mon, 5 Jul 2021 12:48:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7ACC22648C; Mon, 5 Jul 2021 12:48:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165CmuGT012045; Mon, 5 Jul 2021 12:48:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165CmuV7012044; Mon, 5 Jul 2021 12:48:56 GMT (envelope-from git) Date: Mon, 5 Jul 2021 12:48:56 GMT Message-Id: <202107051248.165CmuV7012044@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: 2f6dd4a29198 - stable/12 - refcount: add refcount_releasen MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 2f6dd4a29198ab82ab013e15b4eb2fa6de25d5bf Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 12:48:56 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=2f6dd4a29198ab82ab013e15b4eb2fa6de25d5bf commit 2f6dd4a29198ab82ab013e15b4eb2fa6de25d5bf Author: Mateusz Guzik AuthorDate: 2021-07-05 12:45:32 +0000 Commit: Mateusz Guzik CommitDate: 2021-07-05 12:48:23 +0000 refcount: add refcount_releasen This is a direct commit as the routine in main was added as a side effect of functionality which later got reverted. --- sys/sys/refcount.h | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/sys/sys/refcount.h b/sys/sys/refcount.h index 44edbdf953a0..19a8f38772a0 100644 --- a/sys/sys/refcount.h +++ b/sys/sys/refcount.h @@ -70,26 +70,33 @@ refcount_acquire_checked(volatile u_int *count) } static __inline bool -refcount_release(volatile u_int *count) +refcount_releasen(volatile u_int *count, u_int n) { u_int old; atomic_thread_fence_rel(); - old = atomic_fetchadd_int(count, -1); + old = atomic_fetchadd_int(count, -n); KASSERT(old > 0, ("refcount %p is zero", count)); - if (old > 1) + if (old > n) return (false); /* * Last reference. Signal the user to call the destructor. * - * Ensure that the destructor sees all updates. The fence_rel - * at the start of the function synchronized with this fence. + * Ensure that the destructor sees all updates. This synchronizes with + * release fences from all routines which drop the count. */ atomic_thread_fence_acq(); return (true); } +static __inline bool +refcount_release(volatile u_int *count) +{ + + return (refcount_releasen(count, 1)); +} + /* * This functions returns non-zero if the refcount was * incremented. Else zero is returned. From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 14:38:20 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6185C65DF4C; Mon, 5 Jul 2021 14:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJSwX2Jvjz3sMF; Mon, 5 Jul 2021 14:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 32B1B27A2D; Mon, 5 Jul 2021 14:38:20 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165EcK6u058401; Mon, 5 Jul 2021 14:38:20 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165EcKWh058400; Mon, 5 Jul 2021 14:38:20 GMT (envelope-from git) Date: Mon, 5 Jul 2021 14:38:20 GMT Message-Id: <202107051438.165EcKWh058400@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mike Karels Subject: git: 0a4ad905bf44 - stable/13 - genet: Fix potential crash during attach MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: karels X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0a4ad905bf44d5286ae8e1c4c9693c98fca43183 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 14:38:20 -0000 The branch stable/13 has been updated by karels: URL: https://cgit.FreeBSD.org/src/commit/?id=0a4ad905bf44d5286ae8e1c4c9693c98fca43183 commit 0a4ad905bf44d5286ae8e1c4c9693c98fca43183 Author: Mike Karels AuthorDate: 2021-06-20 16:10:26 +0000 Commit: Mike Karels CommitDate: 2021-07-05 14:12:09 +0000 genet: Fix potential crash during attach As pointed out in the bug, the genet driver (RPi4 Ethernet) was attaching the interrupts before the data structures were fully initialized, causing a crash if an interrupt came in during the attach. Fix by reordering code blocks. PR: 256334 Reported by: < ghuckriede at blackberry.com > Reviewed by: < ghuckriede at blackberry.com > (informally) (cherry picked from commit 13604fb0fd43c85e6bb3a0ad6400a684f150bdea) --- sys/arm64/broadcom/genet/if_genet.c | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c index d0fe2b0d5e71..5d70c7075c3a 100644 --- a/sys/arm64/broadcom/genet/if_genet.c +++ b/sys/arm64/broadcom/genet/if_genet.c @@ -279,21 +279,6 @@ gen_attach(device_t dev) goto fail; } - /* Install interrupt handlers */ - error = bus_setup_intr(dev, sc->res[_RES_IRQ1], - INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr, sc, &sc->ih); - if (error != 0) { - device_printf(dev, "cannot setup interrupt handler1\n"); - goto fail; - } - - error = bus_setup_intr(dev, sc->res[_RES_IRQ2], - INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr2, sc, &sc->ih2); - if (error != 0) { - device_printf(dev, "cannot setup interrupt handler2\n"); - goto fail; - } - /* Setup ethernet interface */ sc->ifp = if_alloc(IFT_ETHER); if_setsoftc(sc->ifp, sc); @@ -310,6 +295,21 @@ gen_attach(device_t dev) IFCAP_HWCSUM_IPV6); if_setcapenable(sc->ifp, if_getcapabilities(sc->ifp)); + /* Install interrupt handlers */ + error = bus_setup_intr(dev, sc->res[_RES_IRQ1], + INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr, sc, &sc->ih); + if (error != 0) { + device_printf(dev, "cannot setup interrupt handler1\n"); + goto fail; + } + + error = bus_setup_intr(dev, sc->res[_RES_IRQ2], + INTR_TYPE_NET | INTR_MPSAFE, NULL, gen_intr2, sc, &sc->ih2); + if (error != 0) { + device_printf(dev, "cannot setup interrupt handler2\n"); + goto fail; + } + /* Attach MII driver */ mii_flags = 0; switch (sc->phy_mode) From owner-dev-commits-src-branches@freebsd.org Mon Jul 5 14:41:46 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3512165DFC0; Mon, 5 Jul 2021 14:41:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJT0V0qPkz3skV; Mon, 5 Jul 2021 14:41:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 04D3D27C6A; Mon, 5 Jul 2021 14:41:46 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 165EfjeV069932; Mon, 5 Jul 2021 14:41:45 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 165EfjH9069931; Mon, 5 Jul 2021 14:41:45 GMT (envelope-from git) Date: Mon, 5 Jul 2021 14:41:45 GMT Message-Id: <202107051441.165EfjH9069931@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mike Karels Subject: git: 2ecf3f2c5774 - stable/13 - genet: pullup minimum header amount for IPv4 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: karels X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 2ecf3f2c57749f9968a50a1ce2c2cc29c6f17823 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jul 2021 14:41:46 -0000 The branch stable/13 has been updated by karels: URL: https://cgit.FreeBSD.org/src/commit/?id=2ecf3f2c57749f9968a50a1ce2c2cc29c6f17823 commit 2ecf3f2c57749f9968a50a1ce2c2cc29c6f17823 Author: Mike Karels AuthorDate: 2021-06-20 17:50:31 +0000 Commit: Mike Karels CommitDate: 2021-07-05 14:39:48 +0000 genet: pullup minimum header amount for IPv4 The genet driver (RPi4 Ethernet) had code to pull headers into the first mbuf if there was only an Ethernet header there. This was originally needed for ICMPv6 replies, then for forwarded IPv6/TCP. Now a situation has been found where it is needed for IPv4, when using NAT with IPFW. Generalize to do this for all protocols. Rather than using an IPv6-related definition for the length, move the length to a variable that can be set with sysctl (hw.genet.tx_hdr_min). Move an old tunable to a new RDTUN variable with a better name. PR: 25607 Reviewers: emaste Differential Revision: https://reviews.freebsd.org/D30831 (cherry picked from commit 184291b0a56a68f2c0dc4388419b0f7d85a08cf4) --- sys/arm64/broadcom/genet/if_genet.c | 53 +++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 23 deletions(-) diff --git a/sys/arm64/broadcom/genet/if_genet.c b/sys/arm64/broadcom/genet/if_genet.c index 5d70c7075c3a..31fab0f9cf09 100644 --- a/sys/arm64/broadcom/genet/if_genet.c +++ b/sys/arm64/broadcom/genet/if_genet.c @@ -49,6 +49,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -97,9 +98,27 @@ __FBSDID("$FreeBSD$"); #define TX_MAX_SEGS 20 -/* Maximum number of mbufs to send to if_input */ +static SYSCTL_NODE(_hw, OID_AUTO, genet, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, + "genet driver parameters"); + +/* Maximum number of mbufs to pass per call to if_input */ static int gen_rx_batch = 16 /* RX_BATCH_DEFAULT */; -TUNABLE_INT("hw.gen.rx_batch", &gen_rx_batch); +SYSCTL_INT(_hw_genet, OID_AUTO, rx_batch, CTLFLAG_RDTUN, + &gen_rx_batch, 0, "max mbufs per call to if_input"); + +TUNABLE_INT("hw.gen.rx_batch", &gen_rx_batch); /* old name/interface */ + +/* + * Transmitting packets with only an Ethernet header in the first mbuf + * fails. Examples include reflected ICMPv6 packets, e.g. echo replies; + * forwarded IPv6/TCP packets; and forwarded IPv4/TCP packets that use NAT + * with IPFW. Pulling up the sizes of ether_header + ip6_hdr + icmp6_hdr + * seems to work for both ICMPv6 and TCP over IPv6, as well as the IPv4/TCP + * case. + */ +static int gen_tx_hdr_min = 56; /* ether_header + ip6_hdr + icmp6_hdr */ +SYSCTL_INT(_hw_genet, OID_AUTO, tx_hdr_min, CTLFLAG_RW, + &gen_tx_hdr_min, 0, "header to add to packets with ether header only"); static struct ofw_compat_data compat_data[] = { { "brcm,genet-v1", 1 }, @@ -995,31 +1014,19 @@ gen_encap(struct gen_softc *sc, struct mbuf **mp) m = *mp; /* - * Reflected ICMPv6 packets, e.g. echo replies, tend to get laid - * out with only the Ethernet header in the first mbuf, and this - * doesn't seem to work. Forwarded TCP packets over IPv6 also - * fail if laid out with only the Ethernet header in the first mbuf. - * For now, pull up any IPv6 packet with that layout. Maybe IPv4 - * needs it but we haven't run into it. Pulling up the sizes of - * ether_header + ip6_header + icmp6_hdr seems to work for both - * ICMPv6 and TCP over IPv6. + * Don't attempt to send packets with only an Ethernet header in + * first mbuf; see comment above with gen_tx_hdr_min. */ -#define IP6_PULLUP_LEN (sizeof(struct ether_header) + \ - sizeof(struct ip6_hdr) + 8) if (m->m_len == sizeof(struct ether_header)) { - int ether_type = mtod(m, struct ether_header *)->ether_type; - if (ntohs(ether_type) == ETHERTYPE_IPV6) { - m = m_pullup(m, MIN(m->m_pkthdr.len, IP6_PULLUP_LEN)); - if (m == NULL) { - if (sc->ifp->if_flags & IFF_DEBUG) - device_printf(sc->dev, - "IPV6 pullup fail\n"); - *mp = NULL; - return (ENOMEM); - } + m = m_pullup(m, MIN(m->m_pkthdr.len, gen_tx_hdr_min)); + if (m == NULL) { + if (sc->ifp->if_flags & IFF_DEBUG) + device_printf(sc->dev, + "header pullup fail\n"); + *mp = NULL; + return (ENOMEM); } } -#undef IP6_PULLUP_LEN if ((if_getcapenable(sc->ifp) & (IFCAP_TXCSUM | IFCAP_TXCSUM_IPV6)) != 0) { From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 06:56:33 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8778A65105A; Tue, 6 Jul 2021 06:56:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtdF3Jfwz4Xm0; Tue, 6 Jul 2021 06:56:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5907D1522D; Tue, 6 Jul 2021 06:56:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1666uXuv062909; Tue, 6 Jul 2021 06:56:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1666uX0s062908; Tue, 6 Jul 2021 06:56:33 GMT (envelope-from git) Date: Tue, 6 Jul 2021 06:56:33 GMT Message-Id: <202107060656.1666uX0s062908@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 3423d44cd152 - stable/13 - libalias: tidy up housekeeping MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3423d44cd152da8e55763f5c6a8b88070316f8e4 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 06:56:33 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=3423d44cd152da8e55763f5c6a8b88070316f8e4 commit 3423d44cd152da8e55763f5c6a8b88070316f8e4 Author: Lutz Donnerhacke AuthorDate: 2021-05-15 15:35:36 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 06:55:52 +0000 libalias: tidy up housekeeping Replace current expensive, but sparsly called housekeeping by a single, repetive action. This is part of a larger restructure of libalias in order to switch to more efficient data structures. The whole restructure process is split into 15 reviews to ease reviewing. All those steps will be squashed into a single commit for MFC in order to hide the intermediate states from production systems. Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D30277 (cherry picked from commit 294799c6b081faece556a5010a4f51552e131c2f) --- sys/netinet/libalias/alias_db.c | 104 ++++++++++++++++--------------------- sys/netinet/libalias/alias_local.h | 6 +-- 2 files changed, 47 insertions(+), 63 deletions(-) diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index 0273cc84773d..1b1cddc0b6fd 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -182,11 +182,6 @@ static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); near relevant functions or structs) */ -/* Parameters used for cleanup of expired links */ -/* NOTE: ALIAS_CLEANUP_INTERVAL_SECS must be less then LINK_TABLE_OUT_SIZE */ -#define ALIAS_CLEANUP_INTERVAL_SECS 64 -#define ALIAS_CLEANUP_MAX_SPOKES (LINK_TABLE_OUT_SIZE/5) - /* Timeouts (in seconds) for different link types */ #define ICMP_EXPIRE_TIME 60 #define UDP_EXPIRE_TIME 60 @@ -329,6 +324,7 @@ struct alias_link { /* Linked list of pointers for input and output lookup tables */ LIST_ENTRY (alias_link) list_out; LIST_ENTRY (alias_link) list_in; + TAILQ_ENTRY (alias_link) list_expire; /* Auxiliary data */ union { char *frag_ptr; @@ -510,7 +506,7 @@ Port Allocation: Link creation and deletion: CleanupAliasData() - remove all link chains from lookup table - IncrementalCleanup() - look for stale links in a single chain + CleanupLink() - look for a stale link DeleteLink() - remove link AddLink() - add link ReLink() - change link @@ -529,8 +525,8 @@ static int GetNewPort(struct libalias *, struct alias_link *, int); static u_short GetSocket(struct libalias *, u_short, int *, int); #endif static void CleanupAliasData(struct libalias *); -static void IncrementalCleanup(struct libalias *); -static void DeleteLink(struct alias_link *); +static void CleanupLink(struct libalias *, struct alias_link **); +static void DeleteLink(struct alias_link **); static struct alias_link * ReLink(struct alias_link *, @@ -807,41 +803,38 @@ FindNewPortGroup(struct libalias *la, static void CleanupAliasData(struct libalias *la) { - struct alias_link *lnk; - int i; + struct alias_link *lnk, *lnk_tmp; LIBALIAS_LOCK_ASSERT(la); - for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) { - lnk = LIST_FIRST(&la->linkTableOut[i]); - while (lnk != NULL) { - struct alias_link *link_next = LIST_NEXT(lnk, list_out); - DeleteLink(lnk); - lnk = link_next; - } - } - la->cleanupIndex = 0; + /* permanent entries may stay */ + TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, list_expire, lnk_tmp) + DeleteLink(&lnk); } static void -IncrementalCleanup(struct libalias *la) +CleanupLink(struct libalias *la, struct alias_link **lnk) { - struct alias_link *lnk, *lnk_tmp; - LIBALIAS_LOCK_ASSERT(la); - LIST_FOREACH_SAFE(lnk, &la->linkTableOut[la->cleanupIndex++], - list_out, lnk_tmp) { - if (la->timeStamp - lnk->timestamp > lnk->expire_time) - DeleteLink(lnk); + + if (lnk == NULL || *lnk == NULL) + return; + + if (la->timeStamp - (*lnk)->timestamp > (*lnk)->expire_time) { + DeleteLink(lnk); + if ((*lnk) == NULL) + return; } - if (la->cleanupIndex == LINK_TABLE_OUT_SIZE) - la->cleanupIndex = 0; + /* move to end, swap may fail on a single entry list */ + TAILQ_REMOVE(&la->checkExpire, (*lnk), list_expire); + TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), list_expire); } static void -DeleteLink(struct alias_link *lnk) +DeleteLink(struct alias_link **plnk) { + struct alias_link *lnk = *plnk; struct libalias *la = lnk->la; LIBALIAS_LOCK_ASSERT(la); @@ -869,6 +862,10 @@ DeleteLink(struct alias_link *lnk) /* Adjust input table pointers */ LIST_REMOVE(lnk, list_in); + + /* remove from housekeeping */ + TAILQ_REMOVE(&la->checkExpire, lnk, list_expire); + #ifndef NO_USE_SOCKETS /* Close socket, if one has been allocated */ if (lnk->sockfd != -1) { @@ -908,6 +905,7 @@ DeleteLink(struct alias_link *lnk) /* Free memory */ free(lnk); + *plnk = NULL; /* Write statistics, if logging enabled */ if (la->packetAliasMode & PKT_ALIAS_LOG) { @@ -1039,6 +1037,9 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, /* Set up pointers for input lookup table */ start_point = StartPointIn(alias_addr, lnk->alias_port, link_type); LIST_INSERT_HEAD(&la->linkTableIn[start_point], lnk, list_in); + + /* Include the element into the housekeeping list */ + TAILQ_INSERT_TAIL(&la->checkExpire, lnk, list_expire); } else { #ifdef LIBALIAS_DEBUG fprintf(stderr, "PacketAlias/AddLink(): "); @@ -1079,7 +1080,7 @@ ReLink(struct alias_link *old_lnk, PunchFWHole(new_lnk); } #endif - DeleteLink(old_lnk); + DeleteLink(&old_lnk); return (new_lnk); } @@ -1102,12 +1103,14 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, lnk->src_port == src_port && lnk->dst_port == dst_port && lnk->link_type == link_type && - lnk->server == NULL) { - lnk->timestamp = la->timeStamp; + lnk->server == NULL) break; - } } + CleanupLink(la, &lnk); + if (lnk != NULL) + lnk->timestamp = la->timeStamp; + /* Search for partially specified links. */ if (lnk == NULL && replace_partial_links) { if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { @@ -1235,6 +1238,7 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, } } + CleanupLink(la, &lnk_fully_specified); if (lnk_fully_specified != NULL) { lnk_fully_specified->timestamp = la->timeStamp; lnk = lnk_fully_specified; @@ -1572,6 +1576,7 @@ FindPptpOutByCallId(struct libalias *la, struct in_addr src_addr, lnk->src_port == src_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -1592,6 +1597,7 @@ FindPptpOutByPeerCallId(struct libalias *la, struct in_addr src_addr, lnk->dst_port == dst_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -1612,6 +1618,7 @@ FindPptpInByCallId(struct libalias *la, struct in_addr dst_addr, lnk->dst_port == dst_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -2038,7 +2045,7 @@ SetExpire(struct alias_link *lnk, int expire) { if (expire == 0) { lnk->flags &= ~LINK_PERMANENT; - DeleteLink(lnk); + DeleteLink(&lnk); } else if (expire == -1) { lnk->flags |= LINK_PERMANENT; } else if (expire > 0) { @@ -2094,7 +2101,7 @@ SetDestCallId(struct alias_link *lnk, u_int16_t cid) void HouseKeeping(struct libalias *la) { - int i, n; + struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); #ifndef _KERNEL struct timeval tv; #endif @@ -2111,25 +2118,7 @@ HouseKeeping(struct libalias *la) gettimeofday(&tv, NULL); la->timeStamp = tv.tv_sec; #endif - - /* Compute number of spokes (output table link chains) to cover */ - n = LINK_TABLE_OUT_SIZE * (la->timeStamp - la->lastCleanupTime); - n /= ALIAS_CLEANUP_INTERVAL_SECS; - - /* Handle different cases */ - if (n > 0) { - if (n > ALIAS_CLEANUP_MAX_SPOKES) - n = ALIAS_CLEANUP_MAX_SPOKES; - la->lastCleanupTime = la->timeStamp; - for (i = 0; i < n; i++) - IncrementalCleanup(la); - } else if (n < 0) { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/HouseKeeping(): "); - fprintf(stderr, "something unexpected in time values\n"); -#endif - la->lastCleanupTime = la->timeStamp; - } + CleanupLink(la, &lnk); } /* Init the log file and enable logging */ @@ -2356,7 +2345,7 @@ LibAliasRedirectDelete(struct libalias *la, struct alias_link *lnk) { LIBALIAS_LOCK(la); la->deleteAllLinks = 1; - DeleteLink(lnk); + DeleteLink(&lnk); la->deleteAllLinks = 0; LIBALIAS_UNLOCK(la); } @@ -2426,17 +2415,16 @@ LibAliasInit(struct libalias *la) #ifdef _KERNEL la->timeStamp = time_uptime; - la->lastCleanupTime = time_uptime; #else gettimeofday(&tv, NULL); la->timeStamp = tv.tv_sec; - la->lastCleanupTime = tv.tv_sec; #endif for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) LIST_INIT(&la->linkTableOut[i]); for (i = 0; i < LINK_TABLE_IN_SIZE; i++) LIST_INIT(&la->linkTableIn[i]); + TAILQ_INIT(&la->checkExpire); #ifdef _KERNEL AliasSctpInit(la); #endif @@ -2466,8 +2454,6 @@ LibAliasInit(struct libalias *la) la->fragmentPtrLinkCount = 0; la->sockCount = 0; - la->cleanupIndex = 0; - la->packetAliasMode = PKT_ALIAS_SAME_PORTS #ifndef NO_USE_SOCKETS | PKT_ALIAS_USE_SOCKETS diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index 61cd30737ce5..fcdaa1690470 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -94,6 +94,8 @@ struct libalias { * output lookup tables. */ LIST_HEAD (, alias_link) linkTableOut[LINK_TABLE_OUT_SIZE]; LIST_HEAD (, alias_link) linkTableIn[LINK_TABLE_IN_SIZE]; + /* HouseKeeping */ + TAILQ_HEAD (, alias_link) checkExpire; /* Link statistics */ int icmpLinkCount; int udpLinkCount; @@ -103,12 +105,8 @@ struct libalias { int fragmentIdLinkCount; int fragmentPtrLinkCount; int sockCount; - /* Index to chain of link table being inspected for old links */ - int cleanupIndex; /* System time in seconds for current packet */ int timeStamp; - /* Last time IncrementalCleanup() was called */ - int lastCleanupTime; /* If equal to zero, DeleteLink() * will not remove permanent links */ int deleteAllLinks; From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 06:56:35 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E7A416512DB; Tue, 6 Jul 2021 06:56:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtdH4xF7z4Xdg; Tue, 6 Jul 2021 06:56:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8F53D15328; Tue, 6 Jul 2021 06:56:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1666uZNZ062957; Tue, 6 Jul 2021 06:56:35 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1666uZ3k062956; Tue, 6 Jul 2021 06:56:35 GMT (envelope-from git) Date: Tue, 6 Jul 2021 06:56:35 GMT Message-Id: <202107060656.1666uZ3k062956@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 390866d47eff - stable/13 - libalias: Promote per instance global variable timeStamp MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 390866d47effe8f5a11f3f852ae891f14dd4d15c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 06:56:36 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=390866d47effe8f5a11f3f852ae891f14dd4d15c commit 390866d47effe8f5a11f3f852ae891f14dd4d15c Author: Lutz Donnerhacke AuthorDate: 2021-05-25 07:07:21 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 06:55:53 +0000 libalias: Promote per instance global variable timeStamp Summary: - Use LibAliasTime as a real global variable for central timekeeping. - Reduce number of syscalls in user space considerably. - Dynamically adjust the packet counters to match the second resolution. - Only check the first few packets after a time increase for expiry. Discussed with: hselasky Differential Revision: https://reviews.freebsd.org/D30566 (cherry picked from commit ef828d39be8e4b6d922db080ed41611702dec56d) --- sys/netinet/libalias/alias_db.c | 60 +++++++++++++++++++++++++------------- sys/netinet/libalias/alias_local.h | 5 ++-- sys/netinet/libalias/alias_sctp.c | 16 +++++----- 3 files changed, 50 insertions(+), 31 deletions(-) diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index 1b1cddc0b6fd..d42948efad8d 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -176,6 +176,7 @@ __FBSDID("$FreeBSD$"); #endif static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); +int LibAliasTime; /* Constants (note: constants are also defined @@ -820,7 +821,7 @@ CleanupLink(struct libalias *la, struct alias_link **lnk) if (lnk == NULL || *lnk == NULL) return; - if (la->timeStamp - (*lnk)->timestamp > (*lnk)->expire_time) { + if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire_time) { DeleteLink(lnk); if ((*lnk) == NULL) return; @@ -940,7 +941,7 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, #endif lnk->flags = 0; lnk->pflags = 0; - lnk->timestamp = la->timeStamp; + lnk->timestamp = LibAliasTime; /* Expiration time */ switch (link_type) { @@ -1109,7 +1110,7 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, CleanupLink(la, &lnk); if (lnk != NULL) - lnk->timestamp = la->timeStamp; + lnk->timestamp = LibAliasTime; /* Search for partially specified links. */ if (lnk == NULL && replace_partial_links) { @@ -1240,7 +1241,7 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, CleanupLink(la, &lnk_fully_specified); if (lnk_fully_specified != NULL) { - lnk_fully_specified->timestamp = la->timeStamp; + lnk_fully_specified->timestamp = LibAliasTime; lnk = lnk_fully_specified; } else if (lnk_unknown_dst_port != NULL) lnk = lnk_unknown_dst_port; @@ -2101,24 +2102,45 @@ SetDestCallId(struct alias_link *lnk, u_int16_t cid) void HouseKeeping(struct libalias *la) { - struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); -#ifndef _KERNEL - struct timeval tv; -#endif + static unsigned int packets = 0; + static unsigned int packet_limit = 1000; LIBALIAS_LOCK_ASSERT(la); + packets++; + /* - * Save system time (seconds) in global variable timeStamp for use - * by other functions. This is done so as not to unnecessarily - * waste timeline by making system calls. + * User space time/gettimeofday/... is very expensive. + * Kernel space cache trashing is unnecessary. + * + * Save system time (seconds) in global variable LibAliasTime + * for use by other functions. This is done so as not to + * unnecessarily waste timeline by making system calls. + * + * Reduce the amount of house keeping work substantially by + * sampling over the packets. */ + if (packets % packet_limit == 0) { + time_t now; + #ifdef _KERNEL - la->timeStamp = time_uptime; + now = time_uptime; #else - gettimeofday(&tv, NULL); - la->timeStamp = tv.tv_sec; + now = time(NULL); #endif - CleanupLink(la, &lnk); + if (now != LibAliasTime) { + /* retry three times a second */ + packet_limit = packets / 3; + packets = 0; + LibAliasTime = now; + } + + } + /* Do a cleanup for the first packets of the new second only */ + if (packets < (la->udpLinkCount + la->tcpLinkCount)) { + struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); + + CleanupLink(la, &lnk); + } } /* Init the log file and enable logging */ @@ -2392,9 +2414,6 @@ struct libalias * LibAliasInit(struct libalias *la) { int i; -#ifndef _KERNEL - struct timeval tv; -#endif if (la == NULL) { #ifdef _KERNEL @@ -2414,10 +2433,9 @@ LibAliasInit(struct libalias *la) LIST_INSERT_HEAD(&instancehead, la, instancelist); #ifdef _KERNEL - la->timeStamp = time_uptime; + LibAliasTime = time_uptime; #else - gettimeofday(&tv, NULL); - la->timeStamp = tv.tv_sec; + LibAliasTime = time(NULL); #endif for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index 8e2fe88fe620..f1b40f0438da 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -105,8 +105,6 @@ struct libalias { unsigned int fragmentIdLinkCount; unsigned int fragmentPtrLinkCount; unsigned int sockCount; - /* System time in seconds for current packet */ - int timeStamp; /* If equal to zero, DeleteLink() * will not remove permanent links */ int deleteAllLinks; @@ -206,6 +204,9 @@ struct libalias { /* Prototypes */ +/* System time in seconds for current packet */ +extern int LibAliasTime; + /* * SctpFunction prototypes * diff --git a/sys/netinet/libalias/alias_sctp.c b/sys/netinet/libalias/alias_sctp.c index 719af1512a05..b88ad2b28ca8 100644 --- a/sys/netinet/libalias/alias_sctp.c +++ b/sys/netinet/libalias/alias_sctp.c @@ -298,10 +298,10 @@ static MALLOC_DEFINE(M_SCTPNAT, "sctpnat", "sctp nat dbs"); #define SN_MAX_TIMER 600 #define SN_TIMER_QUEUE_SIZE SN_MAX_TIMER+2 -#define SN_I_T(la) (la->timeStamp + sysctl_init_timer) /**< INIT State expiration time in seconds */ -#define SN_U_T(la) (la->timeStamp + sysctl_up_timer) /**< UP State expiration time in seconds */ -#define SN_C_T(la) (la->timeStamp + sysctl_shutdown_timer) /**< CL State expiration time in seconds */ -#define SN_X_T(la) (la->timeStamp + sysctl_holddown_timer) /**< Wait after a shutdown complete in seconds */ +#define SN_I_T(la) (LibAliasTime + sysctl_init_timer) /**< INIT State expiration time in seconds */ +#define SN_U_T(la) (LibAliasTime + sysctl_up_timer) /**< UP State expiration time in seconds */ +#define SN_C_T(la) (LibAliasTime + sysctl_shutdown_timer) /**< CL State expiration time in seconds */ +#define SN_X_T(la) (LibAliasTime + sysctl_holddown_timer) /**< Wait after a shutdown complete in seconds */ /** @} * @defgroup sysctl SysCtl Variable and callback function declarations * @@ -667,9 +667,9 @@ AliasSctpInit(struct libalias *la) for (i = 0; i < SN_TIMER_QUEUE_SIZE; i++) LIST_INIT(&la->sctpNatTimer.TimerQ[i]); #ifdef _KERNEL - la->sctpNatTimer.loc_time=time_uptime; /* la->timeStamp is not set yet */ + la->sctpNatTimer.loc_time=time_uptime; /* LibAliasTime is not set yet */ #else - la->sctpNatTimer.loc_time=la->timeStamp; + la->sctpNatTimer.loc_time=LibAliasTime; #endif la->sctpNatTimer.cur_loc = 0; la->sctpLinkCount = 0; @@ -2493,12 +2493,12 @@ sctp_CheckTimers(struct libalias *la) struct sctp_nat_assoc *assoc; LIBALIAS_LOCK_ASSERT(la); - while(la->timeStamp >= la->sctpNatTimer.loc_time) { + while(LibAliasTime >= la->sctpNatTimer.loc_time) { while (!LIST_EMPTY(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc])) { assoc = LIST_FIRST(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc]); //SLIST_REMOVE_HEAD(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc], timer_Q); LIST_REMOVE(assoc, timer_Q); - if (la->timeStamp >= assoc->exp) { /* state expired */ + if (LibAliasTime >= assoc->exp) { /* state expired */ SN_LOG(((assoc->state == SN_CL) ? (SN_LOG_DEBUG) : (SN_LOG_INFO)), logsctperror("Timer Expired", assoc->g_vtag, assoc->state, SN_TO_NODIR)); RmSctpAssoc(la, assoc); From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 06:56:34 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BE5F765148A; Tue, 6 Jul 2021 06:56:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtdG40QGz4XpF; Tue, 6 Jul 2021 06:56:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 71A251522E; Tue, 6 Jul 2021 06:56:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1666uYL5062933; Tue, 6 Jul 2021 06:56:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1666uYMm062932; Tue, 6 Jul 2021 06:56:34 GMT (envelope-from git) Date: Tue, 6 Jul 2021 06:56:34 GMT Message-Id: <202107060656.1666uYMm062932@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 69965155a5c2 - stable/13 - libalias: Stats are unsigned MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 69965155a5c24c7794e61fc5a91ca3ba23468f50 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 06:56:34 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=69965155a5c24c7794e61fc5a91ca3ba23468f50 commit 69965155a5c24c7794e61fc5a91ca3ba23468f50 Author: Lutz Donnerhacke AuthorDate: 2021-05-31 22:08:24 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 06:55:52 +0000 libalias: Stats are unsigned Stats counters are used as unsigned valued (i.e. printf("%u")) but are defined as signed int. This causes trouble later, so fix it early. Differential Revision: https://reviews.freebsd.org/D30587 (cherry picked from commit 3fd20a79e7b55d135bbc5fa197980fe0e9ee0f14) --- sys/netinet/libalias/alias_local.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index fcdaa1690470..8e2fe88fe620 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -97,14 +97,14 @@ struct libalias { /* HouseKeeping */ TAILQ_HEAD (, alias_link) checkExpire; /* Link statistics */ - int icmpLinkCount; - int udpLinkCount; - int tcpLinkCount; - int pptpLinkCount; - int protoLinkCount; - int fragmentIdLinkCount; - int fragmentPtrLinkCount; - int sockCount; + unsigned int icmpLinkCount; + unsigned int udpLinkCount; + unsigned int tcpLinkCount; + unsigned int pptpLinkCount; + unsigned int protoLinkCount; + unsigned int fragmentIdLinkCount; + unsigned int fragmentPtrLinkCount; + unsigned int sockCount; /* System time in seconds for current packet */ int timeStamp; /* If equal to zero, DeleteLink() From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 06:56:37 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1999B65135A; Tue, 6 Jul 2021 06:56:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtdJ5plJz4Xdn; Tue, 6 Jul 2021 06:56:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AD7AF15453; Tue, 6 Jul 2021 06:56:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1666uax7062988; Tue, 6 Jul 2021 06:56:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1666uacA062987; Tue, 6 Jul 2021 06:56:36 GMT (envelope-from git) Date: Tue, 6 Jul 2021 06:56:36 GMT Message-Id: <202107060656.1666uacA062987@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 78d515b2225e - stable/13 - libalias: Restructure MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 78d515b2225ea33aed2d2dc5976ccf4f44cc6a3c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 06:56:37 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=78d515b2225ea33aed2d2dc5976ccf4f44cc6a3c commit 78d515b2225ea33aed2d2dc5976ccf4f44cc6a3c Author: Lutz Donnerhacke AuthorDate: 2021-05-25 13:03:34 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 06:55:53 +0000 libalias: Restructure Clean up the database handling in order to switch to more efficient data structures. The development of this patch was artificially split in to many small steps to ease reviewing. - Common search terms - Separate fully qualified search - Separate table for partial links - Cleanup _FindLinkIn - Factor out the outgoing search function - Factor out a common idiom to return found links - Reorder incoming links by grouping of common search terms - Remove LSNAT from outgoing search - Group internal structure semantically - Separate table for PPTP - Use AliasRange instead of PORT_BASE - Remove temporary state deleteAllLinks from global struct - Avoid uninitialized expiration Discussed with: Dimitry Luhtionov Differential Revision: https://reviews.freebsd.org/D30568 Differential Revision: https://reviews.freebsd.org/D30569 Differential Revision: https://reviews.freebsd.org/D30570 Differential Revision: https://reviews.freebsd.org/D30571 Differential Revision: https://reviews.freebsd.org/D30572 Differential Revision: https://reviews.freebsd.org/D30573 Differential Revision: https://reviews.freebsd.org/D30574 Differential Revision: https://reviews.freebsd.org/D30575 Differential Revision: https://reviews.freebsd.org/D30580 Differential Revision: https://reviews.freebsd.org/D30581 Differential Revision: https://reviews.freebsd.org/D30604 Differential Revision: https://reviews.freebsd.org/D30582 (cherry picked from commit d41044ddfdbc2a026570cae11d5fc2e18f5e4b92) (cherry picked from commit 32f9c2ceb3ec8266e48ec0f6d1556fd98ef12db6) (cherry picked from commit cac129e6030095c33e95c5ce1cdcb9c5c21efce9) (cherry picked from commit 19dcc4f2250b3850991366e3058ca8a64b2f135d) (cherry picked from commit d5419034381d0e8e8c99f0f678118b197144bfff) (cherry picked from commit d4ab07d2aeb13b1aae9ad5d73e5fe77131f2350c) (cherry picked from commit 492d3b7109b45ff30fd199b7eeed797447746e42) (cherry picked from commit 7b44ff4c52ce9e7a89e772566a249e5481fe0ac4) (cherry picked from commit 1178dda53d10b096e0b99e356d15b7c24390e099) (cherry picked from commit 9efcad61d8309ecad3c15392b277fd329a1e45e4) (cherry picked from commit fe83900f9fa931e3d0942ec9c0709896b887c3d7) (cherry picked from commit d989935b5bcd880353f0de89eda958c45e7e3342) (cherry picked from commit b50a4dce185481bebf8096c27588ae04a4a2fd7c) (cherry picked from commit f28455344483310cfd1aa5c0bdd4d014810c0e32) --- sys/netinet/libalias/HISTORY | 4 + sys/netinet/libalias/alias_db.c | 897 ++++++++++++++++++++----------------- sys/netinet/libalias/alias_local.h | 16 +- 3 files changed, 497 insertions(+), 420 deletions(-) diff --git a/sys/netinet/libalias/HISTORY b/sys/netinet/libalias/HISTORY index c5bca59cac1f..8965290a4f27 100644 --- a/sys/netinet/libalias/HISTORY +++ b/sys/netinet/libalias/HISTORY @@ -143,3 +143,7 @@ Version 3.1: May, 2000 (Erik Salander, erik@whistle.com) Version 3.2: July, 2000 (Erik Salander, erik@whistle.com and Junichi Satoh, junichi@junichi.org) - Added support for streaming media (RTSP and PNA) aliasing. + +Version 3.3: May 2021 (donner) + - Dropped LibAliasCheckNewLink + - Refactor the database handling, perfomance improvements. diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index d42948efad8d..b056f55b56a5 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -318,14 +318,23 @@ struct alias_link { #define LINK_UNFIREWALLED 0x08 int timestamp; /* Time link was last accessed */ - int expire_time; /* Expire time for link */ #ifndef NO_USE_SOCKETS int sockfd; /* socket descriptor */ #endif /* Linked list of pointers for input and output lookup tables */ - LIST_ENTRY (alias_link) list_out; - LIST_ENTRY (alias_link) list_in; - TAILQ_ENTRY (alias_link) list_expire; + union { + struct { + LIST_ENTRY(alias_link) in; + LIST_ENTRY(alias_link) out; + } all; + struct { + LIST_ENTRY(alias_link) list; + } pptp; + }; + struct { + TAILQ_ENTRY(alias_link) list; + int time; /* Expire time for link */ + } expire; /* Auxiliary data */ union { char *frag_ptr; @@ -378,12 +387,10 @@ Miscellaneous: */ /* Local prototypes */ -static u_int StartPointIn(struct in_addr, u_short, int); - -static u_int -StartPointOut(struct in_addr, struct in_addr, - u_short, u_short, int); - +static struct group_in * +StartPointIn(struct libalias *, struct in_addr, u_short, int, int); +static u_int +StartPointOut(struct in_addr, struct in_addr, u_short, u_short, int); static int SeqDiff(u_long, u_long); #ifndef NO_FW_PUNCH @@ -401,19 +408,46 @@ static void UninitPacketAliasLog(struct libalias *); void SctpShowAliasStats(struct libalias *la); -static u_int -StartPointIn(struct in_addr alias_addr, - u_short alias_port, - int link_type) +#define INGUARD \ + if (grp->alias_port != alias_port || \ + grp->link_type != link_type || \ + grp->alias_addr.s_addr != alias_addr.s_addr) \ + continue; + +static struct group_in * +StartPointIn(struct libalias *la, + struct in_addr alias_addr, u_short alias_port, int link_type, + int create) { u_int n; + struct group_in *grp, *tmp; n = alias_addr.s_addr; - if (link_type != LINK_PPTP) - n += alias_port; + n += alias_port; n += link_type; - return (n % LINK_TABLE_IN_SIZE); -} + n %= LINK_TABLE_IN_SIZE; + + LIST_FOREACH_SAFE(grp, &la->groupTableIn[n], group_in, tmp) { + /* Auto cleanup */ + if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { + LIST_REMOVE(grp, group_in); + free(grp); + } else { + INGUARD; + return (grp); + } + } + if (!create || (grp = malloc(sizeof(*grp))) == NULL) + return (grp); + grp->alias_addr = alias_addr; + grp->alias_port = alias_port; + grp->link_type = link_type; + LIST_INIT(&grp->full); + LIST_INIT(&grp->partial); + LIST_INSERT_HEAD(&la->groupTableIn[n], grp, group_in); + return (grp); +} +#undef INGUARD static u_int StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, @@ -423,10 +457,8 @@ StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, n = src_addr.s_addr; n += dst_addr.s_addr; - if (link_type != LINK_PPTP) { - n += src_port; - n += dst_port; - } + n += src_port; + n += dst_port; n += link_type; return (n % LINK_TABLE_OUT_SIZE); @@ -525,14 +557,16 @@ static int GetNewPort(struct libalias *, struct alias_link *, int); #ifndef NO_USE_SOCKETS static u_short GetSocket(struct libalias *, u_short, int *, int); #endif -static void CleanupAliasData(struct libalias *); -static void CleanupLink(struct libalias *, struct alias_link **); -static void DeleteLink(struct alias_link **); +static void CleanupAliasData(struct libalias *, int); +static void CleanupLink(struct libalias *, struct alias_link **, int); +static void DeleteLink(struct alias_link **, int); +static struct alias_link * +UseLink(struct libalias *, struct alias_link *); static struct alias_link * ReLink(struct alias_link *, struct in_addr, struct in_addr, struct in_addr, - u_short, u_short, int, int); + u_short, u_short, int, int, int); static struct alias_link * FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); @@ -540,12 +574,20 @@ FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, static struct alias_link * FindLinkIn(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); -#define ALIAS_PORT_BASE 0x08000 -#define ALIAS_PORT_MASK 0x07fff -#define ALIAS_PORT_MASK_EVEN 0x07ffe +static u_short _RandomPort(struct libalias *la); + #define GET_NEW_PORT_MAX_ATTEMPTS 20 -#define FIND_EVEN_ALIAS_BASE 1 +/* get random port in network byte order */ +static u_short +_RandomPort(struct libalias *la) { + u_short port; + + port = la->aliasPortLower + + arc4random_uniform(la->aliasPortLength); + + return ntohs(port); +} /* GetNewPort() allocates port numbers. Note that if a port number is already in use, that does not mean that it cannot be used by @@ -557,8 +599,7 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) { int i; int max_trials; - u_short port_sys; - u_short port_net; + u_short port; LIBALIAS_LOCK_ASSERT(la); /* @@ -566,41 +607,18 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) * this parameter is zero or positive, it precisely specifies * the port number. GetNewPort() will return this number * without check that it is in use. - + * + * The aliasing port is automatically selected by one of + * two methods below: + * * When this parameter is GET_ALIAS_PORT, it indicates to get * a randomly selected port number. */ - if (alias_port_param == GET_ALIAS_PORT) { - /* - * The aliasing port is automatically selected by one of - * two methods below: - */ - max_trials = GET_NEW_PORT_MAX_ATTEMPTS; - - if (la->packetAliasMode & PKT_ALIAS_SAME_PORTS) { - /* - * When the PKT_ALIAS_SAME_PORTS option is chosen, - * the first try will be the actual source port. If - * this is already in use, the remainder of the - * trials will be random. - */ - port_net = lnk->src_port; - port_sys = ntohs(port_net); - } else if (la->aliasPortLower) { - /* First trial is a random port in the aliasing range. */ - port_sys = la->aliasPortLower + - (arc4random() % la->aliasPortLength); - port_net = htons(port_sys); - } else { - /* First trial and all subsequent are random. */ - port_sys = arc4random() & ALIAS_PORT_MASK; - port_sys += ALIAS_PORT_BASE; - port_net = htons(port_sys); - } - } else if (alias_port_param >= 0 && alias_port_param < 0x10000) { + if (alias_port_param >= 0 && alias_port_param < 0x10000) { lnk->alias_port = (u_short) alias_port_param; return (0); - } else { + } + if (alias_port_param != GET_ALIAS_PORT) { #ifdef LIBALIAS_DEBUG fprintf(stderr, "PacketAlias/GetNewPort(): "); fprintf(stderr, "input parameter error\n"); @@ -608,58 +626,57 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) return (-1); } + max_trials = GET_NEW_PORT_MAX_ATTEMPTS; + + /* + * When the PKT_ALIAS_SAME_PORTS option is chosen, + * the first try will be the actual source port. If + * this is already in use, the remainder of the + * trials will be random. + */ + port = (la->packetAliasMode & PKT_ALIAS_SAME_PORTS) + ? lnk->src_port + : _RandomPort(la); + /* Port number search */ - for (i = 0; i < max_trials; i++) { - int go_ahead; + for (i = 0; i < max_trials; i++, port = _RandomPort(la)) { + struct group_in *grp; struct alias_link *search_result; - search_result = FindLinkIn(la, lnk->dst_addr, lnk->alias_addr, - lnk->dst_port, port_net, - lnk->link_type, 0); + grp = StartPointIn(la, lnk->alias_addr, port, lnk->link_type, 0); + if (grp == NULL) + break; + LIST_FOREACH(search_result, &grp->full, all.in) { + if (lnk->dst_addr.s_addr == search_result->dst_addr.s_addr && + lnk->dst_port == search_result->dst_port) + break; /* found match */ + } if (search_result == NULL) - go_ahead = 1; - else if (!(lnk->flags & LINK_PARTIALLY_SPECIFIED) - && (search_result->flags & LINK_PARTIALLY_SPECIFIED)) - go_ahead = 1; - else - go_ahead = 0; + break; + } - if (go_ahead) { -#ifndef NO_USE_SOCKETS - if ((la->packetAliasMode & PKT_ALIAS_USE_SOCKETS) - && (lnk->flags & LINK_PARTIALLY_SPECIFIED) - && ((lnk->link_type == LINK_TCP) || - (lnk->link_type == LINK_UDP))) { - if (GetSocket(la, port_net, &lnk->sockfd, lnk->link_type)) { - lnk->alias_port = port_net; - return (0); - } - } else { + if (i >= max_trials) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/GetNewPort(): "); + fprintf(stderr, "could not find free port\n"); #endif - lnk->alias_port = port_net; - return (0); + return (-1); + } + #ifndef NO_USE_SOCKETS - } -#endif - } - if (la->aliasPortLower) { - port_sys = la->aliasPortLower + - (arc4random() % la->aliasPortLength); - port_net = htons(port_sys); - } else { - port_sys = arc4random() & ALIAS_PORT_MASK; - port_sys += ALIAS_PORT_BASE; - port_net = htons(port_sys); + if ((la->packetAliasMode & PKT_ALIAS_USE_SOCKETS) && + (lnk->flags & LINK_PARTIALLY_SPECIFIED) && + ((lnk->link_type == LINK_TCP) || + (lnk->link_type == LINK_UDP))) { + if (!GetSocket(la, port, &lnk->sockfd, lnk->link_type)) { + return (-1); } } - -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/GetNewPort(): "); - fprintf(stderr, "could not find free port\n"); #endif + lnk->alias_port = port; - return (-1); + return (0); } #ifndef NO_USE_SOCKETS @@ -726,7 +743,7 @@ FindNewPortGroup(struct libalias *la, { int i, j; int max_trials; - u_short port_sys; + u_short port; int link_type; LIBALIAS_LOCK_ASSERT(la); @@ -758,39 +775,31 @@ FindNewPortGroup(struct libalias *la, * try will be the actual source port. If this is already * in use, the remainder of the trials will be random. */ - port_sys = ntohs(src_port); + port = src_port; } else { - /* First trial and all subsequent are random. */ - if (align == FIND_EVEN_ALIAS_BASE) - port_sys = arc4random() & ALIAS_PORT_MASK_EVEN; - else - port_sys = arc4random() & ALIAS_PORT_MASK; - - port_sys += ALIAS_PORT_BASE; + port = _RandomPort(la); } /* Port number search */ - for (i = 0; i < max_trials; i++) { + for (i = 0; i < max_trials; i++, port = _RandomPort(la)) { struct alias_link *search_result; - for (j = 0; j < port_count; j++) + if (align) + port &= htons(0xfffe); + + for (j = 0; j < port_count; j++) { + u_short port_j = ntohs(port) + j; + if ((search_result = FindLinkIn(la, dst_addr, - alias_addr, dst_port, htons(port_sys + j), + alias_addr, dst_port, htons(port_j), link_type, 0)) != NULL) break; + } /* Found a good range, return base */ if (j == port_count) - return (htons(port_sys)); - - /* Find a new base to try */ - if (align == FIND_EVEN_ALIAS_BASE) - port_sys = arc4random() & ALIAS_PORT_MASK_EVEN; - else - port_sys = arc4random() & ALIAS_PORT_MASK; - - port_sys += ALIAS_PORT_BASE; + return (port); } #ifdef LIBALIAS_DEBUG @@ -802,45 +811,65 @@ FindNewPortGroup(struct libalias *la, } static void -CleanupAliasData(struct libalias *la) +CleanupAliasData(struct libalias *la, int deletePermanent) { struct alias_link *lnk, *lnk_tmp; + u_int i; LIBALIAS_LOCK_ASSERT(la); /* permanent entries may stay */ - TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, list_expire, lnk_tmp) - DeleteLink(&lnk); + TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, expire.list, lnk_tmp) + DeleteLink(&lnk, deletePermanent); + + for (i = 0; i < LINK_TABLE_IN_SIZE; i++) { + struct group_in *grp, *grp_tmp; + + LIST_FOREACH_SAFE(grp, &la->groupTableIn[i], group_in, grp_tmp) + if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { + LIST_REMOVE(grp, group_in); + free(grp); + } + } } static void -CleanupLink(struct libalias *la, struct alias_link **lnk) +CleanupLink(struct libalias *la, struct alias_link **lnk, int deletePermanent) { LIBALIAS_LOCK_ASSERT(la); if (lnk == NULL || *lnk == NULL) return; - if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire_time) { - DeleteLink(lnk); + if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire.time) { + DeleteLink(lnk, deletePermanent); if ((*lnk) == NULL) return; } /* move to end, swap may fail on a single entry list */ - TAILQ_REMOVE(&la->checkExpire, (*lnk), list_expire); - TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), list_expire); + TAILQ_REMOVE(&la->checkExpire, (*lnk), expire.list); + TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), expire.list); +} + +static struct alias_link * +UseLink(struct libalias *la, struct alias_link *lnk) +{ + CleanupLink(la, &lnk, 0); + if (lnk != NULL) + lnk->timestamp = LibAliasTime; + return (lnk); } static void -DeleteLink(struct alias_link **plnk) +DeleteLink(struct alias_link **plnk, int deletePermanent) { struct alias_link *lnk = *plnk; struct libalias *la = lnk->la; LIBALIAS_LOCK_ASSERT(la); /* Don't do anything if the link is marked permanent */ - if (la->deleteAllLinks == 0 && lnk->flags & LINK_PERMANENT) + if (!deletePermanent && (lnk->flags & LINK_PERMANENT)) return; #ifndef NO_FW_PUNCH @@ -848,24 +877,32 @@ DeleteLink(struct alias_link **plnk) ClearFWHole(lnk); #endif - /* Free memory allocated for LSNAT server pool */ - if (lnk->server != NULL) { - struct server *head, *curr, *next; + switch (lnk->link_type) { + case LINK_PPTP: + LIST_REMOVE(lnk, pptp.list); + break; + default: + /* Free memory allocated for LSNAT server pool */ + if (lnk->server != NULL) { + struct server *head, *curr, *next; + + head = curr = lnk->server; + do { + next = curr->next; + free(curr); + } while ((curr = next) != head); + } else { + /* Adjust output table pointers */ + LIST_REMOVE(lnk, all.out); + } - head = curr = lnk->server; - do { - next = curr->next; - free(curr); - } while ((curr = next) != head); + /* Adjust input table pointers */ + LIST_REMOVE(lnk, all.in); + break; } - /* Adjust output table pointers */ - LIST_REMOVE(lnk, list_out); - - /* Adjust input table pointers */ - LIST_REMOVE(lnk, list_in); /* remove from housekeeping */ - TAILQ_REMOVE(&la->checkExpire, lnk, list_expire); + TAILQ_REMOVE(&la->checkExpire, lnk, expire.list); #ifndef NO_USE_SOCKETS /* Close socket, if one has been allocated */ @@ -923,133 +960,148 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); + lnk = malloc(sizeof(struct alias_link)); - if (lnk != NULL) { - /* Basic initialization */ - lnk->la = la; - lnk->src_addr = src_addr; - lnk->dst_addr = dst_addr; - lnk->alias_addr = alias_addr; - lnk->proxy_addr.s_addr = INADDR_ANY; - lnk->src_port = src_port; - lnk->dst_port = dst_port; - lnk->proxy_port = 0; - lnk->server = NULL; - lnk->link_type = link_type; + if (lnk == NULL) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/AddLink(): "); + fprintf(stderr, "malloc() call failed.\n"); +#endif + return (NULL); + } + /* Basic initialization */ + lnk->la = la; + lnk->src_addr = src_addr; + lnk->dst_addr = dst_addr; + lnk->alias_addr = alias_addr; + lnk->proxy_addr.s_addr = INADDR_ANY; + lnk->src_port = src_port; + lnk->dst_port = dst_port; + lnk->proxy_port = 0; + lnk->server = NULL; + lnk->link_type = link_type; #ifndef NO_USE_SOCKETS - lnk->sockfd = -1; + lnk->sockfd = -1; #endif - lnk->flags = 0; - lnk->pflags = 0; - lnk->timestamp = LibAliasTime; + lnk->flags = 0; + lnk->pflags = 0; + lnk->timestamp = LibAliasTime; - /* Expiration time */ - switch (link_type) { - case LINK_ICMP: - lnk->expire_time = ICMP_EXPIRE_TIME; - break; - case LINK_UDP: - lnk->expire_time = UDP_EXPIRE_TIME; - break; - case LINK_TCP: - lnk->expire_time = TCP_EXPIRE_INITIAL; - break; - case LINK_PPTP: - lnk->flags |= LINK_PERMANENT; /* no timeout. */ - break; - case LINK_FRAGMENT_ID: - lnk->expire_time = FRAGMENT_ID_EXPIRE_TIME; - break; - case LINK_FRAGMENT_PTR: - lnk->expire_time = FRAGMENT_PTR_EXPIRE_TIME; - break; - case LINK_ADDR: - break; - default: - lnk->expire_time = PROTO_EXPIRE_TIME; - break; - } + /* Expiration time */ + switch (link_type) { + case LINK_ICMP: + lnk->expire.time = ICMP_EXPIRE_TIME; + break; + case LINK_UDP: + lnk->expire.time = UDP_EXPIRE_TIME; + break; + case LINK_TCP: + lnk->expire.time = TCP_EXPIRE_INITIAL; + break; + case LINK_FRAGMENT_ID: + lnk->expire.time = FRAGMENT_ID_EXPIRE_TIME; + break; + case LINK_FRAGMENT_PTR: + lnk->expire.time = FRAGMENT_PTR_EXPIRE_TIME; + break; + default: + lnk->expire.time = PROTO_EXPIRE_TIME; + break; + } + + /* Determine alias flags */ + if (dst_addr.s_addr == INADDR_ANY) + lnk->flags |= LINK_UNKNOWN_DEST_ADDR; + if (dst_port == 0) + lnk->flags |= LINK_UNKNOWN_DEST_PORT; - /* Determine alias flags */ - if (dst_addr.s_addr == INADDR_ANY) - lnk->flags |= LINK_UNKNOWN_DEST_ADDR; - if (dst_port == 0) - lnk->flags |= LINK_UNKNOWN_DEST_PORT; + /* Determine alias port */ + if (GetNewPort(la, lnk, alias_port_param) != 0) { + free(lnk); + return (NULL); + } + /* Link-type dependent initialization */ + switch (link_type) { + case LINK_ICMP: + la->icmpLinkCount++; + break; + case LINK_UDP: + la->udpLinkCount++; + break; + case LINK_TCP: { + struct tcp_dat *aux_tcp; + int i; - /* Determine alias port */ - if (GetNewPort(la, lnk, alias_port_param) != 0) { + aux_tcp = malloc(sizeof(struct tcp_dat)); + if (aux_tcp == NULL) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/AddLink: "); + fprintf(stderr, " cannot allocate auxiliary TCP data\n"); +#endif free(lnk); return (NULL); } - /* Link-type dependent initialization */ - switch (link_type) { - struct tcp_dat *aux_tcp; - case LINK_ICMP: - la->icmpLinkCount++; - break; - case LINK_UDP: - la->udpLinkCount++; - break; - case LINK_TCP: - aux_tcp = malloc(sizeof(struct tcp_dat)); - if (aux_tcp != NULL) { - int i; - - la->tcpLinkCount++; - aux_tcp->state.in = ALIAS_TCP_STATE_NOT_CONNECTED; - aux_tcp->state.out = ALIAS_TCP_STATE_NOT_CONNECTED; - aux_tcp->state.index = 0; - aux_tcp->state.ack_modified = 0; - for (i = 0; i < N_LINK_TCP_DATA; i++) - aux_tcp->ack[i].active = 0; - aux_tcp->fwhole = -1; - lnk->data.tcp = aux_tcp; - } else { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/AddLink: "); - fprintf(stderr, " cannot allocate auxiliary TCP data\n"); -#endif - free(lnk); - return (NULL); - } - break; - case LINK_PPTP: - la->pptpLinkCount++; - break; - case LINK_FRAGMENT_ID: - la->fragmentIdLinkCount++; - break; - case LINK_FRAGMENT_PTR: - la->fragmentPtrLinkCount++; - break; - case LINK_ADDR: - break; - default: - la->protoLinkCount++; - break; + la->tcpLinkCount++; + aux_tcp->state.in = ALIAS_TCP_STATE_NOT_CONNECTED; + aux_tcp->state.out = ALIAS_TCP_STATE_NOT_CONNECTED; + aux_tcp->state.index = 0; + aux_tcp->state.ack_modified = 0; + for (i = 0; i < N_LINK_TCP_DATA; i++) + aux_tcp->ack[i].active = 0; + aux_tcp->fwhole = -1; + lnk->data.tcp = aux_tcp; + } + break; + case LINK_PPTP: + la->pptpLinkCount++; + break; + case LINK_FRAGMENT_ID: + la->fragmentIdLinkCount++; + break; + case LINK_FRAGMENT_PTR: + la->fragmentPtrLinkCount++; + break; + case LINK_ADDR: + break; + default: + la->protoLinkCount++; + break; + } + + switch (link_type) { + case LINK_PPTP: + LIST_INSERT_HEAD(&la->pptpList, lnk, pptp.list); + break; + default: { + struct group_in *grp; + + grp = StartPointIn(la, alias_addr, lnk->alias_port, link_type, 1); + if (grp == NULL) { + free(lnk); + return (NULL); } /* Set up pointers for output lookup table */ start_point = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, list_out); + LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, all.out); /* Set up pointers for input lookup table */ - start_point = StartPointIn(alias_addr, lnk->alias_port, link_type); - LIST_INSERT_HEAD(&la->linkTableIn[start_point], lnk, list_in); - - /* Include the element into the housekeeping list */ - TAILQ_INSERT_TAIL(&la->checkExpire, lnk, list_expire); - } else { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/AddLink(): "); - fprintf(stderr, "malloc() call failed.\n"); -#endif + if (lnk->flags & LINK_PARTIALLY_SPECIFIED) + LIST_INSERT_HEAD(&grp->partial, lnk, all.in); + else + LIST_INSERT_HEAD(&grp->full, lnk, all.in); } - if (la->packetAliasMode & PKT_ALIAS_LOG) { - ShowAliasStats(la); + break; } + + /* Include the element into the housekeeping list */ + TAILQ_INSERT_TAIL(&la->checkExpire, lnk, expire.list); + + if (la->packetAliasMode & PKT_ALIAS_LOG) + ShowAliasStats(la); + return (lnk); } @@ -1065,7 +1117,8 @@ ReLink(struct alias_link *old_lnk, u_short src_port, u_short dst_port, int alias_port_param, - int link_type) + int link_type, + int deletePermanent) { struct alias_link *new_lnk; struct libalias *la = old_lnk->la; @@ -1081,10 +1134,39 @@ ReLink(struct alias_link *old_lnk, PunchFWHole(new_lnk); } #endif - DeleteLink(&old_lnk); + DeleteLink(&old_lnk, deletePermanent); return (new_lnk); } + +#define OUTGUARD \ + if (lnk->src_port != src_port || \ + lnk->src_addr.s_addr != src_addr.s_addr || \ + lnk->dst_addr.s_addr != dst_addr.s_addr || \ + lnk->dst_port != dst_port || \ + lnk->link_type != link_type) \ + continue; + +static struct alias_link * +_SearchLinkOut(struct libalias *la, struct in_addr src_addr, + struct in_addr dst_addr, + u_short src_port, + u_short dst_port, + int link_type) { + u_int i; + struct alias_link *lnk; + + i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); + LIST_FOREACH(lnk, &la->linkTableOut[i], all.out) { + OUTGUARD; + return (UseLink(la, lnk)); + } + + return (NULL); +} + +#undef OUTGUARD + static struct alias_link * _FindLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, @@ -1093,45 +1175,31 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, int link_type, int replace_partial_links) { - u_int i; struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); - i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_FOREACH(lnk, &la->linkTableOut[i], list_out) { - if (lnk->dst_addr.s_addr == dst_addr.s_addr && - lnk->src_addr.s_addr == src_addr.s_addr && - lnk->src_port == src_port && - lnk->dst_port == dst_port && - lnk->link_type == link_type && - lnk->server == NULL) - break; - } - - CleanupLink(la, &lnk); - if (lnk != NULL) - lnk->timestamp = LibAliasTime; + lnk = _SearchLinkOut(la, src_addr, dst_addr, src_port, dst_port, link_type); + if (lnk != NULL || !replace_partial_links) + return (lnk); /* Search for partially specified links. */ - if (lnk == NULL && replace_partial_links) { - if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { - lnk = _FindLinkOut(la, src_addr, dst_addr, src_port, 0, - link_type, 0); - if (lnk == NULL) - lnk = _FindLinkOut(la, src_addr, ANY_ADDR, src_port, - dst_port, link_type, 0); - } - if (lnk == NULL && - (dst_port != 0 || dst_addr.s_addr != INADDR_ANY)) { - lnk = _FindLinkOut(la, src_addr, ANY_ADDR, src_port, 0, - link_type, 0); - } - if (lnk != NULL) { - lnk = ReLink(lnk, - src_addr, dst_addr, lnk->alias_addr, - src_port, dst_port, lnk->alias_port, - link_type); - } + if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { + lnk = _SearchLinkOut(la, src_addr, dst_addr, src_port, 0, + link_type); + if (lnk == NULL) + lnk = _SearchLinkOut(la, src_addr, ANY_ADDR, src_port, + dst_port, link_type); + } + if (lnk == NULL && + (dst_port != 0 || dst_addr.s_addr != INADDR_ANY)) { + lnk = _SearchLinkOut(la, src_addr, ANY_ADDR, src_port, 0, + link_type); + } + if (lnk != NULL) { + lnk = ReLink(lnk, + src_addr, dst_addr, lnk->alias_addr, + src_port, dst_port, lnk->alias_port, + link_type, 0); } return (lnk); } @@ -1175,16 +1243,16 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, int replace_partial_links) { int flags_in; - u_int start_point; + struct group_in *grp; struct alias_link *lnk; - struct alias_link *lnk_fully_specified; struct alias_link *lnk_unknown_all; struct alias_link *lnk_unknown_dst_addr; struct alias_link *lnk_unknown_dst_port; + struct in_addr src_addr; + u_short src_port; LIBALIAS_LOCK_ASSERT(la); /* Initialize pointers */ - lnk_fully_specified = NULL; lnk_unknown_all = NULL; lnk_unknown_dst_addr = NULL; lnk_unknown_dst_port = NULL; @@ -1198,83 +1266,82 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, flags_in |= LINK_UNKNOWN_DEST_PORT; /* Search loop */ - start_point = StartPointIn(alias_addr, alias_port, link_type); - LIST_FOREACH(lnk, &la->linkTableIn[start_point], list_in) { - int flags; - - flags = flags_in | lnk->flags; - if (!(flags & LINK_PARTIALLY_SPECIFIED)) { - if (lnk->alias_addr.s_addr == alias_addr.s_addr - && lnk->alias_port == alias_port - && lnk->dst_addr.s_addr == dst_addr.s_addr *** 448 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 06:56:38 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CCE2265149A; Tue, 6 Jul 2021 06:56:38 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtdK7307z4XmF; Tue, 6 Jul 2021 06:56:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CEEB514DDF; Tue, 6 Jul 2021 06:56:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1666ubEX063012; Tue, 6 Jul 2021 06:56:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1666ubcW063011; Tue, 6 Jul 2021 06:56:37 GMT (envelope-from git) Date: Tue, 6 Jul 2021 06:56:37 GMT Message-Id: <202107060656.1666ubcW063011@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 3a96a25da861 - stable/13 - libalias: Switch to SPLAY trees MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 3a96a25da8614d27f717ba8d29d32bafb04a70e8 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 06:56:38 -0000 The branch stable/13 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=3a96a25da8614d27f717ba8d29d32bafb04a70e8 commit 3a96a25da8614d27f717ba8d29d32bafb04a70e8 Author: Lutz Donnerhacke AuthorDate: 2021-05-27 21:42:54 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 06:55:53 +0000 libalias: Switch to SPLAY trees Current data structure is using a hash of unordered lists. Those unordered lists are quite efficient, because the least recently inserted entries are most likely to be used again. In order to avoid long search times in other cases, the lists are hashed into many buckets. Unfortunatly a search for a miss needs an exhaustive inspection and a careful definition of the hash. Splay trees offer a similar feature - almost O(1) for access of the least recently used entries), and amortized O(ln(n) - for almost all other cases. Get rid of the hash. Now the data structure should able to quickly react to external packets without eating CPU cycles for breakfast, preventing a DoS. PR: 192888 Discussed with: Dimitry Luhtionov Differential Revision: https://reviews.freebsd.org/D30516 Differential Revision: https://reviews.freebsd.org/D30536 Differential Revision: https://reviews.freebsd.org/D30844 (cherry picked from commit 935fc93af157dee352eb4b6c83f8a2a9e7fd9a4e) (cherry picked from commit d261e57deacb0d00d9e827447f235df83dda3e3a) (cherry picked from commit f70c98a2f5d993dc518efd606aa341eda99400ef) (cherry picked from commit 25392fac9488bcae5c451500df2e2945430484a6) (cherry picked from commit 2f4d91f9cb22fc65eb65407e8118b433a5d71976) (cherry picked from commit 4060e77f49d1b9fd2254f3f4da94fd64fce83f72) --- sys/netinet/libalias/HISTORY | 3 +- sys/netinet/libalias/alias_db.c | 502 +++---------------------------------- sys/netinet/libalias/alias_db.h | 443 ++++++++++++++++++++++++++++++++ sys/netinet/libalias/alias_local.h | 10 +- 4 files changed, 489 insertions(+), 469 deletions(-) diff --git a/sys/netinet/libalias/HISTORY b/sys/netinet/libalias/HISTORY index 8965290a4f27..87d5b6f8ed76 100644 --- a/sys/netinet/libalias/HISTORY +++ b/sys/netinet/libalias/HISTORY @@ -146,4 +146,5 @@ Version 3.2: July, 2000 (Erik Salander, erik@whistle.com and Version 3.3: May 2021 (donner) - Dropped LibAliasCheckNewLink - - Refactor the database handling, perfomance improvements. + - Refactor the database handling + - Switch to more effienct SPLAY trees diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index b056f55b56a5..783a8ca3baa8 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -29,120 +29,6 @@ #include __FBSDID("$FreeBSD$"); -/* - Alias_db.c encapsulates all data structures used for storing - packet aliasing data. Other parts of the aliasing software - access data through functions provided in this file. - - Data storage is based on the notion of a "link", which is - established for ICMP echo/reply packets, UDP datagrams and - TCP stream connections. A link stores the original source - and destination addresses. For UDP and TCP, it also stores - source and destination port numbers, as well as an alias - port number. Links are also used to store information about - fragments. - - There is a facility for sweeping through and deleting old - links as new packets are sent through. A simple timeout is - used for ICMP and UDP links. TCP links are left alone unless - there is an incomplete connection, in which case the link - can be deleted after a certain amount of time. - - Initial version: August, 1996 (cjm) - - Version 1.4: September 16, 1996 (cjm) - Facility for handling incoming links added. - - Version 1.6: September 18, 1996 (cjm) - ICMP data handling simplified. - - Version 1.7: January 9, 1997 (cjm) - Fragment handling simplified. - Saves pointers for unresolved fragments. - Permits links for unspecified remote ports - or unspecified remote addresses. - Fixed bug which did not properly zero port - table entries after a link was deleted. - Cleaned up some obsolete comments. - - Version 1.8: January 14, 1997 (cjm) - Fixed data type error in StartPoint(). - (This error did not exist prior to v1.7 - and was discovered and fixed by Ari Suutari) - - Version 1.9: February 1, 1997 - Optionally, connections initiated from packet aliasing host - machine will will not have their port number aliased unless it - conflicts with an aliasing port already being used. (cjm) - - All options earlier being #ifdef'ed are now available through - a new interface, SetPacketAliasMode(). This allows run time - control (which is now available in PPP+pktAlias through the - 'alias' keyword). (ee) - - Added ability to create an alias port without - either destination address or port specified. - port type = ALIAS_PORT_UNKNOWN_DEST_ALL (ee) - - Removed K&R style function headers - and general cleanup. (ee) - - Added packetAliasMode to replace compiler #defines's (ee) - - Allocates sockets for partially specified - ports if ALIAS_USE_SOCKETS defined. (cjm) - - Version 2.0: March, 1997 - SetAliasAddress() will now clean up alias links - if the aliasing address is changed. (cjm) - - PacketAliasPermanentLink() function added to support permanent - links. (J. Fortes suggested the need for this.) - Examples: - - (192.168.0.1, port 23) <-> alias port 6002, unknown dest addr/port - - (192.168.0.2, port 21) <-> alias port 3604, known dest addr - unknown dest port - - These permanent links allow for incoming connections to - machines on the local network. They can be given with a - user-chosen amount of specificity, with increasing specificity - meaning more security. (cjm) - - Quite a bit of rework to the basic engine. The portTable[] - array, which kept track of which ports were in use was replaced - by a table/linked list structure. (cjm) - - SetExpire() function added. (cjm) - - DeleteLink() no longer frees memory association with a pointer - to a fragment (this bug was first recognized by E. Eklund in - v1.9). - - Version 2.1: May, 1997 (cjm) - Packet aliasing engine reworked so that it can handle - multiple external addresses rather than just a single - host address. - - PacketAliasRedirectPort() and PacketAliasRedirectAddr() - added to the API. The first function is a more generalized - version of PacketAliasPermanentLink(). The second function - implements static network address translation. - - Version 3.2: July, 2000 (salander and satoh) - Added FindNewPortGroup to get contiguous range of port values. - - Added QueryUdpTcpIn and QueryUdpTcpOut to look for an aliasing - link but not actually add one. - - Added FindRtspOut, which is closely derived from FindUdpTcpOut, - except that the alias port (from FindNewPortGroup) is provided - as input. - - See HISTORY file for additional revisions. -*/ - #ifdef _KERNEL #include #include @@ -175,177 +61,11 @@ __FBSDID("$FreeBSD$"); #include "alias_mod.h" #endif +#include "alias_db.h" + static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); int LibAliasTime; -/* - Constants (note: constants are also defined - near relevant functions or structs) -*/ - -/* Timeouts (in seconds) for different link types */ -#define ICMP_EXPIRE_TIME 60 -#define UDP_EXPIRE_TIME 60 -#define PROTO_EXPIRE_TIME 60 -#define FRAGMENT_ID_EXPIRE_TIME 10 -#define FRAGMENT_PTR_EXPIRE_TIME 30 - -/* TCP link expire time for different cases */ -/* When the link has been used and closed - minimal grace time to - allow ACKs and potential re-connect in FTP (XXX - is this allowed?) */ -#ifndef TCP_EXPIRE_DEAD -#define TCP_EXPIRE_DEAD 10 -#endif - -/* When the link has been used and closed on one side - the other side - is allowed to still send data */ -#ifndef TCP_EXPIRE_SINGLEDEAD -#define TCP_EXPIRE_SINGLEDEAD 90 -#endif - -/* When the link isn't yet up */ -#ifndef TCP_EXPIRE_INITIAL -#define TCP_EXPIRE_INITIAL 300 -#endif - -/* When the link is up */ -#ifndef TCP_EXPIRE_CONNECTED -#define TCP_EXPIRE_CONNECTED 86400 -#endif - -/* Dummy port number codes used for FindLinkIn/Out() and AddLink(). - These constants can be anything except zero, which indicates an - unknown port number. */ - -#define NO_DEST_PORT 1 -#define NO_SRC_PORT 1 - -/* Matches any/unknown address in FindLinkIn/Out() and AddLink(). */ -static struct in_addr const ANY_ADDR = { INADDR_ANY }; - -/* Data Structures - - The fundamental data structure used in this program is - "struct alias_link". Whenever a TCP connection is made, - a UDP datagram is sent out, or an ICMP echo request is made, - a link record is made (if it has not already been created). - The link record is identified by the source address/port - and the destination address/port. In the case of an ICMP - echo request, the source port is treated as being equivalent - with the 16-bit ID number of the ICMP packet. - - The link record also can store some auxiliary data. For - TCP connections that have had sequence and acknowledgment - modifications, data space is available to track these changes. - A state field is used to keep track in changes to the TCP - connection state. ID numbers of fragments can also be - stored in the auxiliary space. Pointers to unresolved - fragments can also be stored. - - The link records support two independent chainings. Lookup - tables for input and out tables hold the initial pointers - the link chains. On input, the lookup table indexes on alias - port and link type. On output, the lookup table indexes on - source address, destination address, source port, destination - port and link type. -*/ - -/* used to save changes to ACK/sequence numbers */ -struct ack_data_record { - u_long ack_old; - u_long ack_new; - int delta; - int active; -}; - -/* Information about TCP connection */ -struct tcp_state { - int in; /* State for outside -> inside */ - int out; /* State for inside -> outside */ - int index; /* Index to ACK data array */ - /* Indicates whether ACK and sequence numbers been modified */ - int ack_modified; -}; - -/* Number of distinct ACK number changes - * saved for a modified TCP stream */ -#define N_LINK_TCP_DATA 3 -struct tcp_dat { - struct tcp_state state; - struct ack_data_record ack[N_LINK_TCP_DATA]; - /* Which firewall record is used for this hole? */ - int fwhole; -}; - -/* LSNAT server pool (circular list) */ -struct server { - struct in_addr addr; - u_short port; - struct server *next; -}; - -/* Main data structure */ -struct alias_link { - struct libalias *la; - /* Address and port information */ - struct in_addr src_addr; - struct in_addr dst_addr; - struct in_addr alias_addr; - struct in_addr proxy_addr; - u_short src_port; - u_short dst_port; - u_short alias_port; - u_short proxy_port; - struct server *server; - /* Type of link: TCP, UDP, ICMP, proto, frag */ - int link_type; -/* values for link_type */ -#define LINK_ICMP IPPROTO_ICMP -#define LINK_UDP IPPROTO_UDP -#define LINK_TCP IPPROTO_TCP -#define LINK_FRAGMENT_ID (IPPROTO_MAX + 1) -#define LINK_FRAGMENT_PTR (IPPROTO_MAX + 2) -#define LINK_ADDR (IPPROTO_MAX + 3) -#define LINK_PPTP (IPPROTO_MAX + 4) - - int flags; /* indicates special characteristics */ - int pflags; /* protocol-specific flags */ -/* flag bits */ -#define LINK_UNKNOWN_DEST_PORT 0x01 -#define LINK_UNKNOWN_DEST_ADDR 0x02 -#define LINK_PERMANENT 0x04 -#define LINK_PARTIALLY_SPECIFIED 0x03 /* logical-or of first two bits */ -#define LINK_UNFIREWALLED 0x08 - - int timestamp; /* Time link was last accessed */ -#ifndef NO_USE_SOCKETS - int sockfd; /* socket descriptor */ -#endif - /* Linked list of pointers for input and output lookup tables */ - union { - struct { - LIST_ENTRY(alias_link) in; - LIST_ENTRY(alias_link) out; - } all; - struct { - LIST_ENTRY(alias_link) list; - } pptp; - }; - struct { - TAILQ_ENTRY(alias_link) list; - int time; /* Expire time for link */ - } expire; - /* Auxiliary data */ - union { - char *frag_ptr; - struct in_addr frag_addr; - struct tcp_dat *tcp; - } data; -}; - -/* Clean up procedure. */ -static void finishoff(void); - /* Kernel module definition. */ #ifdef _KERNEL MALLOC_DEFINE(M_ALIAS, "libalias", "packet aliasing"); @@ -373,96 +93,32 @@ static moduledata_t alias_mod = { DECLARE_MODULE(alias, alias_mod, SI_SUB_DRIVERS, SI_ORDER_SECOND); #endif -/* Internal utility routines (used only in alias_db.c) - -Lookup table starting points: - StartPointIn() -- link table initial search point for - incoming packets - StartPointOut() -- link table initial search point for - outgoing packets - -Miscellaneous: - SeqDiff() -- difference between two TCP sequences - ShowAliasStats() -- send alias statistics to a monitor file -*/ - -/* Local prototypes */ -static struct group_in * -StartPointIn(struct libalias *, struct in_addr, u_short, int, int); -static u_int -StartPointOut(struct in_addr, struct in_addr, u_short, u_short, int); -static int SeqDiff(u_long, u_long); - -#ifndef NO_FW_PUNCH -/* Firewall control */ -static void InitPunchFW(struct libalias *); -static void UninitPunchFW(struct libalias *); -static void ClearFWHole(struct alias_link *); - -#endif - -/* Log file control */ -static void ShowAliasStats(struct libalias *); -static int InitPacketAliasLog(struct libalias *); -static void UninitPacketAliasLog(struct libalias *); - -void SctpShowAliasStats(struct libalias *la); - -#define INGUARD \ - if (grp->alias_port != alias_port || \ - grp->link_type != link_type || \ - grp->alias_addr.s_addr != alias_addr.s_addr) \ - continue; +SPLAY_GENERATE(splay_out, alias_link, all.out, cmp_out); +SPLAY_GENERATE(splay_in, group_in, in, cmp_in); static struct group_in * StartPointIn(struct libalias *la, struct in_addr alias_addr, u_short alias_port, int link_type, int create) { - u_int n; - struct group_in *grp, *tmp; - - n = alias_addr.s_addr; - n += alias_port; - n += link_type; - n %= LINK_TABLE_IN_SIZE; + struct group_in *grp; + struct group_in needle = { + .alias_addr = alias_addr, + .alias_port = alias_port, + .link_type = link_type + }; - LIST_FOREACH_SAFE(grp, &la->groupTableIn[n], group_in, tmp) { - /* Auto cleanup */ - if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { - LIST_REMOVE(grp, group_in); - free(grp); - } else { - INGUARD; - return (grp); - } - } - if (!create || (grp = malloc(sizeof(*grp))) == NULL) + grp = SPLAY_FIND(splay_in, &la->linkSplayIn, &needle); + if (grp != NULL || !create || (grp = malloc(sizeof(*grp))) == NULL) return (grp); grp->alias_addr = alias_addr; grp->alias_port = alias_port; grp->link_type = link_type; LIST_INIT(&grp->full); LIST_INIT(&grp->partial); - LIST_INSERT_HEAD(&la->groupTableIn[n], grp, group_in); + SPLAY_INSERT(splay_in, &la->linkSplayIn, grp); return (grp); } -#undef INGUARD - -static u_int -StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, - u_short src_port, u_short dst_port, int link_type) -{ - u_int n; - - n = src_addr.s_addr; - n += dst_addr.s_addr; - n += src_port; - n += dst_port; - n += link_type; - - return (n % LINK_TABLE_OUT_SIZE); -} static int SeqDiff(u_long x, u_long y) @@ -531,53 +187,6 @@ void SctpShowAliasStats(struct libalias *la) ShowAliasStats(la); } -/* Internal routines for finding, deleting and adding links - -Port Allocation: - GetNewPort() -- find and reserve new alias port number - GetSocket() -- try to allocate a socket for a given port - -Link creation and deletion: - CleanupAliasData() - remove all link chains from lookup table - CleanupLink() - look for a stale link - DeleteLink() - remove link - AddLink() - add link - ReLink() - change link - -Link search: - FindLinkOut() - find link for outgoing packets - FindLinkIn() - find link for incoming packets - -Port search: - FindNewPortGroup() - find an available group of ports -*/ - -/* Local prototypes */ -static int GetNewPort(struct libalias *, struct alias_link *, int); -#ifndef NO_USE_SOCKETS -static u_short GetSocket(struct libalias *, u_short, int *, int); -#endif -static void CleanupAliasData(struct libalias *, int); -static void CleanupLink(struct libalias *, struct alias_link **, int); -static void DeleteLink(struct alias_link **, int); -static struct alias_link * -UseLink(struct libalias *, struct alias_link *); - -static struct alias_link * -ReLink(struct alias_link *, - struct in_addr, struct in_addr, struct in_addr, - u_short, u_short, int, int, int); - -static struct alias_link * -FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); - -static struct alias_link * -FindLinkIn(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); - -static u_short _RandomPort(struct libalias *la); - -#define GET_NEW_PORT_MAX_ATTEMPTS 20 - /* get random port in network byte order */ static u_short _RandomPort(struct libalias *la) { @@ -814,25 +423,13 @@ static void CleanupAliasData(struct libalias *la, int deletePermanent) { struct alias_link *lnk, *lnk_tmp; - u_int i; LIBALIAS_LOCK_ASSERT(la); /* permanent entries may stay */ TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, expire.list, lnk_tmp) DeleteLink(&lnk, deletePermanent); - - for (i = 0; i < LINK_TABLE_IN_SIZE; i++) { - struct group_in *grp, *grp_tmp; - - LIST_FOREACH_SAFE(grp, &la->groupTableIn[i], group_in, grp_tmp) - if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { - LIST_REMOVE(grp, group_in); - free(grp); - } - } } - static void CleanupLink(struct libalias *la, struct alias_link **lnk, int deletePermanent) { @@ -881,7 +478,9 @@ DeleteLink(struct alias_link **plnk, int deletePermanent) case LINK_PPTP: LIST_REMOVE(lnk, pptp.list); break; - default: + default: { + struct group_in *grp; + /* Free memory allocated for LSNAT server pool */ if (lnk->server != NULL) { struct server *head, *curr, *next; @@ -893,11 +492,21 @@ DeleteLink(struct alias_link **plnk, int deletePermanent) } while ((curr = next) != head); } else { /* Adjust output table pointers */ - LIST_REMOVE(lnk, all.out); + SPLAY_REMOVE(splay_out, &la->linkSplayOut, lnk); } /* Adjust input table pointers */ LIST_REMOVE(lnk, all.in); + + /* Remove intermediate node, if empty */ + grp = StartPointIn(la, lnk->alias_addr, lnk->alias_port, lnk->link_type, 0); + if (grp != NULL && + LIST_EMPTY(&grp->full) && + LIST_EMPTY(&grp->partial)) { + SPLAY_REMOVE(splay_in, &la->linkSplayIn, grp); + free(grp); + } + } break; } @@ -956,7 +565,6 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, struct in_addr alias_addr, u_short src_port, u_short dst_port, int alias_port_param, int link_type) { - u_int start_point; struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); @@ -1083,9 +691,7 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, } /* Set up pointers for output lookup table */ - start_point = StartPointOut(src_addr, dst_addr, - src_port, dst_port, link_type); - LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, all.out); + SPLAY_INSERT(splay_out, &la->linkSplayOut, lnk); /* Set up pointers for input lookup table */ if (lnk->flags & LINK_PARTIALLY_SPECIFIED) @@ -1138,35 +744,25 @@ ReLink(struct alias_link *old_lnk, return (new_lnk); } - -#define OUTGUARD \ - if (lnk->src_port != src_port || \ - lnk->src_addr.s_addr != src_addr.s_addr || \ - lnk->dst_addr.s_addr != dst_addr.s_addr || \ - lnk->dst_port != dst_port || \ - lnk->link_type != link_type) \ - continue; - static struct alias_link * _SearchLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, u_short src_port, u_short dst_port, int link_type) { - u_int i; struct alias_link *lnk; + struct alias_link needle = { + .src_addr = src_addr, + .dst_addr = dst_addr, + .src_port = src_port, + .dst_port = dst_port, + .link_type = link_type + }; - i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_FOREACH(lnk, &la->linkTableOut[i], all.out) { - OUTGUARD; - return (UseLink(la, lnk)); - } - - return (NULL); + lnk = SPLAY_FIND(splay_out, &la->linkSplayOut, &needle); + return (UseLink(la, lnk)); } -#undef OUTGUARD - static struct alias_link * _FindLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, @@ -2331,7 +1927,7 @@ LibAliasAddServer(struct libalias *la, struct alias_link *lnk, struct in_addr ad if (head == NULL) { server->next = server; /* not usable for outgoing connections */ - LIST_REMOVE(lnk, all.out); + SPLAY_REMOVE(splay_out, &la->linkSplayOut, lnk); } else { struct server *s; @@ -2481,8 +2077,6 @@ finishoff(void) struct libalias * LibAliasInit(struct libalias *la) { - int i; - if (la == NULL) { #ifdef _KERNEL #undef malloc /* XXX: ugly */ @@ -2506,10 +2100,8 @@ LibAliasInit(struct libalias *la) LibAliasTime = time(NULL); #endif - for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) - LIST_INIT(&la->linkTableOut[i]); - for (i = 0; i < LINK_TABLE_IN_SIZE; i++) - LIST_INIT(&la->groupTableIn[i]); + SPLAY_INIT(&la->linkSplayIn); + SPLAY_INIT(&la->linkSplayOut); LIST_INIT(&la->pptpList); TAILQ_INIT(&la->checkExpire); #ifdef _KERNEL @@ -2702,20 +2294,6 @@ fill_rule(void *buf, int bufsize, int rulenum, return ((char *)cmd - (char *)buf); } -static void ClearAllFWHoles(struct libalias *la); - -#define fw_setfield(la, field, num) \ -do { \ - (field)[(num) - la->fireWallBaseNum] = 1; \ -} /*lint -save -e717 */ while(0)/* lint -restore */ - -#define fw_clrfield(la, field, num) \ -do { \ - (field)[(num) - la->fireWallBaseNum] = 0; \ -} /*lint -save -e717 */ while(0)/* lint -restore */ - -#define fw_tstfield(la, field, num) ((field)[(num) - la->fireWallBaseNum]) - static void InitPunchFW(struct libalias *la) { diff --git a/sys/netinet/libalias/alias_db.h b/sys/netinet/libalias/alias_db.h new file mode 100644 index 000000000000..971ca305c1a6 --- /dev/null +++ b/sys/netinet/libalias/alias_db.h @@ -0,0 +1,443 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2001 Charles Mott + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* + Alias_db.c encapsulates all data structures used for storing + packet aliasing data. Other parts of the aliasing software + access data through functions provided in this file. + + Data storage is based on the notion of a "link", which is + established for ICMP echo/reply packets, UDP datagrams and + TCP stream connections. A link stores the original source + and destination addresses. For UDP and TCP, it also stores + source and destination port numbers, as well as an alias + port number. Links are also used to store information about + fragments. + + There is a facility for sweeping through and deleting old + links as new packets are sent through. A simple timeout is + used for ICMP and UDP links. TCP links are left alone unless + there is an incomplete connection, in which case the link + can be deleted after a certain amount of time. + + Initial version: August, 1996 (cjm) + + Version 1.4: September 16, 1996 (cjm) + Facility for handling incoming links added. + + Version 1.6: September 18, 1996 (cjm) + ICMP data handling simplified. + + Version 1.7: January 9, 1997 (cjm) + Fragment handling simplified. + Saves pointers for unresolved fragments. + Permits links for unspecified remote ports + or unspecified remote addresses. + Fixed bug which did not properly zero port + table entries after a link was deleted. + Cleaned up some obsolete comments. + + Version 1.8: January 14, 1997 (cjm) + Fixed data type error in StartPoint(). + (This error did not exist prior to v1.7 + and was discovered and fixed by Ari Suutari) + + Version 1.9: February 1, 1997 + Optionally, connections initiated from packet aliasing host + machine will will not have their port number aliased unless it + conflicts with an aliasing port already being used. (cjm) + + All options earlier being #ifdef'ed are now available through + a new interface, SetPacketAliasMode(). This allows run time + control (which is now available in PPP+pktAlias through the + 'alias' keyword). (ee) + + Added ability to create an alias port without + either destination address or port specified. + port type = ALIAS_PORT_UNKNOWN_DEST_ALL (ee) + + Removed K&R style function headers + and general cleanup. (ee) + + Added packetAliasMode to replace compiler #defines's (ee) + + Allocates sockets for partially specified + ports if ALIAS_USE_SOCKETS defined. (cjm) + + Version 2.0: March, 1997 + SetAliasAddress() will now clean up alias links + if the aliasing address is changed. (cjm) + + PacketAliasPermanentLink() function added to support permanent + links. (J. Fortes suggested the need for this.) + Examples: + + (192.168.0.1, port 23) <-> alias port 6002, unknown dest addr/port + + (192.168.0.2, port 21) <-> alias port 3604, known dest addr + unknown dest port + + These permanent links allow for incoming connections to + machines on the local network. They can be given with a + user-chosen amount of specificity, with increasing specificity + meaning more security. (cjm) + + Quite a bit of rework to the basic engine. The portTable[] + array, which kept track of which ports were in use was replaced + by a table/linked list structure. (cjm) + + SetExpire() function added. (cjm) + + DeleteLink() no longer frees memory association with a pointer + to a fragment (this bug was first recognized by E. Eklund in + v1.9). + + Version 2.1: May, 1997 (cjm) + Packet aliasing engine reworked so that it can handle + multiple external addresses rather than just a single + host address. + + PacketAliasRedirectPort() and PacketAliasRedirectAddr() + added to the API. The first function is a more generalized + version of PacketAliasPermanentLink(). The second function + implements static network address translation. + + Version 3.2: July, 2000 (salander and satoh) + Added FindNewPortGroup to get contiguous range of port values. + + Added QueryUdpTcpIn and QueryUdpTcpOut to look for an aliasing + link but not actually add one. + + Added FindRtspOut, which is closely derived from FindUdpTcpOut, + except that the alias port (from FindNewPortGroup) is provided + as input. + + See HISTORY file for additional revisions. +*/ + +#ifndef _ALIAS_DB_H_ +#define _ALIAS_DB_H_ + + +/* + Constants (note: constants are also defined + near relevant functions or structs) +*/ + +/* Timeouts (in seconds) for different link types */ +#define ICMP_EXPIRE_TIME 60 +#define UDP_EXPIRE_TIME 60 +#define PROTO_EXPIRE_TIME 60 +#define FRAGMENT_ID_EXPIRE_TIME 10 +#define FRAGMENT_PTR_EXPIRE_TIME 30 + +/* TCP link expire time for different cases */ +/* When the link has been used and closed - minimal grace time to + allow ACKs and potential re-connect in FTP (XXX - is this allowed?) */ +#ifndef TCP_EXPIRE_DEAD +#define TCP_EXPIRE_DEAD 10 +#endif + +/* When the link has been used and closed on one side - the other side + is allowed to still send data */ +#ifndef TCP_EXPIRE_SINGLEDEAD +#define TCP_EXPIRE_SINGLEDEAD 90 +#endif + +/* When the link isn't yet up */ +#ifndef TCP_EXPIRE_INITIAL +#define TCP_EXPIRE_INITIAL 300 +#endif + +/* When the link is up */ +#ifndef TCP_EXPIRE_CONNECTED +#define TCP_EXPIRE_CONNECTED 86400 +#endif + +/* Dummy port number codes used for FindLinkIn/Out() and AddLink(). + These constants can be anything except zero, which indicates an + unknown port number. */ + +#define NO_DEST_PORT 1 +#define NO_SRC_PORT 1 + +/* Matches any/unknown address in FindLinkIn/Out() and AddLink(). */ +static struct in_addr const ANY_ADDR = { INADDR_ANY }; + +/* Data Structures + + The fundamental data structure used in this program is + "struct alias_link". Whenever a TCP connection is made, + a UDP datagram is sent out, or an ICMP echo request is made, + a link record is made (if it has not already been created). + The link record is identified by the source address/port + and the destination address/port. In the case of an ICMP + echo request, the source port is treated as being equivalent + with the 16-bit ID number of the ICMP packet. + + The link record also can store some auxiliary data. For + TCP connections that have had sequence and acknowledgment + modifications, data space is available to track these changes. + A state field is used to keep track in changes to the TCP + connection state. ID numbers of fragments can also be + stored in the auxiliary space. Pointers to unresolved + fragments can also be stored. + + The link records support two independent chainings. Lookup + tables for input and out tables hold the initial pointers + the link chains. On input, the lookup table indexes on alias + port and link type. On output, the lookup table indexes on + source address, destination address, source port, destination + port and link type. +*/ + +/* used to save changes to ACK/sequence numbers */ +struct ack_data_record { + u_long ack_old; + u_long ack_new; + int delta; + int active; +}; + +/* Information about TCP connection */ +struct tcp_state { + int in; /* State for outside -> inside */ + int out; /* State for inside -> outside */ + int index; /* Index to ACK data array */ + /* Indicates whether ACK and sequence numbers been modified */ + int ack_modified; +}; + +/* Number of distinct ACK number changes + * saved for a modified TCP stream */ +#define N_LINK_TCP_DATA 3 +struct tcp_dat { + struct tcp_state state; + struct ack_data_record ack[N_LINK_TCP_DATA]; + /* Which firewall record is used for this hole? */ + int fwhole; +}; + +/* LSNAT server pool (circular list) */ +struct server { + struct in_addr addr; + u_short port; + struct server *next; +}; + +/* Main data structure */ +struct alias_link { + struct libalias *la; + /* Address and port information */ + struct in_addr src_addr; + struct in_addr dst_addr; + struct in_addr alias_addr; + struct in_addr proxy_addr; + u_short src_port; + u_short dst_port; + u_short alias_port; + u_short proxy_port; + struct server *server; + /* Type of link: TCP, UDP, ICMP, proto, frag */ + int link_type; +/* values for link_type */ +#define LINK_ICMP IPPROTO_ICMP +#define LINK_UDP IPPROTO_UDP +#define LINK_TCP IPPROTO_TCP +#define LINK_FRAGMENT_ID (IPPROTO_MAX + 1) +#define LINK_FRAGMENT_PTR (IPPROTO_MAX + 2) +#define LINK_ADDR (IPPROTO_MAX + 3) +#define LINK_PPTP (IPPROTO_MAX + 4) + + int flags; /* indicates special characteristics */ + int pflags; /* protocol-specific flags */ +/* flag bits */ +#define LINK_UNKNOWN_DEST_PORT 0x01 +#define LINK_UNKNOWN_DEST_ADDR 0x02 +#define LINK_PERMANENT 0x04 *** 203 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 07:11:51 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8C5F6517BC; Tue, 6 Jul 2021 07:11:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtyv4BP6z4ZcM; Tue, 6 Jul 2021 07:11:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 77E041579E; Tue, 6 Jul 2021 07:11:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1667BpeB089644; Tue, 6 Jul 2021 07:11:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1667BpB7089643; Tue, 6 Jul 2021 07:11:51 GMT (envelope-from git) Date: Tue, 6 Jul 2021 07:11:51 GMT Message-Id: <202107060711.1667BpB7089643@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: c207bb80909f - stable/12 - libalias: tidy up housekeeping MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: c207bb80909f437c95a84e3e4e9318d15de444f1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 07:11:51 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=c207bb80909f437c95a84e3e4e9318d15de444f1 commit c207bb80909f437c95a84e3e4e9318d15de444f1 Author: Lutz Donnerhacke AuthorDate: 2021-05-15 15:35:36 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 07:07:43 +0000 libalias: tidy up housekeeping Replace current expensive, but sparsly called housekeeping by a single, repetive action. This is part of a larger restructure of libalias in order to switch to more efficient data structures. The whole restructure process is split into 15 reviews to ease reviewing. All those steps will be squashed into a single commit for MFC in order to hide the intermediate states from production systems. Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D30277 (cherry picked from commit 294799c6b081faece556a5010a4f51552e131c2f) --- sys/netinet/libalias/alias_db.c | 104 ++++++++++++++++--------------------- sys/netinet/libalias/alias_local.h | 6 +-- 2 files changed, 47 insertions(+), 63 deletions(-) diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index 7c2faa62f7fc..d3018a400022 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -182,11 +182,6 @@ static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); near relevant functions or structs) */ -/* Parameters used for cleanup of expired links */ -/* NOTE: ALIAS_CLEANUP_INTERVAL_SECS must be less then LINK_TABLE_OUT_SIZE */ -#define ALIAS_CLEANUP_INTERVAL_SECS 64 -#define ALIAS_CLEANUP_MAX_SPOKES (LINK_TABLE_OUT_SIZE/5) - /* Timeouts (in seconds) for different link types */ #define ICMP_EXPIRE_TIME 60 #define UDP_EXPIRE_TIME 60 @@ -329,6 +324,7 @@ struct alias_link { /* Linked list of pointers for input and output lookup tables */ LIST_ENTRY (alias_link) list_out; LIST_ENTRY (alias_link) list_in; + TAILQ_ENTRY (alias_link) list_expire; /* Auxiliary data */ union { char *frag_ptr; @@ -510,7 +506,7 @@ Port Allocation: Link creation and deletion: CleanupAliasData() - remove all link chains from lookup table - IncrementalCleanup() - look for stale links in a single chain + CleanupLink() - look for a stale link DeleteLink() - remove link AddLink() - add link ReLink() - change link @@ -529,8 +525,8 @@ static int GetNewPort(struct libalias *, struct alias_link *, int); static u_short GetSocket(struct libalias *, u_short, int *, int); #endif static void CleanupAliasData(struct libalias *); -static void IncrementalCleanup(struct libalias *); -static void DeleteLink(struct alias_link *); +static void CleanupLink(struct libalias *, struct alias_link **); +static void DeleteLink(struct alias_link **); static struct alias_link * ReLink(struct alias_link *, @@ -807,41 +803,38 @@ FindNewPortGroup(struct libalias *la, static void CleanupAliasData(struct libalias *la) { - struct alias_link *lnk; - int i; + struct alias_link *lnk, *lnk_tmp; LIBALIAS_LOCK_ASSERT(la); - for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) { - lnk = LIST_FIRST(&la->linkTableOut[i]); - while (lnk != NULL) { - struct alias_link *link_next = LIST_NEXT(lnk, list_out); - DeleteLink(lnk); - lnk = link_next; - } - } - la->cleanupIndex = 0; + /* permanent entries may stay */ + TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, list_expire, lnk_tmp) + DeleteLink(&lnk); } static void -IncrementalCleanup(struct libalias *la) +CleanupLink(struct libalias *la, struct alias_link **lnk) { - struct alias_link *lnk, *lnk_tmp; - LIBALIAS_LOCK_ASSERT(la); - LIST_FOREACH_SAFE(lnk, &la->linkTableOut[la->cleanupIndex++], - list_out, lnk_tmp) { - if (la->timeStamp - lnk->timestamp > lnk->expire_time) - DeleteLink(lnk); + + if (lnk == NULL || *lnk == NULL) + return; + + if (la->timeStamp - (*lnk)->timestamp > (*lnk)->expire_time) { + DeleteLink(lnk); + if ((*lnk) == NULL) + return; } - if (la->cleanupIndex == LINK_TABLE_OUT_SIZE) - la->cleanupIndex = 0; + /* move to end, swap may fail on a single entry list */ + TAILQ_REMOVE(&la->checkExpire, (*lnk), list_expire); + TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), list_expire); } static void -DeleteLink(struct alias_link *lnk) +DeleteLink(struct alias_link **plnk) { + struct alias_link *lnk = *plnk; struct libalias *la = lnk->la; LIBALIAS_LOCK_ASSERT(la); @@ -869,6 +862,10 @@ DeleteLink(struct alias_link *lnk) /* Adjust input table pointers */ LIST_REMOVE(lnk, list_in); + + /* remove from housekeeping */ + TAILQ_REMOVE(&la->checkExpire, lnk, list_expire); + #ifndef NO_USE_SOCKETS /* Close socket, if one has been allocated */ if (lnk->sockfd != -1) { @@ -908,6 +905,7 @@ DeleteLink(struct alias_link *lnk) /* Free memory */ free(lnk); + *plnk = NULL; /* Write statistics, if logging enabled */ if (la->packetAliasMode & PKT_ALIAS_LOG) { @@ -1039,6 +1037,9 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, /* Set up pointers for input lookup table */ start_point = StartPointIn(alias_addr, lnk->alias_port, link_type); LIST_INSERT_HEAD(&la->linkTableIn[start_point], lnk, list_in); + + /* Include the element into the housekeeping list */ + TAILQ_INSERT_TAIL(&la->checkExpire, lnk, list_expire); } else { #ifdef LIBALIAS_DEBUG fprintf(stderr, "PacketAlias/AddLink(): "); @@ -1079,7 +1080,7 @@ ReLink(struct alias_link *old_lnk, PunchFWHole(new_lnk); } #endif - DeleteLink(old_lnk); + DeleteLink(&old_lnk); return (new_lnk); } @@ -1102,12 +1103,14 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, lnk->src_port == src_port && lnk->dst_port == dst_port && lnk->link_type == link_type && - lnk->server == NULL) { - lnk->timestamp = la->timeStamp; + lnk->server == NULL) break; - } } + CleanupLink(la, &lnk); + if (lnk != NULL) + lnk->timestamp = la->timeStamp; + /* Search for partially specified links. */ if (lnk == NULL && replace_partial_links) { if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { @@ -1235,6 +1238,7 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, } } + CleanupLink(la, &lnk_fully_specified); if (lnk_fully_specified != NULL) { lnk_fully_specified->timestamp = la->timeStamp; lnk = lnk_fully_specified; @@ -1572,6 +1576,7 @@ FindPptpOutByCallId(struct libalias *la, struct in_addr src_addr, lnk->src_port == src_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -1592,6 +1597,7 @@ FindPptpOutByPeerCallId(struct libalias *la, struct in_addr src_addr, lnk->dst_port == dst_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -1612,6 +1618,7 @@ FindPptpInByCallId(struct libalias *la, struct in_addr dst_addr, lnk->dst_port == dst_call_id) break; + CleanupLink(la, &lnk); return (lnk); } @@ -2038,7 +2045,7 @@ SetExpire(struct alias_link *lnk, int expire) { if (expire == 0) { lnk->flags &= ~LINK_PERMANENT; - DeleteLink(lnk); + DeleteLink(&lnk); } else if (expire == -1) { lnk->flags |= LINK_PERMANENT; } else if (expire > 0) { @@ -2094,7 +2101,7 @@ SetDestCallId(struct alias_link *lnk, u_int16_t cid) void HouseKeeping(struct libalias *la) { - int i, n; + struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); #ifndef _KERNEL struct timeval tv; #endif @@ -2111,25 +2118,7 @@ HouseKeeping(struct libalias *la) gettimeofday(&tv, NULL); la->timeStamp = tv.tv_sec; #endif - - /* Compute number of spokes (output table link chains) to cover */ - n = LINK_TABLE_OUT_SIZE * (la->timeStamp - la->lastCleanupTime); - n /= ALIAS_CLEANUP_INTERVAL_SECS; - - /* Handle different cases */ - if (n > 0) { - if (n > ALIAS_CLEANUP_MAX_SPOKES) - n = ALIAS_CLEANUP_MAX_SPOKES; - la->lastCleanupTime = la->timeStamp; - for (i = 0; i < n; i++) - IncrementalCleanup(la); - } else if (n < 0) { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/HouseKeeping(): "); - fprintf(stderr, "something unexpected in time values\n"); -#endif - la->lastCleanupTime = la->timeStamp; - } + CleanupLink(la, &lnk); } /* Init the log file and enable logging */ @@ -2356,7 +2345,7 @@ LibAliasRedirectDelete(struct libalias *la, struct alias_link *lnk) { LIBALIAS_LOCK(la); la->deleteAllLinks = 1; - DeleteLink(lnk); + DeleteLink(&lnk); la->deleteAllLinks = 0; LIBALIAS_UNLOCK(la); } @@ -2426,17 +2415,16 @@ LibAliasInit(struct libalias *la) #ifdef _KERNEL la->timeStamp = time_uptime; - la->lastCleanupTime = time_uptime; #else gettimeofday(&tv, NULL); la->timeStamp = tv.tv_sec; - la->lastCleanupTime = tv.tv_sec; #endif for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) LIST_INIT(&la->linkTableOut[i]); for (i = 0; i < LINK_TABLE_IN_SIZE; i++) LIST_INIT(&la->linkTableIn[i]); + TAILQ_INIT(&la->checkExpire); #ifdef _KERNEL AliasSctpInit(la); #endif @@ -2466,8 +2454,6 @@ LibAliasInit(struct libalias *la) la->fragmentPtrLinkCount = 0; la->sockCount = 0; - la->cleanupIndex = 0; - la->packetAliasMode = PKT_ALIAS_SAME_PORTS #ifndef NO_USE_SOCKETS | PKT_ALIAS_USE_SOCKETS diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index 61cd30737ce5..fcdaa1690470 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -94,6 +94,8 @@ struct libalias { * output lookup tables. */ LIST_HEAD (, alias_link) linkTableOut[LINK_TABLE_OUT_SIZE]; LIST_HEAD (, alias_link) linkTableIn[LINK_TABLE_IN_SIZE]; + /* HouseKeeping */ + TAILQ_HEAD (, alias_link) checkExpire; /* Link statistics */ int icmpLinkCount; int udpLinkCount; @@ -103,12 +105,8 @@ struct libalias { int fragmentIdLinkCount; int fragmentPtrLinkCount; int sockCount; - /* Index to chain of link table being inspected for old links */ - int cleanupIndex; /* System time in seconds for current packet */ int timeStamp; - /* Last time IncrementalCleanup() was called */ - int lastCleanupTime; /* If equal to zero, DeleteLink() * will not remove permanent links */ int deleteAllLinks; From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 07:11:53 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 34A7C651A57; Tue, 6 Jul 2021 07:11:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtyw5Qnrz4Zqf; Tue, 6 Jul 2021 07:11:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A05C3157A0; Tue, 6 Jul 2021 07:11:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1667BqmS089668; Tue, 6 Jul 2021 07:11:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1667Bq7d089667; Tue, 6 Jul 2021 07:11:52 GMT (envelope-from git) Date: Tue, 6 Jul 2021 07:11:52 GMT Message-Id: <202107060711.1667Bq7d089667@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: e3c21cc51bec - stable/12 - libalias: Stats are unsigned MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e3c21cc51becd4162b766dd102eef7454891821d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 07:11:53 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=e3c21cc51becd4162b766dd102eef7454891821d commit e3c21cc51becd4162b766dd102eef7454891821d Author: Lutz Donnerhacke AuthorDate: 2021-05-31 22:08:24 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 07:07:46 +0000 libalias: Stats are unsigned Stats counters are used as unsigned valued (i.e. printf("%u")) but are defined as signed int. This causes trouble later, so fix it early. Differential Revision: https://reviews.freebsd.org/D30587 (cherry picked from commit 3fd20a79e7b55d135bbc5fa197980fe0e9ee0f14) --- sys/netinet/libalias/alias_local.h | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index fcdaa1690470..8e2fe88fe620 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -97,14 +97,14 @@ struct libalias { /* HouseKeeping */ TAILQ_HEAD (, alias_link) checkExpire; /* Link statistics */ - int icmpLinkCount; - int udpLinkCount; - int tcpLinkCount; - int pptpLinkCount; - int protoLinkCount; - int fragmentIdLinkCount; - int fragmentPtrLinkCount; - int sockCount; + unsigned int icmpLinkCount; + unsigned int udpLinkCount; + unsigned int tcpLinkCount; + unsigned int pptpLinkCount; + unsigned int protoLinkCount; + unsigned int fragmentIdLinkCount; + unsigned int fragmentPtrLinkCount; + unsigned int sockCount; /* System time in seconds for current packet */ int timeStamp; /* If equal to zero, DeleteLink() From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 07:11:54 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 29023651C33; Tue, 6 Jul 2021 07:11:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtyx6N6rz4ZcS; Tue, 6 Jul 2021 07:11:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BB0ED15900; Tue, 6 Jul 2021 07:11:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1667BrNG089692; Tue, 6 Jul 2021 07:11:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1667Brd7089691; Tue, 6 Jul 2021 07:11:53 GMT (envelope-from git) Date: Tue, 6 Jul 2021 07:11:53 GMT Message-Id: <202107060711.1667Brd7089691@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 8b7b86fa6fe0 - stable/12 - libalias: Promote per instance global variable timeStamp MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 8b7b86fa6fe0761b7a7dc7034c078fdbfa58c4d9 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 07:11:54 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=8b7b86fa6fe0761b7a7dc7034c078fdbfa58c4d9 commit 8b7b86fa6fe0761b7a7dc7034c078fdbfa58c4d9 Author: Lutz Donnerhacke AuthorDate: 2021-05-25 07:07:21 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 07:07:50 +0000 libalias: Promote per instance global variable timeStamp Summary: - Use LibAliasTime as a real global variable for central timekeeping. - Reduce number of syscalls in user space considerably. - Dynamically adjust the packet counters to match the second resolution. - Only check the first few packets after a time increase for expiry. Discussed with: hselasky Differential Revision: https://reviews.freebsd.org/D30566 (cherry picked from commit ef828d39be8e4b6d922db080ed41611702dec56d) --- sys/netinet/libalias/alias_db.c | 60 +++++++++++++++++++++++++------------- sys/netinet/libalias/alias_local.h | 5 ++-- sys/netinet/libalias/alias_sctp.c | 16 +++++----- 3 files changed, 50 insertions(+), 31 deletions(-) diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index d3018a400022..2759c5170b27 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -176,6 +176,7 @@ __FBSDID("$FreeBSD$"); #endif static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); +int LibAliasTime; /* Constants (note: constants are also defined @@ -820,7 +821,7 @@ CleanupLink(struct libalias *la, struct alias_link **lnk) if (lnk == NULL || *lnk == NULL) return; - if (la->timeStamp - (*lnk)->timestamp > (*lnk)->expire_time) { + if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire_time) { DeleteLink(lnk); if ((*lnk) == NULL) return; @@ -940,7 +941,7 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, #endif lnk->flags = 0; lnk->pflags = 0; - lnk->timestamp = la->timeStamp; + lnk->timestamp = LibAliasTime; /* Expiration time */ switch (link_type) { @@ -1109,7 +1110,7 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, CleanupLink(la, &lnk); if (lnk != NULL) - lnk->timestamp = la->timeStamp; + lnk->timestamp = LibAliasTime; /* Search for partially specified links. */ if (lnk == NULL && replace_partial_links) { @@ -1240,7 +1241,7 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, CleanupLink(la, &lnk_fully_specified); if (lnk_fully_specified != NULL) { - lnk_fully_specified->timestamp = la->timeStamp; + lnk_fully_specified->timestamp = LibAliasTime; lnk = lnk_fully_specified; } else if (lnk_unknown_dst_port != NULL) lnk = lnk_unknown_dst_port; @@ -2101,24 +2102,45 @@ SetDestCallId(struct alias_link *lnk, u_int16_t cid) void HouseKeeping(struct libalias *la) { - struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); -#ifndef _KERNEL - struct timeval tv; -#endif + static unsigned int packets = 0; + static unsigned int packet_limit = 1000; LIBALIAS_LOCK_ASSERT(la); + packets++; + /* - * Save system time (seconds) in global variable timeStamp for use - * by other functions. This is done so as not to unnecessarily - * waste timeline by making system calls. + * User space time/gettimeofday/... is very expensive. + * Kernel space cache trashing is unnecessary. + * + * Save system time (seconds) in global variable LibAliasTime + * for use by other functions. This is done so as not to + * unnecessarily waste timeline by making system calls. + * + * Reduce the amount of house keeping work substantially by + * sampling over the packets. */ + if (packets % packet_limit == 0) { + time_t now; + #ifdef _KERNEL - la->timeStamp = time_uptime; + now = time_uptime; #else - gettimeofday(&tv, NULL); - la->timeStamp = tv.tv_sec; + now = time(NULL); #endif - CleanupLink(la, &lnk); + if (now != LibAliasTime) { + /* retry three times a second */ + packet_limit = packets / 3; + packets = 0; + LibAliasTime = now; + } + + } + /* Do a cleanup for the first packets of the new second only */ + if (packets < (la->udpLinkCount + la->tcpLinkCount)) { + struct alias_link * lnk = TAILQ_FIRST(&la->checkExpire); + + CleanupLink(la, &lnk); + } } /* Init the log file and enable logging */ @@ -2392,9 +2414,6 @@ struct libalias * LibAliasInit(struct libalias *la) { int i; -#ifndef _KERNEL - struct timeval tv; -#endif if (la == NULL) { #ifdef _KERNEL @@ -2414,10 +2433,9 @@ LibAliasInit(struct libalias *la) LIST_INSERT_HEAD(&instancehead, la, instancelist); #ifdef _KERNEL - la->timeStamp = time_uptime; + LibAliasTime = time_uptime; #else - gettimeofday(&tv, NULL); - la->timeStamp = tv.tv_sec; + LibAliasTime = time(NULL); #endif for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) diff --git a/sys/netinet/libalias/alias_local.h b/sys/netinet/libalias/alias_local.h index 8e2fe88fe620..f1b40f0438da 100644 --- a/sys/netinet/libalias/alias_local.h +++ b/sys/netinet/libalias/alias_local.h @@ -105,8 +105,6 @@ struct libalias { unsigned int fragmentIdLinkCount; unsigned int fragmentPtrLinkCount; unsigned int sockCount; - /* System time in seconds for current packet */ - int timeStamp; /* If equal to zero, DeleteLink() * will not remove permanent links */ int deleteAllLinks; @@ -206,6 +204,9 @@ struct libalias { /* Prototypes */ +/* System time in seconds for current packet */ +extern int LibAliasTime; + /* * SctpFunction prototypes * diff --git a/sys/netinet/libalias/alias_sctp.c b/sys/netinet/libalias/alias_sctp.c index 2dd57e496f7e..96a0718eae1f 100644 --- a/sys/netinet/libalias/alias_sctp.c +++ b/sys/netinet/libalias/alias_sctp.c @@ -297,10 +297,10 @@ static MALLOC_DEFINE(M_SCTPNAT, "sctpnat", "sctp nat dbs"); #define SN_MAX_TIMER 600 #define SN_TIMER_QUEUE_SIZE SN_MAX_TIMER+2 -#define SN_I_T(la) (la->timeStamp + sysctl_init_timer) /**< INIT State expiration time in seconds */ -#define SN_U_T(la) (la->timeStamp + sysctl_up_timer) /**< UP State expiration time in seconds */ -#define SN_C_T(la) (la->timeStamp + sysctl_shutdown_timer) /**< CL State expiration time in seconds */ -#define SN_X_T(la) (la->timeStamp + sysctl_holddown_timer) /**< Wait after a shutdown complete in seconds */ +#define SN_I_T(la) (LibAliasTime + sysctl_init_timer) /**< INIT State expiration time in seconds */ +#define SN_U_T(la) (LibAliasTime + sysctl_up_timer) /**< UP State expiration time in seconds */ +#define SN_C_T(la) (LibAliasTime + sysctl_shutdown_timer) /**< CL State expiration time in seconds */ +#define SN_X_T(la) (LibAliasTime + sysctl_holddown_timer) /**< Wait after a shutdown complete in seconds */ /** @} * @defgroup sysctl SysCtl Variable and callback function declarations * @@ -642,9 +642,9 @@ AliasSctpInit(struct libalias *la) for (i = 0; i < SN_TIMER_QUEUE_SIZE; i++) LIST_INIT(&la->sctpNatTimer.TimerQ[i]); #ifdef _KERNEL - la->sctpNatTimer.loc_time=time_uptime; /* la->timeStamp is not set yet */ + la->sctpNatTimer.loc_time=time_uptime; /* LibAliasTime is not set yet */ #else - la->sctpNatTimer.loc_time=la->timeStamp; + la->sctpNatTimer.loc_time=LibAliasTime; #endif la->sctpNatTimer.cur_loc = 0; la->sctpLinkCount = 0; @@ -2468,12 +2468,12 @@ sctp_CheckTimers(struct libalias *la) struct sctp_nat_assoc *assoc; LIBALIAS_LOCK_ASSERT(la); - while(la->timeStamp >= la->sctpNatTimer.loc_time) { + while(LibAliasTime >= la->sctpNatTimer.loc_time) { while (!LIST_EMPTY(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc])) { assoc = LIST_FIRST(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc]); //SLIST_REMOVE_HEAD(&la->sctpNatTimer.TimerQ[la->sctpNatTimer.cur_loc], timer_Q); LIST_REMOVE(assoc, timer_Q); - if (la->timeStamp >= assoc->exp) { /* state expired */ + if (LibAliasTime >= assoc->exp) { /* state expired */ SN_LOG(((assoc->state == SN_CL) ? (SN_LOG_DEBUG) : (SN_LOG_INFO)), logsctperror("Timer Expired", assoc->g_vtag, assoc->state, SN_TO_NODIR)); RmSctpAssoc(la, assoc); From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 07:11:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2FD6465187E; Tue, 6 Jul 2021 07:11:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtyz0By9z4Zw0; Tue, 6 Jul 2021 07:11:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DF67A15744; Tue, 6 Jul 2021 07:11:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1667Bs2T089716; Tue, 6 Jul 2021 07:11:54 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1667BsOH089715; Tue, 6 Jul 2021 07:11:54 GMT (envelope-from git) Date: Tue, 6 Jul 2021 07:11:54 GMT Message-Id: <202107060711.1667BsOH089715@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: e0a40dc22771 - stable/12 - libalias: Restructure MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e0a40dc227718f4bc62f279073baa5b6b2cd6a57 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 07:11:55 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=e0a40dc227718f4bc62f279073baa5b6b2cd6a57 commit e0a40dc227718f4bc62f279073baa5b6b2cd6a57 Author: Lutz Donnerhacke AuthorDate: 2021-05-25 13:03:34 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 07:09:53 +0000 libalias: Restructure Clean up the database handling in order to switch to more efficient data structures. The development of this patch was artificially split in to many small steps to ease reviewing. - Common search terms - Separate fully qualified search - Separate table for partial links - Cleanup _FindLinkIn - Factor out the outgoing search function - Factor out a common idiom to return found links - Reorder incoming links by grouping of common search terms - Remove LSNAT from outgoing search - Group internal structure semantically - Separate table for PPTP - Use AliasRange instead of PORT_BASE - Remove temporary state deleteAllLinks from global struct - Avoid uninitialized expiration Discussed with: Dimitry Luhtionov Differential Revision: https://reviews.freebsd.org/D30568 Differential Revision: https://reviews.freebsd.org/D30569 Differential Revision: https://reviews.freebsd.org/D30570 Differential Revision: https://reviews.freebsd.org/D30571 Differential Revision: https://reviews.freebsd.org/D30572 Differential Revision: https://reviews.freebsd.org/D30573 Differential Revision: https://reviews.freebsd.org/D30574 Differential Revision: https://reviews.freebsd.org/D30575 Differential Revision: https://reviews.freebsd.org/D30580 Differential Revision: https://reviews.freebsd.org/D30581 Differential Revision: https://reviews.freebsd.org/D30604 Differential Revision: https://reviews.freebsd.org/D30582 (cherry picked from commit d41044ddfdbc2a026570cae11d5fc2e18f5e4b92) (cherry picked from commit 32f9c2ceb3ec8266e48ec0f6d1556fd98ef12db6) (cherry picked from commit cac129e6030095c33e95c5ce1cdcb9c5c21efce9) (cherry picked from commit 19dcc4f2250b3850991366e3058ca8a64b2f135d) (cherry picked from commit d5419034381d0e8e8c99f0f678118b197144bfff) (cherry picked from commit d4ab07d2aeb13b1aae9ad5d73e5fe77131f2350c) (cherry picked from commit 492d3b7109b45ff30fd199b7eeed797447746e42) (cherry picked from commit 7b44ff4c52ce9e7a89e772566a249e5481fe0ac4) (cherry picked from commit 1178dda53d10b096e0b99e356d15b7c24390e099) (cherry picked from commit 9efcad61d8309ecad3c15392b277fd329a1e45e4) (cherry picked from commit fe83900f9fa931e3d0942ec9c0709896b887c3d7) (cherry picked from commit d989935b5bcd880353f0de89eda958c45e7e3342) (cherry picked from commit b50a4dce185481bebf8096c27588ae04a4a2fd7c) (cherry picked from commit f28455344483310cfd1aa5c0bdd4d014810c0e32) --- sys/netinet/libalias/HISTORY | 4 + sys/netinet/libalias/alias_db.c | 897 ++++++++++++++++++++----------------- sys/netinet/libalias/alias_local.h | 16 +- 3 files changed, 497 insertions(+), 420 deletions(-) diff --git a/sys/netinet/libalias/HISTORY b/sys/netinet/libalias/HISTORY index c5bca59cac1f..8965290a4f27 100644 --- a/sys/netinet/libalias/HISTORY +++ b/sys/netinet/libalias/HISTORY @@ -143,3 +143,7 @@ Version 3.1: May, 2000 (Erik Salander, erik@whistle.com) Version 3.2: July, 2000 (Erik Salander, erik@whistle.com and Junichi Satoh, junichi@junichi.org) - Added support for streaming media (RTSP and PNA) aliasing. + +Version 3.3: May 2021 (donner) + - Dropped LibAliasCheckNewLink + - Refactor the database handling, perfomance improvements. diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index 2759c5170b27..b056f55b56a5 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -318,14 +318,23 @@ struct alias_link { #define LINK_UNFIREWALLED 0x08 int timestamp; /* Time link was last accessed */ - int expire_time; /* Expire time for link */ #ifndef NO_USE_SOCKETS int sockfd; /* socket descriptor */ #endif /* Linked list of pointers for input and output lookup tables */ - LIST_ENTRY (alias_link) list_out; - LIST_ENTRY (alias_link) list_in; - TAILQ_ENTRY (alias_link) list_expire; + union { + struct { + LIST_ENTRY(alias_link) in; + LIST_ENTRY(alias_link) out; + } all; + struct { + LIST_ENTRY(alias_link) list; + } pptp; + }; + struct { + TAILQ_ENTRY(alias_link) list; + int time; /* Expire time for link */ + } expire; /* Auxiliary data */ union { char *frag_ptr; @@ -378,12 +387,10 @@ Miscellaneous: */ /* Local prototypes */ -static u_int StartPointIn(struct in_addr, u_short, int); - -static u_int -StartPointOut(struct in_addr, struct in_addr, - u_short, u_short, int); - +static struct group_in * +StartPointIn(struct libalias *, struct in_addr, u_short, int, int); +static u_int +StartPointOut(struct in_addr, struct in_addr, u_short, u_short, int); static int SeqDiff(u_long, u_long); #ifndef NO_FW_PUNCH @@ -401,19 +408,46 @@ static void UninitPacketAliasLog(struct libalias *); void SctpShowAliasStats(struct libalias *la); -static u_int -StartPointIn(struct in_addr alias_addr, - u_short alias_port, - int link_type) +#define INGUARD \ + if (grp->alias_port != alias_port || \ + grp->link_type != link_type || \ + grp->alias_addr.s_addr != alias_addr.s_addr) \ + continue; + +static struct group_in * +StartPointIn(struct libalias *la, + struct in_addr alias_addr, u_short alias_port, int link_type, + int create) { u_int n; + struct group_in *grp, *tmp; n = alias_addr.s_addr; - if (link_type != LINK_PPTP) - n += alias_port; + n += alias_port; n += link_type; - return (n % LINK_TABLE_IN_SIZE); -} + n %= LINK_TABLE_IN_SIZE; + + LIST_FOREACH_SAFE(grp, &la->groupTableIn[n], group_in, tmp) { + /* Auto cleanup */ + if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { + LIST_REMOVE(grp, group_in); + free(grp); + } else { + INGUARD; + return (grp); + } + } + if (!create || (grp = malloc(sizeof(*grp))) == NULL) + return (grp); + grp->alias_addr = alias_addr; + grp->alias_port = alias_port; + grp->link_type = link_type; + LIST_INIT(&grp->full); + LIST_INIT(&grp->partial); + LIST_INSERT_HEAD(&la->groupTableIn[n], grp, group_in); + return (grp); +} +#undef INGUARD static u_int StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, @@ -423,10 +457,8 @@ StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, n = src_addr.s_addr; n += dst_addr.s_addr; - if (link_type != LINK_PPTP) { - n += src_port; - n += dst_port; - } + n += src_port; + n += dst_port; n += link_type; return (n % LINK_TABLE_OUT_SIZE); @@ -525,14 +557,16 @@ static int GetNewPort(struct libalias *, struct alias_link *, int); #ifndef NO_USE_SOCKETS static u_short GetSocket(struct libalias *, u_short, int *, int); #endif -static void CleanupAliasData(struct libalias *); -static void CleanupLink(struct libalias *, struct alias_link **); -static void DeleteLink(struct alias_link **); +static void CleanupAliasData(struct libalias *, int); +static void CleanupLink(struct libalias *, struct alias_link **, int); +static void DeleteLink(struct alias_link **, int); +static struct alias_link * +UseLink(struct libalias *, struct alias_link *); static struct alias_link * ReLink(struct alias_link *, struct in_addr, struct in_addr, struct in_addr, - u_short, u_short, int, int); + u_short, u_short, int, int, int); static struct alias_link * FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); @@ -540,12 +574,20 @@ FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, static struct alias_link * FindLinkIn(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); -#define ALIAS_PORT_BASE 0x08000 -#define ALIAS_PORT_MASK 0x07fff -#define ALIAS_PORT_MASK_EVEN 0x07ffe +static u_short _RandomPort(struct libalias *la); + #define GET_NEW_PORT_MAX_ATTEMPTS 20 -#define FIND_EVEN_ALIAS_BASE 1 +/* get random port in network byte order */ +static u_short +_RandomPort(struct libalias *la) { + u_short port; + + port = la->aliasPortLower + + arc4random_uniform(la->aliasPortLength); + + return ntohs(port); +} /* GetNewPort() allocates port numbers. Note that if a port number is already in use, that does not mean that it cannot be used by @@ -557,8 +599,7 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) { int i; int max_trials; - u_short port_sys; - u_short port_net; + u_short port; LIBALIAS_LOCK_ASSERT(la); /* @@ -566,41 +607,18 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) * this parameter is zero or positive, it precisely specifies * the port number. GetNewPort() will return this number * without check that it is in use. - + * + * The aliasing port is automatically selected by one of + * two methods below: + * * When this parameter is GET_ALIAS_PORT, it indicates to get * a randomly selected port number. */ - if (alias_port_param == GET_ALIAS_PORT) { - /* - * The aliasing port is automatically selected by one of - * two methods below: - */ - max_trials = GET_NEW_PORT_MAX_ATTEMPTS; - - if (la->packetAliasMode & PKT_ALIAS_SAME_PORTS) { - /* - * When the PKT_ALIAS_SAME_PORTS option is chosen, - * the first try will be the actual source port. If - * this is already in use, the remainder of the - * trials will be random. - */ - port_net = lnk->src_port; - port_sys = ntohs(port_net); - } else if (la->aliasPortLower) { - /* First trial is a random port in the aliasing range. */ - port_sys = la->aliasPortLower + - (arc4random() % la->aliasPortLength); - port_net = htons(port_sys); - } else { - /* First trial and all subsequent are random. */ - port_sys = arc4random() & ALIAS_PORT_MASK; - port_sys += ALIAS_PORT_BASE; - port_net = htons(port_sys); - } - } else if (alias_port_param >= 0 && alias_port_param < 0x10000) { + if (alias_port_param >= 0 && alias_port_param < 0x10000) { lnk->alias_port = (u_short) alias_port_param; return (0); - } else { + } + if (alias_port_param != GET_ALIAS_PORT) { #ifdef LIBALIAS_DEBUG fprintf(stderr, "PacketAlias/GetNewPort(): "); fprintf(stderr, "input parameter error\n"); @@ -608,58 +626,57 @@ GetNewPort(struct libalias *la, struct alias_link *lnk, int alias_port_param) return (-1); } + max_trials = GET_NEW_PORT_MAX_ATTEMPTS; + + /* + * When the PKT_ALIAS_SAME_PORTS option is chosen, + * the first try will be the actual source port. If + * this is already in use, the remainder of the + * trials will be random. + */ + port = (la->packetAliasMode & PKT_ALIAS_SAME_PORTS) + ? lnk->src_port + : _RandomPort(la); + /* Port number search */ - for (i = 0; i < max_trials; i++) { - int go_ahead; + for (i = 0; i < max_trials; i++, port = _RandomPort(la)) { + struct group_in *grp; struct alias_link *search_result; - search_result = FindLinkIn(la, lnk->dst_addr, lnk->alias_addr, - lnk->dst_port, port_net, - lnk->link_type, 0); + grp = StartPointIn(la, lnk->alias_addr, port, lnk->link_type, 0); + if (grp == NULL) + break; + LIST_FOREACH(search_result, &grp->full, all.in) { + if (lnk->dst_addr.s_addr == search_result->dst_addr.s_addr && + lnk->dst_port == search_result->dst_port) + break; /* found match */ + } if (search_result == NULL) - go_ahead = 1; - else if (!(lnk->flags & LINK_PARTIALLY_SPECIFIED) - && (search_result->flags & LINK_PARTIALLY_SPECIFIED)) - go_ahead = 1; - else - go_ahead = 0; + break; + } - if (go_ahead) { -#ifndef NO_USE_SOCKETS - if ((la->packetAliasMode & PKT_ALIAS_USE_SOCKETS) - && (lnk->flags & LINK_PARTIALLY_SPECIFIED) - && ((lnk->link_type == LINK_TCP) || - (lnk->link_type == LINK_UDP))) { - if (GetSocket(la, port_net, &lnk->sockfd, lnk->link_type)) { - lnk->alias_port = port_net; - return (0); - } - } else { + if (i >= max_trials) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/GetNewPort(): "); + fprintf(stderr, "could not find free port\n"); #endif - lnk->alias_port = port_net; - return (0); + return (-1); + } + #ifndef NO_USE_SOCKETS - } -#endif - } - if (la->aliasPortLower) { - port_sys = la->aliasPortLower + - (arc4random() % la->aliasPortLength); - port_net = htons(port_sys); - } else { - port_sys = arc4random() & ALIAS_PORT_MASK; - port_sys += ALIAS_PORT_BASE; - port_net = htons(port_sys); + if ((la->packetAliasMode & PKT_ALIAS_USE_SOCKETS) && + (lnk->flags & LINK_PARTIALLY_SPECIFIED) && + ((lnk->link_type == LINK_TCP) || + (lnk->link_type == LINK_UDP))) { + if (!GetSocket(la, port, &lnk->sockfd, lnk->link_type)) { + return (-1); } } - -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/GetnewPort(): "); - fprintf(stderr, "could not find free port\n"); #endif + lnk->alias_port = port; - return (-1); + return (0); } #ifndef NO_USE_SOCKETS @@ -726,7 +743,7 @@ FindNewPortGroup(struct libalias *la, { int i, j; int max_trials; - u_short port_sys; + u_short port; int link_type; LIBALIAS_LOCK_ASSERT(la); @@ -758,39 +775,31 @@ FindNewPortGroup(struct libalias *la, * try will be the actual source port. If this is already * in use, the remainder of the trials will be random. */ - port_sys = ntohs(src_port); + port = src_port; } else { - /* First trial and all subsequent are random. */ - if (align == FIND_EVEN_ALIAS_BASE) - port_sys = arc4random() & ALIAS_PORT_MASK_EVEN; - else - port_sys = arc4random() & ALIAS_PORT_MASK; - - port_sys += ALIAS_PORT_BASE; + port = _RandomPort(la); } /* Port number search */ - for (i = 0; i < max_trials; i++) { + for (i = 0; i < max_trials; i++, port = _RandomPort(la)) { struct alias_link *search_result; - for (j = 0; j < port_count; j++) + if (align) + port &= htons(0xfffe); + + for (j = 0; j < port_count; j++) { + u_short port_j = ntohs(port) + j; + if ((search_result = FindLinkIn(la, dst_addr, - alias_addr, dst_port, htons(port_sys + j), + alias_addr, dst_port, htons(port_j), link_type, 0)) != NULL) break; + } /* Found a good range, return base */ if (j == port_count) - return (htons(port_sys)); - - /* Find a new base to try */ - if (align == FIND_EVEN_ALIAS_BASE) - port_sys = arc4random() & ALIAS_PORT_MASK_EVEN; - else - port_sys = arc4random() & ALIAS_PORT_MASK; - - port_sys += ALIAS_PORT_BASE; + return (port); } #ifdef LIBALIAS_DEBUG @@ -802,45 +811,65 @@ FindNewPortGroup(struct libalias *la, } static void -CleanupAliasData(struct libalias *la) +CleanupAliasData(struct libalias *la, int deletePermanent) { struct alias_link *lnk, *lnk_tmp; + u_int i; LIBALIAS_LOCK_ASSERT(la); /* permanent entries may stay */ - TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, list_expire, lnk_tmp) - DeleteLink(&lnk); + TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, expire.list, lnk_tmp) + DeleteLink(&lnk, deletePermanent); + + for (i = 0; i < LINK_TABLE_IN_SIZE; i++) { + struct group_in *grp, *grp_tmp; + + LIST_FOREACH_SAFE(grp, &la->groupTableIn[i], group_in, grp_tmp) + if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { + LIST_REMOVE(grp, group_in); + free(grp); + } + } } static void -CleanupLink(struct libalias *la, struct alias_link **lnk) +CleanupLink(struct libalias *la, struct alias_link **lnk, int deletePermanent) { LIBALIAS_LOCK_ASSERT(la); if (lnk == NULL || *lnk == NULL) return; - if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire_time) { - DeleteLink(lnk); + if (LibAliasTime - (*lnk)->timestamp > (*lnk)->expire.time) { + DeleteLink(lnk, deletePermanent); if ((*lnk) == NULL) return; } /* move to end, swap may fail on a single entry list */ - TAILQ_REMOVE(&la->checkExpire, (*lnk), list_expire); - TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), list_expire); + TAILQ_REMOVE(&la->checkExpire, (*lnk), expire.list); + TAILQ_INSERT_TAIL(&la->checkExpire, (*lnk), expire.list); +} + +static struct alias_link * +UseLink(struct libalias *la, struct alias_link *lnk) +{ + CleanupLink(la, &lnk, 0); + if (lnk != NULL) + lnk->timestamp = LibAliasTime; + return (lnk); } static void -DeleteLink(struct alias_link **plnk) +DeleteLink(struct alias_link **plnk, int deletePermanent) { struct alias_link *lnk = *plnk; struct libalias *la = lnk->la; LIBALIAS_LOCK_ASSERT(la); /* Don't do anything if the link is marked permanent */ - if (la->deleteAllLinks == 0 && lnk->flags & LINK_PERMANENT) + if (!deletePermanent && (lnk->flags & LINK_PERMANENT)) return; #ifndef NO_FW_PUNCH @@ -848,24 +877,32 @@ DeleteLink(struct alias_link **plnk) ClearFWHole(lnk); #endif - /* Free memory allocated for LSNAT server pool */ - if (lnk->server != NULL) { - struct server *head, *curr, *next; + switch (lnk->link_type) { + case LINK_PPTP: + LIST_REMOVE(lnk, pptp.list); + break; + default: + /* Free memory allocated for LSNAT server pool */ + if (lnk->server != NULL) { + struct server *head, *curr, *next; + + head = curr = lnk->server; + do { + next = curr->next; + free(curr); + } while ((curr = next) != head); + } else { + /* Adjust output table pointers */ + LIST_REMOVE(lnk, all.out); + } - head = curr = lnk->server; - do { - next = curr->next; - free(curr); - } while ((curr = next) != head); + /* Adjust input table pointers */ + LIST_REMOVE(lnk, all.in); + break; } - /* Adjust output table pointers */ - LIST_REMOVE(lnk, list_out); - - /* Adjust input table pointers */ - LIST_REMOVE(lnk, list_in); /* remove from housekeeping */ - TAILQ_REMOVE(&la->checkExpire, lnk, list_expire); + TAILQ_REMOVE(&la->checkExpire, lnk, expire.list); #ifndef NO_USE_SOCKETS /* Close socket, if one has been allocated */ @@ -923,133 +960,148 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); + lnk = malloc(sizeof(struct alias_link)); - if (lnk != NULL) { - /* Basic initialization */ - lnk->la = la; - lnk->src_addr = src_addr; - lnk->dst_addr = dst_addr; - lnk->alias_addr = alias_addr; - lnk->proxy_addr.s_addr = INADDR_ANY; - lnk->src_port = src_port; - lnk->dst_port = dst_port; - lnk->proxy_port = 0; - lnk->server = NULL; - lnk->link_type = link_type; + if (lnk == NULL) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/AddLink(): "); + fprintf(stderr, "malloc() call failed.\n"); +#endif + return (NULL); + } + /* Basic initialization */ + lnk->la = la; + lnk->src_addr = src_addr; + lnk->dst_addr = dst_addr; + lnk->alias_addr = alias_addr; + lnk->proxy_addr.s_addr = INADDR_ANY; + lnk->src_port = src_port; + lnk->dst_port = dst_port; + lnk->proxy_port = 0; + lnk->server = NULL; + lnk->link_type = link_type; #ifndef NO_USE_SOCKETS - lnk->sockfd = -1; + lnk->sockfd = -1; #endif - lnk->flags = 0; - lnk->pflags = 0; - lnk->timestamp = LibAliasTime; + lnk->flags = 0; + lnk->pflags = 0; + lnk->timestamp = LibAliasTime; - /* Expiration time */ - switch (link_type) { - case LINK_ICMP: - lnk->expire_time = ICMP_EXPIRE_TIME; - break; - case LINK_UDP: - lnk->expire_time = UDP_EXPIRE_TIME; - break; - case LINK_TCP: - lnk->expire_time = TCP_EXPIRE_INITIAL; - break; - case LINK_PPTP: - lnk->flags |= LINK_PERMANENT; /* no timeout. */ - break; - case LINK_FRAGMENT_ID: - lnk->expire_time = FRAGMENT_ID_EXPIRE_TIME; - break; - case LINK_FRAGMENT_PTR: - lnk->expire_time = FRAGMENT_PTR_EXPIRE_TIME; - break; - case LINK_ADDR: - break; - default: - lnk->expire_time = PROTO_EXPIRE_TIME; - break; - } + /* Expiration time */ + switch (link_type) { + case LINK_ICMP: + lnk->expire.time = ICMP_EXPIRE_TIME; + break; + case LINK_UDP: + lnk->expire.time = UDP_EXPIRE_TIME; + break; + case LINK_TCP: + lnk->expire.time = TCP_EXPIRE_INITIAL; + break; + case LINK_FRAGMENT_ID: + lnk->expire.time = FRAGMENT_ID_EXPIRE_TIME; + break; + case LINK_FRAGMENT_PTR: + lnk->expire.time = FRAGMENT_PTR_EXPIRE_TIME; + break; + default: + lnk->expire.time = PROTO_EXPIRE_TIME; + break; + } + + /* Determine alias flags */ + if (dst_addr.s_addr == INADDR_ANY) + lnk->flags |= LINK_UNKNOWN_DEST_ADDR; + if (dst_port == 0) + lnk->flags |= LINK_UNKNOWN_DEST_PORT; - /* Determine alias flags */ - if (dst_addr.s_addr == INADDR_ANY) - lnk->flags |= LINK_UNKNOWN_DEST_ADDR; - if (dst_port == 0) - lnk->flags |= LINK_UNKNOWN_DEST_PORT; + /* Determine alias port */ + if (GetNewPort(la, lnk, alias_port_param) != 0) { + free(lnk); + return (NULL); + } + /* Link-type dependent initialization */ + switch (link_type) { + case LINK_ICMP: + la->icmpLinkCount++; + break; + case LINK_UDP: + la->udpLinkCount++; + break; + case LINK_TCP: { + struct tcp_dat *aux_tcp; + int i; - /* Determine alias port */ - if (GetNewPort(la, lnk, alias_port_param) != 0) { + aux_tcp = malloc(sizeof(struct tcp_dat)); + if (aux_tcp == NULL) { +#ifdef LIBALIAS_DEBUG + fprintf(stderr, "PacketAlias/AddLink: "); + fprintf(stderr, " cannot allocate auxiliary TCP data\n"); +#endif free(lnk); return (NULL); } - /* Link-type dependent initialization */ - switch (link_type) { - struct tcp_dat *aux_tcp; - case LINK_ICMP: - la->icmpLinkCount++; - break; - case LINK_UDP: - la->udpLinkCount++; - break; - case LINK_TCP: - aux_tcp = malloc(sizeof(struct tcp_dat)); - if (aux_tcp != NULL) { - int i; - - la->tcpLinkCount++; - aux_tcp->state.in = ALIAS_TCP_STATE_NOT_CONNECTED; - aux_tcp->state.out = ALIAS_TCP_STATE_NOT_CONNECTED; - aux_tcp->state.index = 0; - aux_tcp->state.ack_modified = 0; - for (i = 0; i < N_LINK_TCP_DATA; i++) - aux_tcp->ack[i].active = 0; - aux_tcp->fwhole = -1; - lnk->data.tcp = aux_tcp; - } else { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/AddLink: "); - fprintf(stderr, " cannot allocate auxiliary TCP data\n"); -#endif - free(lnk); - return (NULL); - } - break; - case LINK_PPTP: - la->pptpLinkCount++; - break; - case LINK_FRAGMENT_ID: - la->fragmentIdLinkCount++; - break; - case LINK_FRAGMENT_PTR: - la->fragmentPtrLinkCount++; - break; - case LINK_ADDR: - break; - default: - la->protoLinkCount++; - break; + la->tcpLinkCount++; + aux_tcp->state.in = ALIAS_TCP_STATE_NOT_CONNECTED; + aux_tcp->state.out = ALIAS_TCP_STATE_NOT_CONNECTED; + aux_tcp->state.index = 0; + aux_tcp->state.ack_modified = 0; + for (i = 0; i < N_LINK_TCP_DATA; i++) + aux_tcp->ack[i].active = 0; + aux_tcp->fwhole = -1; + lnk->data.tcp = aux_tcp; + } + break; + case LINK_PPTP: + la->pptpLinkCount++; + break; + case LINK_FRAGMENT_ID: + la->fragmentIdLinkCount++; + break; + case LINK_FRAGMENT_PTR: + la->fragmentPtrLinkCount++; + break; + case LINK_ADDR: + break; + default: + la->protoLinkCount++; + break; + } + + switch (link_type) { + case LINK_PPTP: + LIST_INSERT_HEAD(&la->pptpList, lnk, pptp.list); + break; + default: { + struct group_in *grp; + + grp = StartPointIn(la, alias_addr, lnk->alias_port, link_type, 1); + if (grp == NULL) { + free(lnk); + return (NULL); } /* Set up pointers for output lookup table */ start_point = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, list_out); + LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, all.out); /* Set up pointers for input lookup table */ - start_point = StartPointIn(alias_addr, lnk->alias_port, link_type); - LIST_INSERT_HEAD(&la->linkTableIn[start_point], lnk, list_in); - - /* Include the element into the housekeeping list */ - TAILQ_INSERT_TAIL(&la->checkExpire, lnk, list_expire); - } else { -#ifdef LIBALIAS_DEBUG - fprintf(stderr, "PacketAlias/AddLink(): "); - fprintf(stderr, "malloc() call failed.\n"); -#endif + if (lnk->flags & LINK_PARTIALLY_SPECIFIED) + LIST_INSERT_HEAD(&grp->partial, lnk, all.in); + else + LIST_INSERT_HEAD(&grp->full, lnk, all.in); } - if (la->packetAliasMode & PKT_ALIAS_LOG) { - ShowAliasStats(la); + break; } + + /* Include the element into the housekeeping list */ + TAILQ_INSERT_TAIL(&la->checkExpire, lnk, expire.list); + + if (la->packetAliasMode & PKT_ALIAS_LOG) + ShowAliasStats(la); + return (lnk); } @@ -1065,7 +1117,8 @@ ReLink(struct alias_link *old_lnk, u_short src_port, u_short dst_port, int alias_port_param, - int link_type) + int link_type, + int deletePermanent) { struct alias_link *new_lnk; struct libalias *la = old_lnk->la; @@ -1081,10 +1134,39 @@ ReLink(struct alias_link *old_lnk, PunchFWHole(new_lnk); } #endif - DeleteLink(&old_lnk); + DeleteLink(&old_lnk, deletePermanent); return (new_lnk); } + +#define OUTGUARD \ + if (lnk->src_port != src_port || \ + lnk->src_addr.s_addr != src_addr.s_addr || \ + lnk->dst_addr.s_addr != dst_addr.s_addr || \ + lnk->dst_port != dst_port || \ + lnk->link_type != link_type) \ + continue; + +static struct alias_link * +_SearchLinkOut(struct libalias *la, struct in_addr src_addr, + struct in_addr dst_addr, + u_short src_port, + u_short dst_port, + int link_type) { + u_int i; + struct alias_link *lnk; + + i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); + LIST_FOREACH(lnk, &la->linkTableOut[i], all.out) { + OUTGUARD; + return (UseLink(la, lnk)); + } + + return (NULL); +} + +#undef OUTGUARD + static struct alias_link * _FindLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, @@ -1093,45 +1175,31 @@ _FindLinkOut(struct libalias *la, struct in_addr src_addr, int link_type, int replace_partial_links) { - u_int i; struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); - i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_FOREACH(lnk, &la->linkTableOut[i], list_out) { - if (lnk->dst_addr.s_addr == dst_addr.s_addr && - lnk->src_addr.s_addr == src_addr.s_addr && - lnk->src_port == src_port && - lnk->dst_port == dst_port && - lnk->link_type == link_type && - lnk->server == NULL) - break; - } - - CleanupLink(la, &lnk); - if (lnk != NULL) - lnk->timestamp = LibAliasTime; + lnk = _SearchLinkOut(la, src_addr, dst_addr, src_port, dst_port, link_type); + if (lnk != NULL || !replace_partial_links) + return (lnk); /* Search for partially specified links. */ - if (lnk == NULL && replace_partial_links) { - if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { - lnk = _FindLinkOut(la, src_addr, dst_addr, src_port, 0, - link_type, 0); - if (lnk == NULL) - lnk = _FindLinkOut(la, src_addr, ANY_ADDR, src_port, - dst_port, link_type, 0); - } - if (lnk == NULL && - (dst_port != 0 || dst_addr.s_addr != INADDR_ANY)) { - lnk = _FindLinkOut(la, src_addr, ANY_ADDR, src_port, 0, - link_type, 0); - } - if (lnk != NULL) { - lnk = ReLink(lnk, - src_addr, dst_addr, lnk->alias_addr, - src_port, dst_port, lnk->alias_port, - link_type); - } + if (dst_port != 0 && dst_addr.s_addr != INADDR_ANY) { + lnk = _SearchLinkOut(la, src_addr, dst_addr, src_port, 0, + link_type); + if (lnk == NULL) + lnk = _SearchLinkOut(la, src_addr, ANY_ADDR, src_port, + dst_port, link_type); + } + if (lnk == NULL && + (dst_port != 0 || dst_addr.s_addr != INADDR_ANY)) { + lnk = _SearchLinkOut(la, src_addr, ANY_ADDR, src_port, 0, + link_type); + } + if (lnk != NULL) { + lnk = ReLink(lnk, + src_addr, dst_addr, lnk->alias_addr, + src_port, dst_port, lnk->alias_port, + link_type, 0); } return (lnk); } @@ -1175,16 +1243,16 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, int replace_partial_links) { int flags_in; - u_int start_point; + struct group_in *grp; struct alias_link *lnk; - struct alias_link *lnk_fully_specified; struct alias_link *lnk_unknown_all; struct alias_link *lnk_unknown_dst_addr; struct alias_link *lnk_unknown_dst_port; + struct in_addr src_addr; + u_short src_port; LIBALIAS_LOCK_ASSERT(la); /* Initialize pointers */ - lnk_fully_specified = NULL; lnk_unknown_all = NULL; lnk_unknown_dst_addr = NULL; lnk_unknown_dst_port = NULL; @@ -1198,83 +1266,82 @@ _FindLinkIn(struct libalias *la, struct in_addr dst_addr, flags_in |= LINK_UNKNOWN_DEST_PORT; /* Search loop */ - start_point = StartPointIn(alias_addr, alias_port, link_type); - LIST_FOREACH(lnk, &la->linkTableIn[start_point], list_in) { - int flags; - - flags = flags_in | lnk->flags; - if (!(flags & LINK_PARTIALLY_SPECIFIED)) { - if (lnk->alias_addr.s_addr == alias_addr.s_addr - && lnk->alias_port == alias_port - && lnk->dst_addr.s_addr == dst_addr.s_addr *** 448 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 07:11:56 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ACED1651F21; Tue, 6 Jul 2021 07:11:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GJtz01HKvz4Zt1; Tue, 6 Jul 2021 07:11:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0519515803; Tue, 6 Jul 2021 07:11:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1667BthX089740; Tue, 6 Jul 2021 07:11:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1667Btd5089739; Tue, 6 Jul 2021 07:11:55 GMT (envelope-from git) Date: Tue, 6 Jul 2021 07:11:55 GMT Message-Id: <202107060711.1667Btd5089739@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Lutz Donnerhacke Subject: git: 1883127de488 - stable/12 - libalias: Switch to SPLAY trees MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: donner X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 1883127de4888b2a30a6cb51e8fb4bdf33b7f411 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 07:11:57 -0000 The branch stable/12 has been updated by donner: URL: https://cgit.FreeBSD.org/src/commit/?id=1883127de4888b2a30a6cb51e8fb4bdf33b7f411 commit 1883127de4888b2a30a6cb51e8fb4bdf33b7f411 Author: Lutz Donnerhacke AuthorDate: 2021-05-27 21:42:54 +0000 Commit: Lutz Donnerhacke CommitDate: 2021-07-06 07:10:02 +0000 libalias: Switch to SPLAY trees Current data structure is using a hash of unordered lists. Those unordered lists are quite efficient, because the least recently inserted entries are most likely to be used again. In order to avoid long search times in other cases, the lists are hashed into many buckets. Unfortunatly a search for a miss needs an exhaustive inspection and a careful definition of the hash. Splay trees offer a similar feature - almost O(1) for access of the least recently used entries), and amortized O(ln(n) - for almost all other cases. Get rid of the hash. Now the data structure should able to quickly react to external packets without eating CPU cycles for breakfast, preventing a DoS. PR: 192888 Discussed with: Dimitry Luhtionov Differential Revision: https://reviews.freebsd.org/D30516 Differential Revision: https://reviews.freebsd.org/D30536 Differential Revision: https://reviews.freebsd.org/D30844 (cherry picked from commit 935fc93af157dee352eb4b6c83f8a2a9e7fd9a4e) (cherry picked from commit d261e57deacb0d00d9e827447f235df83dda3e3a) (cherry picked from commit f70c98a2f5d993dc518efd606aa341eda99400ef) (cherry picked from commit 25392fac9488bcae5c451500df2e2945430484a6) (cherry picked from commit 2f4d91f9cb22fc65eb65407e8118b433a5d71976) (cherry picked from commit 4060e77f49d1b9fd2254f3f4da94fd64fce83f72) --- sys/netinet/libalias/HISTORY | 3 +- sys/netinet/libalias/alias_db.c | 502 +++---------------------------------- sys/netinet/libalias/alias_db.h | 443 ++++++++++++++++++++++++++++++++ sys/netinet/libalias/alias_local.h | 10 +- 4 files changed, 489 insertions(+), 469 deletions(-) diff --git a/sys/netinet/libalias/HISTORY b/sys/netinet/libalias/HISTORY index 8965290a4f27..87d5b6f8ed76 100644 --- a/sys/netinet/libalias/HISTORY +++ b/sys/netinet/libalias/HISTORY @@ -146,4 +146,5 @@ Version 3.2: July, 2000 (Erik Salander, erik@whistle.com and Version 3.3: May 2021 (donner) - Dropped LibAliasCheckNewLink - - Refactor the database handling, perfomance improvements. + - Refactor the database handling + - Switch to more effienct SPLAY trees diff --git a/sys/netinet/libalias/alias_db.c b/sys/netinet/libalias/alias_db.c index b056f55b56a5..783a8ca3baa8 100644 --- a/sys/netinet/libalias/alias_db.c +++ b/sys/netinet/libalias/alias_db.c @@ -29,120 +29,6 @@ #include __FBSDID("$FreeBSD$"); -/* - Alias_db.c encapsulates all data structures used for storing - packet aliasing data. Other parts of the aliasing software - access data through functions provided in this file. - - Data storage is based on the notion of a "link", which is - established for ICMP echo/reply packets, UDP datagrams and - TCP stream connections. A link stores the original source - and destination addresses. For UDP and TCP, it also stores - source and destination port numbers, as well as an alias - port number. Links are also used to store information about - fragments. - - There is a facility for sweeping through and deleting old - links as new packets are sent through. A simple timeout is - used for ICMP and UDP links. TCP links are left alone unless - there is an incomplete connection, in which case the link - can be deleted after a certain amount of time. - - Initial version: August, 1996 (cjm) - - Version 1.4: September 16, 1996 (cjm) - Facility for handling incoming links added. - - Version 1.6: September 18, 1996 (cjm) - ICMP data handling simplified. - - Version 1.7: January 9, 1997 (cjm) - Fragment handling simplified. - Saves pointers for unresolved fragments. - Permits links for unspecified remote ports - or unspecified remote addresses. - Fixed bug which did not properly zero port - table entries after a link was deleted. - Cleaned up some obsolete comments. - - Version 1.8: January 14, 1997 (cjm) - Fixed data type error in StartPoint(). - (This error did not exist prior to v1.7 - and was discovered and fixed by Ari Suutari) - - Version 1.9: February 1, 1997 - Optionally, connections initiated from packet aliasing host - machine will will not have their port number aliased unless it - conflicts with an aliasing port already being used. (cjm) - - All options earlier being #ifdef'ed are now available through - a new interface, SetPacketAliasMode(). This allows run time - control (which is now available in PPP+pktAlias through the - 'alias' keyword). (ee) - - Added ability to create an alias port without - either destination address or port specified. - port type = ALIAS_PORT_UNKNOWN_DEST_ALL (ee) - - Removed K&R style function headers - and general cleanup. (ee) - - Added packetAliasMode to replace compiler #defines's (ee) - - Allocates sockets for partially specified - ports if ALIAS_USE_SOCKETS defined. (cjm) - - Version 2.0: March, 1997 - SetAliasAddress() will now clean up alias links - if the aliasing address is changed. (cjm) - - PacketAliasPermanentLink() function added to support permanent - links. (J. Fortes suggested the need for this.) - Examples: - - (192.168.0.1, port 23) <-> alias port 6002, unknown dest addr/port - - (192.168.0.2, port 21) <-> alias port 3604, known dest addr - unknown dest port - - These permanent links allow for incoming connections to - machines on the local network. They can be given with a - user-chosen amount of specificity, with increasing specificity - meaning more security. (cjm) - - Quite a bit of rework to the basic engine. The portTable[] - array, which kept track of which ports were in use was replaced - by a table/linked list structure. (cjm) - - SetExpire() function added. (cjm) - - DeleteLink() no longer frees memory association with a pointer - to a fragment (this bug was first recognized by E. Eklund in - v1.9). - - Version 2.1: May, 1997 (cjm) - Packet aliasing engine reworked so that it can handle - multiple external addresses rather than just a single - host address. - - PacketAliasRedirectPort() and PacketAliasRedirectAddr() - added to the API. The first function is a more generalized - version of PacketAliasPermanentLink(). The second function - implements static network address translation. - - Version 3.2: July, 2000 (salander and satoh) - Added FindNewPortGroup to get contiguous range of port values. - - Added QueryUdpTcpIn and QueryUdpTcpOut to look for an aliasing - link but not actually add one. - - Added FindRtspOut, which is closely derived from FindUdpTcpOut, - except that the alias port (from FindNewPortGroup) is provided - as input. - - See HISTORY file for additional revisions. -*/ - #ifdef _KERNEL #include #include @@ -175,177 +61,11 @@ __FBSDID("$FreeBSD$"); #include "alias_mod.h" #endif +#include "alias_db.h" + static LIST_HEAD(, libalias) instancehead = LIST_HEAD_INITIALIZER(instancehead); int LibAliasTime; -/* - Constants (note: constants are also defined - near relevant functions or structs) -*/ - -/* Timeouts (in seconds) for different link types */ -#define ICMP_EXPIRE_TIME 60 -#define UDP_EXPIRE_TIME 60 -#define PROTO_EXPIRE_TIME 60 -#define FRAGMENT_ID_EXPIRE_TIME 10 -#define FRAGMENT_PTR_EXPIRE_TIME 30 - -/* TCP link expire time for different cases */ -/* When the link has been used and closed - minimal grace time to - allow ACKs and potential re-connect in FTP (XXX - is this allowed?) */ -#ifndef TCP_EXPIRE_DEAD -#define TCP_EXPIRE_DEAD 10 -#endif - -/* When the link has been used and closed on one side - the other side - is allowed to still send data */ -#ifndef TCP_EXPIRE_SINGLEDEAD -#define TCP_EXPIRE_SINGLEDEAD 90 -#endif - -/* When the link isn't yet up */ -#ifndef TCP_EXPIRE_INITIAL -#define TCP_EXPIRE_INITIAL 300 -#endif - -/* When the link is up */ -#ifndef TCP_EXPIRE_CONNECTED -#define TCP_EXPIRE_CONNECTED 86400 -#endif - -/* Dummy port number codes used for FindLinkIn/Out() and AddLink(). - These constants can be anything except zero, which indicates an - unknown port number. */ - -#define NO_DEST_PORT 1 -#define NO_SRC_PORT 1 - -/* Matches any/unknown address in FindLinkIn/Out() and AddLink(). */ -static struct in_addr const ANY_ADDR = { INADDR_ANY }; - -/* Data Structures - - The fundamental data structure used in this program is - "struct alias_link". Whenever a TCP connection is made, - a UDP datagram is sent out, or an ICMP echo request is made, - a link record is made (if it has not already been created). - The link record is identified by the source address/port - and the destination address/port. In the case of an ICMP - echo request, the source port is treated as being equivalent - with the 16-bit ID number of the ICMP packet. - - The link record also can store some auxiliary data. For - TCP connections that have had sequence and acknowledgment - modifications, data space is available to track these changes. - A state field is used to keep track in changes to the TCP - connection state. ID numbers of fragments can also be - stored in the auxiliary space. Pointers to unresolved - fragments can also be stored. - - The link records support two independent chainings. Lookup - tables for input and out tables hold the initial pointers - the link chains. On input, the lookup table indexes on alias - port and link type. On output, the lookup table indexes on - source address, destination address, source port, destination - port and link type. -*/ - -/* used to save changes to ACK/sequence numbers */ -struct ack_data_record { - u_long ack_old; - u_long ack_new; - int delta; - int active; -}; - -/* Information about TCP connection */ -struct tcp_state { - int in; /* State for outside -> inside */ - int out; /* State for inside -> outside */ - int index; /* Index to ACK data array */ - /* Indicates whether ACK and sequence numbers been modified */ - int ack_modified; -}; - -/* Number of distinct ACK number changes - * saved for a modified TCP stream */ -#define N_LINK_TCP_DATA 3 -struct tcp_dat { - struct tcp_state state; - struct ack_data_record ack[N_LINK_TCP_DATA]; - /* Which firewall record is used for this hole? */ - int fwhole; -}; - -/* LSNAT server pool (circular list) */ -struct server { - struct in_addr addr; - u_short port; - struct server *next; -}; - -/* Main data structure */ -struct alias_link { - struct libalias *la; - /* Address and port information */ - struct in_addr src_addr; - struct in_addr dst_addr; - struct in_addr alias_addr; - struct in_addr proxy_addr; - u_short src_port; - u_short dst_port; - u_short alias_port; - u_short proxy_port; - struct server *server; - /* Type of link: TCP, UDP, ICMP, proto, frag */ - int link_type; -/* values for link_type */ -#define LINK_ICMP IPPROTO_ICMP -#define LINK_UDP IPPROTO_UDP -#define LINK_TCP IPPROTO_TCP -#define LINK_FRAGMENT_ID (IPPROTO_MAX + 1) -#define LINK_FRAGMENT_PTR (IPPROTO_MAX + 2) -#define LINK_ADDR (IPPROTO_MAX + 3) -#define LINK_PPTP (IPPROTO_MAX + 4) - - int flags; /* indicates special characteristics */ - int pflags; /* protocol-specific flags */ -/* flag bits */ -#define LINK_UNKNOWN_DEST_PORT 0x01 -#define LINK_UNKNOWN_DEST_ADDR 0x02 -#define LINK_PERMANENT 0x04 -#define LINK_PARTIALLY_SPECIFIED 0x03 /* logical-or of first two bits */ -#define LINK_UNFIREWALLED 0x08 - - int timestamp; /* Time link was last accessed */ -#ifndef NO_USE_SOCKETS - int sockfd; /* socket descriptor */ -#endif - /* Linked list of pointers for input and output lookup tables */ - union { - struct { - LIST_ENTRY(alias_link) in; - LIST_ENTRY(alias_link) out; - } all; - struct { - LIST_ENTRY(alias_link) list; - } pptp; - }; - struct { - TAILQ_ENTRY(alias_link) list; - int time; /* Expire time for link */ - } expire; - /* Auxiliary data */ - union { - char *frag_ptr; - struct in_addr frag_addr; - struct tcp_dat *tcp; - } data; -}; - -/* Clean up procedure. */ -static void finishoff(void); - /* Kernel module definition. */ #ifdef _KERNEL MALLOC_DEFINE(M_ALIAS, "libalias", "packet aliasing"); @@ -373,96 +93,32 @@ static moduledata_t alias_mod = { DECLARE_MODULE(alias, alias_mod, SI_SUB_DRIVERS, SI_ORDER_SECOND); #endif -/* Internal utility routines (used only in alias_db.c) - -Lookup table starting points: - StartPointIn() -- link table initial search point for - incoming packets - StartPointOut() -- link table initial search point for - outgoing packets - -Miscellaneous: - SeqDiff() -- difference between two TCP sequences - ShowAliasStats() -- send alias statistics to a monitor file -*/ - -/* Local prototypes */ -static struct group_in * -StartPointIn(struct libalias *, struct in_addr, u_short, int, int); -static u_int -StartPointOut(struct in_addr, struct in_addr, u_short, u_short, int); -static int SeqDiff(u_long, u_long); - -#ifndef NO_FW_PUNCH -/* Firewall control */ -static void InitPunchFW(struct libalias *); -static void UninitPunchFW(struct libalias *); -static void ClearFWHole(struct alias_link *); - -#endif - -/* Log file control */ -static void ShowAliasStats(struct libalias *); -static int InitPacketAliasLog(struct libalias *); -static void UninitPacketAliasLog(struct libalias *); - -void SctpShowAliasStats(struct libalias *la); - -#define INGUARD \ - if (grp->alias_port != alias_port || \ - grp->link_type != link_type || \ - grp->alias_addr.s_addr != alias_addr.s_addr) \ - continue; +SPLAY_GENERATE(splay_out, alias_link, all.out, cmp_out); +SPLAY_GENERATE(splay_in, group_in, in, cmp_in); static struct group_in * StartPointIn(struct libalias *la, struct in_addr alias_addr, u_short alias_port, int link_type, int create) { - u_int n; - struct group_in *grp, *tmp; - - n = alias_addr.s_addr; - n += alias_port; - n += link_type; - n %= LINK_TABLE_IN_SIZE; + struct group_in *grp; + struct group_in needle = { + .alias_addr = alias_addr, + .alias_port = alias_port, + .link_type = link_type + }; - LIST_FOREACH_SAFE(grp, &la->groupTableIn[n], group_in, tmp) { - /* Auto cleanup */ - if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { - LIST_REMOVE(grp, group_in); - free(grp); - } else { - INGUARD; - return (grp); - } - } - if (!create || (grp = malloc(sizeof(*grp))) == NULL) + grp = SPLAY_FIND(splay_in, &la->linkSplayIn, &needle); + if (grp != NULL || !create || (grp = malloc(sizeof(*grp))) == NULL) return (grp); grp->alias_addr = alias_addr; grp->alias_port = alias_port; grp->link_type = link_type; LIST_INIT(&grp->full); LIST_INIT(&grp->partial); - LIST_INSERT_HEAD(&la->groupTableIn[n], grp, group_in); + SPLAY_INSERT(splay_in, &la->linkSplayIn, grp); return (grp); } -#undef INGUARD - -static u_int -StartPointOut(struct in_addr src_addr, struct in_addr dst_addr, - u_short src_port, u_short dst_port, int link_type) -{ - u_int n; - - n = src_addr.s_addr; - n += dst_addr.s_addr; - n += src_port; - n += dst_port; - n += link_type; - - return (n % LINK_TABLE_OUT_SIZE); -} static int SeqDiff(u_long x, u_long y) @@ -531,53 +187,6 @@ void SctpShowAliasStats(struct libalias *la) ShowAliasStats(la); } -/* Internal routines for finding, deleting and adding links - -Port Allocation: - GetNewPort() -- find and reserve new alias port number - GetSocket() -- try to allocate a socket for a given port - -Link creation and deletion: - CleanupAliasData() - remove all link chains from lookup table - CleanupLink() - look for a stale link - DeleteLink() - remove link - AddLink() - add link - ReLink() - change link - -Link search: - FindLinkOut() - find link for outgoing packets - FindLinkIn() - find link for incoming packets - -Port search: - FindNewPortGroup() - find an available group of ports -*/ - -/* Local prototypes */ -static int GetNewPort(struct libalias *, struct alias_link *, int); -#ifndef NO_USE_SOCKETS -static u_short GetSocket(struct libalias *, u_short, int *, int); -#endif -static void CleanupAliasData(struct libalias *, int); -static void CleanupLink(struct libalias *, struct alias_link **, int); -static void DeleteLink(struct alias_link **, int); -static struct alias_link * -UseLink(struct libalias *, struct alias_link *); - -static struct alias_link * -ReLink(struct alias_link *, - struct in_addr, struct in_addr, struct in_addr, - u_short, u_short, int, int, int); - -static struct alias_link * -FindLinkOut(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); - -static struct alias_link * -FindLinkIn(struct libalias *, struct in_addr, struct in_addr, u_short, u_short, int, int); - -static u_short _RandomPort(struct libalias *la); - -#define GET_NEW_PORT_MAX_ATTEMPTS 20 - /* get random port in network byte order */ static u_short _RandomPort(struct libalias *la) { @@ -814,25 +423,13 @@ static void CleanupAliasData(struct libalias *la, int deletePermanent) { struct alias_link *lnk, *lnk_tmp; - u_int i; LIBALIAS_LOCK_ASSERT(la); /* permanent entries may stay */ TAILQ_FOREACH_SAFE(lnk, &la->checkExpire, expire.list, lnk_tmp) DeleteLink(&lnk, deletePermanent); - - for (i = 0; i < LINK_TABLE_IN_SIZE; i++) { - struct group_in *grp, *grp_tmp; - - LIST_FOREACH_SAFE(grp, &la->groupTableIn[i], group_in, grp_tmp) - if (LIST_EMPTY(&grp->full) && LIST_EMPTY(&grp->partial)) { - LIST_REMOVE(grp, group_in); - free(grp); - } - } } - static void CleanupLink(struct libalias *la, struct alias_link **lnk, int deletePermanent) { @@ -881,7 +478,9 @@ DeleteLink(struct alias_link **plnk, int deletePermanent) case LINK_PPTP: LIST_REMOVE(lnk, pptp.list); break; - default: + default: { + struct group_in *grp; + /* Free memory allocated for LSNAT server pool */ if (lnk->server != NULL) { struct server *head, *curr, *next; @@ -893,11 +492,21 @@ DeleteLink(struct alias_link **plnk, int deletePermanent) } while ((curr = next) != head); } else { /* Adjust output table pointers */ - LIST_REMOVE(lnk, all.out); + SPLAY_REMOVE(splay_out, &la->linkSplayOut, lnk); } /* Adjust input table pointers */ LIST_REMOVE(lnk, all.in); + + /* Remove intermediate node, if empty */ + grp = StartPointIn(la, lnk->alias_addr, lnk->alias_port, lnk->link_type, 0); + if (grp != NULL && + LIST_EMPTY(&grp->full) && + LIST_EMPTY(&grp->partial)) { + SPLAY_REMOVE(splay_in, &la->linkSplayIn, grp); + free(grp); + } + } break; } @@ -956,7 +565,6 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, struct in_addr alias_addr, u_short src_port, u_short dst_port, int alias_port_param, int link_type) { - u_int start_point; struct alias_link *lnk; LIBALIAS_LOCK_ASSERT(la); @@ -1083,9 +691,7 @@ AddLink(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, } /* Set up pointers for output lookup table */ - start_point = StartPointOut(src_addr, dst_addr, - src_port, dst_port, link_type); - LIST_INSERT_HEAD(&la->linkTableOut[start_point], lnk, all.out); + SPLAY_INSERT(splay_out, &la->linkSplayOut, lnk); /* Set up pointers for input lookup table */ if (lnk->flags & LINK_PARTIALLY_SPECIFIED) @@ -1138,35 +744,25 @@ ReLink(struct alias_link *old_lnk, return (new_lnk); } - -#define OUTGUARD \ - if (lnk->src_port != src_port || \ - lnk->src_addr.s_addr != src_addr.s_addr || \ - lnk->dst_addr.s_addr != dst_addr.s_addr || \ - lnk->dst_port != dst_port || \ - lnk->link_type != link_type) \ - continue; - static struct alias_link * _SearchLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, u_short src_port, u_short dst_port, int link_type) { - u_int i; struct alias_link *lnk; + struct alias_link needle = { + .src_addr = src_addr, + .dst_addr = dst_addr, + .src_port = src_port, + .dst_port = dst_port, + .link_type = link_type + }; - i = StartPointOut(src_addr, dst_addr, src_port, dst_port, link_type); - LIST_FOREACH(lnk, &la->linkTableOut[i], all.out) { - OUTGUARD; - return (UseLink(la, lnk)); - } - - return (NULL); + lnk = SPLAY_FIND(splay_out, &la->linkSplayOut, &needle); + return (UseLink(la, lnk)); } -#undef OUTGUARD - static struct alias_link * _FindLinkOut(struct libalias *la, struct in_addr src_addr, struct in_addr dst_addr, @@ -2331,7 +1927,7 @@ LibAliasAddServer(struct libalias *la, struct alias_link *lnk, struct in_addr ad if (head == NULL) { server->next = server; /* not usable for outgoing connections */ - LIST_REMOVE(lnk, all.out); + SPLAY_REMOVE(splay_out, &la->linkSplayOut, lnk); } else { struct server *s; @@ -2481,8 +2077,6 @@ finishoff(void) struct libalias * LibAliasInit(struct libalias *la) { - int i; - if (la == NULL) { #ifdef _KERNEL #undef malloc /* XXX: ugly */ @@ -2506,10 +2100,8 @@ LibAliasInit(struct libalias *la) LibAliasTime = time(NULL); #endif - for (i = 0; i < LINK_TABLE_OUT_SIZE; i++) - LIST_INIT(&la->linkTableOut[i]); - for (i = 0; i < LINK_TABLE_IN_SIZE; i++) - LIST_INIT(&la->groupTableIn[i]); + SPLAY_INIT(&la->linkSplayIn); + SPLAY_INIT(&la->linkSplayOut); LIST_INIT(&la->pptpList); TAILQ_INIT(&la->checkExpire); #ifdef _KERNEL @@ -2702,20 +2294,6 @@ fill_rule(void *buf, int bufsize, int rulenum, return ((char *)cmd - (char *)buf); } -static void ClearAllFWHoles(struct libalias *la); - -#define fw_setfield(la, field, num) \ -do { \ - (field)[(num) - la->fireWallBaseNum] = 1; \ -} /*lint -save -e717 */ while(0)/* lint -restore */ - -#define fw_clrfield(la, field, num) \ -do { \ - (field)[(num) - la->fireWallBaseNum] = 0; \ -} /*lint -save -e717 */ while(0)/* lint -restore */ - -#define fw_tstfield(la, field, num) ((field)[(num) - la->fireWallBaseNum]) - static void InitPunchFW(struct libalias *la) { diff --git a/sys/netinet/libalias/alias_db.h b/sys/netinet/libalias/alias_db.h new file mode 100644 index 000000000000..971ca305c1a6 --- /dev/null +++ b/sys/netinet/libalias/alias_db.h @@ -0,0 +1,443 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause-FreeBSD + * + * Copyright (c) 2001 Charles Mott + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +/* + Alias_db.c encapsulates all data structures used for storing + packet aliasing data. Other parts of the aliasing software + access data through functions provided in this file. + + Data storage is based on the notion of a "link", which is + established for ICMP echo/reply packets, UDP datagrams and + TCP stream connections. A link stores the original source + and destination addresses. For UDP and TCP, it also stores + source and destination port numbers, as well as an alias + port number. Links are also used to store information about + fragments. + + There is a facility for sweeping through and deleting old + links as new packets are sent through. A simple timeout is + used for ICMP and UDP links. TCP links are left alone unless + there is an incomplete connection, in which case the link + can be deleted after a certain amount of time. + + Initial version: August, 1996 (cjm) + + Version 1.4: September 16, 1996 (cjm) + Facility for handling incoming links added. + + Version 1.6: September 18, 1996 (cjm) + ICMP data handling simplified. + + Version 1.7: January 9, 1997 (cjm) + Fragment handling simplified. + Saves pointers for unresolved fragments. + Permits links for unspecified remote ports + or unspecified remote addresses. + Fixed bug which did not properly zero port + table entries after a link was deleted. + Cleaned up some obsolete comments. + + Version 1.8: January 14, 1997 (cjm) + Fixed data type error in StartPoint(). + (This error did not exist prior to v1.7 + and was discovered and fixed by Ari Suutari) + + Version 1.9: February 1, 1997 + Optionally, connections initiated from packet aliasing host + machine will will not have their port number aliased unless it + conflicts with an aliasing port already being used. (cjm) + + All options earlier being #ifdef'ed are now available through + a new interface, SetPacketAliasMode(). This allows run time + control (which is now available in PPP+pktAlias through the + 'alias' keyword). (ee) + + Added ability to create an alias port without + either destination address or port specified. + port type = ALIAS_PORT_UNKNOWN_DEST_ALL (ee) + + Removed K&R style function headers + and general cleanup. (ee) + + Added packetAliasMode to replace compiler #defines's (ee) + + Allocates sockets for partially specified + ports if ALIAS_USE_SOCKETS defined. (cjm) + + Version 2.0: March, 1997 + SetAliasAddress() will now clean up alias links + if the aliasing address is changed. (cjm) + + PacketAliasPermanentLink() function added to support permanent + links. (J. Fortes suggested the need for this.) + Examples: + + (192.168.0.1, port 23) <-> alias port 6002, unknown dest addr/port + + (192.168.0.2, port 21) <-> alias port 3604, known dest addr + unknown dest port + + These permanent links allow for incoming connections to + machines on the local network. They can be given with a + user-chosen amount of specificity, with increasing specificity + meaning more security. (cjm) + + Quite a bit of rework to the basic engine. The portTable[] + array, which kept track of which ports were in use was replaced + by a table/linked list structure. (cjm) + + SetExpire() function added. (cjm) + + DeleteLink() no longer frees memory association with a pointer + to a fragment (this bug was first recognized by E. Eklund in + v1.9). + + Version 2.1: May, 1997 (cjm) + Packet aliasing engine reworked so that it can handle + multiple external addresses rather than just a single + host address. + + PacketAliasRedirectPort() and PacketAliasRedirectAddr() + added to the API. The first function is a more generalized + version of PacketAliasPermanentLink(). The second function + implements static network address translation. + + Version 3.2: July, 2000 (salander and satoh) + Added FindNewPortGroup to get contiguous range of port values. + + Added QueryUdpTcpIn and QueryUdpTcpOut to look for an aliasing + link but not actually add one. + + Added FindRtspOut, which is closely derived from FindUdpTcpOut, + except that the alias port (from FindNewPortGroup) is provided + as input. + + See HISTORY file for additional revisions. +*/ + +#ifndef _ALIAS_DB_H_ +#define _ALIAS_DB_H_ + + +/* + Constants (note: constants are also defined + near relevant functions or structs) +*/ + +/* Timeouts (in seconds) for different link types */ +#define ICMP_EXPIRE_TIME 60 +#define UDP_EXPIRE_TIME 60 +#define PROTO_EXPIRE_TIME 60 +#define FRAGMENT_ID_EXPIRE_TIME 10 +#define FRAGMENT_PTR_EXPIRE_TIME 30 + +/* TCP link expire time for different cases */ +/* When the link has been used and closed - minimal grace time to + allow ACKs and potential re-connect in FTP (XXX - is this allowed?) */ +#ifndef TCP_EXPIRE_DEAD +#define TCP_EXPIRE_DEAD 10 +#endif + +/* When the link has been used and closed on one side - the other side + is allowed to still send data */ +#ifndef TCP_EXPIRE_SINGLEDEAD +#define TCP_EXPIRE_SINGLEDEAD 90 +#endif + +/* When the link isn't yet up */ +#ifndef TCP_EXPIRE_INITIAL +#define TCP_EXPIRE_INITIAL 300 +#endif + +/* When the link is up */ +#ifndef TCP_EXPIRE_CONNECTED +#define TCP_EXPIRE_CONNECTED 86400 +#endif + +/* Dummy port number codes used for FindLinkIn/Out() and AddLink(). + These constants can be anything except zero, which indicates an + unknown port number. */ + +#define NO_DEST_PORT 1 +#define NO_SRC_PORT 1 + +/* Matches any/unknown address in FindLinkIn/Out() and AddLink(). */ +static struct in_addr const ANY_ADDR = { INADDR_ANY }; + +/* Data Structures + + The fundamental data structure used in this program is + "struct alias_link". Whenever a TCP connection is made, + a UDP datagram is sent out, or an ICMP echo request is made, + a link record is made (if it has not already been created). + The link record is identified by the source address/port + and the destination address/port. In the case of an ICMP + echo request, the source port is treated as being equivalent + with the 16-bit ID number of the ICMP packet. + + The link record also can store some auxiliary data. For + TCP connections that have had sequence and acknowledgment + modifications, data space is available to track these changes. + A state field is used to keep track in changes to the TCP + connection state. ID numbers of fragments can also be + stored in the auxiliary space. Pointers to unresolved + fragments can also be stored. + + The link records support two independent chainings. Lookup + tables for input and out tables hold the initial pointers + the link chains. On input, the lookup table indexes on alias + port and link type. On output, the lookup table indexes on + source address, destination address, source port, destination + port and link type. +*/ + +/* used to save changes to ACK/sequence numbers */ +struct ack_data_record { + u_long ack_old; + u_long ack_new; + int delta; + int active; +}; + +/* Information about TCP connection */ +struct tcp_state { + int in; /* State for outside -> inside */ + int out; /* State for inside -> outside */ + int index; /* Index to ACK data array */ + /* Indicates whether ACK and sequence numbers been modified */ + int ack_modified; +}; + +/* Number of distinct ACK number changes + * saved for a modified TCP stream */ +#define N_LINK_TCP_DATA 3 +struct tcp_dat { + struct tcp_state state; + struct ack_data_record ack[N_LINK_TCP_DATA]; + /* Which firewall record is used for this hole? */ + int fwhole; +}; + +/* LSNAT server pool (circular list) */ +struct server { + struct in_addr addr; + u_short port; + struct server *next; +}; + +/* Main data structure */ +struct alias_link { + struct libalias *la; + /* Address and port information */ + struct in_addr src_addr; + struct in_addr dst_addr; + struct in_addr alias_addr; + struct in_addr proxy_addr; + u_short src_port; + u_short dst_port; + u_short alias_port; + u_short proxy_port; + struct server *server; + /* Type of link: TCP, UDP, ICMP, proto, frag */ + int link_type; +/* values for link_type */ +#define LINK_ICMP IPPROTO_ICMP +#define LINK_UDP IPPROTO_UDP +#define LINK_TCP IPPROTO_TCP +#define LINK_FRAGMENT_ID (IPPROTO_MAX + 1) +#define LINK_FRAGMENT_PTR (IPPROTO_MAX + 2) +#define LINK_ADDR (IPPROTO_MAX + 3) +#define LINK_PPTP (IPPROTO_MAX + 4) + + int flags; /* indicates special characteristics */ + int pflags; /* protocol-specific flags */ +/* flag bits */ +#define LINK_UNKNOWN_DEST_PORT 0x01 +#define LINK_UNKNOWN_DEST_ADDR 0x02 +#define LINK_PERMANENT 0x04 *** 203 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 14:56:34 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 303EA659407; Tue, 6 Jul 2021 14:56:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK5H60Gdlz3J5w; Tue, 6 Jul 2021 14:56:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E5B911BA90; Tue, 6 Jul 2021 14:56:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166EuXKj003394; Tue, 6 Jul 2021 14:56:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166EuXsa003393; Tue, 6 Jul 2021 14:56:33 GMT (envelope-from git) Date: Tue, 6 Jul 2021 14:56:33 GMT Message-Id: <202107061456.166EuXsa003393@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: 185dcb1072e3 - stable/13 - md5: Improve compatibility with coreutils and format fix MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 185dcb1072e3ce627df787ecc99671bded1fb922 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 14:56:34 -0000 The branch stable/13 has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=185dcb1072e3ce627df787ecc99671bded1fb922 commit 185dcb1072e3ce627df787ecc99671bded1fb922 Author: Stefan Eßer AuthorDate: 2021-06-29 07:01:13 +0000 Commit: Stefan Eßer CommitDate: 2021-07-06 14:56:06 +0000 md5: Improve compatibility with coreutils and format fix The previous changes that added support for the coreutils -c option modified the output generated by passing -r to match that of the coreutils versions. The difference is that coreutils separates the hash from the file name by two blanks " " (or a blank followed by an asterisk " *" with the -b option denoting). While most scripts or users will not notice the difference, it might be considered a violation of POLA and this commit reverts the change for the non-sum programs. These will print a single blank " " as the separator, as they die before the previous commit. In order to still generate output that is identical to that of the coreutils programs, this commit generates the " " or " *" separator used by them for the -sum versions, depending on the presence of the -b option. (cherry picked from commit b33d1898c1b0e6d76b52eb48078260588802dc95) --- sbin/md5/md5.1 | 25 ++++++++++++++++--------- sbin/md5/md5.c | 10 +++++++++- 2 files changed, 25 insertions(+), 10 deletions(-) diff --git a/sbin/md5/md5.1 b/sbin/md5/md5.1 index da034d1298c5..2b1da85c080f 100644 --- a/sbin/md5/md5.1 +++ b/sbin/md5/md5.1 @@ -1,5 +1,5 @@ .\" $FreeBSD$ -.Dd June 24, 2021 +.Dd June 29, 2021 .Dt MD5 1 .Os .Sh NAME @@ -77,9 +77,11 @@ The hexadecimal checksum of each file listed on the command line is printed after the options are processed. .Bl -tag -width indent .It Fl b -Ignored for compatibility with the coreutils +Make the .Nm -sum -programs. +programs separate hash and digest with a blank followed by an asterisk instead +of by 2 blank characters for full compatibility with the output generated by the +coreutils versions of these programs. .It Fl c Ar string If the program was called with a name that does not end in .Nm sum , @@ -88,13 +90,13 @@ compare the digest of the file against this string. .It Fl c Ar file If the program was called with a name that does end in .Nm sum , -the file passed as argument must contain digest lines generated by the same digest algorithm -with or without the +the file passed as argument must contain digest lines generated by the same +digest algorithm with or without the .Fl r option .Pq i.e. in either classical BSD format or in GNU coreutils format . -A line with file name followed by -.Dq : +A line with the file name followed by a colon +.Dq ":" and either OK or FAILED is written for each well-formed line in the digest file. If applicable, the number of failed comparisons and the number of lines that were skipped since they were not well-formed are printed at the end. @@ -157,8 +159,13 @@ $ echo -n Hello | md5 Calculate the checksum of multiple files reversing the output: .Bd -literal -offset indent $ md5 -r /boot/loader.conf /etc/rc.conf -ada5f60f23af88ff95b8091d6d67bef6 /boot/loader.conf -d80bf36c332dc0fdc479366ec3fa44cd /etc/rc.conf +ada5f60f23af88ff95b8091d6d67bef6 /boot/loader.conf +d80bf36c332dc0fdc479366ec3fa44cd /etc/rc.conf +.Pd +The +.Nm -sum +variants put 2 blank characters between hash and file name for full compatibilty +with the coreutils versions of these commands. .Ed .Pp Write the digest for diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index 43a76227a9d2..7235e6e0a39f 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -54,6 +54,7 @@ __FBSDID("$FreeBSD$"); #define TEST_BLOCK_COUNT 100000 #define MDTESTCOUNT 8 +static int bflag; static int cflag; static int pflag; static int qflag; @@ -301,6 +302,7 @@ main(int argc, char *argv[]) while ((ch = getopt(argc, argv, "bc:pqrs:tx")) != -1) switch (ch) { case 'b': + bflag = 1; break; case 'c': cflag = 1; @@ -436,7 +438,13 @@ MDOutput(const Algorithm_t *alg, char *p, char *argv[]) printf("%s\n", p); } else { if (rflag) - printf("%s %s", p, *argv); + if (gnu_emu) + if (bflag) + printf("%s *%s", p, *argv); + else + printf("%s %s", p, *argv); + else + printf("%s %s", p, *argv); else printf("%s (%s) = %s", alg->name, *argv, p); if (checkAgainst) { From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 15:28:16 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1677A65C547; Tue, 6 Jul 2021 15:28:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK5zh05L5z3QGV; Tue, 6 Jul 2021 15:28:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E03401BFD7; Tue, 6 Jul 2021 15:28:15 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166FSFp4051637; Tue, 6 Jul 2021 15:28:15 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166FSF8s051636; Tue, 6 Jul 2021 15:28:15 GMT (envelope-from git) Date: Tue, 6 Jul 2021 15:28:15 GMT Message-Id: <202107061528.166FSF8s051636@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: 546cc4ae8586 - stable/12 - sbin/md5: Create /usr/tests/sbin/md5 directory for placing tests MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 546cc4ae8586dcf052c9d8b17a1bc1d09d87b3b7 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 15:28:16 -0000 The branch stable/12 has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=546cc4ae8586dcf052c9d8b17a1bc1d09d87b3b7 commit 546cc4ae8586dcf052c9d8b17a1bc1d09d87b3b7 Author: Li-Wen Hsu AuthorDate: 2021-06-25 10:29:46 +0000 Commit: Stefan Eßer CommitDate: 2021-07-06 15:26:18 +0000 sbin/md5: Create /usr/tests/sbin/md5 directory for placing tests Author: Li-Wen Hsu Sponsored by: The FreeBSD Foundation (cherry picked from commit cb194afef50dc4e1473413a69462aaf03cfe0ce4) sbin/md5: add tests While the correctness of the supported hash algorithms can be tested with the built-in self-test feature, these test cases are meant to detect changes in the output format. A follow-up commit will improve the compatibility with the coreutils versions of the hash programs, and these tests should detect any unintended side-effects of such a change on existing functionality. (cherry picked from commit 884fc5527a3ffd9830b941cbec328de1623044cc) sbin/md5: fix test case to not write outside temp directory The bsd-p-test did create files 1.out .. 8.out in the test directory. This has been fixed and the temporary output files are now written to the temporary directory. (cherry picked from commit 95d3044bbdadfca6120a3e7c61c9ebb9a476f369) --- etc/mtree/BSD.tests.dist | 2 ++ sbin/md5/Makefile | 5 ++++ sbin/md5/tests/1.inp | 0 sbin/md5/tests/1.sha512-p.chk | 1 + sbin/md5/tests/1.sha512sum-p.chk | 1 + sbin/md5/tests/2.inp | 1 + sbin/md5/tests/2.sha512-p.chk | 1 + sbin/md5/tests/2.sha512sum-p.chk | 1 + sbin/md5/tests/3.inp | 1 + sbin/md5/tests/3.sha512-p.chk | 1 + sbin/md5/tests/3.sha512sum-p.chk | 1 + sbin/md5/tests/4.inp | 1 + sbin/md5/tests/4.sha512-p.chk | 1 + sbin/md5/tests/4.sha512sum-p.chk | 1 + sbin/md5/tests/5.inp | 1 + sbin/md5/tests/5.sha512-p.chk | 1 + sbin/md5/tests/5.sha512sum-p.chk | 1 + sbin/md5/tests/6.inp | 1 + sbin/md5/tests/6.sha512-p.chk | 1 + sbin/md5/tests/6.sha512sum-p.chk | 1 + sbin/md5/tests/7.inp | 1 + sbin/md5/tests/7.sha512-p.chk | 1 + sbin/md5/tests/7.sha512sum-p.chk | 1 + sbin/md5/tests/8.inp | 1 + sbin/md5/tests/8.sha512-p.chk | 1 + sbin/md5/tests/8.sha512sum-p.chk | 1 + sbin/md5/tests/Makefile | 41 +++++++++++++++++++++++++++++++++ sbin/md5/tests/algorithms.txt | 11 +++++++++ sbin/md5/tests/bsd-c-test.SH | 23 ++++++++++++++++++ sbin/md5/tests/bsd-p-test.SH | 24 +++++++++++++++++++ sbin/md5/tests/bsd-s-test.SH | 30 ++++++++++++++++++++++++ sbin/md5/tests/md5.digest | 8 +++++++ sbin/md5/tests/md5sum.digest | 8 +++++++ sbin/md5/tests/rmd160.digest | 8 +++++++ sbin/md5/tests/rmd160sum.digest | 8 +++++++ sbin/md5/tests/self-test.SH | 8 +++++++ sbin/md5/tests/self-test.md5.chk | 9 ++++++++ sbin/md5/tests/self-test.rmd160.chk | 9 ++++++++ sbin/md5/tests/self-test.sh_inp | 8 +++++++ sbin/md5/tests/self-test.sha1.chk | 9 ++++++++ sbin/md5/tests/self-test.sha224.chk | 9 ++++++++ sbin/md5/tests/self-test.sha256.chk | 9 ++++++++ sbin/md5/tests/self-test.sha384.chk | 9 ++++++++ sbin/md5/tests/self-test.sha512.chk | 9 ++++++++ sbin/md5/tests/self-test.sha512t256.chk | 9 ++++++++ sbin/md5/tests/self-test.skein1024.chk | 9 ++++++++ sbin/md5/tests/self-test.skein256.chk | 9 ++++++++ sbin/md5/tests/self-test.skein512.chk | 9 ++++++++ sbin/md5/tests/sha1.digest | 8 +++++++ sbin/md5/tests/sha1sum.digest | 8 +++++++ sbin/md5/tests/sha224.digest | 8 +++++++ sbin/md5/tests/sha224sum.digest | 8 +++++++ sbin/md5/tests/sha256.digest | 8 +++++++ sbin/md5/tests/sha256sum.digest | 8 +++++++ sbin/md5/tests/sha384.digest | 8 +++++++ sbin/md5/tests/sha384sum.digest | 8 +++++++ sbin/md5/tests/sha512.digest | 8 +++++++ sbin/md5/tests/sha512sum.digest | 8 +++++++ sbin/md5/tests/sha512t256.digest | 8 +++++++ sbin/md5/tests/sha512t256sum.digest | 8 +++++++ sbin/md5/tests/skein1024.digest | 8 +++++++ sbin/md5/tests/skein1024sum.digest | 8 +++++++ sbin/md5/tests/skein256.digest | 8 +++++++ sbin/md5/tests/skein256sum.digest | 8 +++++++ sbin/md5/tests/skein512.digest | 8 +++++++ sbin/md5/tests/skein512sum.digest | 8 +++++++ 66 files changed, 450 insertions(+) diff --git a/etc/mtree/BSD.tests.dist b/etc/mtree/BSD.tests.dist index df03a6375779..4997d67296ba 100644 --- a/etc/mtree/BSD.tests.dist +++ b/etc/mtree/BSD.tests.dist @@ -432,6 +432,8 @@ .. ifconfig .. + md5 + .. mdconfig .. nvmecontrol diff --git a/sbin/md5/Makefile b/sbin/md5/Makefile index dcbd94dcd381..6e29d94eb92a 100644 --- a/sbin/md5/Makefile +++ b/sbin/md5/Makefile @@ -36,4 +36,9 @@ LIBADD= md CFLAGS+=-DHAVE_CAPSICUM .endif +.include + +HAS_TESTS= +SUBDIR.${MK_TESTS}+= tests + .include diff --git a/sbin/md5/tests/1.inp b/sbin/md5/tests/1.inp new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/sbin/md5/tests/1.sha512-p.chk b/sbin/md5/tests/1.sha512-p.chk new file mode 100644 index 000000000000..c2f1924ca315 --- /dev/null +++ b/sbin/md5/tests/1.sha512-p.chk @@ -0,0 +1 @@ +cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e diff --git a/sbin/md5/tests/1.sha512sum-p.chk b/sbin/md5/tests/1.sha512sum-p.chk new file mode 100644 index 000000000000..c2f1924ca315 --- /dev/null +++ b/sbin/md5/tests/1.sha512sum-p.chk @@ -0,0 +1 @@ +cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e diff --git a/sbin/md5/tests/2.inp b/sbin/md5/tests/2.inp new file mode 100644 index 000000000000..2e65efe2a145 --- /dev/null +++ b/sbin/md5/tests/2.inp @@ -0,0 +1 @@ +a \ No newline at end of file diff --git a/sbin/md5/tests/2.sha512-p.chk b/sbin/md5/tests/2.sha512-p.chk new file mode 100644 index 000000000000..be4d8addd112 --- /dev/null +++ b/sbin/md5/tests/2.sha512-p.chk @@ -0,0 +1 @@ +a1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75 diff --git a/sbin/md5/tests/2.sha512sum-p.chk b/sbin/md5/tests/2.sha512sum-p.chk new file mode 100644 index 000000000000..be4d8addd112 --- /dev/null +++ b/sbin/md5/tests/2.sha512sum-p.chk @@ -0,0 +1 @@ +a1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75 diff --git a/sbin/md5/tests/3.inp b/sbin/md5/tests/3.inp new file mode 100644 index 000000000000..f2ba8f84ab5c --- /dev/null +++ b/sbin/md5/tests/3.inp @@ -0,0 +1 @@ +abc \ No newline at end of file diff --git a/sbin/md5/tests/3.sha512-p.chk b/sbin/md5/tests/3.sha512-p.chk new file mode 100644 index 000000000000..3971db2fddea --- /dev/null +++ b/sbin/md5/tests/3.sha512-p.chk @@ -0,0 +1 @@ +abcddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f diff --git a/sbin/md5/tests/3.sha512sum-p.chk b/sbin/md5/tests/3.sha512sum-p.chk new file mode 100644 index 000000000000..3971db2fddea --- /dev/null +++ b/sbin/md5/tests/3.sha512sum-p.chk @@ -0,0 +1 @@ +abcddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f diff --git a/sbin/md5/tests/4.inp b/sbin/md5/tests/4.inp new file mode 100644 index 000000000000..e407fe0e6143 --- /dev/null +++ b/sbin/md5/tests/4.inp @@ -0,0 +1 @@ +message digest \ No newline at end of file diff --git a/sbin/md5/tests/4.sha512-p.chk b/sbin/md5/tests/4.sha512-p.chk new file mode 100644 index 000000000000..cd5bad00b50d --- /dev/null +++ b/sbin/md5/tests/4.sha512-p.chk @@ -0,0 +1 @@ +message digest107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c13492ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c diff --git a/sbin/md5/tests/4.sha512sum-p.chk b/sbin/md5/tests/4.sha512sum-p.chk new file mode 100644 index 000000000000..cd5bad00b50d --- /dev/null +++ b/sbin/md5/tests/4.sha512sum-p.chk @@ -0,0 +1 @@ +message digest107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c13492ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c diff --git a/sbin/md5/tests/5.inp b/sbin/md5/tests/5.inp new file mode 100644 index 000000000000..e85d5b45283a --- /dev/null +++ b/sbin/md5/tests/5.inp @@ -0,0 +1 @@ +abcdefghijklmnopqrstuvwxyz \ No newline at end of file diff --git a/sbin/md5/tests/5.sha512-p.chk b/sbin/md5/tests/5.sha512-p.chk new file mode 100644 index 000000000000..77f97a5a0117 --- /dev/null +++ b/sbin/md5/tests/5.sha512-p.chk @@ -0,0 +1 @@ +abcdefghijklmnopqrstuvwxyz4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1 diff --git a/sbin/md5/tests/5.sha512sum-p.chk b/sbin/md5/tests/5.sha512sum-p.chk new file mode 100644 index 000000000000..77f97a5a0117 --- /dev/null +++ b/sbin/md5/tests/5.sha512sum-p.chk @@ -0,0 +1 @@ +abcdefghijklmnopqrstuvwxyz4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1 diff --git a/sbin/md5/tests/6.inp b/sbin/md5/tests/6.inp new file mode 100644 index 000000000000..9f75c3756298 --- /dev/null +++ b/sbin/md5/tests/6.inp @@ -0,0 +1 @@ +ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789 \ No newline at end of file diff --git a/sbin/md5/tests/6.sha512-p.chk b/sbin/md5/tests/6.sha512-p.chk new file mode 100644 index 000000000000..d14aad405215 --- /dev/null +++ b/sbin/md5/tests/6.sha512-p.chk @@ -0,0 +1 @@ +ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567891e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894 diff --git a/sbin/md5/tests/6.sha512sum-p.chk b/sbin/md5/tests/6.sha512sum-p.chk new file mode 100644 index 000000000000..d14aad405215 --- /dev/null +++ b/sbin/md5/tests/6.sha512sum-p.chk @@ -0,0 +1 @@ +ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234567891e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894 diff --git a/sbin/md5/tests/7.inp b/sbin/md5/tests/7.inp new file mode 100644 index 000000000000..15e0acf537d8 --- /dev/null +++ b/sbin/md5/tests/7.inp @@ -0,0 +1 @@ +12345678901234567890123456789012345678901234567890123456789012345678901234567890 \ No newline at end of file diff --git a/sbin/md5/tests/7.sha512-p.chk b/sbin/md5/tests/7.sha512-p.chk new file mode 100644 index 000000000000..efd5828f95ff --- /dev/null +++ b/sbin/md5/tests/7.sha512-p.chk @@ -0,0 +1 @@ +1234567890123456789012345678901234567890123456789012345678901234567890123456789072ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d1914042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843 diff --git a/sbin/md5/tests/7.sha512sum-p.chk b/sbin/md5/tests/7.sha512sum-p.chk new file mode 100644 index 000000000000..efd5828f95ff --- /dev/null +++ b/sbin/md5/tests/7.sha512sum-p.chk @@ -0,0 +1 @@ +1234567890123456789012345678901234567890123456789012345678901234567890123456789072ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d1914042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843 diff --git a/sbin/md5/tests/8.inp b/sbin/md5/tests/8.inp new file mode 100644 index 000000000000..689a689eceea --- /dev/null +++ b/sbin/md5/tests/8.inp @@ -0,0 +1 @@ +MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt \ No newline at end of file diff --git a/sbin/md5/tests/8.sha512-p.chk b/sbin/md5/tests/8.sha512-p.chk new file mode 100644 index 000000000000..d53bfc56d752 --- /dev/null +++ b/sbin/md5/tests/8.sha512-p.chk @@ -0,0 +1 @@ +MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubte8a835195e039708b13d9131e025f4441dbdc521ce625f245a436dcd762f54bf5cb298d96235e6c6a304e087ec8189b9512cbdf6427737ea82793460c367b9c3 diff --git a/sbin/md5/tests/8.sha512sum-p.chk b/sbin/md5/tests/8.sha512sum-p.chk new file mode 100644 index 000000000000..d53bfc56d752 --- /dev/null +++ b/sbin/md5/tests/8.sha512sum-p.chk @@ -0,0 +1 @@ +MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubte8a835195e039708b13d9131e025f4441dbdc521ce625f245a436dcd762f54bf5cb298d96235e6c6a304e087ec8189b9512cbdf6427737ea82793460c367b9c3 diff --git a/sbin/md5/tests/Makefile b/sbin/md5/tests/Makefile new file mode 100644 index 000000000000..b37332aa9894 --- /dev/null +++ b/sbin/md5/tests/Makefile @@ -0,0 +1,41 @@ +# $FreeBSD$ + +.include + +PACKAGE= tests + +TEST_DIR= ${SRCTOP}/sbin/md5/tests + +.PATH: ${TEST_DIR} + +FILESGROUPS+= FILESinputs +FILESinputsPACKAGE= ${PACKAGE} +FILESinputsDIR= ${TESTSDIR} +FILESinputs!= echo ${TEST_DIR}/*.inp + +FILESGROUPS+= FILESchkfiles +FILESchkfilesPACKAGE= ${PACKAGE} +FILESchkfilesDIR= ${TESTSDIR} +FILESchkfiles!= echo ${TEST_DIR}/*.chk + +FILESGROUPS+= FILESdigests +FILESdigestsPACKAGE= ${PACKAGE} +FILESdigestsDIR= ${TESTSDIR} +FILESdigests!= echo ${TEST_DIR}/*.digest + +FILESGROUPS+= FILESparam +FILESparamPACKAGE= ${PACKAGE} +FILESparamDIR= ${TESTSDIR} +FILESparam!= echo ${TEST_DIR}/*.txt + +PLAIN_TESTS_SH+= self-test +PLAIN_TESTS_SH+= bsd-c-test +PLAIN_TESTS_SH+= bsd-p-test +PLAIN_TESTS_SH+= bsd-s-test + +.SUFFIXES: .SH + +.SH.sh: + sed 's|%%TESTSBASE%%|${TESTSBASE}|g' ${.ALLSRC} > ${.TARGET} + +.include diff --git a/sbin/md5/tests/algorithms.txt b/sbin/md5/tests/algorithms.txt new file mode 100644 index 000000000000..88f0ff5b783a --- /dev/null +++ b/sbin/md5/tests/algorithms.txt @@ -0,0 +1,11 @@ +md5 +rmd160 +sha1 +sha224 +sha256 +sha384 +sha512 +sha512t256 +skein1024 +skein256 +skein512 diff --git a/sbin/md5/tests/bsd-c-test.SH b/sbin/md5/tests/bsd-c-test.SH new file mode 100644 index 000000000000..0327b95f0c0e --- /dev/null +++ b/sbin/md5/tests/bsd-c-test.SH @@ -0,0 +1,23 @@ +#!/bin/sh + +exitcode=0 + +testloop () { + opt=$1 + + while read algorithm; do + n=0 + for f in %%TESTSBASE%%/sbin/md5/*.inp; do + n=$((n + 1)) + expected=$(head -$n %%TESTSBASE%%/sbin/md5/$algorithm.digest | tail -1 | cut -w -f4) + hash=$($algorithm $opt -c "$expected" $f) || exitcode=1 + done + done < %%TESTSBASE%%/sbin/md5/algorithms.txt +} + +testloop "" +testloop -q +testloop -r +testloop -qr + +exit $exitcode diff --git a/sbin/md5/tests/bsd-p-test.SH b/sbin/md5/tests/bsd-p-test.SH new file mode 100644 index 000000000000..77a725377e2c --- /dev/null +++ b/sbin/md5/tests/bsd-p-test.SH @@ -0,0 +1,24 @@ +#!/bin/sh + +exitcode=0 + +testloop () { + opt=$1 + sum=$2 + + for algorithm in sha512; do + for f in %%TESTSBASE%%/sbin/md5/*.inp; do + outbase=$(basename $f .inp) + $algorithm$sum $opt -p < $f > $outbase.out + diff %%TESTSBASE%%/sbin/md5/$outbase.$algorithm$sum-p.chk $outbase.out || exitcode=1 + done + done +} + +testloop "" "" +testloop -q "" +testloop -qr "" +testloop "" sum +testloop -q sum + +exit $exitcode diff --git a/sbin/md5/tests/bsd-s-test.SH b/sbin/md5/tests/bsd-s-test.SH new file mode 100644 index 000000000000..83267711137f --- /dev/null +++ b/sbin/md5/tests/bsd-s-test.SH @@ -0,0 +1,30 @@ +#!/bin/sh + +exitcode=0 + +testloop () { + opt=$1 + sum=$2 + field=$3 + + while read algorithm; do + n=0 + for f in %%TESTSBASE%%/sbin/md5/*.inp; do + n=$((n + 1)) + read text < $f + hash=$($algorithm$sum $opt -s "$text" | cut -d= -f$field) + hash=$(echo $hash | cut -w -f1) + expected=$(head -$n %%TESTSBASE%%/sbin/md5/$algorithm.digest | tail -1 | cut -w -f4) + [ "$hash" = "$expected" ] || exitcode=1 + done + done < %%TESTSBASE%%/sbin/md5/algorithms.txt +} + +testloop "" "" 2 +testloop -q "" 1 +testloop -r "" 1 +testloop -qr "" 1 +testloop "" sum 1 +testloop -q sum 1 + +exit $exitcode diff --git a/sbin/md5/tests/md5.digest b/sbin/md5/tests/md5.digest new file mode 100644 index 000000000000..38b8dcb6e705 --- /dev/null +++ b/sbin/md5/tests/md5.digest @@ -0,0 +1,8 @@ +MD5 (1.inp) = d41d8cd98f00b204e9800998ecf8427e +MD5 (2.inp) = 0cc175b9c0f1b6a831c399e269772661 +MD5 (3.inp) = 900150983cd24fb0d6963f7d28e17f72 +MD5 (4.inp) = f96b697d7cb7938d525a2f31aaf161d0 +MD5 (5.inp) = c3fcd3d76192e4007dfb496cca67e13b +MD5 (6.inp) = d174ab98d277d9f5a5611c2c9f419d9f +MD5 (7.inp) = 57edf4a22be3c955ac49da2e2107b67a +MD5 (8.inp) = b50663f41d44d92171cb9976bc118538 diff --git a/sbin/md5/tests/md5sum.digest b/sbin/md5/tests/md5sum.digest new file mode 100644 index 000000000000..c560afa0d7f0 --- /dev/null +++ b/sbin/md5/tests/md5sum.digest @@ -0,0 +1,8 @@ +d41d8cd98f00b204e9800998ecf8427e 1.inp +0cc175b9c0f1b6a831c399e269772661 2.inp +900150983cd24fb0d6963f7d28e17f72 3.inp +f96b697d7cb7938d525a2f31aaf161d0 4.inp +c3fcd3d76192e4007dfb496cca67e13b 5.inp +d174ab98d277d9f5a5611c2c9f419d9f 6.inp +57edf4a22be3c955ac49da2e2107b67a 7.inp +b50663f41d44d92171cb9976bc118538 8.inp diff --git a/sbin/md5/tests/rmd160.digest b/sbin/md5/tests/rmd160.digest new file mode 100644 index 000000000000..b2ffd3c0ce45 --- /dev/null +++ b/sbin/md5/tests/rmd160.digest @@ -0,0 +1,8 @@ +RMD160 (1.inp) = 9c1185a5c5e9fc54612808977ee8f548b2258d31 +RMD160 (2.inp) = 0bdc9d2d256b3ee9daae347be6f4dc835a467ffe +RMD160 (3.inp) = 8eb208f7e05d987a9b044a8e98c6b087f15a0bfc +RMD160 (4.inp) = 5d0689ef49d2fae572b881b123a85ffa21595f36 +RMD160 (5.inp) = f71c27109c692c1b56bbdceb5b9d2865b3708dbc +RMD160 (6.inp) = b0e20b6e3116640286ed3a87a5713079b21f5189 +RMD160 (7.inp) = 9b752e45573d4b39f4dbd3323cab82bf63326bfb +RMD160 (8.inp) = 5feb69c6bf7c29d95715ad55f57d8ac5b2b7dd32 diff --git a/sbin/md5/tests/rmd160sum.digest b/sbin/md5/tests/rmd160sum.digest new file mode 100644 index 000000000000..4864ff39e050 --- /dev/null +++ b/sbin/md5/tests/rmd160sum.digest @@ -0,0 +1,8 @@ +9c1185a5c5e9fc54612808977ee8f548b2258d31 1.inp +0bdc9d2d256b3ee9daae347be6f4dc835a467ffe 2.inp +8eb208f7e05d987a9b044a8e98c6b087f15a0bfc 3.inp +5d0689ef49d2fae572b881b123a85ffa21595f36 4.inp +f71c27109c692c1b56bbdceb5b9d2865b3708dbc 5.inp +b0e20b6e3116640286ed3a87a5713079b21f5189 6.inp +9b752e45573d4b39f4dbd3323cab82bf63326bfb 7.inp +5feb69c6bf7c29d95715ad55f57d8ac5b2b7dd32 8.inp diff --git a/sbin/md5/tests/self-test.SH b/sbin/md5/tests/self-test.SH new file mode 100644 index 000000000000..33bd5569c23c --- /dev/null +++ b/sbin/md5/tests/self-test.SH @@ -0,0 +1,8 @@ +#!/bin/sh + +while read algorithm; do + $algorithm -x > self-test.$algorithm.out || exitcode=$? + diff %%TESTSBASE%%/sbin/md5/self-test.$algorithm.chk self-test.$algorithm.out +done < %%TESTSBASE%%/sbin/md5/algorithms.txt + +exit $exitcode diff --git a/sbin/md5/tests/self-test.md5.chk b/sbin/md5/tests/self-test.md5.chk new file mode 100644 index 000000000000..d4eb05560b53 --- /dev/null +++ b/sbin/md5/tests/self-test.md5.chk @@ -0,0 +1,9 @@ +MD5 test suite: +MD5 ("") = d41d8cd98f00b204e9800998ecf8427e - verified correct +MD5 ("a") = 0cc175b9c0f1b6a831c399e269772661 - verified correct +MD5 ("abc") = 900150983cd24fb0d6963f7d28e17f72 - verified correct +MD5 ("message digest") = f96b697d7cb7938d525a2f31aaf161d0 - verified correct +MD5 ("abcdefghijklmnopqrstuvwxyz") = c3fcd3d76192e4007dfb496cca67e13b - verified correct +MD5 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = d174ab98d277d9f5a5611c2c9f419d9f - verified correct +MD5 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 57edf4a22be3c955ac49da2e2107b67a - verified correct +MD5 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = b50663f41d44d92171cb9976bc118538 - verified correct diff --git a/sbin/md5/tests/self-test.rmd160.chk b/sbin/md5/tests/self-test.rmd160.chk new file mode 100644 index 000000000000..a0680ac7f647 --- /dev/null +++ b/sbin/md5/tests/self-test.rmd160.chk @@ -0,0 +1,9 @@ +RMD160 test suite: +RMD160 ("") = 9c1185a5c5e9fc54612808977ee8f548b2258d31 - verified correct +RMD160 ("a") = 0bdc9d2d256b3ee9daae347be6f4dc835a467ffe - verified correct +RMD160 ("abc") = 8eb208f7e05d987a9b044a8e98c6b087f15a0bfc - verified correct +RMD160 ("message digest") = 5d0689ef49d2fae572b881b123a85ffa21595f36 - verified correct +RMD160 ("abcdefghijklmnopqrstuvwxyz") = f71c27109c692c1b56bbdceb5b9d2865b3708dbc - verified correct +RMD160 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = b0e20b6e3116640286ed3a87a5713079b21f5189 - verified correct +RMD160 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 9b752e45573d4b39f4dbd3323cab82bf63326bfb - verified correct +RMD160 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = 5feb69c6bf7c29d95715ad55f57d8ac5b2b7dd32 - verified correct diff --git a/sbin/md5/tests/self-test.sh_inp b/sbin/md5/tests/self-test.sh_inp new file mode 100644 index 000000000000..33bd5569c23c --- /dev/null +++ b/sbin/md5/tests/self-test.sh_inp @@ -0,0 +1,8 @@ +#!/bin/sh + +while read algorithm; do + $algorithm -x > self-test.$algorithm.out || exitcode=$? + diff %%TESTSBASE%%/sbin/md5/self-test.$algorithm.chk self-test.$algorithm.out +done < %%TESTSBASE%%/sbin/md5/algorithms.txt + +exit $exitcode diff --git a/sbin/md5/tests/self-test.sha1.chk b/sbin/md5/tests/self-test.sha1.chk new file mode 100644 index 000000000000..e463947feb36 --- /dev/null +++ b/sbin/md5/tests/self-test.sha1.chk @@ -0,0 +1,9 @@ +SHA1 test suite: +SHA1 ("") = da39a3ee5e6b4b0d3255bfef95601890afd80709 - verified correct +SHA1 ("a") = 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8 - verified correct +SHA1 ("abc") = a9993e364706816aba3e25717850c26c9cd0d89d - verified correct +SHA1 ("message digest") = c12252ceda8be8994d5fa0290a47231c1d16aae3 - verified correct +SHA1 ("abcdefghijklmnopqrstuvwxyz") = 32d10c7b8cf96570ca04ce37f2a19d84240d3a89 - verified correct +SHA1 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = 761c457bf73b14d27e9e9265c46f4b4dda11f940 - verified correct +SHA1 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 50abf5706a150990a08b2c5ea40fa0e585554732 - verified correct +SHA1 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = 18eca4333979c4181199b7b4fab8786d16cf2846 - verified correct diff --git a/sbin/md5/tests/self-test.sha224.chk b/sbin/md5/tests/self-test.sha224.chk new file mode 100644 index 000000000000..e74cccafa6d7 --- /dev/null +++ b/sbin/md5/tests/self-test.sha224.chk @@ -0,0 +1,9 @@ +SHA224 test suite: +SHA224 ("") = d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f - verified correct +SHA224 ("a") = abd37534c7d9a2efb9465de931cd7055ffdb8879563ae98078d6d6d5 - verified correct +SHA224 ("abc") = 23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7 - verified correct +SHA224 ("message digest") = 2cb21c83ae2f004de7e81c3c7019cbcb65b71ab656b22d6d0c39b8eb - verified correct +SHA224 ("abcdefghijklmnopqrstuvwxyz") = 45a5f72c39c5cff2522eb3429799e49e5f44b356ef926bcf390dccc2 - verified correct +SHA224 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = bff72b4fcb7d75e5632900ac5f90d219e05e97a7bde72e740db393d9 - verified correct +SHA224 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = b50aecbe4e9bb0b57bc5f3ae760a8e01db24f203fb3cdcd13148046e - verified correct +SHA224 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = 5ae55f3779c8a1204210d7ed7689f661fbe140f96f272ab79e19d470 - verified correct diff --git a/sbin/md5/tests/self-test.sha256.chk b/sbin/md5/tests/self-test.sha256.chk new file mode 100644 index 000000000000..ba640b6a9217 --- /dev/null +++ b/sbin/md5/tests/self-test.sha256.chk @@ -0,0 +1,9 @@ +SHA256 test suite: +SHA256 ("") = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 - verified correct +SHA256 ("a") = ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb - verified correct +SHA256 ("abc") = ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad - verified correct +SHA256 ("message digest") = f7846f55cf23e14eebeab5b4e1550cad5b509e3348fbc4efa3a1413d393cb650 - verified correct +SHA256 ("abcdefghijklmnopqrstuvwxyz") = 71c480df93d6ae2f1efad1447c66c9525e316218cf51fc8d9ed832f2daf18b73 - verified correct +SHA256 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = db4bfcbd4da0cd85a60c3c37d3fbd8805c77f15fc6b1fdfe614ee0a7c8fdb4c0 - verified correct +SHA256 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = f371bc4a311f2b009eef952dd83ca80e2b60026c8e935592d0f9c308453c813e - verified correct +SHA256 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = e6eae09f10ad4122a0e2a4075761d185a272ebd9f5aa489e998ff2f09cbfdd9f - verified correct diff --git a/sbin/md5/tests/self-test.sha384.chk b/sbin/md5/tests/self-test.sha384.chk new file mode 100644 index 000000000000..a493cadc1964 --- /dev/null +++ b/sbin/md5/tests/self-test.sha384.chk @@ -0,0 +1,9 @@ +SHA384 test suite: +SHA384 ("") = 38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b - verified correct +SHA384 ("a") = 54a59b9f22b0b80880d8427e548b7c23abd873486e1f035dce9cd697e85175033caa88e6d57bc35efae0b5afd3145f31 - verified correct +SHA384 ("abc") = cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7 - verified correct +SHA384 ("message digest") = 473ed35167ec1f5d8e550368a3db39be54639f828868e9454c239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5 - verified correct +SHA384 ("abcdefghijklmnopqrstuvwxyz") = feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4 - verified correct +SHA384 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = 1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039c1464ee8732f11a5341a6f41e0c202294736ed64db1a84 - verified correct +SHA384 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026 - verified correct +SHA384 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = 99428d401bf4abcd4ee0695248c9858b7503853acfae21a9cffa7855f46d1395ef38596fcd06d5a8c32d41a839cc5dfb - verified correct diff --git a/sbin/md5/tests/self-test.sha512.chk b/sbin/md5/tests/self-test.sha512.chk new file mode 100644 index 000000000000..f36e455cdaa5 --- /dev/null +++ b/sbin/md5/tests/self-test.sha512.chk @@ -0,0 +1,9 @@ +SHA512 test suite: +SHA512 ("") = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e - verified correct +SHA512 ("a") = 1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75 - verified correct +SHA512 ("abc") = ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f - verified correct +SHA512 ("message digest") = 107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c13492ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c - verified correct +SHA512 ("abcdefghijklmnopqrstuvwxyz") = 4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1 - verified correct +SHA512 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = 1e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894 - verified correct +SHA512 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 72ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d1914042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843 - verified correct +SHA512 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = e8a835195e039708b13d9131e025f4441dbdc521ce625f245a436dcd762f54bf5cb298d96235e6c6a304e087ec8189b9512cbdf6427737ea82793460c367b9c3 - verified correct diff --git a/sbin/md5/tests/self-test.sha512t256.chk b/sbin/md5/tests/self-test.sha512t256.chk new file mode 100644 index 000000000000..25612ace484a --- /dev/null +++ b/sbin/md5/tests/self-test.sha512t256.chk @@ -0,0 +1,9 @@ +SHA512t256 test suite: +SHA512t256 ("") = c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a - verified correct +SHA512t256 ("a") = 455e518824bc0601f9fb858ff5c37d417d67c2f8e0df2babe4808858aea830f8 - verified correct +SHA512t256 ("abc") = 53048e2681941ef99b2e29b76b4c7dabe4c2d0c634fc6d46e0e2f13107e7af23 - verified correct +SHA512t256 ("message digest") = 0cf471fd17ed69d990daf3433c89b16d63dec1bb9cb42a6094604ee5d7b4e9fb - verified correct +SHA512t256 ("abcdefghijklmnopqrstuvwxyz") = fc3189443f9c268f626aea08a756abe7b726b05f701cb08222312ccfd6710a26 - verified correct +SHA512t256 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = cdf1cc0effe26ecc0c13758f7b4a48e000615df241284185c39eb05d355bb9c8 - verified correct +SHA512t256 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 2c9fdbc0c90bdd87612ee8455474f9044850241dc105b1e8b94b8ddf5fac9148 - verified correct +SHA512t256 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = dd095fc859b336c30a52548b3dc59fcc0d1be8616ebcf3368fad23107db2d736 - verified correct diff --git a/sbin/md5/tests/self-test.skein1024.chk b/sbin/md5/tests/self-test.skein1024.chk new file mode 100644 index 000000000000..8cfc5997afde --- /dev/null +++ b/sbin/md5/tests/self-test.skein1024.chk @@ -0,0 +1,9 @@ +Skein1024 test suite: +Skein1024 ("") = 0fff9563bb3279289227ac77d319b6fff8d7e9f09da1247b72a0a265cd6d2a62645ad547ed8193db48cff847c06494a03f55666d3b47eb4c20456c9373c86297d630d5578ebd34cb40991578f9f52b18003efa35d3da6553ff35db91b81ab890bec1b189b7f52cb2a783ebb7d823d725b0b4a71f6824e88f68f982eefc6d19c6 - verified correct +Skein1024 ("a") = 6ab4c4ba9814a3d976ec8bffa7fcc638ceba0544a97b3c98411323ffd2dc936315d13dc93c13c4e88cda6f5bac6f2558b2d8694d3b6143e40d644ae43ca940685cb37f809d3d0550c56cba8036dee729a4f8fb960732e59e64d57f7f7710f8670963cdcdc95b41daab4855fcf8b6762a64b173ee61343a2c7689af1d293eba97 - verified correct +Skein1024 ("abc") = 35a599a0f91abcdb4cb73c19b8cb8d947742d82c309137a7caed29e8e0a2ca7a9ff9a90c34c1908cc7e7fd99bb15032fb86e76df21b72628399b5f7c3cc209d7bb31c99cd4e19465622a049afbb87c03b5ce3888d17e6e667279ec0aa9b3e2712624c01b5f5bbe1a564220bdcf6990af0c2539019f313fdd7406cca3892a1f1f - verified correct +Skein1024 ("message digest") = ea891f5268acd0fac97467fc1aa89d1ce8681a9992a42540e53babee861483110c2d16f49e73bac27653ff173003e40cfb08516cd34262e6af95a5d8645c9c1abb3e813604d508b8511b30f9a5c1b352aa0791c7d2f27b2706dccea54bc7de6555b5202351751c3299f97c09cf89c40f67187e2521c0fad82b30edbb224f0458 - verified correct +Skein1024 ("abcdefghijklmnopqrstuvwxyz") = f23d95c2a25fbcd0e797cd058fec39d3c52d2b5afd7a9af1df934e63257d1d3dcf3246e7329c0f1104c1e51e3d22e300507b0c3b9f985bb1f645ef49835080536becf83788e17fed09c9982ba65c3cb7ffe6a5f745b911c506962adf226e435c42f6f6bc08d288f9c810e807e3216ef444f3db22744441deefa4900982a1371f - verified correct +Skein1024 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = cf3889e8a8d11bfd3938055d7d061437962bc5eac8ae83b1b71c94be201b8cf657fdbfc38674997a008c0c903f56a23feb3ae30e012377f1cfa080a9ca7fe8b96138662653fb3335c7d06595bf8baf65e215307532094cfdfa056bd8052ab792a3944a2adaa47b30335b8badb8fe9eb94fe329cdca04e58bbc530f0af709f469 - verified correct +Skein1024 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = cf21a613620e6c119eca31fdfaad449a8e02f95ca256c21d2a105f8e4157048f9fe1e897893ea18b64e0e37cb07d5ac947f27ba544caf7cbc1ad094e675aed77a366270f7eb7f46543bccfa61c526fd628408058ed00ed566ac35a9761d002e629c4fb0d430b2f4ad016fcc49c44d2981c4002da0eecc42144160e2eaea4855a - verified correct +Skein1024 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = e6799b78db54085a2be7ff4c8007f147fa88d326abab30be0560b953396d8802feee9a15419b48a467574e9283be15685ca8a079ee52b27166b64dd70b124b1d4e4f6aca37224c3f2685e67e67baef9f94b905698adc794a09672aba977a61b20966912acdb08c21a2c37001785355dc884751a21f848ab36e590331ff938138 - verified correct diff --git a/sbin/md5/tests/self-test.skein256.chk b/sbin/md5/tests/self-test.skein256.chk new file mode 100644 index 000000000000..41d1fdcb8cc8 --- /dev/null +++ b/sbin/md5/tests/self-test.skein256.chk @@ -0,0 +1,9 @@ +Skein256 test suite: +Skein256 ("") = c8877087da56e072870daa843f176e9453115929094c3a40c463a196c29bf7ba - verified correct +Skein256 ("a") = 7fba44ff1a31d71a0c1f82e6e82fb5e9ac6c92a39c9185b9951fed82d82fe635 - verified correct +Skein256 ("abc") = 258bdec343b9fde1639221a5ae0144a96e552e5288753c5fec76c05fc2fc1870 - verified correct +Skein256 ("message digest") = 4d2ce0062b5eb3a4db95bc1117dd8aa014f6cd50fdc8e64f31f7d41f9231e488 - verified correct +Skein256 ("abcdefghijklmnopqrstuvwxyz") = 46d8440685461b00e3ddb891b2ecc6855287d2bd8834a95fb1c1708b00ea5e82 - verified correct +Skein256 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = 7c5eb606389556b33d34eb2536459528dc0af97adbcd0ce273aeb650f598d4b2 - verified correct +Skein256 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 4def7a7e5464a140ae9c3a80279fbebce4bd00f9faad819ab7e001512f67a10d - verified correct +Skein256 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = d9c017dbe355f318d036469eb9b5fbe129fc2b5786a9dc6746a516eab6fe0126 - verified correct diff --git a/sbin/md5/tests/self-test.skein512.chk b/sbin/md5/tests/self-test.skein512.chk new file mode 100644 index 000000000000..437220e17adb --- /dev/null +++ b/sbin/md5/tests/self-test.skein512.chk @@ -0,0 +1,9 @@ +Skein512 test suite: +Skein512 ("") = bc5b4c50925519c290cc634277ae3d6257212395cba733bbad37a4af0fa06af41fca7903d06564fea7a2d3730dbdb80c1f85562dfcc070334ea4d1d9e72cba7a - verified correct +Skein512 ("a") = b1cd8d33f61b3737adfd59bb13ad82f4a9548e92f22956a8976cca3fdb7fee4fe91698146c4197cec85d38b83c5d93bdba92c01fd9a53870d0c7f967bc62bdce - verified correct +Skein512 ("abc") = 8f5dd9ec798152668e35129496b029a960c9a9b88662f7f9482f110b31f9f93893ecfb25c009baad9e46737197d5630379816a886aa05526d3a70df272d96e75 - verified correct +Skein512 ("message digest") = 15b73c158ffb875fed4d72801ded0794c720b121c0c78edf45f900937e6933d9e21a3a984206933d504b5dbb2368000411477ee1b204c986068df77886542fcc - verified correct +Skein512 ("abcdefghijklmnopqrstuvwxyz") = 23793ad900ef12f9165c8080da6fdfd2c8354a2929b8aadf83aa82a3c6470342f57cf8c035ec0d97429b626c4d94f28632c8f5134fd367dca5cf293d2ec13f8c - verified correct +Skein512 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789") = 0c6bed927e022f5ddcf81877d42e5f75798a9f8fd3ede3d83baac0a2f364b082e036c11af35fe478745459dd8f5c0b73efe3c56ba5bb2009208d5a29cc6e469c - verified correct +Skein512 ("12345678901234567890123456789012345678901234567890123456789012345678901234567890") = 2ca9fcffb3456f297d1b5f407014ecb856f0baac8eb540f534b1f187196f21e88f31103128c2f03fcc9857d7a58eb66f9525e2302d88833ee069295537a434ce - verified correct +Skein512 ("MD5 has not yet (2001-09-03) been broken, but sufficient attacks have been made that its security is in some doubt") = 1131f2aaa0e97126c9314f9f968cc827259bbfabced2943bb8c9274448998fb3b78738b4580dd500c76105fd3c03e465e1414f2c29664286b1f79d3e51128125 - verified correct diff --git a/sbin/md5/tests/sha1.digest b/sbin/md5/tests/sha1.digest new file mode 100644 index 000000000000..10ce642c0883 --- /dev/null +++ b/sbin/md5/tests/sha1.digest @@ -0,0 +1,8 @@ +SHA1 (1.inp) = da39a3ee5e6b4b0d3255bfef95601890afd80709 +SHA1 (2.inp) = 86f7e437faa5a7fce15d1ddcb9eaeaea377667b8 +SHA1 (3.inp) = a9993e364706816aba3e25717850c26c9cd0d89d +SHA1 (4.inp) = c12252ceda8be8994d5fa0290a47231c1d16aae3 +SHA1 (5.inp) = 32d10c7b8cf96570ca04ce37f2a19d84240d3a89 +SHA1 (6.inp) = 761c457bf73b14d27e9e9265c46f4b4dda11f940 +SHA1 (7.inp) = 50abf5706a150990a08b2c5ea40fa0e585554732 +SHA1 (8.inp) = 18eca4333979c4181199b7b4fab8786d16cf2846 diff --git a/sbin/md5/tests/sha1sum.digest b/sbin/md5/tests/sha1sum.digest new file mode 100644 index 000000000000..d82d2889ff5d --- /dev/null +++ b/sbin/md5/tests/sha1sum.digest @@ -0,0 +1,8 @@ +da39a3ee5e6b4b0d3255bfef95601890afd80709 1.inp +86f7e437faa5a7fce15d1ddcb9eaeaea377667b8 2.inp +a9993e364706816aba3e25717850c26c9cd0d89d 3.inp +c12252ceda8be8994d5fa0290a47231c1d16aae3 4.inp +32d10c7b8cf96570ca04ce37f2a19d84240d3a89 5.inp +761c457bf73b14d27e9e9265c46f4b4dda11f940 6.inp +50abf5706a150990a08b2c5ea40fa0e585554732 7.inp +18eca4333979c4181199b7b4fab8786d16cf2846 8.inp diff --git a/sbin/md5/tests/sha224.digest b/sbin/md5/tests/sha224.digest new file mode 100644 index 000000000000..a1007511a978 --- /dev/null +++ b/sbin/md5/tests/sha224.digest @@ -0,0 +1,8 @@ +SHA224 (1.inp) = d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f +SHA224 (2.inp) = abd37534c7d9a2efb9465de931cd7055ffdb8879563ae98078d6d6d5 +SHA224 (3.inp) = 23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7 +SHA224 (4.inp) = 2cb21c83ae2f004de7e81c3c7019cbcb65b71ab656b22d6d0c39b8eb +SHA224 (5.inp) = 45a5f72c39c5cff2522eb3429799e49e5f44b356ef926bcf390dccc2 +SHA224 (6.inp) = bff72b4fcb7d75e5632900ac5f90d219e05e97a7bde72e740db393d9 +SHA224 (7.inp) = b50aecbe4e9bb0b57bc5f3ae760a8e01db24f203fb3cdcd13148046e +SHA224 (8.inp) = 5ae55f3779c8a1204210d7ed7689f661fbe140f96f272ab79e19d470 diff --git a/sbin/md5/tests/sha224sum.digest b/sbin/md5/tests/sha224sum.digest new file mode 100644 index 000000000000..37fd833032fb --- /dev/null +++ b/sbin/md5/tests/sha224sum.digest @@ -0,0 +1,8 @@ +d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f 1.inp +abd37534c7d9a2efb9465de931cd7055ffdb8879563ae98078d6d6d5 2.inp +23097d223405d8228642a477bda255b32aadbce4bda0b3f7e36c9da7 3.inp +2cb21c83ae2f004de7e81c3c7019cbcb65b71ab656b22d6d0c39b8eb 4.inp +45a5f72c39c5cff2522eb3429799e49e5f44b356ef926bcf390dccc2 5.inp +bff72b4fcb7d75e5632900ac5f90d219e05e97a7bde72e740db393d9 6.inp +b50aecbe4e9bb0b57bc5f3ae760a8e01db24f203fb3cdcd13148046e 7.inp +5ae55f3779c8a1204210d7ed7689f661fbe140f96f272ab79e19d470 8.inp diff --git a/sbin/md5/tests/sha256.digest b/sbin/md5/tests/sha256.digest new file mode 100644 index 000000000000..b6c4411261a8 --- /dev/null +++ b/sbin/md5/tests/sha256.digest @@ -0,0 +1,8 @@ +SHA256 (1.inp) = e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 +SHA256 (2.inp) = ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb +SHA256 (3.inp) = ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad +SHA256 (4.inp) = f7846f55cf23e14eebeab5b4e1550cad5b509e3348fbc4efa3a1413d393cb650 +SHA256 (5.inp) = 71c480df93d6ae2f1efad1447c66c9525e316218cf51fc8d9ed832f2daf18b73 +SHA256 (6.inp) = db4bfcbd4da0cd85a60c3c37d3fbd8805c77f15fc6b1fdfe614ee0a7c8fdb4c0 +SHA256 (7.inp) = f371bc4a311f2b009eef952dd83ca80e2b60026c8e935592d0f9c308453c813e +SHA256 (8.inp) = e6eae09f10ad4122a0e2a4075761d185a272ebd9f5aa489e998ff2f09cbfdd9f diff --git a/sbin/md5/tests/sha256sum.digest b/sbin/md5/tests/sha256sum.digest new file mode 100644 index 000000000000..35ce99a0c6c1 --- /dev/null +++ b/sbin/md5/tests/sha256sum.digest @@ -0,0 +1,8 @@ +e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 1.inp +ca978112ca1bbdcafac231b39a23dc4da786eff8147c4e72b9807785afee48bb 2.inp +ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad 3.inp +f7846f55cf23e14eebeab5b4e1550cad5b509e3348fbc4efa3a1413d393cb650 4.inp +71c480df93d6ae2f1efad1447c66c9525e316218cf51fc8d9ed832f2daf18b73 5.inp +db4bfcbd4da0cd85a60c3c37d3fbd8805c77f15fc6b1fdfe614ee0a7c8fdb4c0 6.inp +f371bc4a311f2b009eef952dd83ca80e2b60026c8e935592d0f9c308453c813e 7.inp +e6eae09f10ad4122a0e2a4075761d185a272ebd9f5aa489e998ff2f09cbfdd9f 8.inp diff --git a/sbin/md5/tests/sha384.digest b/sbin/md5/tests/sha384.digest new file mode 100644 index 000000000000..b56ef8b708e4 --- /dev/null +++ b/sbin/md5/tests/sha384.digest @@ -0,0 +1,8 @@ +SHA384 (1.inp) = 38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b +SHA384 (2.inp) = 54a59b9f22b0b80880d8427e548b7c23abd873486e1f035dce9cd697e85175033caa88e6d57bc35efae0b5afd3145f31 +SHA384 (3.inp) = cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7 +SHA384 (4.inp) = 473ed35167ec1f5d8e550368a3db39be54639f828868e9454c239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5 +SHA384 (5.inp) = feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4 +SHA384 (6.inp) = 1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039c1464ee8732f11a5341a6f41e0c202294736ed64db1a84 +SHA384 (7.inp) = b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026 +SHA384 (8.inp) = 99428d401bf4abcd4ee0695248c9858b7503853acfae21a9cffa7855f46d1395ef38596fcd06d5a8c32d41a839cc5dfb diff --git a/sbin/md5/tests/sha384sum.digest b/sbin/md5/tests/sha384sum.digest new file mode 100644 index 000000000000..7d669e0dda08 --- /dev/null +++ b/sbin/md5/tests/sha384sum.digest @@ -0,0 +1,8 @@ +38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b 1.inp +54a59b9f22b0b80880d8427e548b7c23abd873486e1f035dce9cd697e85175033caa88e6d57bc35efae0b5afd3145f31 2.inp +cb00753f45a35e8bb5a03d699ac65007272c32ab0eded1631a8b605a43ff5bed8086072ba1e7cc2358baeca134c825a7 3.inp +473ed35167ec1f5d8e550368a3db39be54639f828868e9454c239fc8b52e3c61dbd0d8b4de1390c256dcbb5d5fd99cd5 4.inp +feb67349df3db6f5924815d6c3dc133f091809213731fe5c7b5f4999e463479ff2877f5f2936fa63bb43784b12f3ebb4 5.inp +1761336e3f7cbfe51deb137f026f89e01a448e3b1fafa64039c1464ee8732f11a5341a6f41e0c202294736ed64db1a84 6.inp +b12932b0627d1c060942f5447764155655bd4da0c9afa6dd9b9ef53129af1b8fb0195996d2de9ca0df9d821ffee67026 7.inp +99428d401bf4abcd4ee0695248c9858b7503853acfae21a9cffa7855f46d1395ef38596fcd06d5a8c32d41a839cc5dfb 8.inp diff --git a/sbin/md5/tests/sha512.digest b/sbin/md5/tests/sha512.digest new file mode 100644 index 000000000000..dd82845b8131 --- /dev/null +++ b/sbin/md5/tests/sha512.digest @@ -0,0 +1,8 @@ +SHA512 (1.inp) = cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e +SHA512 (2.inp) = 1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75 +SHA512 (3.inp) = ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f +SHA512 (4.inp) = 107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c13492ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c +SHA512 (5.inp) = 4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1 +SHA512 (6.inp) = 1e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894 +SHA512 (7.inp) = 72ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d1914042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843 +SHA512 (8.inp) = e8a835195e039708b13d9131e025f4441dbdc521ce625f245a436dcd762f54bf5cb298d96235e6c6a304e087ec8189b9512cbdf6427737ea82793460c367b9c3 diff --git a/sbin/md5/tests/sha512sum.digest b/sbin/md5/tests/sha512sum.digest new file mode 100644 index 000000000000..55bbd509e4df --- /dev/null +++ b/sbin/md5/tests/sha512sum.digest @@ -0,0 +1,8 @@ +cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e 1.inp +1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75 2.inp +ddaf35a193617abacc417349ae20413112e6fa4e89a97ea20a9eeee64b55d39a2192992a274fc1a836ba3c23a3feebbd454d4423643ce80e2a9ac94fa54ca49f 3.inp +107dbf389d9e9f71a3a95f6c055b9251bc5268c2be16d6c13492ea45b0199f3309e16455ab1e96118e8a905d5597b72038ddb372a89826046de66687bb420e7c 4.inp +4dbff86cc2ca1bae1e16468a05cb9881c97f1753bce3619034898faa1aabe429955a1bf8ec483d7421fe3c1646613a59ed5441fb0f321389f77f48a879c7b1f1 5.inp +1e07be23c26a86ea37ea810c8ec7809352515a970e9253c26f536cfc7a9996c45c8370583e0a78fa4a90041d71a4ceab7423f19c71b9d5a3e01249f0bebd5894 6.inp +72ec1ef1124a45b047e8b7c75a932195135bb61de24ec0d1914042246e0aec3a2354e093d76f3048b456764346900cb130d2a4fd5dd16abb5e30bcb850dee843 7.inp +e8a835195e039708b13d9131e025f4441dbdc521ce625f245a436dcd762f54bf5cb298d96235e6c6a304e087ec8189b9512cbdf6427737ea82793460c367b9c3 8.inp diff --git a/sbin/md5/tests/sha512t256.digest b/sbin/md5/tests/sha512t256.digest new file mode 100644 index 000000000000..3e7df9eb98db --- /dev/null +++ b/sbin/md5/tests/sha512t256.digest @@ -0,0 +1,8 @@ +SHA512t256 (1.inp) = c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a +SHA512t256 (2.inp) = 455e518824bc0601f9fb858ff5c37d417d67c2f8e0df2babe4808858aea830f8 +SHA512t256 (3.inp) = 53048e2681941ef99b2e29b76b4c7dabe4c2d0c634fc6d46e0e2f13107e7af23 +SHA512t256 (4.inp) = 0cf471fd17ed69d990daf3433c89b16d63dec1bb9cb42a6094604ee5d7b4e9fb +SHA512t256 (5.inp) = fc3189443f9c268f626aea08a756abe7b726b05f701cb08222312ccfd6710a26 +SHA512t256 (6.inp) = cdf1cc0effe26ecc0c13758f7b4a48e000615df241284185c39eb05d355bb9c8 +SHA512t256 (7.inp) = 2c9fdbc0c90bdd87612ee8455474f9044850241dc105b1e8b94b8ddf5fac9148 +SHA512t256 (8.inp) = dd095fc859b336c30a52548b3dc59fcc0d1be8616ebcf3368fad23107db2d736 diff --git a/sbin/md5/tests/sha512t256sum.digest b/sbin/md5/tests/sha512t256sum.digest new file mode 100644 index 000000000000..425e31006d6e --- /dev/null +++ b/sbin/md5/tests/sha512t256sum.digest @@ -0,0 +1,8 @@ +c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a 1.inp +455e518824bc0601f9fb858ff5c37d417d67c2f8e0df2babe4808858aea830f8 2.inp +53048e2681941ef99b2e29b76b4c7dabe4c2d0c634fc6d46e0e2f13107e7af23 3.inp +0cf471fd17ed69d990daf3433c89b16d63dec1bb9cb42a6094604ee5d7b4e9fb 4.inp +fc3189443f9c268f626aea08a756abe7b726b05f701cb08222312ccfd6710a26 5.inp +cdf1cc0effe26ecc0c13758f7b4a48e000615df241284185c39eb05d355bb9c8 6.inp +2c9fdbc0c90bdd87612ee8455474f9044850241dc105b1e8b94b8ddf5fac9148 7.inp +dd095fc859b336c30a52548b3dc59fcc0d1be8616ebcf3368fad23107db2d736 8.inp diff --git a/sbin/md5/tests/skein1024.digest b/sbin/md5/tests/skein1024.digest new file mode 100644 index 000000000000..dea354221b26 --- /dev/null +++ b/sbin/md5/tests/skein1024.digest @@ -0,0 +1,8 @@ +Skein1024 (1.inp) = 0fff9563bb3279289227ac77d319b6fff8d7e9f09da1247b72a0a265cd6d2a62645ad547ed8193db48cff847c06494a03f55666d3b47eb4c20456c9373c86297d630d5578ebd34cb40991578f9f52b18003efa35d3da6553ff35db91b81ab890bec1b189b7f52cb2a783ebb7d823d725b0b4a71f6824e88f68f982eefc6d19c6 +Skein1024 (2.inp) = 6ab4c4ba9814a3d976ec8bffa7fcc638ceba0544a97b3c98411323ffd2dc936315d13dc93c13c4e88cda6f5bac6f2558b2d8694d3b6143e40d644ae43ca940685cb37f809d3d0550c56cba8036dee729a4f8fb960732e59e64d57f7f7710f8670963cdcdc95b41daab4855fcf8b6762a64b173ee61343a2c7689af1d293eba97 +Skein1024 (3.inp) = 35a599a0f91abcdb4cb73c19b8cb8d947742d82c309137a7caed29e8e0a2ca7a9ff9a90c34c1908cc7e7fd99bb15032fb86e76df21b72628399b5f7c3cc209d7bb31c99cd4e19465622a049afbb87c03b5ce3888d17e6e667279ec0aa9b3e2712624c01b5f5bbe1a564220bdcf6990af0c2539019f313fdd7406cca3892a1f1f +Skein1024 (4.inp) = ea891f5268acd0fac97467fc1aa89d1ce8681a9992a42540e53babee861483110c2d16f49e73bac27653ff173003e40cfb08516cd34262e6af95a5d8645c9c1abb3e813604d508b8511b30f9a5c1b352aa0791c7d2f27b2706dccea54bc7de6555b5202351751c3299f97c09cf89c40f67187e2521c0fad82b30edbb224f0458 +Skein1024 (5.inp) = f23d95c2a25fbcd0e797cd058fec39d3c52d2b5afd7a9af1df934e63257d1d3dcf3246e7329c0f1104c1e51e3d22e300507b0c3b9f985bb1f645ef49835080536becf83788e17fed09c9982ba65c3cb7ffe6a5f745b911c506962adf226e435c42f6f6bc08d288f9c810e807e3216ef444f3db22744441deefa4900982a1371f +Skein1024 (6.inp) = cf3889e8a8d11bfd3938055d7d061437962bc5eac8ae83b1b71c94be201b8cf657fdbfc38674997a008c0c903f56a23feb3ae30e012377f1cfa080a9ca7fe8b96138662653fb3335c7d06595bf8baf65e215307532094cfdfa056bd8052ab792a3944a2adaa47b30335b8badb8fe9eb94fe329cdca04e58bbc530f0af709f469 +Skein1024 (7.inp) = cf21a613620e6c119eca31fdfaad449a8e02f95ca256c21d2a105f8e4157048f9fe1e897893ea18b64e0e37cb07d5ac947f27ba544caf7cbc1ad094e675aed77a366270f7eb7f46543bccfa61c526fd628408058ed00ed566ac35a9761d002e629c4fb0d430b2f4ad016fcc49c44d2981c4002da0eecc42144160e2eaea4855a +Skein1024 (8.inp) = e6799b78db54085a2be7ff4c8007f147fa88d326abab30be0560b953396d8802feee9a15419b48a467574e9283be15685ca8a079ee52b27166b64dd70b124b1d4e4f6aca37224c3f2685e67e67baef9f94b905698adc794a09672aba977a61b20966912acdb08c21a2c37001785355dc884751a21f848ab36e590331ff938138 diff --git a/sbin/md5/tests/skein1024sum.digest b/sbin/md5/tests/skein1024sum.digest new file mode 100644 index 000000000000..dca058ab55b7 --- /dev/null +++ b/sbin/md5/tests/skein1024sum.digest @@ -0,0 +1,8 @@ +0fff9563bb3279289227ac77d319b6fff8d7e9f09da1247b72a0a265cd6d2a62645ad547ed8193db48cff847c06494a03f55666d3b47eb4c20456c9373c86297d630d5578ebd34cb40991578f9f52b18003efa35d3da6553ff35db91b81ab890bec1b189b7f52cb2a783ebb7d823d725b0b4a71f6824e88f68f982eefc6d19c6 1.inp +6ab4c4ba9814a3d976ec8bffa7fcc638ceba0544a97b3c98411323ffd2dc936315d13dc93c13c4e88cda6f5bac6f2558b2d8694d3b6143e40d644ae43ca940685cb37f809d3d0550c56cba8036dee729a4f8fb960732e59e64d57f7f7710f8670963cdcdc95b41daab4855fcf8b6762a64b173ee61343a2c7689af1d293eba97 2.inp +35a599a0f91abcdb4cb73c19b8cb8d947742d82c309137a7caed29e8e0a2ca7a9ff9a90c34c1908cc7e7fd99bb15032fb86e76df21b72628399b5f7c3cc209d7bb31c99cd4e19465622a049afbb87c03b5ce3888d17e6e667279ec0aa9b3e2712624c01b5f5bbe1a564220bdcf6990af0c2539019f313fdd7406cca3892a1f1f 3.inp +ea891f5268acd0fac97467fc1aa89d1ce8681a9992a42540e53babee861483110c2d16f49e73bac27653ff173003e40cfb08516cd34262e6af95a5d8645c9c1abb3e813604d508b8511b30f9a5c1b352aa0791c7d2f27b2706dccea54bc7de6555b5202351751c3299f97c09cf89c40f67187e2521c0fad82b30edbb224f0458 4.inp +f23d95c2a25fbcd0e797cd058fec39d3c52d2b5afd7a9af1df934e63257d1d3dcf3246e7329c0f1104c1e51e3d22e300507b0c3b9f985bb1f645ef49835080536becf83788e17fed09c9982ba65c3cb7ffe6a5f745b911c506962adf226e435c42f6f6bc08d288f9c810e807e3216ef444f3db22744441deefa4900982a1371f 5.inp +cf3889e8a8d11bfd3938055d7d061437962bc5eac8ae83b1b71c94be201b8cf657fdbfc38674997a008c0c903f56a23feb3ae30e012377f1cfa080a9ca7fe8b96138662653fb3335c7d06595bf8baf65e215307532094cfdfa056bd8052ab792a3944a2adaa47b30335b8badb8fe9eb94fe329cdca04e58bbc530f0af709f469 6.inp +cf21a613620e6c119eca31fdfaad449a8e02f95ca256c21d2a105f8e4157048f9fe1e897893ea18b64e0e37cb07d5ac947f27ba544caf7cbc1ad094e675aed77a366270f7eb7f46543bccfa61c526fd628408058ed00ed566ac35a9761d002e629c4fb0d430b2f4ad016fcc49c44d2981c4002da0eecc42144160e2eaea4855a 7.inp +e6799b78db54085a2be7ff4c8007f147fa88d326abab30be0560b953396d8802feee9a15419b48a467574e9283be15685ca8a079ee52b27166b64dd70b124b1d4e4f6aca37224c3f2685e67e67baef9f94b905698adc794a09672aba977a61b20966912acdb08c21a2c37001785355dc884751a21f848ab36e590331ff938138 8.inp diff --git a/sbin/md5/tests/skein256.digest b/sbin/md5/tests/skein256.digest new file mode 100644 index 000000000000..ee17f8df1532 --- /dev/null +++ b/sbin/md5/tests/skein256.digest @@ -0,0 +1,8 @@ +Skein256 (1.inp) = c8877087da56e072870daa843f176e9453115929094c3a40c463a196c29bf7ba +Skein256 (2.inp) = 7fba44ff1a31d71a0c1f82e6e82fb5e9ac6c92a39c9185b9951fed82d82fe635 +Skein256 (3.inp) = 258bdec343b9fde1639221a5ae0144a96e552e5288753c5fec76c05fc2fc1870 +Skein256 (4.inp) = 4d2ce0062b5eb3a4db95bc1117dd8aa014f6cd50fdc8e64f31f7d41f9231e488 +Skein256 (5.inp) = 46d8440685461b00e3ddb891b2ecc6855287d2bd8834a95fb1c1708b00ea5e82 +Skein256 (6.inp) = 7c5eb606389556b33d34eb2536459528dc0af97adbcd0ce273aeb650f598d4b2 +Skein256 (7.inp) = 4def7a7e5464a140ae9c3a80279fbebce4bd00f9faad819ab7e001512f67a10d +Skein256 (8.inp) = d9c017dbe355f318d036469eb9b5fbe129fc2b5786a9dc6746a516eab6fe0126 diff --git a/sbin/md5/tests/skein256sum.digest b/sbin/md5/tests/skein256sum.digest new file mode 100644 index 000000000000..2e1c8d1bc60d --- /dev/null +++ b/sbin/md5/tests/skein256sum.digest @@ -0,0 +1,8 @@ +c8877087da56e072870daa843f176e9453115929094c3a40c463a196c29bf7ba 1.inp +7fba44ff1a31d71a0c1f82e6e82fb5e9ac6c92a39c9185b9951fed82d82fe635 2.inp +258bdec343b9fde1639221a5ae0144a96e552e5288753c5fec76c05fc2fc1870 3.inp +4d2ce0062b5eb3a4db95bc1117dd8aa014f6cd50fdc8e64f31f7d41f9231e488 4.inp +46d8440685461b00e3ddb891b2ecc6855287d2bd8834a95fb1c1708b00ea5e82 5.inp +7c5eb606389556b33d34eb2536459528dc0af97adbcd0ce273aeb650f598d4b2 6.inp +4def7a7e5464a140ae9c3a80279fbebce4bd00f9faad819ab7e001512f67a10d 7.inp +d9c017dbe355f318d036469eb9b5fbe129fc2b5786a9dc6746a516eab6fe0126 8.inp diff --git a/sbin/md5/tests/skein512.digest b/sbin/md5/tests/skein512.digest new file mode 100644 index 000000000000..6b2e9ad0f723 --- /dev/null +++ b/sbin/md5/tests/skein512.digest @@ -0,0 +1,8 @@ +Skein512 (1.inp) = bc5b4c50925519c290cc634277ae3d6257212395cba733bbad37a4af0fa06af41fca7903d06564fea7a2d3730dbdb80c1f85562dfcc070334ea4d1d9e72cba7a +Skein512 (2.inp) = b1cd8d33f61b3737adfd59bb13ad82f4a9548e92f22956a8976cca3fdb7fee4fe91698146c4197cec85d38b83c5d93bdba92c01fd9a53870d0c7f967bc62bdce +Skein512 (3.inp) = 8f5dd9ec798152668e35129496b029a960c9a9b88662f7f9482f110b31f9f93893ecfb25c009baad9e46737197d5630379816a886aa05526d3a70df272d96e75 +Skein512 (4.inp) = 15b73c158ffb875fed4d72801ded0794c720b121c0c78edf45f900937e6933d9e21a3a984206933d504b5dbb2368000411477ee1b204c986068df77886542fcc +Skein512 (5.inp) = 23793ad900ef12f9165c8080da6fdfd2c8354a2929b8aadf83aa82a3c6470342f57cf8c035ec0d97429b626c4d94f28632c8f5134fd367dca5cf293d2ec13f8c +Skein512 (6.inp) = 0c6bed927e022f5ddcf81877d42e5f75798a9f8fd3ede3d83baac0a2f364b082e036c11af35fe478745459dd8f5c0b73efe3c56ba5bb2009208d5a29cc6e469c +Skein512 (7.inp) = 2ca9fcffb3456f297d1b5f407014ecb856f0baac8eb540f534b1f187196f21e88f31103128c2f03fcc9857d7a58eb66f9525e2302d88833ee069295537a434ce +Skein512 (8.inp) = 1131f2aaa0e97126c9314f9f968cc827259bbfabced2943bb8c9274448998fb3b78738b4580dd500c76105fd3c03e465e1414f2c29664286b1f79d3e51128125 diff --git a/sbin/md5/tests/skein512sum.digest b/sbin/md5/tests/skein512sum.digest new file mode 100644 index 000000000000..3ee9b67038ad --- /dev/null +++ b/sbin/md5/tests/skein512sum.digest @@ -0,0 +1,8 @@ +bc5b4c50925519c290cc634277ae3d6257212395cba733bbad37a4af0fa06af41fca7903d06564fea7a2d3730dbdb80c1f85562dfcc070334ea4d1d9e72cba7a 1.inp +b1cd8d33f61b3737adfd59bb13ad82f4a9548e92f22956a8976cca3fdb7fee4fe91698146c4197cec85d38b83c5d93bdba92c01fd9a53870d0c7f967bc62bdce 2.inp +8f5dd9ec798152668e35129496b029a960c9a9b88662f7f9482f110b31f9f93893ecfb25c009baad9e46737197d5630379816a886aa05526d3a70df272d96e75 3.inp +15b73c158ffb875fed4d72801ded0794c720b121c0c78edf45f900937e6933d9e21a3a984206933d504b5dbb2368000411477ee1b204c986068df77886542fcc 4.inp +23793ad900ef12f9165c8080da6fdfd2c8354a2929b8aadf83aa82a3c6470342f57cf8c035ec0d97429b626c4d94f28632c8f5134fd367dca5cf293d2ec13f8c 5.inp +0c6bed927e022f5ddcf81877d42e5f75798a9f8fd3ede3d83baac0a2f364b082e036c11af35fe478745459dd8f5c0b73efe3c56ba5bb2009208d5a29cc6e469c 6.inp +2ca9fcffb3456f297d1b5f407014ecb856f0baac8eb540f534b1f187196f21e88f31103128c2f03fcc9857d7a58eb66f9525e2302d88833ee069295537a434ce 7.inp +1131f2aaa0e97126c9314f9f968cc827259bbfabced2943bb8c9274448998fb3b78738b4580dd500c76105fd3c03e465e1414f2c29664286b1f79d3e51128125 8.inp From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 15:38:48 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7770665C967; Tue, 6 Jul 2021 15:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK6Cr2xx5z3h01; Tue, 6 Jul 2021 15:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 498301C159; Tue, 6 Jul 2021 15:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166FcmOw065467; Tue, 6 Jul 2021 15:38:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166FcmPq065466; Tue, 6 Jul 2021 15:38:48 GMT (envelope-from git) Date: Tue, 6 Jul 2021 15:38:48 GMT Message-Id: <202107061538.166FcmPq065466@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: 5f13ee9179e1 - stable/12 - md5: Create md5sum, etc compatible programs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 5f13ee9179e15702129d5ea330c599ae9a3b871f Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 15:38:48 -0000 The branch stable/12 has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=5f13ee9179e15702129d5ea330c599ae9a3b871f commit 5f13ee9179e15702129d5ea330c599ae9a3b871f Author: Warner Losh AuthorDate: 2021-05-19 17:26:20 +0000 Commit: Stefan Eßer CommitDate: 2021-07-06 15:37:24 +0000 md5: Create md5sum, etc compatible programs On Linux, there's a similar set of programs to ours, but that end in the letters 'sum'. These act basically like FreeBSD versions run with the -r option. Add code so that when the program ends in 'sum' you get the linux -r behavior. This is enough to make most things that use sha*sum work correctly (the -c / --check options, as well as the long args are not implemented). When running with the -sum programs, ignore -t instead of running internal speed tests and make -c an error. Reviewed by: sef, and kp and allanjude (earlier version) Relnotes: yes Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D30309 --- sbin/md5/Makefile | 30 ++++++++++++++++++++++++++---- sbin/md5/md5.1 | 32 +++++++++++++++++++++++++++++++- sbin/md5/md5.c | 35 ++++++++++++++++++++++++++++++----- 3 files changed, 87 insertions(+), 10 deletions(-) diff --git a/sbin/md5/Makefile b/sbin/md5/Makefile index 6e29d94eb92a..e499967d23d5 100644 --- a/sbin/md5/Makefile +++ b/sbin/md5/Makefile @@ -4,27 +4,49 @@ PACKAGE=runtime PROG= md5 -LINKS= ${BINDIR}/md5 ${BINDIR}/rmd160 \ +LINKS= ${BINDIR}/md5 ${BINDIR}/md5sum \ + ${BINDIR}/md5 ${BINDIR}/rmd160 \ + ${BINDIR}/md5 ${BINDIR}/rmd160sum \ ${BINDIR}/md5 ${BINDIR}/sha1 \ + ${BINDIR}/md5 ${BINDIR}/sha1sum \ ${BINDIR}/md5 ${BINDIR}/sha224 \ + ${BINDIR}/md5 ${BINDIR}/sha224sum \ ${BINDIR}/md5 ${BINDIR}/sha256 \ + ${BINDIR}/md5 ${BINDIR}/sha256sum \ ${BINDIR}/md5 ${BINDIR}/sha384 \ + ${BINDIR}/md5 ${BINDIR}/sha384sum \ ${BINDIR}/md5 ${BINDIR}/sha512 \ + ${BINDIR}/md5 ${BINDIR}/sha512sum \ ${BINDIR}/md5 ${BINDIR}/sha512t256 \ + ${BINDIR}/md5 ${BINDIR}/sha512t256sum \ ${BINDIR}/md5 ${BINDIR}/skein256 \ + ${BINDIR}/md5 ${BINDIR}/skein256sum \ ${BINDIR}/md5 ${BINDIR}/skein512 \ - ${BINDIR}/md5 ${BINDIR}/skein1024 + ${BINDIR}/md5 ${BINDIR}/skein512sum \ + ${BINDIR}/md5 ${BINDIR}/skein1024 \ + ${BINDIR}/md5 ${BINDIR}/skein1024sum -MLINKS= md5.1 rmd160.1 \ +MLINKS= md5.1 md5sum.1 \ + md5.1 rmd160.1 \ + md5.1 rmd160sum.1 \ md5.1 sha1.1 \ + md5.1 sha1sum.1 \ md5.1 sha224.1 \ + md5.1 sha224sum.1 \ md5.1 sha256.1 \ + md5.1 sha256sum.1 \ md5.1 sha384.1 \ + md5.1 sha384sum.1 \ md5.1 sha512.1 \ + md5.1 sha512sum.1 \ md5.1 sha512t256.1 \ + md5.1 sha512t256sum.1 \ md5.1 skein256.1 \ + md5.1 skein256sum.1 \ md5.1 skein512.1 \ - md5.1 skein1024.1 + md5.1 skein512sum.1 \ + md5.1 skein1024.1 \ + md5.1 skein1024sum.1 LIBADD= md diff --git a/sbin/md5/md5.1 b/sbin/md5/md5.1 index 1cf0db333b74..af4e606b62e7 100644 --- a/sbin/md5/md5.1 +++ b/sbin/md5/md5.1 @@ -4,7 +4,9 @@ .Os .Sh NAME .Nm md5 , sha1 , sha224 , sha256 , sha384 , sha512 , sha512t256 , rmd160 , -.Nm skein256 , skein512 , skein1024 +.Nm skein256 , skein512 , skein1024 , +.Nm md5sum , sha1sum , sha224sum , sha256sum , sha384sum , sha512sum , +.Nm sha512t256sum , rmd160sum , skein256sum , skein512sum , skein1024sum .Nd calculate a message-digest fingerprint (checksum) for a file .Sh SYNOPSIS .Nm @@ -26,6 +28,15 @@ output a or .Dq message digest of the input. +The +.Nm md5sum , sha1sum , sha224sum , sha256sum , sha384sum , sha512sum , +.Nm sha512t256sum , rmd160sum , skein256sum , skein512sum , +and +.Nm skein1024sum +utilities do the same, but default to the reversed format of +the +.Fl r +flag. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. @@ -73,9 +84,18 @@ precede any files named on the command line. The hexadecimal checksum of each file listed on the command line is printed after the options are processed. .Bl -tag -width indent +.It Fl b +Ignored for compatibility with the coreutils +.Nm -sum +programs. .It Fl c Ar string Compare the digest of the file against this string. .Pq Note that this option is not yet useful if multiple files are specified. +This option causes an error in for the +.Nm -sum +programs because it check the checksums listed in a file for the coreutils +.Nm -sum +programs that is not yet implemented. .It Fl s Ar string Print a checksum of the given .Ar string . @@ -95,6 +115,9 @@ when combined with the options. .It Fl t Run a built-in time trial. +For the +.Nm -sum +versions, this is a nop for compatibility with coreutils. .It Fl x Run a built-in test script. .El @@ -144,6 +167,13 @@ Secure Hash Standard (SHS): .Pp The RIPEMD-160 page: .Pa http://www.esat.kuleuven.ac.be/~bosselae/ripemd160.html . +.Sh BUGS +All of the utilities that end in +.Sq sum +are intended to be compatible with the GNU coreutils programs. +However, the long arguments and the +.Fl -check +functionality are not provided. .Sh ACKNOWLEDGMENTS This program is placed in the public domain for free general use by RSA Data Security. diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index 96dfcede9711..4381ef124c7b 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -177,13 +177,32 @@ main(int argc, char *argv[]) char buf[HEX_DIGEST_LENGTH]; size_t len; unsigned digest; - const char* progname; + char *progname; + bool gnu_emu = false; if ((progname = strrchr(argv[0], '/')) == NULL) progname = argv[0]; else progname++; + /* + * GNU coreutils has a number of programs named *sum. These produce + * similar results to the BSD version, but in a different format, + * similar to BSD's -r flag. We install links to this program with + * ending 'sum' to provide this compatibility. Check here to see if the + * name of the program ends in 'sum', set the flag and drop the 'sum' so + * the digest lookup works. Also, make -t a nop when running in this mode + * since that means 'text file' there (though it's a nop in coreutils + * on unix-like systems). The -c flag conflicts, so it's just disabled + * in this mode (though in the future it might be implemented). + */ + len = strlen(progname); + if (len > 3 && strcmp(progname + len - 3, "sum") == 0) { + progname[len - 3] = '\0'; + rflag = 1; + gnu_emu = true; + } + for (digest = 0; digest < sizeof(Algorithm)/sizeof(*Algorithm); digest++) if (strcasecmp(Algorithm[digest].progname, progname) == 0) break; @@ -195,9 +214,13 @@ main(int argc, char *argv[]) checkAgainst = NULL; checksFailed = 0; skip = 0; - while ((ch = getopt(argc, argv, "c:pqrs:tx")) != -1) + while ((ch = getopt(argc, argv, "bc:pqrs:tx")) != -1) switch (ch) { + case 'b': + break; case 'c': + if (gnu_emu) + errx(1, "-c check option not supported"); checkAgainst = optarg; break; case 'p': @@ -214,8 +237,10 @@ main(int argc, char *argv[]) string = optarg; break; case 't': - MDTimeTrial(&Algorithm[digest]); - skip = 1; + if (!gnu_emu) { + MDTimeTrial(&Algorithm[digest]); + skip = 1; + } /* else: text mode is a nop */ break; case 'x': MDTestSuite(&Algorithm[digest]); @@ -348,7 +373,7 @@ MDTimeTrial(const Algorithm_t *alg) printf(" done\n"); printf("Digest = %s", p); printf("\nTime = %f seconds\n", seconds); - printf("Speed = %f MiB/second\n", (float) TEST_BLOCK_LEN * + printf("Speed = %f MiB/second\n", (float) TEST_BLOCK_LEN * (float) TEST_BLOCK_COUNT / seconds / (1 << 20)); } /* From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 15:38:49 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9A13865CC93; Tue, 6 Jul 2021 15:38:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK6Cs3dLTz3h55; Tue, 6 Jul 2021 15:38:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 61BA01C500; Tue, 6 Jul 2021 15:38:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166FcnHO065499; Tue, 6 Jul 2021 15:38:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166FcnVo065498; Tue, 6 Jul 2021 15:38:49 GMT (envelope-from git) Date: Tue, 6 Jul 2021 15:38:49 GMT Message-Id: <202107061538.166FcnVo065498@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: 04beb393e366 - stable/12 - md5: portability fix -- include stdbool.h explicitly MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 04beb393e36691f415fbaf1746d4b981784ab734 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 15:38:49 -0000 The branch stable/12 has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=04beb393e36691f415fbaf1746d4b981784ab734 commit 04beb393e36691f415fbaf1746d4b981784ab734 Author: Warner Losh AuthorDate: 2021-05-20 17:26:46 +0000 Commit: Stefan Eßer CommitDate: 2021-07-06 15:38:03 +0000 md5: portability fix -- include stdbool.h explicitly stdbool.h needs to be included to use type bool variables. Due to namespace pollution, this gets brought in on FreeBSD, but not on other systems. Include it explicilty. Noticed by: arichards@ Sponsored by: Netflix --- sbin/md5/md5.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index 4381ef124c7b..e553d4c56e85 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 15:47:27 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2E6D865CAEF; Tue, 6 Jul 2021 15:47:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK6Pq0nDzz3hs1; Tue, 6 Jul 2021 15:47:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 02D9E1C5C0; Tue, 6 Jul 2021 15:47:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166FlQAR078704; Tue, 6 Jul 2021 15:47:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166FlQsD078703; Tue, 6 Jul 2021 15:47:26 GMT (envelope-from git) Date: Tue, 6 Jul 2021 15:47:26 GMT Message-Id: <202107061547.166FlQsD078703@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: =?utf-8?Q?Stefan E=C3=9Fer?= Subject: git: b7c17fd730ca - stable/12 - sbin/md5: add md5sum to synopsis in the man-page MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: se X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: b7c17fd730ca4e8806a506b617ad32218aa0b826 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 15:47:27 -0000 The branch stable/12 has been updated by se: URL: https://cgit.FreeBSD.org/src/commit/?id=b7c17fd730ca4e8806a506b617ad32218aa0b826 commit b7c17fd730ca4e8806a506b617ad32218aa0b826 Author: Stefan Eßer AuthorDate: 2021-06-19 11:27:16 +0000 Commit: Stefan Eßer CommitDate: 2021-07-06 15:44:14 +0000 sbin/md5: add md5sum to synopsis in the man-page When invoked with "sum" following the hash name, the -c option takes a file argument, not a string argument. (cherry picked from commit 84f35b6f86efd004a8f25ef6eecea0f5d02733c4) sbin/md5: improve compatibility with coreutils -c option The -c option expects a digest file in either BSD or coreutils format. The output for matched and mismatched files is identical to that of the coreutils version. The review of these changes included test cases that have already been committed for the functionality that existed before. Another test script is added to cover the coreutils compatible extension implemented by this patch. This commit contains a tests/Makefile that has been cleaned up compared to the review version, using an implicit rule to apply the TESTBASE path at build time (and the scripts have been renamed to have an extension of .SH instead of .sh to trigger this rule). Reviewed by: imp Differential Revision: https://reviews.freebsd.org/D30812 (cherry picked from commit b33d1898c1b0e6d76b52eb48078260588802dc95) md5: Improve compatibility with coreutils and format fix The previous changes that added support for the coreutils -c option modified the output generated by passing -r to match that of the coreutils versions. The difference is that coreutils separates the hash from the file name by two blanks " " (or a blank followed by an asterisk " *" with the -b option denoting). While most scripts or users will not notice the difference, it might be considered a violation of POLA and this commit reverts the change for the non-sum programs. These will print a single blank " " as the separator, as they die before the previous commit. In order to still generate output that is identical to that of the coreutils programs, this commit generates the " " or " *" separator used by them for the -sum versions, depending on the presence of the -b option. (cherry picked from commit b33d1898c1b0e6d76b52eb48078260588802dc95) md5: Fix cross-build after c2870e576bd2 On macOS and Linux the current set of headers do not end up pulling in sys/param.h, causing MAXPATHLEN to not be defined and the build to fail. (cherry picked from commit cab31e0e216c7defefd4aba14693ba2252ea7308) --- sbin/md5/md5.1 | 114 ++++++++++++++++++++++--- sbin/md5/md5.c | 171 ++++++++++++++++++++++++++++++++----- sbin/md5/tests/Makefile | 1 + sbin/md5/tests/coreutils-c-test.SH | 21 +++++ 4 files changed, 274 insertions(+), 33 deletions(-) diff --git a/sbin/md5/md5.1 b/sbin/md5/md5.1 index af4e606b62e7..f91e0b759dc7 100644 --- a/sbin/md5/md5.1 +++ b/sbin/md5/md5.1 @@ -15,6 +15,12 @@ .Op Fl s Ar string .Op Ar .Pp +.Nm md5sum +.Op Fl pqrtx +.Op Fl c Ar file +.Op Fl s Ar string +.Op Ar +.Pp (All other hashes have the same options and usage.) .Sh DESCRIPTION The @@ -85,17 +91,34 @@ The hexadecimal checksum of each file listed on the command line is printed after the options are processed. .Bl -tag -width indent .It Fl b -Ignored for compatibility with the coreutils +Make the .Nm -sum -programs. +programs separate hash and digest with a blank followed by an asterisk instead +of by 2 blank characters for full compatibility with the output generated by the +coreutils versions of these programs. .It Fl c Ar string -Compare the digest of the file against this string. +If the program was called with a name that does not end in +.Nm sum , +compare the digest of the file against this string. .Pq Note that this option is not yet useful if multiple files are specified. -This option causes an error in for the -.Nm -sum -programs because it check the checksums listed in a file for the coreutils -.Nm -sum -programs that is not yet implemented. +.It Fl c Ar file +If the program was called with a name that does end in +.Nm sum , +the file passed as argument must contain digest lines generated by the same +digest algorithm with or without the +.Fl r +option +.Pq i.e. in either classical BSD format or in GNU coreutils format . +A line with the file name followed by a colon +.Dq ":" +and either OK or FAILED is written for each well-formed line in the digest file. +If applicable, the number of failed comparisons and the number of lines that were +skipped since they were not well-formed are printed at the end. +The +.Fl q +option can be used to quiesce the output unless there are mismatched entries in +the digest. +.Pp .It Fl s Ar string Print a checksum of the given .Ar string . @@ -132,6 +155,77 @@ utilities exit 0 on success, and 2 if at least one file does not have the same hash as the .Fl c option. +<<<<<<< HEAD +======= +.Sh EXAMPLES +Calculate the MD5 checksum of the string +.Dq Hello . +.Bd -literal -offset indent +$ md5 -s Hello +MD5 ("Hello") = 8b1a9953c4611296a827abf8c47804d7 +.Ed +.Pp +Same as above, but note the absence of the newline character in the input +string: +.Bd -literal -offset indent +$ echo -n Hello | md5 +8b1a9953c4611296a827abf8c47804d7 +.Ed +.Pp +Calculate the checksum of multiple files reversing the output: +.Bd -literal -offset indent +$ md5 -r /boot/loader.conf /etc/rc.conf +ada5f60f23af88ff95b8091d6d67bef6 /boot/loader.conf +d80bf36c332dc0fdc479366ec3fa44cd /etc/rc.conf +.Pd +The +.Nm -sum +variants put 2 blank characters between hash and file name for full compatibilty +with the coreutils versions of these commands. +.Ed +.Pp +Write the digest for +.Pa /boot/loader.conf +in a file named +.Pa digest . +Then calculate the checksum again and validate it against the checksum string +extracted from the +.Pa digest +file: +.Bd -literal -offset indent +$ md5 /boot/loader.conf > digest && md5 -c $(cut -f2 -d= digest) /boot/loader.conf +MD5 (/boot/loader.conf) = ada5f60f23af88ff95b8091d6d67bef6 +.Ed +.Pp +Same as above but comparing the digest against an invalid string +.Pq Dq randomstring , +which results in a failure. +.Bd -literal -offset indent +$ md5 -c randomstring /boot/loader.conf +MD5 (/boot/loader.conf) = ada5f60f23af88ff95b8091d6d67bef6 [ Failed ] +.Ed +.Pp +If invoked with a name ending in +.Nm -sum +the +.Fl c +option does not compare against a hash string passed as parameter. +Instead, it expects a digest file, as created under the name +.Pa digest +for +.Pa /boot/loader.conf +in the example above. +.Bd -literal -offset indent +$ md5 -c digest /boot/loader.conf +/boot/loader.conf: OK +.Ed +.Pp +The digest file may contain any number of lines in the format generated with or without the +.Fl r +option +.Pq i.e. in either classical BSD format or in GNU coreutils format . +If a hash value does not match the file, FAILED is printed instead of OK. +>>>>>>> c2870e576bd2 (sbin/md5: improve compatibility with coreutils -c option) .Sh SEE ALSO .Xr cksum 1 , .Xr md5 3 , @@ -171,9 +265,7 @@ The RIPEMD-160 page: All of the utilities that end in .Sq sum are intended to be compatible with the GNU coreutils programs. -However, the long arguments and the -.Fl -check -functionality are not provided. +However, the long option functionality is not provided. .Sh ACKNOWLEDGMENTS This program is placed in the public domain for free general use by RSA Data Security. diff --git a/sbin/md5/md5.c b/sbin/md5/md5.c index e553d4c56e85..7235e6e0a39f 100644 --- a/sbin/md5/md5.c +++ b/sbin/md5/md5.c @@ -21,6 +21,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -53,6 +54,8 @@ __FBSDID("$FreeBSD$"); #define TEST_BLOCK_COUNT 100000 #define MDTESTCOUNT 8 +static int bflag; +static int cflag; static int pflag; static int qflag; static int rflag; @@ -152,12 +155,93 @@ static const struct Algorithm_t Algorithm[] = { (DIGEST_End*)&SKEIN1024_End, &SKEIN1024_Data, &SKEIN1024_Fd } }; +static unsigned digest; +static unsigned malformed; +static bool gnu_emu = false; + static void MD5_Update(MD5_CTX *c, const unsigned char *data, size_t len) { MD5Update(c, data, len); } +struct chksumrec { + char *filename; + char *chksum; + struct chksumrec *next; +}; + +static struct chksumrec *head = NULL; +static struct chksumrec **next = &head; + +#define PADDING 7 /* extra padding for "SHA512t256 (...) = ...\n" style */ +#define CHKFILELINELEN (HEX_DIGEST_LENGTH + MAXPATHLEN + PADDING) + +static int gnu_check(const char *checksumsfile) +{ + FILE *inp; + char linebuf[CHKFILELINELEN]; + int linelen; + int lineno; + char *filename; + char *hashstr; + struct chksumrec *rec; + const char *digestname; + int digestnamelen; + int hashstrlen; + + if ((inp = fopen(checksumsfile, "r")) == NULL) + err(1, "%s", checksumsfile); + digestname = Algorithm[digest].name; + digestnamelen = strlen(digestname); + hashstrlen = strlen(*(Algorithm[digest].TestOutput[0])); + lineno = 1; + while (fgets(linebuf, sizeof(linebuf), inp) != NULL) { + linelen = strlen(linebuf) - 1; + if (linelen <= 0) + break; + if (linebuf[linelen] != '\n') + errx(1, "malformed input line %d (len=%d)", lineno, linelen); + linebuf[linelen] = '\0'; + filename = linebuf + digestnamelen + 2; + hashstr = linebuf + linelen - hashstrlen; + /* + * supported formats: + * BSD: (): + * GNU: [ *] + */ + if (linelen >= digestnamelen + hashstrlen + 6 && + strncmp(linebuf, digestname, digestnamelen) == 0 && + strncmp(filename - 2, " (", 2) == 0 && + strncmp(hashstr - 4, ") = ", 4) == 0) { + *(hashstr - 4) = '\0'; + } else if (linelen >= hashstrlen + 3 && + linebuf[hashstrlen] == ' ') { + linebuf[hashstrlen] = '\0'; + hashstr = linebuf; + filename = linebuf + hashstrlen + 1; + if (*filename == ' ' || *filename == '*') + filename++; + } else { + malformed++; + continue; + } + rec = malloc(sizeof (*rec)); + if (rec == NULL) + errx(1, "malloc failed"); + rec->chksum = strdup(hashstr); + rec->filename = strdup(filename); + if (rec->chksum == NULL || rec->filename == NULL) + errx(1, "malloc failed"); + rec->next = NULL; + *next = rec; + next = &rec->next; + lineno++; + } + fclose(inp); + return (lineno - 1); +} + /* Main driver. Arguments (may be any combination): @@ -177,9 +261,9 @@ main(int argc, char *argv[]) char *p, *string; char buf[HEX_DIGEST_LENGTH]; size_t len; - unsigned digest; char *progname; - bool gnu_emu = false; + struct chksumrec *rec; + int numrecs; if ((progname = strrchr(argv[0], '/')) == NULL) progname = argv[0]; @@ -199,13 +283,13 @@ main(int argc, char *argv[]) */ len = strlen(progname); if (len > 3 && strcmp(progname + len - 3, "sum") == 0) { - progname[len - 3] = '\0'; + len -= 3; rflag = 1; gnu_emu = true; } for (digest = 0; digest < sizeof(Algorithm)/sizeof(*Algorithm); digest++) - if (strcasecmp(Algorithm[digest].progname, progname) == 0) + if (strncasecmp(Algorithm[digest].progname, progname, len) == 0) break; if (digest == sizeof(Algorithm)/sizeof(*Algorithm)) @@ -218,11 +302,14 @@ main(int argc, char *argv[]) while ((ch = getopt(argc, argv, "bc:pqrs:tx")) != -1) switch (ch) { case 'b': + bflag = 1; break; case 'c': + cflag = 1; if (gnu_emu) - errx(1, "-c check option not supported"); - checkAgainst = optarg; + numrecs = gnu_check(optarg); + else + checkAgainst = optarg; break; case 'p': pflag = 1; @@ -258,6 +345,20 @@ main(int argc, char *argv[]) err(1, "unable to limit rights for stdio"); #endif + if (cflag && gnu_emu) { + /* + * Replace argv by an array of filenames from the digest file + */ + argc = 0; + argv = (char**)calloc(sizeof(char *), numrecs + 1); + for (rec = head; rec != NULL; rec = rec->next) { + argv[argc] = rec->filename; + argc++; + } + argv[argc] = NULL; + rec = head; + } + if (*argv) { do { if ((fd = open(*argv, O_RDONLY)) < 0) { @@ -279,11 +380,15 @@ main(int argc, char *argv[]) err(1, "capsicum"); #endif } + if (cflag && gnu_emu) { + checkAgainst = rec->chksum; + rec = rec->next; + } p = Algorithm[digest].Fd(fd, buf); (void)close(fd); MDOutput(&Algorithm[digest], p, argv); } while (*++argv); - } else if (!sflag && !skip) { + } else if (!cflag && !sflag && !skip) { #ifdef HAVE_CAPSICUM if (caph_limit_stdin() < 0 || caph_enter() < 0) err(1, "capsicum"); @@ -295,7 +400,12 @@ main(int argc, char *argv[]) p = Algorithm[digest].Data(string, len, buf); MDOutput(&Algorithm[digest], p, &string); } - + if (gnu_emu) { + if (malformed > 0) + warnx("WARNING: %d lines are improperly formatted", malformed); + if (checksFailed > 0) + warnx("WARNING: %d computed checksums did NOT match", checksFailed); + } if (failed != 0) return (1); if (checksFailed != 0) @@ -310,6 +420,8 @@ main(int argc, char *argv[]) static void MDOutput(const Algorithm_t *alg, char *p, char *argv[]) { + bool checkfailed = false; + if (p == NULL) { warn("%s", *argv); failed++; @@ -318,21 +430,33 @@ MDOutput(const Algorithm_t *alg, char *p, char *argv[]) * If argv is NULL we are reading from stdin, where the output * format has always been just the hash. */ - if (qflag || argv == NULL) - printf("%s", p); - else if (rflag) - printf("%s %s", p, *argv); - else - printf("%s (%s) = %s", - alg->name, *argv, p); - if (checkAgainst && strcasecmp(checkAgainst, p) != 0) - { - checksFailed++; - if (!qflag) - printf(" [ Failed ]"); + if (cflag && gnu_emu) { + checkfailed = strcasecmp(checkAgainst, p) != 0; + if (!qflag || checkfailed) + printf("%s: %s\n", *argv, checkfailed ? "FAILED" : "OK"); + } else if (qflag || argv == NULL) { + printf("%s\n", p); + } else { + if (rflag) + if (gnu_emu) + if (bflag) + printf("%s *%s", p, *argv); + else + printf("%s %s", p, *argv); + else + printf("%s %s", p, *argv); + else + printf("%s (%s) = %s", alg->name, *argv, p); + if (checkAgainst) { + checkfailed = strcasecmp(checkAgainst, p) != 0; + if (!qflag && checkfailed) + printf(" [ Failed ]"); + } + printf("\n"); } - printf("\n"); } + if (checkfailed) + checksFailed++; } /* @@ -559,6 +683,9 @@ static void usage(const Algorithm_t *alg) { - fprintf(stderr, "usage: %s [-pqrtx] [-c string] [-s string] [files ...]\n", alg->progname); + if (gnu_emu) + fprintf(stderr, "usage: %ssum [-pqrtx] [-c file] [-s string] [files ...]\n", alg->progname); + else + fprintf(stderr, "usage: %s [-pqrtx] [-c string] [-s string] [files ...]\n", alg->progname); exit(1); } diff --git a/sbin/md5/tests/Makefile b/sbin/md5/tests/Makefile index b37332aa9894..93f3913c687d 100644 --- a/sbin/md5/tests/Makefile +++ b/sbin/md5/tests/Makefile @@ -32,6 +32,7 @@ PLAIN_TESTS_SH+= self-test PLAIN_TESTS_SH+= bsd-c-test PLAIN_TESTS_SH+= bsd-p-test PLAIN_TESTS_SH+= bsd-s-test +PLAIN_TESTS_SH+= coreutils-c-test .SUFFIXES: .SH diff --git a/sbin/md5/tests/coreutils-c-test.SH b/sbin/md5/tests/coreutils-c-test.SH new file mode 100644 index 000000000000..5996e533c627 --- /dev/null +++ b/sbin/md5/tests/coreutils-c-test.SH @@ -0,0 +1,21 @@ +#!/bin/sh + +/bin/cp %%TESTSBASE%%/sbin/md5/*.inp . || exit 127 + +exitcode=0 + +testloop () { + opt=$1 + + while read algorithm; do + ${algorithm}sum -c %%TESTSBASE%%/sbin/md5/${algorithm}.digest || exitcode=1 + ${algorithm}sum -c %%TESTSBASE%%/sbin/md5/${algorithm}sum.digest || exitcode=1 + done < %%TESTSBASE%%/sbin/md5/algorithms.txt +} + +testloop "" +testloop -q +testloop -r +testloop -qr + +exit $exitcode From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 17:38:29 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3507C664929; Tue, 6 Jul 2021 17:38:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK8sx0qj2z4XMS; Tue, 6 Jul 2021 17:38:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 04A9F1DE89; Tue, 6 Jul 2021 17:38:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166HcSCQ026380; Tue, 6 Jul 2021 17:38:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166HcSjN026379; Tue, 6 Jul 2021 17:38:28 GMT (envelope-from git) Date: Tue, 6 Jul 2021 17:38:28 GMT Message-Id: <202107061738.166HcSjN026379@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ryan Moeller Subject: git: 4b74458dddba - stable/13 - bsdinstall: Avoid double-mounting /dev MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: freqlabs X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4b74458dddba20cf83e27ba2834aba61e37d0324 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 17:38:29 -0000 The branch stable/13 has been updated by freqlabs: URL: https://cgit.FreeBSD.org/src/commit/?id=4b74458dddba20cf83e27ba2834aba61e37d0324 commit 4b74458dddba20cf83e27ba2834aba61e37d0324 Author: Ryan Moeller AuthorDate: 2021-06-23 13:42:43 +0000 Commit: Ryan Moeller CommitDate: 2021-07-06 17:04:28 +0000 bsdinstall: Avoid double-mounting /dev After 34766aa8cb514472c571f8b0e90e833833acef51 we are mounting and unmounting devfs elsewhere already. Reviewed by: nwhitehorn MFC after: 1 week Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D30877 (cherry picked from commit b50db44f021c12283a2e140063a6b6fcc30295e5) --- usr.sbin/bsdinstall/scripts/script | 2 -- 1 file changed, 2 deletions(-) diff --git a/usr.sbin/bsdinstall/scripts/script b/usr.sbin/bsdinstall/scripts/script index 131ef008d35f..9ed06c1bd36b 100755 --- a/usr.sbin/bsdinstall/scripts/script +++ b/usr.sbin/bsdinstall/scripts/script @@ -162,9 +162,7 @@ fi if [ -f $TMPDIR/bsdinstall-installscript-ab ]; then cp $TMPDIR/bsdinstall-installscript-ab $BSDINSTALL_CHROOT/tmp/installscript chmod a+x $BSDINSTALL_CHROOT/tmp/installscript - mount -t devfs devfs "$BSDINSTALL_CHROOT/dev" chroot $BSDINSTALL_CHROOT /tmp/installscript $@ 2>&1 - umount "$BSDINSTALL_CHROOT/dev" rm $BSDINSTALL_CHROOT/tmp/installscript fi From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 17:53:25 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0EE8F6648F2; Tue, 6 Jul 2021 17:53:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9C86s9Mz4YrH; Tue, 6 Jul 2021 17:53:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D465B1DE76; Tue, 6 Jul 2021 17:53:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166HrO4S052702; Tue, 6 Jul 2021 17:53:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166HrO1O052701; Tue, 6 Jul 2021 17:53:24 GMT (envelope-from git) Date: Tue, 6 Jul 2021 17:53:24 GMT Message-Id: <202107061753.166HrO1O052701@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: f98545c7e991 - stable/13 - libnv: fix memory leaks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f98545c7e99195d0f6e4be2068331421c853be17 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 17:53:25 -0000 The branch stable/13 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=f98545c7e99195d0f6e4be2068331421c853be17 commit f98545c7e99195d0f6e4be2068331421c853be17 Author: Mariusz Zaborski AuthorDate: 2019-02-10 23:28:55 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:53:07 +0000 libnv: fix memory leaks nvpair_create_stringv: free the temporary string; this fix affects nvlist_add_stringf() and nvlist_add_stringv(). nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain of nvpairs (as resetting it prevents nvlist_destroy() from freeing it). Note: freeing the chain in nvlist_destroy() is not sufficient, because it would still leak through nvlist_take_nvlist_array(). This affects all nvlist_*_nvlist_array() use Submitted by: Mindaugas Rasiukevicius Reported by: clang/gcc ASAN MFC after: 2 weeks (cherry picked from commit b5d787d93b3d83f28e87e1f8cc740cb160f8f0ac) --- sys/contrib/libnv/bsd_nvpair.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/sys/contrib/libnv/bsd_nvpair.c b/sys/contrib/libnv/bsd_nvpair.c index 6405dcd1c516..556203f740fc 100644 --- a/sys/contrib/libnv/bsd_nvpair.c +++ b/sys/contrib/libnv/bsd_nvpair.c @@ -229,8 +229,16 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_set_array_next(nvlarray[i], NULL); - nvlist_set_parent(nvlarray[i], NULL); + nvlist_t *nvl; + nvpair_t *nnvp; + + nvl = nvlarray[i]; + nnvp = nvlist_get_array_next_nvpair(nvl); + if (nnvp != NULL) { + nvpair_free_structure(nnvp); + } + nvlist_set_array_next(nvl, NULL); + nvlist_set_parent(nvl, NULL); } } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 17:53:26 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49C3E664DB3; Tue, 6 Jul 2021 17:53:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9CB0vJwz4YNy; Tue, 6 Jul 2021 17:53:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0177A1E408; Tue, 6 Jul 2021 17:53:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166HrP0D052726; Tue, 6 Jul 2021 17:53:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166HrPl9052725; Tue, 6 Jul 2021 17:53:25 GMT (envelope-from git) Date: Tue, 6 Jul 2021 17:53:25 GMT Message-Id: <202107061753.166HrPl9052725@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: 4c79391a7fa4 - stable/13 - libnv: fix double free MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4c79391a7fa47c22429d0f9c8877a27065794cf1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 17:53:26 -0000 The branch stable/13 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=4c79391a7fa47c22429d0f9c8877a27065794cf1 commit 4c79391a7fa47c22429d0f9c8877a27065794cf1 Author: Mariusz Zaborski AuthorDate: 2019-02-17 18:26:27 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:53:08 +0000 libnv: fix double free In r343986 we introduced a double free. The structure was already freed fixed in the r302966. This problem was introduced because the GitHub version was out of sync with the FreeBSD one. Submitted by: Mindaugas Rasiukevicius MFC with: r343986 (cherry picked from commit d97753b5c8f1d32fbcdcbb0d129b49f808245865) --- sys/contrib/libnv/bsd_nvpair.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/sys/contrib/libnv/bsd_nvpair.c b/sys/contrib/libnv/bsd_nvpair.c index 556203f740fc..34ea8c6bc1d5 100644 --- a/sys/contrib/libnv/bsd_nvpair.c +++ b/sys/contrib/libnv/bsd_nvpair.c @@ -229,14 +229,6 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_t *nvl; - nvpair_t *nnvp; - - nvl = nvlarray[i]; - nnvp = nvlist_get_array_next_nvpair(nvl); - if (nnvp != NULL) { - nvpair_free_structure(nnvp); - } nvlist_set_array_next(nvl, NULL); nvlist_set_parent(nvl, NULL); } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 17:53:27 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DB83F664771; Tue, 6 Jul 2021 17:53:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9CC5VLvz4YrS; Tue, 6 Jul 2021 17:53:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 286281E306; Tue, 6 Jul 2021 17:53:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166HrRu9052750; Tue, 6 Jul 2021 17:53:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166HrRUc052749; Tue, 6 Jul 2021 17:53:27 GMT (envelope-from git) Date: Tue, 6 Jul 2021 17:53:27 GMT Message-Id: <202107061753.166HrRUc052749@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: b907e9d9a149 - stable/13 - libnv: fix revert MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: b907e9d9a1492711e5e342b2bc33b30670a7222d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 17:53:28 -0000 The branch stable/13 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=b907e9d9a1492711e5e342b2bc33b30670a7222d commit b907e9d9a1492711e5e342b2bc33b30670a7222d Author: Mariusz Zaborski AuthorDate: 2019-02-17 18:32:19 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:53:08 +0000 libnv: fix revert Reported by: jenkins (cherry picked from commit 3bea7b5b05f200df4cabee12e405b8feade16f0e) --- sys/contrib/libnv/bsd_nvpair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/contrib/libnv/bsd_nvpair.c b/sys/contrib/libnv/bsd_nvpair.c index 34ea8c6bc1d5..6405dcd1c516 100644 --- a/sys/contrib/libnv/bsd_nvpair.c +++ b/sys/contrib/libnv/bsd_nvpair.c @@ -229,8 +229,8 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_set_array_next(nvl, NULL); - nvlist_set_parent(nvl, NULL); + nvlist_set_array_next(nvlarray[i], NULL); + nvlist_set_parent(nvlarray[i], NULL); } } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 17:53:28 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 879DC664F1B; Tue, 6 Jul 2021 17:53:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9CD3H77z4YX6; Tue, 6 Jul 2021 17:53:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4F6D41E307; Tue, 6 Jul 2021 17:53:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166HrS0m052778; Tue, 6 Jul 2021 17:53:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166HrSwV052777; Tue, 6 Jul 2021 17:53:28 GMT (envelope-from git) Date: Tue, 6 Jul 2021 17:53:28 GMT Message-Id: <202107061753.166HrSwV052777@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: ff339365be13 - stable/13 - libnv: optimize nvlist size calculation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ff339365be130f155f94f9a5d8daf69b695cfc55 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 17:53:28 -0000 The branch stable/13 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=ff339365be130f155f94f9a5d8daf69b695cfc55 commit ff339365be130f155f94f9a5d8daf69b695cfc55 Author: Mariusz Zaborski AuthorDate: 2021-06-11 15:35:36 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:53:08 +0000 libnv: optimize nvlist size calculation If we had a multiple nvlist, during nvlist_pack, we calculated the size of every nvlist separately. For example, if we had a nvlist with three nodes each containing another (A contains B, and B contains C), we first calculated the size of nvlist A (which contains B, C), then we calculate the size of B (which contains C, notice that we already did the calculation of B, when we calculate A), and finally C. This means that this calculation was O(N!). This was done because each time we pack nvlist, we have to put its size in the header (the separate header for A, B, and C). To not break the ABI and to reduce the complexity of nvlist_size, instead of calculating the nvlist size when requested, we track the size of each nvlist. Reported by: pjd, kp Tested by: kp (cherry picked from commit 89d5cbb82294c8624e66f920d50353057ccab14b) --- sys/contrib/libnv/nvlist.c | 124 +++++++++++++++++++-------------------------- 1 file changed, 53 insertions(+), 71 deletions(-) diff --git a/sys/contrib/libnv/nvlist.c b/sys/contrib/libnv/nvlist.c index 311325d822ce..31ab62abeb67 100644 --- a/sys/contrib/libnv/nvlist.c +++ b/sys/contrib/libnv/nvlist.c @@ -99,6 +99,7 @@ struct nvlist { int nvl_magic; int nvl_error; int nvl_flags; + size_t nvl_datasize; nvpair_t *nvl_parent; nvpair_t *nvl_array_next; struct nvl_head nvl_head; @@ -139,6 +140,7 @@ nvlist_create(int flags) nvl->nvl_flags = flags; nvl->nvl_parent = NULL; nvl->nvl_array_next = NULL; + nvl->nvl_datasize = sizeof(struct nvlist_header); TAILQ_INIT(&nvl->nvl_head); nvl->nvl_magic = NVLIST_MAGIC; @@ -247,6 +249,51 @@ nvlist_set_array_next(nvlist_t *nvl, nvpair_t *ele) nvl->nvl_array_next = ele; } +static void +nvlist_update_size(nvlist_t *nvl, nvpair_t *new, ssize_t mul) +{ + ssize_t size; + size_t nitems; + const nvlist_t *nvlistnew; + const nvlist_t * const *nvlarray; + nvlist_t *parent; + unsigned int ii; + + NVLIST_ASSERT(nvl); + NVPAIR_ASSERT(new); + PJDLOG_ASSERT(mul == 1 || mul == -1); + + size = nvpair_header_size(); + size += strlen(nvpair_name(new)) + 1; + + if (nvpair_type(new) == NV_TYPE_NVLIST) { + nvlistnew = nvpair_get_nvlist(new); + size += nvlistnew->nvl_datasize; + size += nvpair_header_size() + 1; + } else if (nvpair_type(new) == NV_TYPE_NVLIST_ARRAY) { + nvlarray = nvpair_get_nvlist_array(new, &nitems); + PJDLOG_ASSERT(nitems > 0); + + size += (nvpair_header_size() + 1) * nitems; + for (ii = 0; ii < nitems; ii++) { + PJDLOG_ASSERT(nvlarray[ii]->nvl_error == 0); + size += nvlarray[ii]->nvl_datasize; + } + } else { + size += nvpair_size(new); + } + + size *= mul; + + nvl->nvl_datasize += size; + + parent = nvl; + while ((parent = __DECONST(nvlist_t *, + nvlist_get_parent(parent, NULL))) != NULL) { + parent->nvl_datasize += size; + } +} + nvpair_t * nvlist_get_array_next_nvpair(nvlist_t *nvl) { @@ -640,78 +687,8 @@ nvlist_fdump(const nvlist_t *nvl, FILE *fp) size_t nvlist_size(const nvlist_t *nvl) { - const nvlist_t *tmpnvl; - const nvlist_t * const *nvlarray; - const nvpair_t *nvp, *tmpnvp; - void *cookie; - size_t size, nitems; - unsigned int ii; - - NVLIST_ASSERT(nvl); - PJDLOG_ASSERT(nvl->nvl_error == 0); - size = sizeof(struct nvlist_header); - nvp = nvlist_first_nvpair(nvl); - while (nvp != NULL) { - size += nvpair_header_size(); - size += strlen(nvpair_name(nvp)) + 1; - if (nvpair_type(nvp) == NV_TYPE_NVLIST) { - size += sizeof(struct nvlist_header); - size += nvpair_header_size() + 1; - tmpnvl = nvpair_get_nvlist(nvp); - PJDLOG_ASSERT(tmpnvl->nvl_error == 0); - tmpnvp = nvlist_first_nvpair(tmpnvl); - if (tmpnvp != NULL) { - nvl = tmpnvl; - nvp = tmpnvp; - continue; - } - } else if (nvpair_type(nvp) == NV_TYPE_NVLIST_ARRAY) { - nvlarray = nvpair_get_nvlist_array(nvp, &nitems); - PJDLOG_ASSERT(nitems > 0); - - size += (nvpair_header_size() + 1) * nitems; - size += sizeof(struct nvlist_header) * nitems; - - tmpnvl = NULL; - tmpnvp = NULL; - for (ii = 0; ii < nitems; ii++) { - PJDLOG_ASSERT(nvlarray[ii]->nvl_error == 0); - tmpnvp = nvlist_first_nvpair(nvlarray[ii]); - if (tmpnvp != NULL) { - tmpnvl = nvlarray[ii]; - break; - } - } - if (tmpnvp != NULL) { - nvp = tmpnvp; - nvl = tmpnvl; - continue; - } - - } else { - size += nvpair_size(nvp); - } - - while ((nvp = nvlist_next_nvpair(nvl, nvp)) == NULL) { - do { - cookie = NULL; - nvl = nvlist_get_pararr(nvl, &cookie); - if (nvl == NULL) - goto out; - if (nvlist_in_array(nvl) && cookie == NULL) { - nvp = nvlist_first_nvpair(nvl); - } else { - nvp = cookie; - } - } while (nvp == NULL); - if (nvlist_in_array(nvl) && cookie == NULL) - break; - } - } - -out: - return (size); + return (nvl->nvl_datasize); } #ifndef _KERNEL @@ -1483,6 +1460,7 @@ nvlist_add_nvpair(nvlist_t *nvl, const nvpair_t *nvp) } nvpair_insert(&nvl->nvl_head, newnvp, nvl); + nvlist_update_size(nvl, newnvp, 1); } void @@ -1631,10 +1609,12 @@ nvlist_append_##type##_array(nvlist_t *nvl, const char *name, vtype value)\ nvlist_add_##type##_array(nvl, name, &value, 1); \ return; \ } \ + nvlist_update_size(nvl, nvp, -1); \ if (nvpair_append_##type##_array(nvp, value) == -1) { \ nvl->nvl_error = ERRNO_OR_DEFAULT(ENOMEM); \ ERRNO_SET(nvl->nvl_error); \ } \ + nvlist_update_size(nvl, nvp, 1); \ } NVLIST_APPEND_ARRAY(const bool, bool, BOOL) @@ -1669,6 +1649,7 @@ nvlist_move_nvpair(nvlist_t *nvl, nvpair_t *nvp) } nvpair_insert(&nvl->nvl_head, nvp, nvl); + nvlist_update_size(nvl, nvp, 1); return (true); } @@ -2020,6 +2001,7 @@ nvlist_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) PJDLOG_ASSERT(nvpair_nvlist(nvp) == nvl); nvpair_remove(&nvl->nvl_head, nvp, nvl); + nvlist_update_size(nvl, nvp, -1); } void From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 18:25:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 20E5E6651EB; Tue, 6 Jul 2021 18:25:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9wg0PQKz4ccK; Tue, 6 Jul 2021 18:25:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E943F1E7D4; Tue, 6 Jul 2021 18:25:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166IPsjn093214; Tue, 6 Jul 2021 18:25:54 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166IPsi5093213; Tue, 6 Jul 2021 18:25:54 GMT (envelope-from git) Date: Tue, 6 Jul 2021 18:25:54 GMT Message-Id: <202107061825.166IPsi5093213@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: dc876a5bcd46 - stable/12 - libnv: fix memory leaks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: dc876a5bcd46b486103c9127f51e01663504195b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 18:25:55 -0000 The branch stable/12 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=dc876a5bcd46b486103c9127f51e01663504195b commit dc876a5bcd46b486103c9127f51e01663504195b Author: Mariusz Zaborski AuthorDate: 2019-02-10 23:28:55 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:58:08 +0000 libnv: fix memory leaks nvpair_create_stringv: free the temporary string; this fix affects nvlist_add_stringf() and nvlist_add_stringv(). nvpair_remove_nvlist_array (NV_TYPE_NVLIST_ARRAY case): free the chain of nvpairs (as resetting it prevents nvlist_destroy() from freeing it). Note: freeing the chain in nvlist_destroy() is not sufficient, because it would still leak through nvlist_take_nvlist_array(). This affects all nvlist_*_nvlist_array() use Submitted by: Mindaugas Rasiukevicius Reported by: clang/gcc ASAN MFC after: 2 weeks (cherry picked from commit b5d787d93b3d83f28e87e1f8cc740cb160f8f0ac) --- lib/libnv/tests/nvlist_send_recv_test.c | 2 ++ sys/contrib/libnv/nv_impl.h | 1 + sys/contrib/libnv/nvlist.c | 9 +++++++++ sys/contrib/libnv/nvpair.c | 15 +++++++++++---- 4 files changed, 23 insertions(+), 4 deletions(-) diff --git a/lib/libnv/tests/nvlist_send_recv_test.c b/lib/libnv/tests/nvlist_send_recv_test.c index 1b673b5d4259..fbc918102b50 100644 --- a/lib/libnv/tests/nvlist_send_recv_test.c +++ b/lib/libnv/tests/nvlist_send_recv_test.c @@ -304,6 +304,8 @@ parent(int sock) name = nvlist_next(nvl, &type, &cookie); CHECK(name == NULL); + + nvlist_destroy(nvl); } static void diff --git a/sys/contrib/libnv/nv_impl.h b/sys/contrib/libnv/nv_impl.h index a67cc2ca0883..1875c739beee 100644 --- a/sys/contrib/libnv/nv_impl.h +++ b/sys/contrib/libnv/nv_impl.h @@ -103,6 +103,7 @@ bool nvlist_move_nvpair(nvlist_t *nvl, nvpair_t *nvp); void nvlist_set_parent(nvlist_t *nvl, nvpair_t *parent); void nvlist_set_array_next(nvlist_t *nvl, nvpair_t *ele); +nvpair_t *nvlist_get_array_next_nvpair(nvlist_t *nvl); const nvpair_t *nvlist_get_nvpair(const nvlist_t *nvl, const char *name); diff --git a/sys/contrib/libnv/nvlist.c b/sys/contrib/libnv/nvlist.c index 69b32a62a4cc..311325d822ce 100644 --- a/sys/contrib/libnv/nvlist.c +++ b/sys/contrib/libnv/nvlist.c @@ -247,6 +247,15 @@ nvlist_set_array_next(nvlist_t *nvl, nvpair_t *ele) nvl->nvl_array_next = ele; } +nvpair_t * +nvlist_get_array_next_nvpair(nvlist_t *nvl) +{ + + NVLIST_ASSERT(nvl); + + return (nvl->nvl_array_next); +} + bool nvlist_in_array(const nvlist_t *nvl) { diff --git a/sys/contrib/libnv/nvpair.c b/sys/contrib/libnv/nvpair.c index ed01da2c5040..b767b9bbf972 100644 --- a/sys/contrib/libnv/nvpair.c +++ b/sys/contrib/libnv/nvpair.c @@ -229,8 +229,16 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_set_array_next(nvlarray[i], NULL); - nvlist_set_parent(nvlarray[i], NULL); + nvlist_t *nvl; + nvpair_t *nnvp; + + nvl = nvlarray[i]; + nnvp = nvlist_get_array_next_nvpair(nvl); + if (nnvp != NULL) { + nvpair_free_structure(nnvp); + } + nvlist_set_array_next(nvl, NULL); + nvlist_set_parent(nvl, NULL); } } @@ -1194,8 +1202,7 @@ nvpair_create_stringv(const char *name, const char *valuefmt, va_list valueap) if (len < 0) return (NULL); nvp = nvpair_create_string(name, str); - if (nvp == NULL) - nv_free(str); + nv_free(str); return (nvp); } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 18:25:56 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 44CC26651EC; Tue, 6 Jul 2021 18:25:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9wh1P5Dz4chk; Tue, 6 Jul 2021 18:25:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 17B061E672; Tue, 6 Jul 2021 18:25:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166IPuta093239; Tue, 6 Jul 2021 18:25:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166IPtT0093238; Tue, 6 Jul 2021 18:25:55 GMT (envelope-from git) Date: Tue, 6 Jul 2021 18:25:55 GMT Message-Id: <202107061825.166IPtT0093238@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: bc2bc8ee6e18 - stable/12 - libnv: fix memory leaks MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: bc2bc8ee6e1814361125c189591917be0fa5e4c1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 18:25:56 -0000 The branch stable/12 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=bc2bc8ee6e1814361125c189591917be0fa5e4c1 commit bc2bc8ee6e1814361125c189591917be0fa5e4c1 Author: Mariusz Zaborski AuthorDate: 2019-02-10 23:30:54 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:58:08 +0000 libnv: fix memory leaks Free the data array for NV_TYPE_DESCRIPTOR_ARRAY case. MFC after: 2 weeks (cherry picked from commit 0020c845a086766b3315372f006363f8ad76ac54) --- sys/contrib/libnv/nvpair.c | 1 + 1 file changed, 1 insertion(+) diff --git a/sys/contrib/libnv/nvpair.c b/sys/contrib/libnv/nvpair.c index b767b9bbf972..556203f740fc 100644 --- a/sys/contrib/libnv/nvpair.c +++ b/sys/contrib/libnv/nvpair.c @@ -2062,6 +2062,7 @@ nvpair_free(nvpair_t *nvp) case NV_TYPE_DESCRIPTOR_ARRAY: for (i = 0; i < nvp->nvp_nitems; i++) close(((int *)(intptr_t)nvp->nvp_data)[i]); + nv_free((int *)(intptr_t)nvp->nvp_data); break; #endif case NV_TYPE_NVLIST: From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 18:25:57 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E346F6658A5; Tue, 6 Jul 2021 18:25:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9wj2y2kz4cRY; Tue, 6 Jul 2021 18:25:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 43A501E999; Tue, 6 Jul 2021 18:25:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166IPvtl093263; Tue, 6 Jul 2021 18:25:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166IPv9H093262; Tue, 6 Jul 2021 18:25:57 GMT (envelope-from git) Date: Tue, 6 Jul 2021 18:25:57 GMT Message-Id: <202107061825.166IPv9H093262@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: 6c33105c1a81 - stable/12 - libnv: fix double free MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 6c33105c1a817e4cc3d6e59998fb53e8d9bd5748 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 18:25:58 -0000 The branch stable/12 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=6c33105c1a817e4cc3d6e59998fb53e8d9bd5748 commit 6c33105c1a817e4cc3d6e59998fb53e8d9bd5748 Author: Mariusz Zaborski AuthorDate: 2019-02-17 18:26:27 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:58:09 +0000 libnv: fix double free In r343986 we introduced a double free. The structure was already freed fixed in the r302966. This problem was introduced because the GitHub version was out of sync with the FreeBSD one. Submitted by: Mindaugas Rasiukevicius MFC with: r343986 (cherry picked from commit d97753b5c8f1d32fbcdcbb0d129b49f808245865) --- sys/contrib/libnv/nvpair.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/sys/contrib/libnv/nvpair.c b/sys/contrib/libnv/nvpair.c index 556203f740fc..34ea8c6bc1d5 100644 --- a/sys/contrib/libnv/nvpair.c +++ b/sys/contrib/libnv/nvpair.c @@ -229,14 +229,6 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_t *nvl; - nvpair_t *nnvp; - - nvl = nvlarray[i]; - nnvp = nvlist_get_array_next_nvpair(nvl); - if (nnvp != NULL) { - nvpair_free_structure(nnvp); - } nvlist_set_array_next(nvl, NULL); nvlist_set_parent(nvl, NULL); } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 18:25:58 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E93A066562D; Tue, 6 Jul 2021 18:25:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9wk3jFBz4cTg; Tue, 6 Jul 2021 18:25:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5D9CB1E97F; Tue, 6 Jul 2021 18:25:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166IPwrY093287; Tue, 6 Jul 2021 18:25:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166IPwZk093286; Tue, 6 Jul 2021 18:25:58 GMT (envelope-from git) Date: Tue, 6 Jul 2021 18:25:58 GMT Message-Id: <202107061825.166IPwZk093286@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: ddab4ee61002 - stable/12 - libnv: fix revert MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: ddab4ee6100216900d6bbc9f3da4d98fde844812 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 18:25:59 -0000 The branch stable/12 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=ddab4ee6100216900d6bbc9f3da4d98fde844812 commit ddab4ee6100216900d6bbc9f3da4d98fde844812 Author: Mariusz Zaborski AuthorDate: 2019-02-17 18:32:19 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:58:09 +0000 libnv: fix revert Reported by: jenkins (cherry picked from commit 3bea7b5b05f200df4cabee12e405b8feade16f0e) --- sys/contrib/libnv/nvpair.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/contrib/libnv/nvpair.c b/sys/contrib/libnv/nvpair.c index 34ea8c6bc1d5..6405dcd1c516 100644 --- a/sys/contrib/libnv/nvpair.c +++ b/sys/contrib/libnv/nvpair.c @@ -229,8 +229,8 @@ nvpair_remove_nvlist_array(nvpair_t *nvp) nvlarray = __DECONST(nvlist_t **, nvpair_get_nvlist_array(nvp, &count)); for (i = 0; i < count; i++) { - nvlist_set_array_next(nvl, NULL); - nvlist_set_parent(nvl, NULL); + nvlist_set_array_next(nvlarray[i], NULL); + nvlist_set_parent(nvlarray[i], NULL); } } From owner-dev-commits-src-branches@freebsd.org Tue Jul 6 18:26:00 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7663D665918; Tue, 6 Jul 2021 18:26:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GK9wm0SFwz4clY; Tue, 6 Jul 2021 18:25:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 81BB31E99A; Tue, 6 Jul 2021 18:25:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 166IPxMj093311; Tue, 6 Jul 2021 18:25:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 166IPxAs093310; Tue, 6 Jul 2021 18:25:59 GMT (envelope-from git) Date: Tue, 6 Jul 2021 18:25:59 GMT Message-Id: <202107061825.166IPxAs093310@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mariusz Zaborski Subject: git: eed85dd1ad31 - stable/12 - libnv: optimize nvlist size calculation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: oshogbo X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: eed85dd1ad31fc83aa3bc1bd8b4c9249616c8b78 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2021 18:26:00 -0000 The branch stable/12 has been updated by oshogbo: URL: https://cgit.FreeBSD.org/src/commit/?id=eed85dd1ad31fc83aa3bc1bd8b4c9249616c8b78 commit eed85dd1ad31fc83aa3bc1bd8b4c9249616c8b78 Author: Mariusz Zaborski AuthorDate: 2021-06-11 15:35:36 +0000 Commit: Mariusz Zaborski CommitDate: 2021-07-06 17:58:09 +0000 libnv: optimize nvlist size calculation If we had a multiple nvlist, during nvlist_pack, we calculated the size of every nvlist separately. For example, if we had a nvlist with three nodes each containing another (A contains B, and B contains C), we first calculated the size of nvlist A (which contains B, C), then we calculate the size of B (which contains C, notice that we already did the calculation of B, when we calculate A), and finally C. This means that this calculation was O(N!). This was done because each time we pack nvlist, we have to put its size in the header (the separate header for A, B, and C). To not break the ABI and to reduce the complexity of nvlist_size, instead of calculating the nvlist size when requested, we track the size of each nvlist. Reported by: pjd, kp Tested by: kp (cherry picked from commit 89d5cbb82294c8624e66f920d50353057ccab14b) --- sys/contrib/libnv/nvlist.c | 124 +++++++++++++++++++-------------------------- 1 file changed, 53 insertions(+), 71 deletions(-) diff --git a/sys/contrib/libnv/nvlist.c b/sys/contrib/libnv/nvlist.c index 311325d822ce..31ab62abeb67 100644 --- a/sys/contrib/libnv/nvlist.c +++ b/sys/contrib/libnv/nvlist.c @@ -99,6 +99,7 @@ struct nvlist { int nvl_magic; int nvl_error; int nvl_flags; + size_t nvl_datasize; nvpair_t *nvl_parent; nvpair_t *nvl_array_next; struct nvl_head nvl_head; @@ -139,6 +140,7 @@ nvlist_create(int flags) nvl->nvl_flags = flags; nvl->nvl_parent = NULL; nvl->nvl_array_next = NULL; + nvl->nvl_datasize = sizeof(struct nvlist_header); TAILQ_INIT(&nvl->nvl_head); nvl->nvl_magic = NVLIST_MAGIC; @@ -247,6 +249,51 @@ nvlist_set_array_next(nvlist_t *nvl, nvpair_t *ele) nvl->nvl_array_next = ele; } +static void +nvlist_update_size(nvlist_t *nvl, nvpair_t *new, ssize_t mul) +{ + ssize_t size; + size_t nitems; + const nvlist_t *nvlistnew; + const nvlist_t * const *nvlarray; + nvlist_t *parent; + unsigned int ii; + + NVLIST_ASSERT(nvl); + NVPAIR_ASSERT(new); + PJDLOG_ASSERT(mul == 1 || mul == -1); + + size = nvpair_header_size(); + size += strlen(nvpair_name(new)) + 1; + + if (nvpair_type(new) == NV_TYPE_NVLIST) { + nvlistnew = nvpair_get_nvlist(new); + size += nvlistnew->nvl_datasize; + size += nvpair_header_size() + 1; + } else if (nvpair_type(new) == NV_TYPE_NVLIST_ARRAY) { + nvlarray = nvpair_get_nvlist_array(new, &nitems); + PJDLOG_ASSERT(nitems > 0); + + size += (nvpair_header_size() + 1) * nitems; + for (ii = 0; ii < nitems; ii++) { + PJDLOG_ASSERT(nvlarray[ii]->nvl_error == 0); + size += nvlarray[ii]->nvl_datasize; + } + } else { + size += nvpair_size(new); + } + + size *= mul; + + nvl->nvl_datasize += size; + + parent = nvl; + while ((parent = __DECONST(nvlist_t *, + nvlist_get_parent(parent, NULL))) != NULL) { + parent->nvl_datasize += size; + } +} + nvpair_t * nvlist_get_array_next_nvpair(nvlist_t *nvl) { @@ -640,78 +687,8 @@ nvlist_fdump(const nvlist_t *nvl, FILE *fp) size_t nvlist_size(const nvlist_t *nvl) { - const nvlist_t *tmpnvl; - const nvlist_t * const *nvlarray; - const nvpair_t *nvp, *tmpnvp; - void *cookie; - size_t size, nitems; - unsigned int ii; - - NVLIST_ASSERT(nvl); - PJDLOG_ASSERT(nvl->nvl_error == 0); - size = sizeof(struct nvlist_header); - nvp = nvlist_first_nvpair(nvl); - while (nvp != NULL) { - size += nvpair_header_size(); - size += strlen(nvpair_name(nvp)) + 1; - if (nvpair_type(nvp) == NV_TYPE_NVLIST) { - size += sizeof(struct nvlist_header); - size += nvpair_header_size() + 1; - tmpnvl = nvpair_get_nvlist(nvp); - PJDLOG_ASSERT(tmpnvl->nvl_error == 0); - tmpnvp = nvlist_first_nvpair(tmpnvl); - if (tmpnvp != NULL) { - nvl = tmpnvl; - nvp = tmpnvp; - continue; - } - } else if (nvpair_type(nvp) == NV_TYPE_NVLIST_ARRAY) { - nvlarray = nvpair_get_nvlist_array(nvp, &nitems); - PJDLOG_ASSERT(nitems > 0); - - size += (nvpair_header_size() + 1) * nitems; - size += sizeof(struct nvlist_header) * nitems; - - tmpnvl = NULL; - tmpnvp = NULL; - for (ii = 0; ii < nitems; ii++) { - PJDLOG_ASSERT(nvlarray[ii]->nvl_error == 0); - tmpnvp = nvlist_first_nvpair(nvlarray[ii]); - if (tmpnvp != NULL) { - tmpnvl = nvlarray[ii]; - break; - } - } - if (tmpnvp != NULL) { - nvp = tmpnvp; - nvl = tmpnvl; - continue; - } - - } else { - size += nvpair_size(nvp); - } - - while ((nvp = nvlist_next_nvpair(nvl, nvp)) == NULL) { - do { - cookie = NULL; - nvl = nvlist_get_pararr(nvl, &cookie); - if (nvl == NULL) - goto out; - if (nvlist_in_array(nvl) && cookie == NULL) { - nvp = nvlist_first_nvpair(nvl); - } else { - nvp = cookie; - } - } while (nvp == NULL); - if (nvlist_in_array(nvl) && cookie == NULL) - break; - } - } - -out: - return (size); + return (nvl->nvl_datasize); } #ifndef _KERNEL @@ -1483,6 +1460,7 @@ nvlist_add_nvpair(nvlist_t *nvl, const nvpair_t *nvp) } nvpair_insert(&nvl->nvl_head, newnvp, nvl); + nvlist_update_size(nvl, newnvp, 1); } void @@ -1631,10 +1609,12 @@ nvlist_append_##type##_array(nvlist_t *nvl, const char *name, vtype value)\ nvlist_add_##type##_array(nvl, name, &value, 1); \ return; \ } \ + nvlist_update_size(nvl, nvp, -1); \ if (nvpair_append_##type##_array(nvp, value) == -1) { \ nvl->nvl_error = ERRNO_OR_DEFAULT(ENOMEM); \ ERRNO_SET(nvl->nvl_error); \ } \ + nvlist_update_size(nvl, nvp, 1); \ } NVLIST_APPEND_ARRAY(const bool, bool, BOOL) @@ -1669,6 +1649,7 @@ nvlist_move_nvpair(nvlist_t *nvl, nvpair_t *nvp) } nvpair_insert(&nvl->nvl_head, nvp, nvl); + nvlist_update_size(nvl, nvp, 1); return (true); } @@ -2020,6 +2001,7 @@ nvlist_remove_nvpair(nvlist_t *nvl, nvpair_t *nvp) PJDLOG_ASSERT(nvpair_nvlist(nvp) == nvl); nvpair_remove(&nvl->nvl_head, nvp, nvl); + nvlist_update_size(nvl, nvp, -1); } void From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:10:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4CE9A653E89; Wed, 7 Jul 2021 11:10:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDH1m6rz4fXc; Wed, 7 Jul 2021 11:10:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 24FB043D8; Wed, 7 Jul 2021 11:10:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BAtAV040780; Wed, 7 Jul 2021 11:10:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BAt9r040779; Wed, 7 Jul 2021 11:10:55 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:10:55 GMT Message-Id: <202107071110.167BAt9r040779@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 1fc1d45374d5 - stable/13 - seqc: add symbolic name for in-modify bit MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 1fc1d45374d58bb24c93f6eed66e1503d10cd4ee Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:10:55 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=1fc1d45374d58bb24c93f6eed66e1503d10cd4ee commit 1fc1d45374d58bb24c93f6eed66e1503d10cd4ee Author: Konstantin Belousov AuthorDate: 2021-04-29 22:39:23 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:20 +0000 seqc: add symbolic name for in-modify bit (cherry picked from commit 71faea93070e6df371bf73ed12e2a1da27dc8e2d) --- sys/sys/seqc.h | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/sys/sys/seqc.h b/sys/sys/seqc.h index 3cfaaf3acbda..fd9f196aceca 100644 --- a/sys/sys/seqc.h +++ b/sys/sys/seqc.h @@ -45,13 +45,15 @@ #include +#define SEQC_MOD 1 + /* * Predicts from inline functions are not honored by clang. */ #define seqc_in_modify(seqc) ({ \ seqc_t __seqc = (seqc); \ \ - __predict_false(__seqc & 1); \ + __predict_false(__seqc & SEQC_MOD); \ }) static __inline void @@ -60,7 +62,7 @@ seqc_write_begin(seqc_t *seqcp) critical_enter(); MPASS(!seqc_in_modify(*seqcp)); - *seqcp += 1; + *seqcp += SEQC_MOD; atomic_thread_fence_rel(); } @@ -69,7 +71,7 @@ seqc_write_end(seqc_t *seqcp) { atomic_thread_fence_rel(); - *seqcp += 1; + *seqcp += SEQC_MOD; MPASS(!seqc_in_modify(*seqcp)); critical_exit(); } @@ -85,7 +87,7 @@ static __inline seqc_t seqc_read_notmodify(const seqc_t *seqcp) { - return (atomic_load_acq_int(__DECONST(seqc_t *, seqcp)) & ~1); + return (atomic_load_acq_int(__DECONST(seqc_t *, seqcp)) & ~SEQC_MOD); } static __inline seqc_t @@ -126,7 +128,7 @@ seqc_sleepable_write_begin(seqc_t *seqcp) { MPASS(!seqc_in_modify(*seqcp)); - *seqcp += 1; + *seqcp += SEQC_MOD; atomic_thread_fence_rel(); } @@ -135,7 +137,7 @@ seqc_sleepable_write_end(seqc_t *seqcp) { atomic_thread_fence_rel(); - *seqcp += 1; + *seqcp += SEQC_MOD; MPASS(!seqc_in_modify(*seqcp)); } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:10:56 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D04E653A4E; Wed, 7 Jul 2021 11:10:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDJ2t5Yz4fmV; Wed, 7 Jul 2021 11:10:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4753345BF; Wed, 7 Jul 2021 11:10:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BAueW040804; Wed, 7 Jul 2021 11:10:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BAuL8040803; Wed, 7 Jul 2021 11:10:56 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:10:56 GMT Message-Id: <202107071110.167BAuL8040803@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 893aecc35acc - stable/13 - namei: add NDPREINIT() macro MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 893aecc35acc603ba9ca00addefd6c6ead9f403c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:10:56 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=893aecc35acc603ba9ca00addefd6c6ead9f403c commit 893aecc35acc603ba9ca00addefd6c6ead9f403c Author: Konstantin Belousov AuthorDate: 2021-06-14 19:03:41 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:25 +0000 namei: add NDPREINIT() macro (cherry picked from commit 802cf4ab0ee9cae7af523cfcf1221febd2143210) --- sys/kern/vfs_syscalls.c | 9 +++++++++ sys/sys/namei.h | 3 +++ 2 files changed, 12 insertions(+) diff --git a/sys/kern/vfs_syscalls.c b/sys/kern/vfs_syscalls.c index c18bb56fc632..7f2f3480a5af 100644 --- a/sys/kern/vfs_syscalls.c +++ b/sys/kern/vfs_syscalls.c @@ -1329,6 +1329,7 @@ kern_mknodat(struct thread *td, int fd, const char *path, enum uio_seg pathseg, } if (error != 0) return (error); + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | @@ -1437,6 +1438,7 @@ kern_mkfifoat(struct thread *td, int fd, const char *path, int error; AUDIT_ARG_MODE(mode); + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | @@ -1568,6 +1570,7 @@ kern_linkat(struct thread *td, int fd1, int fd2, const char *path1, struct nameidata nd; int error; + NDPREINIT(&nd); do { bwillwrite(); NDINIT_ATRIGHTS(&nd, LOOKUP, follow | AUDITVNODE1, segflag, @@ -1715,6 +1718,7 @@ kern_symlinkat(struct thread *td, const char *path1, int fd, const char *path2, syspath = tmppath; } AUDIT_ARG_TEXT(syspath); + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | @@ -1779,6 +1783,7 @@ sys_undelete(struct thread *td, struct undelete_args *uap) struct nameidata nd; int error; + NDPREINIT(&nd); restart: bwillwrite(); NDINIT(&nd, DELETE, LOCKPARENT | DOWHITEOUT | AUDITVNODE1, @@ -1892,6 +1897,7 @@ kern_funlinkat(struct thread *td, int dfd, const char *path, int fd, return (error); } + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, DELETE, LOCKPARENT | LOCKLEAF | AUDITVNODE1 | @@ -3423,6 +3429,7 @@ kern_truncate(struct thread *td, const char *path, enum uio_seg pathseg, if (length < 0) return (EINVAL); + NDPREINIT(&nd); retry: NDINIT(&nd, LOOKUP, FOLLOW | AUDITVNODE1, pathseg, path, td); if ((error = namei(&nd)) != 0) @@ -3790,6 +3797,7 @@ kern_mkdirat(struct thread *td, int fd, const char *path, enum uio_seg segflg, int error; AUDIT_ARG_MODE(mode); + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, CREATE, LOCKPARENT | SAVENAME | AUDITVNODE1 | @@ -3860,6 +3868,7 @@ kern_frmdirat(struct thread *td, int dfd, const char *path, int fd, return (error); } + NDPREINIT(&nd); restart: bwillwrite(); NDINIT_ATRIGHTS(&nd, DELETE, LOCKPARENT | LOCKLEAF | AUDITVNODE1 | diff --git a/sys/sys/namei.h b/sys/sys/namei.h index 5f3d917083a5..b4db0e758e2b 100644 --- a/sys/sys/namei.h +++ b/sys/sys/namei.h @@ -265,6 +265,9 @@ do { \ _ndp->ni_startdir = NULL; \ } while (0) +#define NDPREINIT(ndp) do { \ +} while (0) + #define NDF_NO_DVP_RELE 0x00000001 #define NDF_NO_DVP_UNLOCK 0x00000002 #define NDF_NO_DVP_PUT 0x00000003 From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:10:57 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5983653D11; Wed, 7 Jul 2021 11:10:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDK4YYGz4fs0; Wed, 7 Jul 2021 11:10:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6C63345C0; Wed, 7 Jul 2021 11:10:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BAv4s040828; Wed, 7 Jul 2021 11:10:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BAvce040827; Wed, 7 Jul 2021 11:10:57 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:10:57 GMT Message-Id: <202107071110.167BAvce040827@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: eee344ccd142 - stable/13 - unionfs: do not use bare struct componentname MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: eee344ccd1420498eb7860999d115ea1de8c68e6 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:10:58 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=eee344ccd1420498eb7860999d115ea1de8c68e6 commit eee344ccd1420498eb7860999d115ea1de8c68e6 Author: Konstantin Belousov AuthorDate: 2021-06-14 18:45:23 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:30 +0000 unionfs: do not use bare struct componentname (cherry picked from commit 190110f2eba1551793f290a9f01e52ffe015a5da) --- sys/fs/unionfs/union_subr.c | 70 ++++++++++++++++++++++++--------------------- 1 file changed, 38 insertions(+), 32 deletions(-) diff --git a/sys/fs/unionfs/union_subr.c b/sys/fs/unionfs/union_subr.c index 30b637171021..93e4f50d98c5 100644 --- a/sys/fs/unionfs/union_subr.c +++ b/sys/fs/unionfs/union_subr.c @@ -760,7 +760,7 @@ unionfs_mkshadowdir(struct unionfs_mount *ump, struct vnode *udvp, struct vnode *uvp; struct vattr va; struct vattr lva; - struct componentname cn; + struct nameidata nd; struct mount *mp; struct ucred *cred; struct ucred *credbk; @@ -787,12 +787,14 @@ unionfs_mkshadowdir(struct unionfs_mount *ump, struct vnode *udvp, uifree(rootinfo); cnp->cn_cred = cred; - memset(&cn, 0, sizeof(cn)); + memset(&nd.ni_cnd, 0, sizeof(struct componentname)); + NDPREINIT(&nd); if ((error = VOP_GETATTR(lvp, &lva, cnp->cn_cred))) goto unionfs_mkshadowdir_abort; - if ((error = unionfs_relookup(udvp, &uvp, cnp, &cn, td, cnp->cn_nameptr, cnp->cn_namelen, CREATE))) + if ((error = unionfs_relookup(udvp, &uvp, cnp, &nd.ni_cnd, td, + cnp->cn_nameptr, cnp->cn_namelen, CREATE))) goto unionfs_mkshadowdir_abort; if (uvp != NULLVP) { if (udvp == uvp) @@ -808,7 +810,7 @@ unionfs_mkshadowdir(struct unionfs_mount *ump, struct vnode *udvp, goto unionfs_mkshadowdir_free_out; unionfs_create_uppervattr_core(ump, &lva, &va, td); - error = VOP_MKDIR(udvp, &uvp, &cn, &va); + error = VOP_MKDIR(udvp, &uvp, &nd.ni_cnd, &va); if (!error) { unionfs_node_update(unp, uvp, td); @@ -818,14 +820,14 @@ unionfs_mkshadowdir(struct unionfs_mount *ump, struct vnode *udvp, * Ignore errors. */ va.va_type = VNON; - VOP_SETATTR(uvp, &va, cn.cn_cred); + VOP_SETATTR(uvp, &va, nd.ni_cnd.cn_cred); } vn_finished_write(mp); unionfs_mkshadowdir_free_out: - if (cn.cn_flags & HASBUF) { - uma_zfree(namei_zone, cn.cn_pnbuf); - cn.cn_flags &= ~HASBUF; + if (nd.ni_cnd.cn_flags & HASBUF) { + uma_zfree(namei_zone, nd.ni_cnd.cn_pnbuf); + nd.ni_cnd.cn_flags &= ~HASBUF; } unionfs_mkshadowdir_abort: @@ -847,19 +849,21 @@ unionfs_mkwhiteout(struct vnode *dvp, struct componentname *cnp, { int error; struct vnode *wvp; - struct componentname cn; + struct nameidata nd; struct mount *mp; if (path == NULL) path = cnp->cn_nameptr; wvp = NULLVP; - if ((error = unionfs_relookup(dvp, &wvp, cnp, &cn, td, path, strlen(path), CREATE))) + NDPREINIT(&nd); + if ((error = unionfs_relookup(dvp, &wvp, cnp, &nd.ni_cnd, td, path, + strlen(path), CREATE))) return (error); if (wvp != NULLVP) { - if (cn.cn_flags & HASBUF) { - uma_zfree(namei_zone, cn.cn_pnbuf); - cn.cn_flags &= ~HASBUF; + if (nd.ni_cnd.cn_flags & HASBUF) { + uma_zfree(namei_zone, nd.ni_cnd.cn_pnbuf); + nd.ni_cnd.cn_flags &= ~HASBUF; } if (dvp == wvp) vrele(wvp); @@ -871,14 +875,14 @@ unionfs_mkwhiteout(struct vnode *dvp, struct componentname *cnp, if ((error = vn_start_write(dvp, &mp, V_WAIT | PCATCH))) goto unionfs_mkwhiteout_free_out; - error = VOP_WHITEOUT(dvp, &cn, CREATE); + error = VOP_WHITEOUT(dvp, &nd.ni_cnd, CREATE); vn_finished_write(mp); unionfs_mkwhiteout_free_out: - if (cn.cn_flags & HASBUF) { - uma_zfree(namei_zone, cn.cn_pnbuf); - cn.cn_flags &= ~HASBUF; + if (nd.ni_cnd.cn_flags & HASBUF) { + uma_zfree(namei_zone, nd.ni_cnd.cn_pnbuf); + nd.ni_cnd.cn_flags &= ~HASBUF; } return (error); @@ -904,7 +908,7 @@ unionfs_vn_create_on_upper(struct vnode **vpp, struct vnode *udvp, struct vattr lva; int fmode; int error; - struct componentname cn; + struct nameidata nd; ump = MOUNTTOUNIONFSMOUNT(UNIONFSTOV(unp)->v_mount); vp = NULLVP; @@ -920,18 +924,20 @@ unionfs_vn_create_on_upper(struct vnode **vpp, struct vnode *udvp, if (unp->un_path == NULL) panic("unionfs: un_path is null"); - cn.cn_namelen = strlen(unp->un_path); - cn.cn_pnbuf = uma_zalloc(namei_zone, M_WAITOK); - bcopy(unp->un_path, cn.cn_pnbuf, cn.cn_namelen + 1); - cn.cn_nameiop = CREATE; - cn.cn_flags = (LOCKPARENT | LOCKLEAF | HASBUF | SAVENAME | ISLASTCN); - cn.cn_lkflags = LK_EXCLUSIVE; - cn.cn_thread = td; - cn.cn_cred = cred; - cn.cn_nameptr = cn.cn_pnbuf; + nd.ni_cnd.cn_namelen = strlen(unp->un_path); + nd.ni_cnd.cn_pnbuf = uma_zalloc(namei_zone, M_WAITOK); + bcopy(unp->un_path, nd.ni_cnd.cn_pnbuf, nd.ni_cnd.cn_namelen + 1); + nd.ni_cnd.cn_nameiop = CREATE; + nd.ni_cnd.cn_flags = LOCKPARENT | LOCKLEAF | HASBUF | SAVENAME | + ISLASTCN; + nd.ni_cnd.cn_lkflags = LK_EXCLUSIVE; + nd.ni_cnd.cn_thread = td; + nd.ni_cnd.cn_cred = cred; + nd.ni_cnd.cn_nameptr = nd.ni_cnd.cn_pnbuf; + NDPREINIT(&nd); vref(udvp); - if ((error = relookup(udvp, &vp, &cn)) != 0) + if ((error = relookup(udvp, &vp, &nd.ni_cnd)) != 0) goto unionfs_vn_create_on_upper_free_out2; vrele(udvp); @@ -944,7 +950,7 @@ unionfs_vn_create_on_upper(struct vnode **vpp, struct vnode *udvp, goto unionfs_vn_create_on_upper_free_out1; } - if ((error = VOP_CREATE(udvp, &vp, &cn, uvap)) != 0) + if ((error = VOP_CREATE(udvp, &vp, &nd.ni_cnd, uvap)) != 0) goto unionfs_vn_create_on_upper_free_out1; if ((error = VOP_OPEN(vp, fmode, cred, td, NULL)) != 0) { @@ -964,9 +970,9 @@ unionfs_vn_create_on_upper_free_out1: VOP_UNLOCK(udvp); unionfs_vn_create_on_upper_free_out2: - if (cn.cn_flags & HASBUF) { - uma_zfree(namei_zone, cn.cn_pnbuf); - cn.cn_flags &= ~HASBUF; + if (nd.ni_cnd.cn_flags & HASBUF) { + uma_zfree(namei_zone, nd.ni_cnd.cn_pnbuf); + nd.ni_cnd.cn_flags &= ~HASBUF; } return (error); From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:10:59 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 35463653CD3; Wed, 7 Jul 2021 11:10:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDL5THQz4fs5; Wed, 7 Jul 2021 11:10:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8ECE645C1; Wed, 7 Jul 2021 11:10:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BAwvX040852; Wed, 7 Jul 2021 11:10:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BAw13040851; Wed, 7 Jul 2021 11:10:58 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:10:58 GMT Message-Id: <202107071110.167BAw13040851@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: b3c1b988ded5 - stable/13 - ufs_vnops.c: style MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: b3c1b988ded5d80e0fd3425580716f1110629bfb Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:10:59 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=b3c1b988ded5d80e0fd3425580716f1110629bfb commit b3c1b988ded5d80e0fd3425580716f1110629bfb Author: Konstantin Belousov AuthorDate: 2021-05-01 21:40:50 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:35 +0000 ufs_vnops.c: style (cherry picked from commit b2b40b28b1784fb153f8aefb4f86e085a149e91e) --- sys/ufs/ufs/ufs_vnops.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/ufs/ufs/ufs_vnops.c b/sys/ufs/ufs/ufs_vnops.c index ef288a32e815..00249e3902c8 100644 --- a/sys/ufs/ufs/ufs_vnops.c +++ b/sys/ufs/ufs/ufs_vnops.c @@ -105,14 +105,16 @@ VFS_SMR_DECLARE; static vop_accessx_t ufs_accessx; static vop_fplookup_vexec_t ufs_fplookup_vexec; static int ufs_chmod(struct vnode *, int, struct ucred *, struct thread *); -static int ufs_chown(struct vnode *, uid_t, gid_t, struct ucred *, struct thread *); +static int ufs_chown(struct vnode *, uid_t, gid_t, struct ucred *, + struct thread *); static vop_close_t ufs_close; static vop_create_t ufs_create; static vop_stat_t ufs_stat; static vop_getattr_t ufs_getattr; static vop_ioctl_t ufs_ioctl; static vop_link_t ufs_link; -static int ufs_makeinode(int mode, struct vnode *, struct vnode **, struct componentname *, const char *); +static int ufs_makeinode(int mode, struct vnode *, struct vnode **, + struct componentname *, const char *); static vop_mmapped_t ufs_mmapped; static vop_mkdir_t ufs_mkdir; static vop_mknod_t ufs_mknod; From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:00 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42B03653D1B; Wed, 7 Jul 2021 11:11:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDM6Rjsz4fXp; Wed, 7 Jul 2021 11:10:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BA0423DBC; Wed, 7 Jul 2021 11:10:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BAxBj040880; Wed, 7 Jul 2021 11:10:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BAxsQ040879; Wed, 7 Jul 2021 11:10:59 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:10:59 GMT Message-Id: <202107071110.167BAxsQ040879@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 91ef0d6ac65f - stable/13 - ffs: reduce number of dvp relocks in softdep_prelink() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 91ef0d6ac65f20fe677802146f5fa308793c6443 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:00 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=91ef0d6ac65f20fe677802146f5fa308793c6443 commit 91ef0d6ac65f20fe677802146f5fa308793c6443 Author: Konstantin Belousov AuthorDate: 2021-04-29 22:21:33 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:40 +0000 ffs: reduce number of dvp relocks in softdep_prelink() (cherry picked from commit d0929a990c80b78bc84326234cf54cd4382d42fd) --- sys/ufs/ffs/ffs_softdep.c | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index 2cc16ab2a2c1..dbd08905d953 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -3392,10 +3392,9 @@ softdep_prelink(dvp, vp) if (vp != NULL) VOP_UNLOCK(vp); ffs_syncvnode(dvp, MNT_WAIT, 0); - VOP_UNLOCK(dvp); - /* Process vp before dvp as it may create .. removes. */ if (vp != NULL) { + VOP_UNLOCK(dvp); vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); if (vp->v_data == NULL) { vn_lock_pair(dvp, false, vp, true); @@ -3406,12 +3405,11 @@ softdep_prelink(dvp, vp) process_truncates(vp); FREE_LOCK(ump); VOP_UNLOCK(vp); - } - - vn_lock(dvp, LK_EXCLUSIVE | LK_RETRY); - if (dvp->v_data == NULL) { - vn_lock_pair(dvp, true, vp, false); - return (ERELOOKUP); + vn_lock(dvp, LK_EXCLUSIVE | LK_RETRY); + if (dvp->v_data == NULL) { + vn_lock_pair(dvp, true, vp, false); + return (ERELOOKUP); + } } ACQUIRE_LOCK(ump); From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:01 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E0C7653247; Wed, 7 Jul 2021 11:11:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDP2pVgz4fdM; Wed, 7 Jul 2021 11:11:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DE6B143DA; Wed, 7 Jul 2021 11:11:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB0xj040922; Wed, 7 Jul 2021 11:11:00 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB0Ls040921; Wed, 7 Jul 2021 11:11:00 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:00 GMT Message-Id: <202107071111.167BB0Ls040921@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: ac6447c08aec - stable/13 - ufs_rename(): softdep_prerename() does something only for SU+J MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ac6447c08aec6e389ae87369af221dfe2ecd2f16 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:01 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=ac6447c08aec6e389ae87369af221dfe2ecd2f16 commit ac6447c08aec6e389ae87369af221dfe2ecd2f16 Author: Konstantin Belousov AuthorDate: 2021-04-29 22:23:38 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:45 +0000 ufs_rename(): softdep_prerename() does something only for SU+J (cherry picked from commit 5eacde3eb83de6ef1e0b126c019c759bf411e36a) --- sys/ufs/ufs/ufs_vnops.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sys/ufs/ufs/ufs_vnops.c b/sys/ufs/ufs/ufs_vnops.c index 00249e3902c8..2154f7aa7f95 100644 --- a/sys/ufs/ufs/ufs_vnops.c +++ b/sys/ufs/ufs/ufs_vnops.c @@ -1364,7 +1364,7 @@ relock: } } - if (DOINGSOFTDEP(fdvp)) { + if (DOINGSUJ(fdvp)) { error = softdep_prerename(fdvp, fvp, tdvp, tvp); if (error != 0) { if (error == ERELOOKUP) { From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:02 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5DF09653A5C; Wed, 7 Jul 2021 11:11:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDQ19MJz4fsG; Wed, 7 Jul 2021 11:11:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0A22743DC; Wed, 7 Jul 2021 11:11:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB1OS040946; Wed, 7 Jul 2021 11:11:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB1C1040945; Wed, 7 Jul 2021 11:11:01 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:01 GMT Message-Id: <202107071111.167BB1C1040945@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: a7a074d50aec - stable/13 - ffs: mark block (re-)allocations as seqc writes MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a7a074d50aecfe5717d469a81018678ce2880cf1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:02 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=a7a074d50aecfe5717d469a81018678ce2880cf1 commit a7a074d50aecfe5717d469a81018678ce2880cf1 Author: Konstantin Belousov AuthorDate: 2021-05-21 14:12:08 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:53 +0000 ffs: mark block (re-)allocations as seqc writes (cherry picked from commit d4d289cd51078de9e82c9d83977cfa614032cd06) --- sys/ufs/ffs/ffs_alloc.c | 9 ++-- sys/ufs/ffs/ffs_balloc.c | 116 ++++++++++++++++++++++++++++------------------- 2 files changed, 75 insertions(+), 50 deletions(-) diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index c895c8c7bf07..771377b7cefc 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -517,6 +517,7 @@ ffs_reallocblks(ap) } */ *ap; { struct ufsmount *ump; + int error; /* * We used to skip reallocating the blocks of a file into a @@ -546,9 +547,11 @@ ffs_reallocblks(ap) if (DOINGSUJ(ap->a_vp)) if (softdep_prealloc(ap->a_vp, MNT_NOWAIT) != 0) return (ENOSPC); - if (ump->um_fstype == UFS1) - return (ffs_reallocblks_ufs1(ap)); - return (ffs_reallocblks_ufs2(ap)); + vn_seqc_write_begin(ap->a_vp); + error = ump->um_fstype == UFS1 ? ffs_reallocblks_ufs1(ap) : + ffs_reallocblks_ufs2(ap); + vn_seqc_write_end(ap->a_vp); + return (error); } static int diff --git a/sys/ufs/ffs/ffs_balloc.c b/sys/ufs/ffs/ffs_balloc.c index 1b53a90a48c4..d7718e56dd87 100644 --- a/sys/ufs/ffs/ffs_balloc.c +++ b/sys/ufs/ffs/ffs_balloc.c @@ -128,6 +128,8 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, return (EFBIG); gbflags = (flags & BA_UNMAPPED) != 0 ? GB_UNMAPPED : 0; + vn_seqc_write_begin(vp); + /* * If the next write will extend the file into a new block, * and the file is currently composed of a fragment @@ -144,7 +146,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, &dp->di_db[0]), osize, (int)fs->fs_bsize, flags, cred, &bp); if (error) - return (error); + goto done; if (DOINGSOFTDEP(vp)) softdep_setup_allocdirect(ip, nb, dbtofsb(fs, bp->b_blkno), dp->di_db[nb], @@ -174,17 +176,20 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, error = bread(vp, lbn, fs->fs_bsize, NOCRED, &bp); if (error != 0) - return (error); + goto done; } else { bp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); - if (bp == NULL) - return (EIO); + if (bp == NULL) { + error = EIO; + goto done; + } vfs_bio_clrbuf(bp); } bp->b_blkno = fsbtodb(fs, nb); *bpp = bp; - return (0); + error = 0; + goto done; } if (nb != 0) { /* @@ -194,9 +199,8 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, nsize = fragroundup(fs, size); if (nsize <= osize) { error = bread(vp, lbn, osize, NOCRED, &bp); - if (error) { - return (error); - } + if (error) + goto done; bp->b_blkno = fsbtodb(fs, nb); } else { UFS_LOCK(ump); @@ -205,7 +209,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, &dp->di_db[0]), osize, nsize, flags, cred, &bp); if (error) - return (error); + goto done; if (DOINGSOFTDEP(vp)) softdep_setup_allocdirect(ip, lbn, dbtofsb(fs, bp->b_blkno), nb, @@ -221,7 +225,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, ffs_blkpref_ufs1(ip, lbn, (int)lbn, &dp->di_db[0]), nsize, flags, cred, &newb); if (error) - return (error); + goto done; bp = getblk(vp, lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); if (flags & BA_CLRBUF) @@ -233,14 +237,15 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, dp->di_db[lbn] = dbtofsb(fs, bp->b_blkno); UFS_INODE_SET_FLAG(ip, IN_CHANGE | IN_UPDATE | IN_IBLKDATA); *bpp = bp; - return (0); + error = 0; + goto done; } /* * Determine the number of levels of indirection. */ pref = 0; if ((error = ufs_getlbns(vp, lbn, indirs, &num)) != 0) - return(error); + goto done; #ifdef INVARIANTS if (num < 1) panic ("ffs_balloc_ufs1: ufs_getlbns returned indirect block"); @@ -261,7 +266,7 @@ ffs_balloc_ufs1(struct vnode *vp, off_t startoffset, int size, if ((error = ffs_alloc(ip, lbn, pref, (int)fs->fs_bsize, flags, cred, &newb)) != 0) { curthread_pflags_restore(saved_inbdflush); - return (error); + goto done; } pref = newb + fs->fs_frag; nb = newb; @@ -387,7 +392,8 @@ retry: if (flags & BA_METAONLY) { curthread_pflags_restore(saved_inbdflush); *bpp = bp; - return (0); + error = 0; + goto done; } /* * Get the data block, allocating if necessary. @@ -453,7 +459,8 @@ retry: } curthread_pflags_restore(saved_inbdflush); *bpp = nbp; - return (0); + error = 0; + goto done; } brelse(bp); if (flags & BA_CLRBUF) { @@ -478,7 +485,8 @@ retry: } curthread_pflags_restore(saved_inbdflush); *bpp = nbp; - return (0); + error = 0; + goto done; fail: curthread_pflags_restore(saved_inbdflush); /* @@ -486,7 +494,7 @@ fail: * This is the usual case and avoids the need to fsync the file. */ if (allocblk == allociblk && allocib == NULL && unwindidx == -1) - return (error); + goto done; /* * If we have failed part way through block allocation, we * have to deallocate any indirect blocks that we have allocated. @@ -574,6 +582,8 @@ fail: ffs_blkfree(ump, fs, ump->um_devvp, *blkp, fs->fs_bsize, ip->i_number, vp->v_type, NULL, SINGLETON_KEY); } +done: + vn_seqc_write_end(vp); return (error); } @@ -619,12 +629,17 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, return (EFBIG); gbflags = (flags & BA_UNMAPPED) != 0 ? GB_UNMAPPED : 0; + vn_seqc_write_begin(vp); + /* * Check for allocating external data. */ if (flags & IO_EXT) { - if (lbn >= UFS_NXADDR) - return (EFBIG); + if (lbn >= UFS_NXADDR) { + error = EFBIG; + goto done; + } + /* * If the next write will extend the data into a new block, * and the data is currently composed of a fragment @@ -642,7 +657,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, &dp->di_extb[0]), osize, (int)fs->fs_bsize, flags, cred, &bp); if (error) - return (error); + goto done; if (DOINGSOFTDEP(vp)) softdep_setup_allocext(ip, nb, dbtofsb(fs, bp->b_blkno), @@ -668,13 +683,12 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, if (nb != 0 && dp->di_extsize >= smalllblktosize(fs, lbn + 1)) { error = bread_gb(vp, -1 - lbn, fs->fs_bsize, NOCRED, gbflags, &bp); - if (error) { - return (error); - } + if (error) + goto done; bp->b_blkno = fsbtodb(fs, nb); bp->b_xflags |= BX_ALTDATA; *bpp = bp; - return (0); + goto done; } if (nb != 0) { /* @@ -685,9 +699,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, if (nsize <= osize) { error = bread_gb(vp, -1 - lbn, osize, NOCRED, gbflags, &bp); - if (error) { - return (error); - } + if (error) + goto done; bp->b_blkno = fsbtodb(fs, nb); bp->b_xflags |= BX_ALTDATA; } else { @@ -698,7 +711,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, &dp->di_extb[0]), osize, nsize, flags, cred, &bp); if (error) - return (error); + goto done; bp->b_xflags |= BX_ALTDATA; if (DOINGSOFTDEP(vp)) softdep_setup_allocext(ip, lbn, @@ -715,7 +728,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, ffs_blkpref_ufs2(ip, lbn, (int)lbn, &dp->di_extb[0]), nsize, flags, cred, &newb); if (error) - return (error); + goto done; bp = getblk(vp, -1 - lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); bp->b_xflags |= BX_ALTDATA; @@ -728,7 +741,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, dp->di_extb[lbn] = dbtofsb(fs, bp->b_blkno); UFS_INODE_SET_FLAG(ip, IN_CHANGE | IN_IBLKDATA); *bpp = bp; - return (0); + error = 0; + goto done; } /* * If the next write will extend the file into a new block, @@ -746,7 +760,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, &dp->di_db[0]), osize, (int)fs->fs_bsize, flags, cred, &bp); if (error) - return (error); + goto done; if (DOINGSOFTDEP(vp)) softdep_setup_allocdirect(ip, nb, dbtofsb(fs, bp->b_blkno), @@ -775,17 +789,20 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, error = bread_gb(vp, lbn, fs->fs_bsize, NOCRED, gbflags, &bp); if (error != 0) - return (error); + goto done; } else { bp = getblk(vp, lbn, fs->fs_bsize, 0, 0, gbflags); - if (bp == NULL) - return (EIO); + if (bp == NULL) { + error = EIO; + goto done; + } vfs_bio_clrbuf(bp); } bp->b_blkno = fsbtodb(fs, nb); *bpp = bp; - return (0); + error = 0; + goto done; } if (nb != 0) { /* @@ -796,9 +813,8 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, if (nsize <= osize) { error = bread_gb(vp, lbn, osize, NOCRED, gbflags, &bp); - if (error) { - return (error); - } + if (error) + goto done; bp->b_blkno = fsbtodb(fs, nb); } else { UFS_LOCK(ump); @@ -807,7 +823,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, &dp->di_db[0]), osize, nsize, flags, cred, &bp); if (error) - return (error); + goto done; if (DOINGSOFTDEP(vp)) softdep_setup_allocdirect(ip, lbn, dbtofsb(fs, bp->b_blkno), nb, @@ -823,7 +839,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, ffs_blkpref_ufs2(ip, lbn, (int)lbn, &dp->di_db[0]), nsize, flags, cred, &newb); if (error) - return (error); + goto done; bp = getblk(vp, lbn, nsize, 0, 0, gbflags); bp->b_blkno = fsbtodb(fs, newb); if (flags & BA_CLRBUF) @@ -835,14 +851,15 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, dp->di_db[lbn] = dbtofsb(fs, bp->b_blkno); UFS_INODE_SET_FLAG(ip, IN_CHANGE | IN_UPDATE | IN_IBLKDATA); *bpp = bp; - return (0); + error = 0; + goto done; } /* * Determine the number of levels of indirection. */ pref = 0; if ((error = ufs_getlbns(vp, lbn, indirs, &num)) != 0) - return(error); + goto done; #ifdef INVARIANTS if (num < 1) panic ("ffs_balloc_ufs2: ufs_getlbns returned indirect block"); @@ -863,7 +880,7 @@ ffs_balloc_ufs2(struct vnode *vp, off_t startoffset, int size, if ((error = ffs_alloc(ip, lbn, pref, (int)fs->fs_bsize, flags, cred, &newb)) != 0) { curthread_pflags_restore(saved_inbdflush); - return (error); + goto done; } pref = newb + fs->fs_frag; nb = newb; @@ -991,7 +1008,8 @@ retry: if (flags & BA_METAONLY) { curthread_pflags_restore(saved_inbdflush); *bpp = bp; - return (0); + error = 0; + goto done; } /* * Get the data block, allocating if necessary. @@ -1057,7 +1075,8 @@ retry: } curthread_pflags_restore(saved_inbdflush); *bpp = nbp; - return (0); + error = 0; + goto done; } brelse(bp); /* @@ -1088,7 +1107,8 @@ retry: } curthread_pflags_restore(saved_inbdflush); *bpp = nbp; - return (0); + error = 0; + goto done; fail: curthread_pflags_restore(saved_inbdflush); /* @@ -1096,7 +1116,7 @@ fail: * This is the usual case and avoids the need to fsync the file. */ if (allocblk == allociblk && allocib == NULL && unwindidx == -1) - return (error); + goto done; /* * If we have failed part way through block allocation, we * have to deallocate any indirect blocks that we have allocated. @@ -1184,5 +1204,7 @@ fail: ffs_blkfree(ump, fs, ump->um_devvp, *blkp, fs->fs_bsize, ip->i_number, vp->v_type, NULL, SINGLETON_KEY); } +done: + vn_seqc_write_end(vp); return (error); } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:04 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A1E49653C5E; Wed, 7 Jul 2021 11:11:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDS3B26z4fY6; Wed, 7 Jul 2021 11:11:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4CA1943DE; Wed, 7 Jul 2021 11:11:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB47q040994; Wed, 7 Jul 2021 11:11:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB4BB040993; Wed, 7 Jul 2021 11:11:04 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:04 GMT Message-Id: <202107071111.167BB4BB040993@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 2622570aeb3d - stable/13 - softdep_prelink(): only do sync if other thread changed the vnode metadata since previous prelink MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 2622570aeb3d162812d72f7ef192c322cd8b73ef Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:04 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=2622570aeb3d162812d72f7ef192c322cd8b73ef commit 2622570aeb3d162812d72f7ef192c322cd8b73ef Author: Konstantin Belousov AuthorDate: 2021-05-01 21:53:21 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:50:13 +0000 softdep_prelink(): only do sync if other thread changed the vnode metadata since previous prelink FreeBSD_version is bumped due to struct namei size change (cherry picked from commit 64b494a1050ae2cf2412edc19b57dc80f49eeda1) --- sys/sys/namei.h | 12 +++++++++++- sys/sys/param.h | 2 +- sys/ufs/ffs/ffs_extern.h | 3 ++- sys/ufs/ffs/ffs_softdep.c | 32 +++++++++++++++++++++++++++----- sys/ufs/ufs/ufs_vnops.c | 12 ++++++------ 5 files changed, 47 insertions(+), 14 deletions(-) diff --git a/sys/sys/namei.h b/sys/sys/namei.h index b4db0e758e2b..9e0a82ea1659 100644 --- a/sys/sys/namei.h +++ b/sys/sys/namei.h @@ -38,6 +38,7 @@ #include #include #include +#include #include enum nameiop { LOOKUP, CREATE, DELETE, RENAME }; @@ -111,6 +112,12 @@ struct nameidata { */ struct componentname ni_cnd; struct nameicap_tracker_head ni_cap_tracker; + /* + * Private helper data for UFS, must be at the end. See + * NDINIT_PREFILL(). + */ + seqc_t ni_dvp_seqc; + seqc_t ni_vp_seqc; }; #ifdef _KERNEL @@ -224,7 +231,8 @@ int cache_fplookup(struct nameidata *ndp, enum cache_fpl_status *status, * Note the constant pattern may *hide* bugs. */ #ifdef INVARIANTS -#define NDINIT_PREFILL(arg) memset(arg, 0xff, sizeof(*arg)) +#define NDINIT_PREFILL(arg) memset(arg, 0xff, offsetof(struct nameidata, \ + ni_dvp_seqc)) #define NDINIT_DBG(arg) { (arg)->ni_debugflags = NAMEI_DBG_INITED; } #define NDREINIT_DBG(arg) { \ if (((arg)->ni_debugflags & NAMEI_DBG_INITED) == 0) \ @@ -266,6 +274,8 @@ do { \ } while (0) #define NDPREINIT(ndp) do { \ + (ndp)->ni_dvp_seqc = SEQC_MOD; \ + (ndp)->ni_vp_seqc = SEQC_MOD; \ } while (0) #define NDF_NO_DVP_RELE 0x00000001 diff --git a/sys/sys/param.h b/sys/sys/param.h index 41db9a1675f7..4c43839c297e 100644 --- a/sys/sys/param.h +++ b/sys/sys/param.h @@ -60,7 +60,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 1300510 /* Master, propagated to newvers */ +#define __FreeBSD_version 1300511 /* Master, propagated to newvers */ /* * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD, diff --git a/sys/ufs/ffs/ffs_extern.h b/sys/ufs/ffs/ffs_extern.h index 544012089046..0365cb76601e 100644 --- a/sys/ufs/ffs/ffs_extern.h +++ b/sys/ufs/ffs/ffs_extern.h @@ -178,7 +178,8 @@ int softdep_request_cleanup(struct fs *, struct vnode *, struct ucred *, int); int softdep_prerename(struct vnode *, struct vnode *, struct vnode *, struct vnode *); -int softdep_prelink(struct vnode *, struct vnode *); +int softdep_prelink(struct vnode *, struct vnode *, + struct componentname *); void softdep_setup_freeblocks(struct inode *, off_t, int); void softdep_setup_inomapdep(struct buf *, struct inode *, ino_t, int); void softdep_setup_blkmapdep(struct buf *, struct mount *, ufs2_daddr_t, diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index dbd08905d953..f6dd5c776611 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -621,9 +621,10 @@ softdep_prerename(fdvp, fvp, tdvp, tvp) } int -softdep_prelink(dvp, vp) +softdep_prelink(dvp, vp, cnp) struct vnode *dvp; struct vnode *vp; + struct componentname *cnp; { panic("softdep_prelink called"); @@ -3361,11 +3362,13 @@ softdep_prerename(fdvp, fvp, tdvp, tvp) * syscall must be restarted at top level from the lookup. */ int -softdep_prelink(dvp, vp) +softdep_prelink(dvp, vp, cnp) struct vnode *dvp; struct vnode *vp; + struct componentname *cnp; { struct ufsmount *ump; + struct nameidata *ndp; ASSERT_VOP_ELOCKED(dvp, "prelink dvp"); if (vp != NULL) @@ -3381,13 +3384,28 @@ softdep_prelink(dvp, vp) if (journal_space(ump, 0) || (vp != NULL && IS_SNAPSHOT(VTOI(vp)))) return (0); + /* + * Check if the journal space consumption can in theory be + * accounted on dvp and vp. If the vnodes metadata was not + * changed comparing with the previous round-trip into + * softdep_prelink(), as indicated by the seqc generation + * recorded in the nameidata, then there is no point in + * starting the sync. + */ + ndp = __containerof(cnp, struct nameidata, ni_cnd); + if (!seqc_in_modify(ndp->ni_dvp_seqc) && + vn_seqc_consistent(dvp, ndp->ni_dvp_seqc) && + (vp == NULL || (!seqc_in_modify(ndp->ni_vp_seqc) && + vn_seqc_consistent(vp, ndp->ni_vp_seqc)))) + return (0); + stat_journal_low++; if (vp != NULL) { VOP_UNLOCK(dvp); ffs_syncvnode(vp, MNT_NOWAIT, 0); vn_lock_pair(dvp, false, vp, true); if (dvp->v_data == NULL) - return (ERELOOKUP); + goto out; } if (vp != NULL) VOP_UNLOCK(vp); @@ -3398,7 +3416,7 @@ softdep_prelink(dvp, vp) vn_lock(vp, LK_EXCLUSIVE | LK_RETRY); if (vp->v_data == NULL) { vn_lock_pair(dvp, false, vp, true); - return (ERELOOKUP); + goto out; } ACQUIRE_LOCK(ump); process_removes(vp); @@ -3408,7 +3426,7 @@ softdep_prelink(dvp, vp) vn_lock(dvp, LK_EXCLUSIVE | LK_RETRY); if (dvp->v_data == NULL) { vn_lock_pair(dvp, true, vp, false); - return (ERELOOKUP); + goto out; } } @@ -3427,6 +3445,10 @@ softdep_prelink(dvp, vp) FREE_LOCK(ump); vn_lock_pair(dvp, false, vp, false); +out: + ndp->ni_dvp_seqc = vn_seqc_read_any(dvp); + if (vp != NULL) + ndp->ni_vp_seqc = vn_seqc_read_any(vp); return (ERELOOKUP); } diff --git a/sys/ufs/ufs/ufs_vnops.c b/sys/ufs/ufs/ufs_vnops.c index b0fb1b74b900..2dfc2e24f772 100644 --- a/sys/ufs/ufs/ufs_vnops.c +++ b/sys/ufs/ufs/ufs_vnops.c @@ -1010,7 +1010,7 @@ ufs_remove(ap) (VTOI(dvp)->i_flags & APPEND)) return (EPERM); if (DOINGSUJ(dvp)) { - error = softdep_prelink(dvp, vp); + error = softdep_prelink(dvp, vp, ap->a_cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); @@ -1075,7 +1075,7 @@ ufs_link(ap) #endif if (DOINGSUJ(tdvp)) { - error = softdep_prelink(tdvp, vp); + error = softdep_prelink(tdvp, vp, cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); @@ -1147,7 +1147,7 @@ ufs_whiteout(ap) if (DOINGSUJ(dvp) && (ap->a_flags == CREATE || ap->a_flags == DELETE)) { - error = softdep_prelink(dvp, NULL); + error = softdep_prelink(dvp, NULL, cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); @@ -1962,7 +1962,7 @@ ufs_mkdir(ap) } if (DOINGSUJ(dvp)) { - error = softdep_prelink(dvp, NULL); + error = softdep_prelink(dvp, NULL, cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); @@ -2226,7 +2226,7 @@ ufs_rmdir(ap) goto out; } if (DOINGSUJ(dvp)) { - error = softdep_prelink(dvp, vp); + error = softdep_prelink(dvp, vp, cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); @@ -2751,7 +2751,7 @@ ufs_makeinode(mode, dvp, vpp, cnp, callfunc) return (EINVAL); } if (DOINGSUJ(dvp)) { - error = softdep_prelink(dvp, NULL); + error = softdep_prelink(dvp, NULL, cnp); if (error != 0) { MPASS(error == ERELOOKUP); return (error); From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:06 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 42AC3653250; Wed, 7 Jul 2021 11:11:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDT4YW5z4fq5; Wed, 7 Jul 2021 11:11:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 70D5C40B6; Wed, 7 Jul 2021 11:11:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB5RA041018; Wed, 7 Jul 2021 11:11:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB564041017; Wed, 7 Jul 2021 11:11:05 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:05 GMT Message-Id: <202107071111.167BB564041017@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: ecb63094dce6 - stable/13 - ffs_softdep.c: add journal_check_space() helper MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ecb63094dce6c4a77c19eba046e61ccd4094381b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:06 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=ecb63094dce6c4a77c19eba046e61ccd4094381b commit ecb63094dce6c4a77c19eba046e61ccd4094381b Author: Konstantin Belousov AuthorDate: 2021-06-15 10:23:44 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:50:43 +0000 ffs_softdep.c: add journal_check_space() helper (cherry picked from commit 2126f103e0434db6ca34f0a5167bf5f03d4f02ad) --- sys/ufs/ffs/ffs_softdep.c | 31 ++++++++++++++++--------------- 1 file changed, 16 insertions(+), 15 deletions(-) diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index f6dd5c776611..fe3af651632e 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -3199,6 +3199,19 @@ journal_unsuspend(struct ufsmount *ump) return (0); } +static void +journal_check_space(ump) + struct ufsmount *ump; +{ + LOCK_OWNED(ump); + + if (journal_space(ump, 0) == 0) { + softdep_speedup(ump); + if (journal_space(ump, 1) == 0) + journal_suspend(ump); + } +} + /* * Called before any allocation function to be certain that there is * sufficient space in the journal prior to creating any new records. @@ -3249,11 +3262,7 @@ softdep_prealloc(vp, waitok) ACQUIRE_LOCK(ump); process_removes(vp); process_truncates(vp); - if (journal_space(ump, 0) == 0) { - softdep_speedup(ump); - if (journal_space(ump, 1) == 0) - journal_suspend(ump); - } + journal_check_space(ump); FREE_LOCK(ump); return (0); @@ -3340,11 +3349,7 @@ softdep_prerename(fdvp, fvp, tdvp, tvp) ACQUIRE_LOCK(ump); softdep_speedup(ump); process_worklist_item(UFSTOVFS(ump), 2, LK_NOWAIT); - if (journal_space(ump, 0) == 0) { - softdep_speedup(ump); - if (journal_space(ump, 1) == 0) - journal_suspend(ump); - } + journal_check_space(ump); FREE_LOCK(ump); return (ERELOOKUP); } @@ -3437,11 +3442,7 @@ softdep_prelink(dvp, vp, cnp) softdep_speedup(ump); process_worklist_item(UFSTOVFS(ump), 2, LK_NOWAIT); - if (journal_space(ump, 0) == 0) { - softdep_speedup(ump); - if (journal_space(ump, 1) == 0) - journal_suspend(ump); - } + journal_check_space(ump); FREE_LOCK(ump); vn_lock_pair(dvp, false, vp, false); From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:04 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8A43365324D; Wed, 7 Jul 2021 11:11:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDR6kLkz4fnC; Wed, 7 Jul 2021 11:11:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 258704606; Wed, 7 Jul 2021 11:11:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB3FA040970; Wed, 7 Jul 2021 11:11:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB3tB040969; Wed, 7 Jul 2021 11:11:03 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:03 GMT Message-Id: <202107071111.167BB3tB040969@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: 9487bd8ee37e - stable/13 - ufs_rename(): only do softdep_prerename() when other thread changed a vnode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 9487bd8ee37e6a0394946909636267f644ee8df0 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:04 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=9487bd8ee37e6a0394946909636267f644ee8df0 commit 9487bd8ee37e6a0394946909636267f644ee8df0 Author: Konstantin Belousov AuthorDate: 2021-04-29 22:39:57 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:23:58 +0000 ufs_rename(): only do softdep_prerename() when other thread changed a vnode (cherry picked from commit f7565466622a411a50522f23528faeb1e57d4571) --- sys/ufs/ufs/ufs_vnops.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/sys/ufs/ufs/ufs_vnops.c b/sys/ufs/ufs/ufs_vnops.c index 2154f7aa7f95..b0fb1b74b900 100644 --- a/sys/ufs/ufs/ufs_vnops.c +++ b/sys/ufs/ufs/ufs_vnops.c @@ -1248,6 +1248,7 @@ ufs_rename(ap) int error = 0; struct mount *mp; ino_t ino; + seqc_t fdvp_s, fvp_s, tdvp_s, tvp_s; bool want_seqc_end; want_seqc_end = false; @@ -1271,6 +1272,8 @@ ufs_rename(ap) mp = NULL; goto releout; } + + fdvp_s = fvp_s = tdvp_s = tvp_s = SEQC_MOD; relock: /* * We need to acquire 2 to 4 locks depending on whether tvp is NULL @@ -1364,10 +1367,20 @@ relock: } } - if (DOINGSUJ(fdvp)) { + if (DOINGSUJ(fdvp) && + (seqc_in_modify(fdvp_s) || !vn_seqc_consistent(fdvp, fdvp_s) || + seqc_in_modify(fvp_s) || !vn_seqc_consistent(fvp, fvp_s) || + seqc_in_modify(tdvp_s) || !vn_seqc_consistent(tdvp, tdvp_s) || + (tvp != NULL && (seqc_in_modify(tvp_s) || + !vn_seqc_consistent(tvp, tvp_s))))) { error = softdep_prerename(fdvp, fvp, tdvp, tvp); if (error != 0) { if (error == ERELOOKUP) { + fdvp_s = vn_seqc_read_any(fdvp); + fvp_s = vn_seqc_read_any(fvp); + tdvp_s = vn_seqc_read_any(tdvp); + if (tvp != NULL) + tvp_s = vn_seqc_read_any(tvp); atomic_add_int(&rename_restarts, 1); goto relock; } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 11:11:08 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9DABC653E28; Wed, 7 Jul 2021 11:11:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKcDW5KWjz4fsT; Wed, 7 Jul 2021 11:11:07 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9970D44A4; Wed, 7 Jul 2021 11:11:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167BB67G041048; Wed, 7 Jul 2021 11:11:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167BB6O5041047; Wed, 7 Jul 2021 11:11:06 GMT (envelope-from git) Date: Wed, 7 Jul 2021 11:11:06 GMT Message-Id: <202107071111.167BB6O5041047@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Konstantin Belousov Subject: git: d36505493d2a - stable/13 - ffs_softdep: force sync if journal is low in journal_check_space MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d36505493d2a876b37da0c7850ef906b1f32df08 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 11:11:08 -0000 The branch stable/13 has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=d36505493d2a876b37da0c7850ef906b1f32df08 commit d36505493d2a876b37da0c7850ef906b1f32df08 Author: Konstantin Belousov AuthorDate: 2021-06-15 10:27:37 +0000 Commit: Konstantin Belousov CommitDate: 2021-07-07 10:50:44 +0000 ffs_softdep: force sync if journal is low in journal_check_space (cherry picked from commit 50acaaef54b4d7811393eb8c05a398d7a1882418) --- sys/ufs/ffs/ffs_softdep.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/sys/ufs/ffs/ffs_softdep.c b/sys/ufs/ffs/ffs_softdep.c index fe3af651632e..446fc69a6a32 100644 --- a/sys/ufs/ffs/ffs_softdep.c +++ b/sys/ufs/ffs/ffs_softdep.c @@ -3203,10 +3203,17 @@ static void journal_check_space(ump) struct ufsmount *ump; { + struct mount *mp; + LOCK_OWNED(ump); if (journal_space(ump, 0) == 0) { softdep_speedup(ump); + mp = UFSTOVFS(ump); + FREE_LOCK(ump); + VFS_SYNC(mp, MNT_NOWAIT); + ffs_sbupdate(ump, MNT_WAIT, 0); + ACQUIRE_LOCK(ump); if (journal_space(ump, 1) == 0) journal_suspend(ump); } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 13:18:23 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C36D655976; Wed, 7 Jul 2021 13:18:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKg3M3rq2z3Cmt; Wed, 7 Jul 2021 13:18:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6C3EB5B6D; Wed, 7 Jul 2021 13:18:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167DINI8007801; Wed, 7 Jul 2021 13:18:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167DINdW007800; Wed, 7 Jul 2021 13:18:23 GMT (envelope-from git) Date: Wed, 7 Jul 2021 13:18:23 GMT Message-Id: <202107071318.167DINdW007800@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ed Maste Subject: git: 9e06b34bb5d3 - stable/13 - Add deprecation notice for WITH_PROFILE option MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 9e06b34bb5d3544915ebea1a3f343f915004bbe1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 13:18:23 -0000 The branch stable/13 has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=9e06b34bb5d3544915ebea1a3f343f915004bbe1 commit 9e06b34bb5d3544915ebea1a3f343f915004bbe1 Author: Ed Maste AuthorDate: 2021-06-27 17:21:26 +0000 Commit: Ed Maste CommitDate: 2021-07-07 13:17:06 +0000 Add deprecation notice for WITH_PROFILE option As discussed on freebsd-current [1] and freebsd-arch [2] and review D30833, FreeBSD 14 will ship without the _p.a libraries built with -pg. Both upstream and base system (in commit b762974cf4b9) Clang have been modified to remove the special case for linking against these libraries. Clang's -pg support and mcount() remain, so building with -pg can still be used on code that the user builds; we just do not provide prebuilt libraries compiled with -pg. A similar change is still needed for GCC. [1] https://lists.freebsd.org/pipermail/freebsd-current/2020-January/075105.html [2] https://lists.freebsd.org/archives/freebsd-arch/2021-June/000016.html MFC after: 1 week Sponsored by: The FreeBSD Foundation (cherry picked from commit 175841285e289edebb6603da39f02549521ce950) Clarify notice for profiled libraries in FreeBSD 14 Reported by: kevans Fixes: 175841285e28 ("Add deprecation notice for...") Sponsored by: The FreeBSD Foundation (cherry picked from commit f94360971e649fa684ef3b7e72839b59c7242bdb) --- share/man/man5/src.conf.5 | 2 ++ tools/build/options/WITH_PROFILE | 2 ++ 2 files changed, 4 insertions(+) diff --git a/share/man/man5/src.conf.5 b/share/man/man5/src.conf.5 index dad8d465e3fd..4f44bda093ce 100644 --- a/share/man/man5/src.conf.5 +++ b/share/man/man5/src.conf.5 @@ -1374,6 +1374,8 @@ mips/mips64. .It Va WITH_PROFILE Set to build profiled libraries for use with .Xr gprof 8 . +This option is deprecated and is not present in +.Fx 14 . .Pp This is a default setting on amd64/amd64, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mips, powerpc/powerpc, powerpc/powerpc64, riscv/riscv64 and riscv/riscv64sf. diff --git a/tools/build/options/WITH_PROFILE b/tools/build/options/WITH_PROFILE index b8880d9459cc..347a3db000d1 100644 --- a/tools/build/options/WITH_PROFILE +++ b/tools/build/options/WITH_PROFILE @@ -1,3 +1,5 @@ .\" $FreeBSD$ Set to build profiled libraries for use with .Xr gprof 8 . +This option is deprecated and is not present in +.Fx 14 . From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 13:19:37 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2439C656304; Wed, 7 Jul 2021 13:19:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKg4n0ZNTz3DHF; Wed, 7 Jul 2021 13:19:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E62265AC6; Wed, 7 Jul 2021 13:19:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167DJaip008016; Wed, 7 Jul 2021 13:19:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167DJaRV008015; Wed, 7 Jul 2021 13:19:36 GMT (envelope-from git) Date: Wed, 7 Jul 2021 13:19:36 GMT Message-Id: <202107071319.167DJaRV008015@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ed Maste Subject: git: c3e6df3967f9 - stable/12 - Add deprecation notice for WITH_PROFILE option MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: c3e6df3967f9a64322ad493e0ecd7e8cffba018f Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 13:19:37 -0000 The branch stable/12 has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=c3e6df3967f9a64322ad493e0ecd7e8cffba018f commit c3e6df3967f9a64322ad493e0ecd7e8cffba018f Author: Ed Maste AuthorDate: 2021-06-27 17:21:26 +0000 Commit: Ed Maste CommitDate: 2021-07-07 13:19:00 +0000 Add deprecation notice for WITH_PROFILE option As discussed on freebsd-current [1] and freebsd-arch [2] and review D30833, FreeBSD 14 will ship without the _p.a libraries built with -pg. Both upstream and base system (in commit b762974cf4b9) Clang have been modified to remove the special case for linking against these libraries. Clang's -pg support and mcount() remain, so building with -pg can still be used on code that the user builds; we just do not provide prebuilt libraries compiled with -pg. A similar change is still needed for GCC. [1] https://lists.freebsd.org/pipermail/freebsd-current/2020-January/075105.html [2] https://lists.freebsd.org/archives/freebsd-arch/2021-June/000016.html MFC after: 1 week Sponsored by: The FreeBSD Foundation (cherry picked from commit 175841285e289edebb6603da39f02549521ce950) Clarify notice for profiled libraries in FreeBSD 14 Reported by: kevans Fixes: 175841285e28 ("Add deprecation notice for...") Sponsored by: The FreeBSD Foundation (cherry picked from commit f94360971e649fa684ef3b7e72839b59c7242bdb) --- share/man/man5/src.conf.5 | 4 +++- tools/build/options/WITH_PROFILE | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/share/man/man5/src.conf.5 b/share/man/man5/src.conf.5 index 0e36281c78df..606e61a4e9c4 100644 --- a/share/man/man5/src.conf.5 +++ b/share/man/man5/src.conf.5 @@ -1,6 +1,6 @@ .\" DO NOT EDIT-- this file is @generated by tools/build/options/makeman. .\" $FreeBSD$ -.Dd December 29, 2020 +.Dd June 28, 2021 .Dt SRC.CONF 5 .Os .Sh NAME @@ -1660,6 +1660,8 @@ mips/mips64el, mips/mips64, mips/mips64elhf and mips/mips64hf. .It Va WITH_PROFILE Set to build profiled libraries for use with .Xr gprof 8 . +This option is deprecated and is not present in +.Fx 14 . .Pp This is a default setting on amd64/amd64, arm/arm, arm/armv6, arm/armv7, arm64/aarch64, i386/i386, mips/mipsel, mips/mips, mips/mipsn32, mips/mipselhf, mips/mipshf, powerpc/powerpc, powerpc/powerpc64, powerpc/powerpcspe, riscv/riscv64, riscv/riscv64sf and sparc64/sparc64. diff --git a/tools/build/options/WITH_PROFILE b/tools/build/options/WITH_PROFILE index b8880d9459cc..347a3db000d1 100644 --- a/tools/build/options/WITH_PROFILE +++ b/tools/build/options/WITH_PROFILE @@ -1,3 +1,5 @@ .\" $FreeBSD$ Set to build profiled libraries for use with .Xr gprof 8 . +This option is deprecated and is not present in +.Fx 14 . From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 20:10:04 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CEC3865C16B; Wed, 7 Jul 2021 20:10:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKrBN5LC7z4tgj; Wed, 7 Jul 2021 20:10:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F943136E0; Wed, 7 Jul 2021 20:10:04 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167KA40j056366; Wed, 7 Jul 2021 20:10:04 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167KA4QU056363; Wed, 7 Jul 2021 20:10:04 GMT (envelope-from git) Date: Wed, 7 Jul 2021 20:10:04 GMT Message-Id: <202107072010.167KA4QU056363@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: e991afea1e79 - stable/13 - kern: fail getgroup and setgroup with negative int MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e991afea1e794885931d17310019e4c067501981 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 20:10:04 -0000 The branch stable/13 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=e991afea1e794885931d17310019e4c067501981 commit e991afea1e794885931d17310019e4c067501981 Author: Moritz Buhl AuthorDate: 2019-07-09 15:03:37 +0000 Commit: Warner Losh CommitDate: 2021-07-07 20:09:44 +0000 kern: fail getgroup and setgroup with negative int Found using https://github.com/NetBSD/src/blob/trunk/tests/lib/libc/sys/t_getgroups.c getgroups/setgroups want an int and therefore casting it to u_int resulted in `getgroups(-1, ...)` not returning -1 / errno = EINVAL. imp@ updated syscall.master and made changes markj@ suggested PR: 189941 Tested by: imp@ Reviewed by: markj@ Pull Request: https://github.com/freebsd/freebsd-src/pull/407 Differential Revision: https://reviews.freebsd.org/D30617 (cherry picked from commit 4bc2174a1b489c36195ccc8cfc15e0775b817c69) --- sys/kern/kern_prot.c | 12 +++++------- sys/kern/syscalls.master | 4 ++-- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/sys/kern/kern_prot.c b/sys/kern/kern_prot.c index 170e9598835e..08fc3c8062db 100644 --- a/sys/kern/kern_prot.c +++ b/sys/kern/kern_prot.c @@ -288,7 +288,7 @@ sys_getegid(struct thread *td, struct getegid_args *uap) #ifndef _SYS_SYSPROTO_H_ struct getgroups_args { - u_int gidsetsize; + int gidsetsize; gid_t *gidset; }; #endif @@ -296,8 +296,7 @@ int sys_getgroups(struct thread *td, struct getgroups_args *uap) { struct ucred *cred; - u_int ngrp; - int error; + int ngrp, error; cred = td->td_ucred; ngrp = cred->cr_ngroups; @@ -791,7 +790,7 @@ fail: #ifndef _SYS_SYSPROTO_H_ struct setgroups_args { - u_int gidsetsize; + int gidsetsize; gid_t *gidset; }; #endif @@ -801,11 +800,10 @@ sys_setgroups(struct thread *td, struct setgroups_args *uap) { gid_t smallgroups[XU_NGROUPS]; gid_t *groups; - u_int gidsetsize; - int error; + int gidsetsize, error; gidsetsize = uap->gidsetsize; - if (gidsetsize > ngroups_max + 1) + if (gidsetsize > ngroups_max + 1 || gidsetsize < 0) return (EINVAL); if (gidsetsize > XU_NGROUPS) diff --git a/sys/kern/syscalls.master b/sys/kern/syscalls.master index 5c6ebeb9b52f..5b8e8049927c 100644 --- a/sys/kern/syscalls.master +++ b/sys/kern/syscalls.master @@ -526,13 +526,13 @@ } 79 AUE_GETGROUPS STD { int getgroups( - u_int gidsetsize, + int gidsetsize, _Out_writes_opt_(gidsetsize) gid_t *gidset ); } 80 AUE_SETGROUPS STD { int setgroups( - u_int gidsetsize, + int gidsetsize, _In_reads_(gidsetsize) gid_t *gidset ); } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 20:10:06 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0BD0065C3CE; Wed, 7 Jul 2021 20:10:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKrBP6mLnz4tjW; Wed, 7 Jul 2021 20:10:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C064313BAF; Wed, 7 Jul 2021 20:10:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167KA5lS056585; Wed, 7 Jul 2021 20:10:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167KA5U8056582; Wed, 7 Jul 2021 20:10:05 GMT (envelope-from git) Date: Wed, 7 Jul 2021 20:10:05 GMT Message-Id: <202107072010.167KA5U8056582@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 0f4cb36cef40 - stable/13 - t_getgroups: No longer expected to fail MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 0f4cb36cef406cc7dae7100df82297f1a4b16585 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 20:10:06 -0000 The branch stable/13 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=0f4cb36cef406cc7dae7100df82297f1a4b16585 commit 0f4cb36cef406cc7dae7100df82297f1a4b16585 Author: Warner Losh AuthorDate: 2021-06-02 18:02:56 +0000 Commit: Warner Losh CommitDate: 2021-07-07 20:09:44 +0000 t_getgroups: No longer expected to fail Sponsored by: Netflix (cherry picked from commit bf26ea77553931c22e72ddf1f9df6fb51fcbadfe) --- contrib/netbsd-tests/lib/libc/sys/t_getgroups.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/contrib/netbsd-tests/lib/libc/sys/t_getgroups.c b/contrib/netbsd-tests/lib/libc/sys/t_getgroups.c index 7dedca445288..9a8ec8ede271 100644 --- a/contrib/netbsd-tests/lib/libc/sys/t_getgroups.c +++ b/contrib/netbsd-tests/lib/libc/sys/t_getgroups.c @@ -57,9 +57,6 @@ ATF_TC_BODY(getgroups_err, tc) errno = 0; -#ifdef __FreeBSD__ - atf_tc_expect_fail("Reported as kern/189941"); -#endif ATF_REQUIRE(getgroups(-1, gidset) == -1); ATF_REQUIRE(errno == EINVAL); } From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 20:24:02 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0D3B665C75A; Wed, 7 Jul 2021 20:24:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKrVT714mz4vdZ; Wed, 7 Jul 2021 20:24:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D925C1410E; Wed, 7 Jul 2021 20:24:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167KO1KW080440; Wed, 7 Jul 2021 20:24:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167KO1rR080439; Wed, 7 Jul 2021 20:24:01 GMT (envelope-from git) Date: Wed, 7 Jul 2021 20:24:01 GMT Message-Id: <202107072024.167KO1rR080439@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Houchard Subject: git: d2e48ad87d3e - stable/13 - arm64: Make sure COMPAT_FREEBSD32 handles thumb entry point. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cognet X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d2e48ad87d3e0f63f4ac73c524fb18afce4a2d53 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 20:24:02 -0000 The branch stable/13 has been updated by cognet: URL: https://cgit.FreeBSD.org/src/commit/?id=d2e48ad87d3e0f63f4ac73c524fb18afce4a2d53 commit d2e48ad87d3e0f63f4ac73c524fb18afce4a2d53 Author: Olivier Houchard AuthorDate: 2021-06-30 12:50:47 +0000 Commit: Olivier Houchard CommitDate: 2021-07-07 20:22:18 +0000 arm64: Make sure COMPAT_FREEBSD32 handles thumb entry point. If the entry point for the binary executed is a thumb 2 entry point, make sure we set the PSR_T bit, or the CPU will interpret it as arm32 code and bad things will happen. PR: 256899 MFC after: 1 week (cherry picked from commit 712c060c94fd447c91b0e6218c12a431206b487a) Signed-off-by: Olivier Houchard --- sys/arm64/arm64/elf32_machdep.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/arm64/arm64/elf32_machdep.c b/sys/arm64/arm64/elf32_machdep.c index b9669616e1dd..6c98f7c839ca 100644 --- a/sys/arm64/arm64/elf32_machdep.c +++ b/sys/arm64/arm64/elf32_machdep.c @@ -255,6 +255,8 @@ freebsd32_setregs(struct thread *td, struct image_params *imgp, tf->tf_x[14] = imgp->entry_addr; tf->tf_elr = imgp->entry_addr; tf->tf_spsr = PSR_M_32; + if ((uint32_t)imgp->entry_addr & 1) + tf->tf_spsr |= PSR_T; #ifdef VFP vfp_reset_state(td, pcb); From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 20:24:03 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3B98965C53E; Wed, 7 Jul 2021 20:24:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKrVW0wYHz4vkV; Wed, 7 Jul 2021 20:24:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 07B4F13F25; Wed, 7 Jul 2021 20:24:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167KO2VK080464; Wed, 7 Jul 2021 20:24:02 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167KO2AW080463; Wed, 7 Jul 2021 20:24:02 GMT (envelope-from git) Date: Wed, 7 Jul 2021 20:24:02 GMT Message-Id: <202107072024.167KO2AW080463@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Houchard Subject: git: 82a1d67f018c - stable/13 - arm: Make sure we can handle a thumb entry point. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cognet X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 82a1d67f018c7178485166ae3e06f29bff292263 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 20:24:03 -0000 The branch stable/13 has been updated by cognet: URL: https://cgit.FreeBSD.org/src/commit/?id=82a1d67f018c7178485166ae3e06f29bff292263 commit 82a1d67f018c7178485166ae3e06f29bff292263 Author: Olivier Houchard AuthorDate: 2021-06-30 20:56:50 +0000 Commit: Olivier Houchard CommitDate: 2021-07-07 20:22:41 +0000 arm: Make sure we can handle a thumb entry point. Similarly to what's been done on arm64 with commit 712c060c94fd447c91b0e6218c12a431206b487a, when executing a binary, if the entry point is a thumb symbol, then make sure we set the PSL_T flag, otherwise the CPU will interpret it in ARM mode, and that will likely leads to an undefined instruction. PR: 256899 MFC after: 1 week (cherry picked from commit 8c3bd133dd52824e427e350c65eae1fd9eb5a3cd) Signed-off-by: Olivier Houchard --- sys/arm/arm/machdep.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/arm/arm/machdep.c b/sys/arm/arm/machdep.c index 3296adaaa44b..ea5b5a0de251 100644 --- a/sys/arm/arm/machdep.c +++ b/sys/arm/arm/machdep.c @@ -351,6 +351,8 @@ exec_setregs(struct thread *td, struct image_params *imgp, uintptr_t stack) tf->tf_svc_lr = 0x77777777; tf->tf_pc = imgp->entry_addr; tf->tf_spsr = PSR_USR32_MODE; + if ((register_t)imgp->entry_addr & 1) + tf->tf_spsr |= PSR_T; } #ifdef VFP From owner-dev-commits-src-branches@freebsd.org Wed Jul 7 20:25:28 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AAB2765C8CC; Wed, 7 Jul 2021 20:25:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GKrX83fSwz3Cd9; Wed, 7 Jul 2021 20:25:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 56AB614119; Wed, 7 Jul 2021 20:25:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 167KPSl1080705; Wed, 7 Jul 2021 20:25:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 167KPSqH080704; Wed, 7 Jul 2021 20:25:28 GMT (envelope-from git) Date: Wed, 7 Jul 2021 20:25:28 GMT Message-Id: <202107072025.167KPSqH080704@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Olivier Houchard Subject: git: 9bc7bbcf214b - stable/12 - arm: Make sure we can handle a thumb entry point. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cognet X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 9bc7bbcf214b9e15b67e42706cef770ede541913 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jul 2021 20:25:28 -0000 The branch stable/12 has been updated by cognet: URL: https://cgit.FreeBSD.org/src/commit/?id=9bc7bbcf214b9e15b67e42706cef770ede541913 commit 9bc7bbcf214b9e15b67e42706cef770ede541913 Author: Olivier Houchard AuthorDate: 2021-06-30 20:56:50 +0000 Commit: Olivier Houchard CommitDate: 2021-07-07 20:24:56 +0000 arm: Make sure we can handle a thumb entry point. Similarly to what's been done on arm64 with commit 712c060c94fd447c91b0e6218c12a431206b487a, when executing a binary, if the entry point is a thumb symbol, then make sure we set the PSL_T flag, otherwise the CPU will interpret it in ARM mode, and that will likely leads to an undefined instruction. PR: 256899 MFC after: 1 week (cherry picked from commit 8c3bd133dd52824e427e350c65eae1fd9eb5a3cd) Signed-off-by: Olivier Houchard --- sys/arm/arm/machdep.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/sys/arm/arm/machdep.c b/sys/arm/arm/machdep.c index f24df96ca3d6..a1354f303b3a 100644 --- a/sys/arm/arm/machdep.c +++ b/sys/arm/arm/machdep.c @@ -425,6 +425,8 @@ exec_setregs(struct thread *td, struct image_params *imgp, u_long stack) tf->tf_svc_lr = 0x77777777; tf->tf_pc = imgp->entry_addr; tf->tf_spsr = PSR_USR32_MODE; + if ((register_t)imgp->entry_addr & 1) + tf->tf_spsr |= PSR_T; } From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 05:10:25 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 301676626FD; Thu, 8 Jul 2021 05:10:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GL49s0wTSz5820; Thu, 8 Jul 2021 05:10:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 07DC91AAF8; Thu, 8 Jul 2021 05:10:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1685AOHR079663; Thu, 8 Jul 2021 05:10:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1685AOAM079662; Thu, 8 Jul 2021 05:10:24 GMT (envelope-from git) Date: Thu, 8 Jul 2021 05:10:24 GMT Message-Id: <202107080510.1685AOAM079662@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 427654eb6a14 - stable/12 - mergemaster: handle symbolic links during update. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 427654eb6a1435acb15c272731fd9518b11961d7 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 05:10:25 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=427654eb6a1435acb15c272731fd9518b11961d7 commit 427654eb6a1435acb15c272731fd9518b11961d7 Author: Warner Losh AuthorDate: 2020-11-18 19:22:24 +0000 Commit: Warner Losh CommitDate: 2021-07-08 05:10:05 +0000 mergemaster: handle symbolic links during update. /etc/os-release is now a symbolic link to a generated file. Make mergemaster cope with symbolic links generically. I'm no longer a big mergemaster user, so this has only been lightly tested by me, though Kimura-san has ran it through its paces. Submitted by: Yasushiro KIMURA-san PR: 242212 MFC After: 2 weeks (cherry picked from commit 30a56f9ef72705f2f3646ce4330cbc20675a465e) --- usr.sbin/mergemaster/mergemaster.sh | 118 +++++++++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 2 deletions(-) diff --git a/usr.sbin/mergemaster/mergemaster.sh b/usr.sbin/mergemaster/mergemaster.sh index 9360ac98bdae..7703e2856111 100755 --- a/usr.sbin/mergemaster/mergemaster.sh +++ b/usr.sbin/mergemaster/mergemaster.sh @@ -1093,6 +1093,7 @@ for COMPFILE in `find . | sort` ; do fi done +# Compare regular files for COMPFILE in `find . -type f | sort`; do # First, check to see if the file exists in DESTDIR. If not, the @@ -1182,6 +1183,119 @@ for COMPFILE in `find . -type f | sort`; do fi # Yes, the file still remains to be checked done # This is for the for way up there at the beginning of the comparison +ask_answer_for_symbolic_link () { + HANDLE_COMPSYMLINK='' + while true; do + echo " Use 'd' to delete the temporary ${COMPSYMLINK}" + echo " Use 'i' to install the temporary ${COMPSYMLINK}" + echo '' + echo " Default is to leave the temporary symbolic link to deal with by hand" + echo '' + echo -n "How should I deal with this? [Leave it for later] " + read HANDLE_COMPSYMLINK + case ${HANDLE_COMPSYMLINK} in + ''|[dDiI]) + break + ;; + *) + echo "invalid choice: ${HANDLE_COMPSYMLINK}" + echo '' + HANDLE_COMPSYMLINK='' + ;; + esac + done +} + +install_symbolic_link () { + rm -f ${DESTDIR}${COMPSYMLINK#.} > /dev/null 2>&1 + if [ -L ${DESTDIR}${COMPSYMLINK#.} ]; then + return 1 + fi + cp -a ${COMPSYMLINK} ${DESTDIR}${COMPSYMLINK#.} > /dev/null 2>&1 + if [ ! -L ${DESTDIR}${COMPSYMLINK#.} ]; then + return 1 + fi + return 0 +} + +handle_symbolic_link () { + case ${HANDLE_COMPSYMLINK} in + [dD]) + rm ${COMPSYMLINK} + echo '' + echo " *** Deleting ${COMPSYMLINK}" + echo '' + return 1 + ;; + [iI]) + echo '' + if install_symbolic_link; then + rm ${COMPSYMLINK} + echo " *** ${COMPSYMLINK} installed successfully" + return 2 + else + echo " *** Problem installing ${COMPSYMLINK}, it will remain to merge by hand" + return 3 + fi + echo '' + ;; + '') + echo '' + echo " *** ${COMPSYMLINK} will remain for your consideration" + echo '' + return 0 + ;; + esac +} + +# Compare symblic links +for COMPSYMLINK in `find . -type l | sort`; do + if [ ! -L "${DESTDIR}${COMPSYMLINK#.}" ]; then + if [ -n "${AUTO_RUN}" -a -z "${AUTO_INSTALL}" ]; then + echo " *** ${COMPSYMLINK} will remain for your consideration" + continue + else + echo '' + echo " *** There is no installed version of ${COMPSYMLINK}" + echo '' + if [ -n "${AUTO_INSTALL}" ]; then + HANDLE_COMPSYMLINK="i" + else + ask_answer_for_symbolic_link + fi + handle_symbolic_link + if [ -n "${AUTO_INSTALL}" -a $? -eq 2 ]; then + AUTO_INSTALLED_FILES="${AUTO_INSTALLED_FILES} ${DESTDIR}${COMPSYMLINK#.} +" + fi + fi + elif [ $(readlink ${COMPSYMLINK}) = $(readlink ${DESTDIR}${COMPSYMLINK#.}) ]; then + echo " *** Temp ${COMPSYMLINK} and installed are the same, deleting" + rm ${COMPSYMLINK} + else + if [ -n "${AUTO_RUN}" -a -z "${AUTO_UPGRADE}" ]; then + echo " *** ${COMPSYMLINK} will remain for your consideration" + continue + else + echo '' + echo " *** Target of temp symbolic link is differnt from that of installed one" + echo " Temp (${COMPSYMLINK}): $(readlink ${COMPSYMLINK})" + echo " Installed (${DESTDIR}${COMPSYMLINK#.})): $(readlink ${DESTDIR}${COMPSYMLINK#.})" + echo '' + if [ -n "${AUTO_UPGRADE}" ]; then + HANDLE_COMPSYMLINK="i" + else + ask_answer_for_symbolic_link + fi + handle_symbolic_link + if [ -n "${AUTO_UPGRADE}" -a $? -eq 2 ]; then + AUTO_UPGRADED_FILES="${AUTO_UPGRADED_FILES} ${DESTDIR}${COMPSYMLINK#.} +" + fi + fi + fi +done + echo '' echo "*** Comparison complete" @@ -1193,10 +1307,10 @@ fi echo '' -TEST_FOR_FILES=`find ${TEMPROOT} -type f -size +0 2>/dev/null` +TEST_FOR_FILES=`find ${TEMPROOT} -type f -size +0 -or -type l 2>/dev/null` if [ -n "${TEST_FOR_FILES}" ]; then echo "*** Files that remain for you to merge by hand:" - find "${TEMPROOT}" -type f -size +0 | sort + find "${TEMPROOT}" -type f -size +0 -or -type l | sort echo '' case "${AUTO_RUN}" in From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 05:13:29 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9845C662EB9; Thu, 8 Jul 2021 05:13:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GL4FP2vYxz58T4; Thu, 8 Jul 2021 05:13:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4C3FF1AFE1; Thu, 8 Jul 2021 05:13:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1685DTXF083957; Thu, 8 Jul 2021 05:13:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1685DTrK083956; Thu, 8 Jul 2021 05:13:29 GMT (envelope-from git) Date: Thu, 8 Jul 2021 05:13:29 GMT Message-Id: <202107080513.1685DTrK083956@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 27e18c06f08b - stable/11 - mergemaster: handle symbolic links during update. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/11 X-Git-Reftype: branch X-Git-Commit: 27e18c06f08b413cd6d90922971194ae65921805 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 05:13:29 -0000 The branch stable/11 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=27e18c06f08b413cd6d90922971194ae65921805 commit 27e18c06f08b413cd6d90922971194ae65921805 Author: Warner Losh AuthorDate: 2020-11-18 19:22:24 +0000 Commit: Warner Losh CommitDate: 2021-07-08 05:13:18 +0000 mergemaster: handle symbolic links during update. /etc/os-release is now a symbolic link to a generated file. Make mergemaster cope with symbolic links generically. I'm no longer a big mergemaster user, so this has only been lightly tested by me, though Kimura-san has ran it through its paces. Submitted by: Yasushiro KIMURA-san PR: 242212 MFC After: 2 weeks (cherry picked from commit 30a56f9ef72705f2f3646ce4330cbc20675a465e) --- usr.sbin/mergemaster/mergemaster.sh | 118 +++++++++++++++++++++++++++++++++++- 1 file changed, 116 insertions(+), 2 deletions(-) diff --git a/usr.sbin/mergemaster/mergemaster.sh b/usr.sbin/mergemaster/mergemaster.sh index 086ee11db0f6..fdccf81802ea 100755 --- a/usr.sbin/mergemaster/mergemaster.sh +++ b/usr.sbin/mergemaster/mergemaster.sh @@ -1072,6 +1072,7 @@ for COMPFILE in `find . | sort` ; do fi done +# Compare regular files for COMPFILE in `find . -type f | sort`; do # First, check to see if the file exists in DESTDIR. If not, the @@ -1161,6 +1162,119 @@ for COMPFILE in `find . -type f | sort`; do fi # Yes, the file still remains to be checked done # This is for the for way up there at the beginning of the comparison +ask_answer_for_symbolic_link () { + HANDLE_COMPSYMLINK='' + while true; do + echo " Use 'd' to delete the temporary ${COMPSYMLINK}" + echo " Use 'i' to install the temporary ${COMPSYMLINK}" + echo '' + echo " Default is to leave the temporary symbolic link to deal with by hand" + echo '' + echo -n "How should I deal with this? [Leave it for later] " + read HANDLE_COMPSYMLINK + case ${HANDLE_COMPSYMLINK} in + ''|[dDiI]) + break + ;; + *) + echo "invalid choice: ${HANDLE_COMPSYMLINK}" + echo '' + HANDLE_COMPSYMLINK='' + ;; + esac + done +} + +install_symbolic_link () { + rm -f ${DESTDIR}${COMPSYMLINK#.} > /dev/null 2>&1 + if [ -L ${DESTDIR}${COMPSYMLINK#.} ]; then + return 1 + fi + cp -a ${COMPSYMLINK} ${DESTDIR}${COMPSYMLINK#.} > /dev/null 2>&1 + if [ ! -L ${DESTDIR}${COMPSYMLINK#.} ]; then + return 1 + fi + return 0 +} + +handle_symbolic_link () { + case ${HANDLE_COMPSYMLINK} in + [dD]) + rm ${COMPSYMLINK} + echo '' + echo " *** Deleting ${COMPSYMLINK}" + echo '' + return 1 + ;; + [iI]) + echo '' + if install_symbolic_link; then + rm ${COMPSYMLINK} + echo " *** ${COMPSYMLINK} installed successfully" + return 2 + else + echo " *** Problem installing ${COMPSYMLINK}, it will remain to merge by hand" + return 3 + fi + echo '' + ;; + '') + echo '' + echo " *** ${COMPSYMLINK} will remain for your consideration" + echo '' + return 0 + ;; + esac +} + +# Compare symblic links +for COMPSYMLINK in `find . -type l | sort`; do + if [ ! -L "${DESTDIR}${COMPSYMLINK#.}" ]; then + if [ -n "${AUTO_RUN}" -a -z "${AUTO_INSTALL}" ]; then + echo " *** ${COMPSYMLINK} will remain for your consideration" + continue + else + echo '' + echo " *** There is no installed version of ${COMPSYMLINK}" + echo '' + if [ -n "${AUTO_INSTALL}" ]; then + HANDLE_COMPSYMLINK="i" + else + ask_answer_for_symbolic_link + fi + handle_symbolic_link + if [ -n "${AUTO_INSTALL}" -a $? -eq 2 ]; then + AUTO_INSTALLED_FILES="${AUTO_INSTALLED_FILES} ${DESTDIR}${COMPSYMLINK#.} +" + fi + fi + elif [ $(readlink ${COMPSYMLINK}) = $(readlink ${DESTDIR}${COMPSYMLINK#.}) ]; then + echo " *** Temp ${COMPSYMLINK} and installed are the same, deleting" + rm ${COMPSYMLINK} + else + if [ -n "${AUTO_RUN}" -a -z "${AUTO_UPGRADE}" ]; then + echo " *** ${COMPSYMLINK} will remain for your consideration" + continue + else + echo '' + echo " *** Target of temp symbolic link is differnt from that of installed one" + echo " Temp (${COMPSYMLINK}): $(readlink ${COMPSYMLINK})" + echo " Installed (${DESTDIR}${COMPSYMLINK#.})): $(readlink ${DESTDIR}${COMPSYMLINK#.})" + echo '' + if [ -n "${AUTO_UPGRADE}" ]; then + HANDLE_COMPSYMLINK="i" + else + ask_answer_for_symbolic_link + fi + handle_symbolic_link + if [ -n "${AUTO_UPGRADE}" -a $? -eq 2 ]; then + AUTO_UPGRADED_FILES="${AUTO_UPGRADED_FILES} ${DESTDIR}${COMPSYMLINK#.} +" + fi + fi + fi +done + echo '' echo "*** Comparison complete" @@ -1172,10 +1286,10 @@ fi echo '' -TEST_FOR_FILES=`find ${TEMPROOT} -type f -size +0 2>/dev/null` +TEST_FOR_FILES=`find ${TEMPROOT} -type f -size +0 -or -type l 2>/dev/null` if [ -n "${TEST_FOR_FILES}" ]; then echo "*** Files that remain for you to merge by hand:" - find "${TEMPROOT}" -type f -size +0 | sort + find "${TEMPROOT}" -type f -size +0 -or -type l | sort echo '' case "${AUTO_RUN}" in From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 05:51:22 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DEC166633A5; Thu, 8 Jul 2021 05:51:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GL5565jdCz3F0m; Thu, 8 Jul 2021 05:51:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id ACBCF1B55A; Thu, 8 Jul 2021 05:51:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1685pMlu035365; Thu, 8 Jul 2021 05:51:22 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1685pMOb035364; Thu, 8 Jul 2021 05:51:22 GMT (envelope-from git) Date: Thu, 8 Jul 2021 05:51:22 GMT Message-Id: <202107080551.1685pMOb035364@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: c978000da5c6 - stable/12 - Mount and unmount devfs around calls to add packages. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: c978000da5c6a909e16c0996a93d91068cafb03e Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 05:51:22 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=c978000da5c6a909e16c0996a93d91068cafb03e commit c978000da5c6a909e16c0996a93d91068cafb03e Author: Warner Losh AuthorDate: 2019-06-21 03:49:36 +0000 Commit: Warner Losh CommitDate: 2021-07-08 05:51:03 +0000 Mount and unmount devfs around calls to add packages. pkg now uses /dev/null for some of its operations. NanoBSD's packaging stuff didn't mount that for the chroot it ran in, so any config that added packages would see the error: pkg: Cannot open /dev/null:No such file or directory when trying to actually add those packages. It's easy enough for nanobsd to mount /dev and it won't hurt anything that was already working and may help things that weren't (like this). I moved the mount/unmount pair to be in the right push/pop order from the submitted patch. PR: 238727 Submitted by: mike tancsa Tested by: Karl Denninger (cherry picked from commit 804b78634d543667a37ac459e692fe42c9dd30e8) --- tools/tools/nanobsd/defaults.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tools/tools/nanobsd/defaults.sh b/tools/tools/nanobsd/defaults.sh index 0002373be658..86acab65f67d 100755 --- a/tools/tools/nanobsd/defaults.sh +++ b/tools/tools/nanobsd/defaults.sh @@ -778,8 +778,9 @@ cust_pkgng ( ) ( # Mount packages into chroot mkdir -p ${NANO_WORLDDIR}/_.p mount -t nullfs -o noatime -o ro ${NANO_PACKAGE_DIR} ${NANO_WORLDDIR}/_.p + mount -t devfs devfs ${NANO_WORLDDIR}/dev - trap "umount ${NANO_WORLDDIR}/_.p ; rm -rf ${NANO_WORLDDIR}/_.p" 1 2 15 EXIT + trap "umount ${NANO_WORLDDIR}/dev; umount ${NANO_WORLDDIR}/_.p ; rm -rf ${NANO_WORLDDIR}/_.p" 1 2 15 EXIT # Install pkg-* package CR "${PKGCMD} add /_.p/${_NANO_PKG_PACKAGE}" @@ -804,6 +805,7 @@ cust_pkgng ( ) ( CR0 "${PKGCMD} info" trap - 1 2 15 EXIT + umount ${NANO_WORLDDIR}/dev umount ${NANO_WORLDDIR}/_.p rm -rf ${NANO_WORLDDIR}/_.p ) From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:29 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EEE05668213; Thu, 8 Jul 2021 11:50:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3T6F76z4k11; Thu, 8 Jul 2021 11:50:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BE16C205C1; Thu, 8 Jul 2021 11:50:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoT3n011093; Thu, 8 Jul 2021 11:50:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoTSW011092; Thu, 8 Jul 2021 11:50:29 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:29 GMT Message-Id: <202107081150.168BoTSW011092@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 1975b8347df6 - stable/13 - dummynet: fix sysctls MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 1975b8347df66301e4760df04f4fa74e8619fa07 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:30 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=1975b8347df66301e4760df04f4fa74e8619fa07 commit 1975b8347df66301e4760df04f4fa74e8619fa07 Author: Kristof Provost AuthorDate: 2021-07-01 07:42:34 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:46:45 +0000 dummynet: fix sysctls The sysctl nodes which use V_dn_cfg must be marked as CTLFLAG_VNET so that we use the correct per-vnet offset PR: 256819 Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30974 (cherry picked from commit 8f76eebce424de064f65fec5cdd105446a2de3bd) --- sys/netpfil/ipfw/ip_dn_io.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/sys/netpfil/ipfw/ip_dn_io.c b/sys/netpfil/ipfw/ip_dn_io.c index 39bea3eb99dd..11357b44e05e 100644 --- a/sys/netpfil/ipfw/ip_dn_io.c +++ b/sys/netpfil/ipfw/ip_dn_io.c @@ -169,55 +169,55 @@ SYSCTL_PROC(_net_inet_ip_dummynet, OID_AUTO, pipe_byte_limit, 0, 0, sysctl_limits, "L", "Upper limit in bytes for pipe queue."); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, io_fast, - CTLFLAG_RW, DC(io_fast), 0, "Enable fast dummynet io."); + CTLFLAG_RW | CTLFLAG_VNET, DC(io_fast), 0, "Enable fast dummynet io."); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, debug, - CTLFLAG_RW, DC(debug), 0, "Dummynet debug level"); + CTLFLAG_RW | CTLFLAG_VNET, DC(debug), 0, "Dummynet debug level"); /* RED parameters */ SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_lookup_depth, - CTLFLAG_RD, DC(red_lookup_depth), 0, "Depth of RED lookup table"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_lookup_depth), 0, "Depth of RED lookup table"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_avg_pkt_size, - CTLFLAG_RD, DC(red_avg_pkt_size), 0, "RED Medium packet size"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_avg_pkt_size), 0, "RED Medium packet size"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_max_pkt_size, - CTLFLAG_RD, DC(red_max_pkt_size), 0, "RED Max packet size"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_max_pkt_size), 0, "RED Max packet size"); /* time adjustment */ SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_delta, - CTLFLAG_RD, DC(tick_delta), 0, "Last vs standard tick difference (usec)."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_delta), 0, "Last vs standard tick difference (usec)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_delta_sum, - CTLFLAG_RD, DC(tick_delta_sum), 0, "Accumulated tick difference (usec)."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_delta_sum), 0, "Accumulated tick difference (usec)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_adjustment, - CTLFLAG_RD, DC(tick_adjustment), 0, "Tick adjustments done."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_adjustment), 0, "Tick adjustments done."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_diff, - CTLFLAG_RD, DC(tick_diff), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_diff), 0, "Adjusted vs non-adjusted curr_time difference (ticks)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_lost, - CTLFLAG_RD, DC(tick_lost), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_lost), 0, "Number of ticks coalesced by dummynet taskqueue."); /* Drain parameters */ SYSCTL_UINT(_net_inet_ip_dummynet, OID_AUTO, expire, - CTLFLAG_RW, DC(expire), 0, "Expire empty queues/pipes"); + CTLFLAG_RW | CTLFLAG_VNET, DC(expire), 0, "Expire empty queues/pipes"); SYSCTL_UINT(_net_inet_ip_dummynet, OID_AUTO, expire_cycle, - CTLFLAG_RD, DC(expire_cycle), 0, "Expire cycle for queues/pipes"); + CTLFLAG_RD | CTLFLAG_VNET, DC(expire_cycle), 0, "Expire cycle for queues/pipes"); /* statistics */ SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, schk_count, - CTLFLAG_RD, DC(schk_count), 0, "Number of schedulers"); + CTLFLAG_RD | CTLFLAG_VNET, DC(schk_count), 0, "Number of schedulers"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, si_count, - CTLFLAG_RD, DC(si_count), 0, "Number of scheduler instances"); + CTLFLAG_RD | CTLFLAG_VNET, DC(si_count), 0, "Number of scheduler instances"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, fsk_count, - CTLFLAG_RD, DC(fsk_count), 0, "Number of flowsets"); + CTLFLAG_RD | CTLFLAG_VNET, DC(fsk_count), 0, "Number of flowsets"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, queue_count, - CTLFLAG_RD, DC(queue_count), 0, "Number of queues"); + CTLFLAG_RD | CTLFLAG_VNET, DC(queue_count), 0, "Number of queues"); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt, - CTLFLAG_RD, DC(io_pkt), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt), 0, "Number of packets passed to dummynet."); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt_fast, - CTLFLAG_RD, DC(io_pkt_fast), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt_fast), 0, "Number of packets bypassed dummynet scheduler."); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt_drop, - CTLFLAG_RD, DC(io_pkt_drop), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt_drop), 0, "Number of packets dropped by dummynet."); #undef DC SYSEND From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:31 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1610A667BE0; Thu, 8 Jul 2021 11:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3W00pfz4k13; Thu, 8 Jul 2021 11:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DD83F20689; Thu, 8 Jul 2021 11:50:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoUei011128; Thu, 8 Jul 2021 11:50:30 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoUPN011127; Thu, 8 Jul 2021 11:50:30 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:30 GMT Message-Id: <202107081150.168BoUPN011127@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 4f5a2009ad8a - stable/13 - ftp-proxy: Revert incorrect migration to libpfctl MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4f5a2009ad8ad98a457ddecb63fe1ed8a968226d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:31 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=4f5a2009ad8ad98a457ddecb63fe1ed8a968226d commit 4f5a2009ad8ad98a457ddecb63fe1ed8a968226d Author: Kristof Provost AuthorDate: 2021-07-01 15:16:10 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:46:54 +0000 ftp-proxy: Revert incorrect migration to libpfctl libpfctl supports creating rules, but not (yet) adding addresses to a pool. Adding addresses certainly does not work through adding a rule. PR: 256917 MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 8923ea6c867fd75b08b76883ec122c429a4018f9) --- contrib/pf/ftp-proxy/filter.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/contrib/pf/ftp-proxy/filter.c b/contrib/pf/ftp-proxy/filter.c index dad6324808bc..e4787985e99f 100644 --- a/contrib/pf/ftp-proxy/filter.c +++ b/contrib/pf/ftp-proxy/filter.c @@ -103,8 +103,7 @@ add_nat(u_int32_t id, struct sockaddr *src, struct sockaddr *dst, &satosin6(nat)->sin6_addr.s6_addr, 16); memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16); } - if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call, - pfticket, pfpool_ticket)) + if (ioctl(dev, DIOCADDADDR, &pfp) == -1) return (-1); pfrule.rpool.proxy_port[0] = nat_range_low; @@ -138,8 +137,7 @@ add_rdr(u_int32_t id, struct sockaddr *src, struct sockaddr *dst, &satosin6(rdr)->sin6_addr.s6_addr, 16); memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16); } - if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call, - pfticket, pfpool_ticket)) + if (ioctl(dev, DIOCADDADDR, &pfp) == -1) return (-1); pfrule.rpool.proxy_port[0] = rdr_port; From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:31 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6BC8F6680D6; Thu, 8 Jul 2021 11:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3W2P33z4k3V; Thu, 8 Jul 2021 11:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3AAF420155; Thu, 8 Jul 2021 11:50:31 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoVG6011211; Thu, 8 Jul 2021 11:50:31 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoVVw011210; Thu, 8 Jul 2021 11:50:31 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:31 GMT Message-Id: <202107081150.168BoVVw011210@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: aae1a783dd44 - stable/12 - dummynet: fix sysctls MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: aae1a783dd444060f9a9ed9cf0fbb0c3ba7c6800 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:31 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=aae1a783dd444060f9a9ed9cf0fbb0c3ba7c6800 commit aae1a783dd444060f9a9ed9cf0fbb0c3ba7c6800 Author: Kristof Provost AuthorDate: 2021-07-01 07:42:34 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:49:56 +0000 dummynet: fix sysctls The sysctl nodes which use V_dn_cfg must be marked as CTLFLAG_VNET so that we use the correct per-vnet offset PR: 256819 Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30974 (cherry picked from commit 8f76eebce424de064f65fec5cdd105446a2de3bd) --- sys/netpfil/ipfw/ip_dn_io.c | 38 +++++++++++++++++++------------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/sys/netpfil/ipfw/ip_dn_io.c b/sys/netpfil/ipfw/ip_dn_io.c index d276f85311d1..114cfac440ee 100644 --- a/sys/netpfil/ipfw/ip_dn_io.c +++ b/sys/netpfil/ipfw/ip_dn_io.c @@ -165,55 +165,55 @@ SYSCTL_PROC(_net_inet_ip_dummynet, OID_AUTO, pipe_byte_limit, CTLTYPE_LONG | CTLFLAG_RW, 0, 0, sysctl_limits, "L", "Upper limit in bytes for pipe queue."); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, io_fast, - CTLFLAG_RW, DC(io_fast), 0, "Enable fast dummynet io."); + CTLFLAG_RW | CTLFLAG_VNET, DC(io_fast), 0, "Enable fast dummynet io."); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, debug, - CTLFLAG_RW, DC(debug), 0, "Dummynet debug level"); + CTLFLAG_RW | CTLFLAG_VNET, DC(debug), 0, "Dummynet debug level"); /* RED parameters */ SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_lookup_depth, - CTLFLAG_RD, DC(red_lookup_depth), 0, "Depth of RED lookup table"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_lookup_depth), 0, "Depth of RED lookup table"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_avg_pkt_size, - CTLFLAG_RD, DC(red_avg_pkt_size), 0, "RED Medium packet size"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_avg_pkt_size), 0, "RED Medium packet size"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, red_max_pkt_size, - CTLFLAG_RD, DC(red_max_pkt_size), 0, "RED Max packet size"); + CTLFLAG_RD | CTLFLAG_VNET, DC(red_max_pkt_size), 0, "RED Max packet size"); /* time adjustment */ SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_delta, - CTLFLAG_RD, DC(tick_delta), 0, "Last vs standard tick difference (usec)."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_delta), 0, "Last vs standard tick difference (usec)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_delta_sum, - CTLFLAG_RD, DC(tick_delta_sum), 0, "Accumulated tick difference (usec)."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_delta_sum), 0, "Accumulated tick difference (usec)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_adjustment, - CTLFLAG_RD, DC(tick_adjustment), 0, "Tick adjustments done."); + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_adjustment), 0, "Tick adjustments done."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_diff, - CTLFLAG_RD, DC(tick_diff), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_diff), 0, "Adjusted vs non-adjusted curr_time difference (ticks)."); SYSCTL_LONG(_net_inet_ip_dummynet, OID_AUTO, tick_lost, - CTLFLAG_RD, DC(tick_lost), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(tick_lost), 0, "Number of ticks coalesced by dummynet taskqueue."); /* Drain parameters */ SYSCTL_UINT(_net_inet_ip_dummynet, OID_AUTO, expire, - CTLFLAG_RW, DC(expire), 0, "Expire empty queues/pipes"); + CTLFLAG_RW | CTLFLAG_VNET, DC(expire), 0, "Expire empty queues/pipes"); SYSCTL_UINT(_net_inet_ip_dummynet, OID_AUTO, expire_cycle, - CTLFLAG_RD, DC(expire_cycle), 0, "Expire cycle for queues/pipes"); + CTLFLAG_RD | CTLFLAG_VNET, DC(expire_cycle), 0, "Expire cycle for queues/pipes"); /* statistics */ SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, schk_count, - CTLFLAG_RD, DC(schk_count), 0, "Number of schedulers"); + CTLFLAG_RD | CTLFLAG_VNET, DC(schk_count), 0, "Number of schedulers"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, si_count, - CTLFLAG_RD, DC(si_count), 0, "Number of scheduler instances"); + CTLFLAG_RD | CTLFLAG_VNET, DC(si_count), 0, "Number of scheduler instances"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, fsk_count, - CTLFLAG_RD, DC(fsk_count), 0, "Number of flowsets"); + CTLFLAG_RD | CTLFLAG_VNET, DC(fsk_count), 0, "Number of flowsets"); SYSCTL_INT(_net_inet_ip_dummynet, OID_AUTO, queue_count, - CTLFLAG_RD, DC(queue_count), 0, "Number of queues"); + CTLFLAG_RD | CTLFLAG_VNET, DC(queue_count), 0, "Number of queues"); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt, - CTLFLAG_RD, DC(io_pkt), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt), 0, "Number of packets passed to dummynet."); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt_fast, - CTLFLAG_RD, DC(io_pkt_fast), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt_fast), 0, "Number of packets bypassed dummynet scheduler."); SYSCTL_ULONG(_net_inet_ip_dummynet, OID_AUTO, io_pkt_drop, - CTLFLAG_RD, DC(io_pkt_drop), 0, + CTLFLAG_RD | CTLFLAG_VNET, DC(io_pkt_drop), 0, "Number of packets dropped by dummynet."); #undef DC SYSEND From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:33 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 28A8B667E3A; Thu, 8 Jul 2021 11:50:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3X49Vvz4k3f; Thu, 8 Jul 2021 11:50:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 469D720635; Thu, 8 Jul 2021 11:50:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoW3a011260; Thu, 8 Jul 2021 11:50:32 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoW7l011259; Thu, 8 Jul 2021 11:50:32 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:32 GMT Message-Id: <202107081150.168BoW7l011259@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: d442d97b7632 - stable/12 - ftp-proxy: Revert incorrect migration to libpfctl MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: d442d97b7632628ac3cf3e591ddbd8b872c14818 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:33 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=d442d97b7632628ac3cf3e591ddbd8b872c14818 commit d442d97b7632628ac3cf3e591ddbd8b872c14818 Author: Kristof Provost AuthorDate: 2021-07-01 15:16:10 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:49:56 +0000 ftp-proxy: Revert incorrect migration to libpfctl libpfctl supports creating rules, but not (yet) adding addresses to a pool. Adding addresses certainly does not work through adding a rule. PR: 256917 MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit 8923ea6c867fd75b08b76883ec122c429a4018f9) --- contrib/pf/ftp-proxy/filter.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/contrib/pf/ftp-proxy/filter.c b/contrib/pf/ftp-proxy/filter.c index dad6324808bc..e4787985e99f 100644 --- a/contrib/pf/ftp-proxy/filter.c +++ b/contrib/pf/ftp-proxy/filter.c @@ -103,8 +103,7 @@ add_nat(u_int32_t id, struct sockaddr *src, struct sockaddr *dst, &satosin6(nat)->sin6_addr.s6_addr, 16); memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16); } - if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call, - pfticket, pfpool_ticket)) + if (ioctl(dev, DIOCADDADDR, &pfp) == -1) return (-1); pfrule.rpool.proxy_port[0] = nat_range_low; @@ -138,8 +137,7 @@ add_rdr(u_int32_t id, struct sockaddr *src, struct sockaddr *dst, &satosin6(rdr)->sin6_addr.s6_addr, 16); memset(&pfp.addr.addr.v.a.mask.addr8, 255, 16); } - if (pfctl_add_rule(dev, &pfrule, pfanchor, pfanchor_call, - pfticket, pfpool_ticket)) + if (ioctl(dev, DIOCADDADDR, &pfp) == -1) return (-1); pfrule.rpool.proxy_port[0] = rdr_port; From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:33 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id F2FBF668063; Thu, 8 Jul 2021 11:50:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3X2KZKz4kCm; Thu, 8 Jul 2021 11:50:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0F048205C2; Thu, 8 Jul 2021 11:50:32 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoV89011236; Thu, 8 Jul 2021 11:50:31 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoVfd011235; Thu, 8 Jul 2021 11:50:31 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:31 GMT Message-Id: <202107081150.168BoVfd011235@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: f6d448caf698 - stable/13 - pf tests: ftp-proxy test MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f6d448caf69889eca9e41a0815db65a22fe7652d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:33 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=f6d448caf69889eca9e41a0815db65a22fe7652d commit f6d448caf69889eca9e41a0815db65a22fe7652d Author: Kristof Provost AuthorDate: 2021-07-01 15:15:36 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:47:09 +0000 pf tests: ftp-proxy test Basic test case for ftp-proxy PR: 256917 MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit dd82fd3543022017b84007ac1a0d45fc683f9850) --- tests/sys/netpfil/pf/Makefile | 1 + tests/sys/netpfil/pf/proxy.sh | 95 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Makefile index e21f78e761b6..b693ca63adcb 100644 --- a/tests/sys/netpfil/pf/Makefile +++ b/tests/sys/netpfil/pf/Makefile @@ -17,6 +17,7 @@ ATF_TESTS_SH+= anchor \ nat \ pass_block \ pfsync \ + proxy \ rdr \ route_to \ rules_counter \ diff --git a/tests/sys/netpfil/pf/proxy.sh b/tests/sys/netpfil/pf/proxy.sh new file mode 100644 index 000000000000..bb5286103e85 --- /dev/null +++ b/tests/sys/netpfil/pf/proxy.sh @@ -0,0 +1,95 @@ +# $FreeBSD$ +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +. $(atf_get_srcdir)/utils.subr + +common_dir=$(atf_get_srcdir)/../common + +atf_test_case "ftp" "cleanup" +ftp_head() +{ + atf_set descr 'Test ftp-proxy' + atf_set require.user root + atf_set require.progs twistd +} + +ftp_body() +{ + pft_init + + epair_client=$(vnet_mkepair) + epair_link=$(vnet_mkepair) + + ifconfig ${epair_client}a 192.0.2.2/24 up + route add -net 198.51.100.0/24 192.0.2.1 + + vnet_mkjail fwd ${epair_client}b ${epair_link}a + jexec fwd ifconfig ${epair_client}b 192.0.2.1/24 up + jexec fwd ifconfig ${epair_link}a 198.51.100.1/24 up + jexec fwd ifconfig lo0 127.0.0.1/8 up + jexec fwd sysctl net.inet.ip.forwarding=1 + + vnet_mkjail srv ${epair_link}b + jexec srv ifconfig ${epair_link}b 198.51.100.2/24 up + jexec srv route add default 198.51.100.1 + + # Start FTP server in srv + jexec srv twistd ftp -r `pwd` -p 21 + + # Sanity check + atf_check -s exit:0 -o ignore ping -c 1 198.51.100.2 + + jexec fwd pfctl -e + pft_set_rules fwd \ + "nat on ${epair_link}a inet from 192.0.2.0/24 to any -> (${epair_link}a)" \ + "nat-anchor \"ftp-proxy/*\"" \ + "rdr-anchor \"ftp-proxy/*\"" \ + "rdr pass on ${epair_client}b proto tcp from 192.0.2.0/24 to any port 21 -> 127.0.0.1 port 8021" \ + "anchor \"ftp-proxy/*\"" \ + "pass out proto tcp from 127.0.0.1 to any port 21" + jexec fwd /usr/sbin/ftp-proxy + + # Create a dummy file to download + echo 'foo' > remote.txt + echo 'get remote.txt local.txt' | ftp -a 198.51.100.2 + + # Compare the downloaded file to the original + if ! diff -q local.txt remote.txt; + then + atf_fail 'Failed to retrieve file' + fi +} + +ftp_cleanup() +{ + pft_cleanup +} + +atf_init_test_cases() +{ + atf_add_test_case "ftp" +} From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 11:50:33 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C7440667BE6; Thu, 8 Jul 2021 11:50:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLF3Y4Mn6z4jxM; Thu, 8 Jul 2021 11:50:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6FF34203CA; Thu, 8 Jul 2021 11:50:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168BoXh8011338; Thu, 8 Jul 2021 11:50:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168BoX5v011337; Thu, 8 Jul 2021 11:50:33 GMT (envelope-from git) Date: Thu, 8 Jul 2021 11:50:33 GMT Message-Id: <202107081150.168BoX5v011337@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: e26de428fa54 - stable/12 - pf tests: ftp-proxy test MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e26de428fa542c0fd62a696e2d721ff6f8dbc5db Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 11:50:33 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=e26de428fa542c0fd62a696e2d721ff6f8dbc5db commit e26de428fa542c0fd62a696e2d721ff6f8dbc5db Author: Kristof Provost AuthorDate: 2021-07-01 15:15:36 +0000 Commit: Kristof Provost CommitDate: 2021-07-08 06:49:56 +0000 pf tests: ftp-proxy test Basic test case for ftp-proxy PR: 256917 MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") (cherry picked from commit dd82fd3543022017b84007ac1a0d45fc683f9850) --- tests/sys/netpfil/pf/Makefile | 1 + tests/sys/netpfil/pf/proxy.sh | 95 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 96 insertions(+) diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Makefile index f67f510fbf29..a8c5aac4af08 100644 --- a/tests/sys/netpfil/pf/Makefile +++ b/tests/sys/netpfil/pf/Makefile @@ -23,6 +23,7 @@ ATF_TESTS_SH+= anchor \ synproxy \ set_skip \ pfsync \ + proxy \ table \ tos diff --git a/tests/sys/netpfil/pf/proxy.sh b/tests/sys/netpfil/pf/proxy.sh new file mode 100644 index 000000000000..bb5286103e85 --- /dev/null +++ b/tests/sys/netpfil/pf/proxy.sh @@ -0,0 +1,95 @@ +# $FreeBSD$ +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +. $(atf_get_srcdir)/utils.subr + +common_dir=$(atf_get_srcdir)/../common + +atf_test_case "ftp" "cleanup" +ftp_head() +{ + atf_set descr 'Test ftp-proxy' + atf_set require.user root + atf_set require.progs twistd +} + +ftp_body() +{ + pft_init + + epair_client=$(vnet_mkepair) + epair_link=$(vnet_mkepair) + + ifconfig ${epair_client}a 192.0.2.2/24 up + route add -net 198.51.100.0/24 192.0.2.1 + + vnet_mkjail fwd ${epair_client}b ${epair_link}a + jexec fwd ifconfig ${epair_client}b 192.0.2.1/24 up + jexec fwd ifconfig ${epair_link}a 198.51.100.1/24 up + jexec fwd ifconfig lo0 127.0.0.1/8 up + jexec fwd sysctl net.inet.ip.forwarding=1 + + vnet_mkjail srv ${epair_link}b + jexec srv ifconfig ${epair_link}b 198.51.100.2/24 up + jexec srv route add default 198.51.100.1 + + # Start FTP server in srv + jexec srv twistd ftp -r `pwd` -p 21 + + # Sanity check + atf_check -s exit:0 -o ignore ping -c 1 198.51.100.2 + + jexec fwd pfctl -e + pft_set_rules fwd \ + "nat on ${epair_link}a inet from 192.0.2.0/24 to any -> (${epair_link}a)" \ + "nat-anchor \"ftp-proxy/*\"" \ + "rdr-anchor \"ftp-proxy/*\"" \ + "rdr pass on ${epair_client}b proto tcp from 192.0.2.0/24 to any port 21 -> 127.0.0.1 port 8021" \ + "anchor \"ftp-proxy/*\"" \ + "pass out proto tcp from 127.0.0.1 to any port 21" + jexec fwd /usr/sbin/ftp-proxy + + # Create a dummy file to download + echo 'foo' > remote.txt + echo 'get remote.txt local.txt' | ftp -a 198.51.100.2 + + # Compare the downloaded file to the original + if ! diff -q local.txt remote.txt; + then + atf_fail 'Failed to retrieve file' + fi +} + +ftp_cleanup() +{ + pft_cleanup +} + +atf_init_test_cases() +{ + atf_add_test_case "ftp" +} From owner-dev-commits-src-branches@freebsd.org Thu Jul 8 22:26:28 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8FFD0654BE4; Thu, 8 Jul 2021 22:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLW9J3HVnz3NpJ; Thu, 8 Jul 2021 22:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 58C481035; Thu, 8 Jul 2021 22:26:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 168MQSTO061066; Thu, 8 Jul 2021 22:26:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 168MQSwP061065; Thu, 8 Jul 2021 22:26:28 GMT (envelope-from git) Date: Thu, 8 Jul 2021 22:26:28 GMT Message-Id: <202107082226.168MQSwP061065@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: f00dd44f0a24 - stable/13 - devmatch: improve naming of devmatch config variable MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: f00dd44f0a249019e5d4045c296bb17086c8f5e2 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jul 2021 22:26:28 -0000 The branch stable/13 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=f00dd44f0a249019e5d4045c296bb17086c8f5e2 commit f00dd44f0a249019e5d4045c296bb17086c8f5e2 Author: Ceri Davies AuthorDate: 2021-06-18 12:17:30 +0000 Commit: Warner Losh CommitDate: 2021-07-08 22:14:07 +0000 devmatch: improve naming of devmatch config variable Accept the old rc.conf variable if the new one is not present for compatability. Approved by: imp Differential Revision: https://reviews.freebsd.org/D30806 (cherry picked from commit c43b0081faab742eb93c3d064b552b65f926b86e) --- libexec/rc/rc.conf | 2 +- libexec/rc/rc.d/devmatch | 4 ++-- share/man/man5/rc.conf.5 | 10 +++++++++- 3 files changed, 12 insertions(+), 4 deletions(-) diff --git a/libexec/rc/rc.conf b/libexec/rc/rc.conf index 53bd8c81c2d7..89bb8e65d7b8 100644 --- a/libexec/rc/rc.conf +++ b/libexec/rc/rc.conf @@ -45,7 +45,7 @@ ddb_config="/etc/ddb.conf" # ddb(8) config file. devd_enable="YES" # Run devd, to trigger programs on device tree changes. devd_flags="" # Additional flags for devd(8). devmatch_enable="YES" # Demand load kernel modules based on device ids. -devmatch_blacklist="" # List of modules (w/o .ko) to exclude from devmatch. +devmatch_blocklist="" # List of modules (w/o .ko) to exclude from devmatch. #kld_list="" # Kernel modules to load after local disks are mounted kldxref_enable="YES" # Build linker.hints files with kldxref(8). kldxref_clobber="NO" # Overwrite old linker.hints at boot. diff --git a/libexec/rc/rc.d/devmatch b/libexec/rc/rc.d/devmatch index 491bc94c4a16..0b2e3719dabb 100755 --- a/libexec/rc/rc.d/devmatch +++ b/libexec/rc/rc.d/devmatch @@ -59,9 +59,9 @@ devmatch_start() # or drivers that have symbolic links that # confuse devmatch by running it -n. # Finally, we filter out all items in the - # devmactch_blacklist. + # devmatch_blocklist. devctl freeze - x=$(echo ${devmatch_blacklist} | tr ' ' '#') + x=$(echo ${devmatch_blocklist:-${devmatch_blacklist}} | tr ' ' '#') for m in ${list}; do case "#${x}#" in *"#${m}#"*) continue ;; diff --git a/share/man/man5/rc.conf.5 b/share/man/man5/rc.conf.5 index 473406660b8b..5f0022ca1a78 100644 --- a/share/man/man5/rc.conf.5 +++ b/share/man/man5/rc.conf.5 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 10, 2021 +.Dd July 8, 2021 .Dt RC.CONF 5 .Os .Sh NAME @@ -257,8 +257,16 @@ If set to .Dq Li NO , disable auto-loading of kernel modules with .Xr devmatch 8 . +.It Va devmatch_blocklist +.Pq Vt str +A whitespace-separated list of kernel modules to be ignored by +.Xr devmatch 8 . .It Va devmatch_blacklist .Pq Vt str +This variable is deprecated. +Use +.Va devmatch_blocklist +instead. A whitespace-separated list of kernel modules to be ignored by .Xr devmatch 8 . .It Va kld_list From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 01:38:37 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C9436573A8; Fri, 9 Jul 2021 01:38:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLbR12dXcz4bLT; Fri, 9 Jul 2021 01:38:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 418FF3ABE; Fri, 9 Jul 2021 01:38:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1691cbRm014749; Fri, 9 Jul 2021 01:38:37 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1691cbYk014748; Fri, 9 Jul 2021 01:38:37 GMT (envelope-from git) Date: Fri, 9 Jul 2021 01:38:37 GMT Message-Id: <202107090138.1691cbYk014748@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 4a58aca40df9 - stable/13 - Allow sleepq_signal() to drop the lock. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4a58aca40df9a5a5e5db430f3744a2712f955ed7 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 01:38:37 -0000 The branch stable/13 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=4a58aca40df9a5a5e5db430f3744a2712f955ed7 commit 4a58aca40df9a5a5e5db430f3744a2712f955ed7 Author: Alexander Motin AuthorDate: 2021-06-25 17:52:58 +0000 Commit: Alexander Motin CommitDate: 2021-07-09 01:38:32 +0000 Allow sleepq_signal() to drop the lock. Introduce SLEEPQ_DROP sleepq_signal() flag, allowing one to drop the sleep queue chain lock before returning. Reduced lock scope allows significantly reduce lock contention inside taskqueue_enqueue() for ZFS worker threads doing ~350K disk reads/s on 40-thread system. MFC after: 2 weeks Sponsored by: iXsystems, Inc. (cherry picked from commit 6df35af4d85c6311d8e762521580e7176b69394e) --- sys/kern/kern_synch.c | 8 +++----- sys/kern/subr_sleepqueue.c | 8 ++++++-- sys/sys/sleepqueue.h | 1 + 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/sys/kern/kern_synch.c b/sys/kern/kern_synch.c index 4c0491ab6e85..b63877e26b68 100644 --- a/sys/kern/kern_synch.c +++ b/sys/kern/kern_synch.c @@ -366,8 +366,7 @@ wakeup_one(const void *ident) int wakeup_swapper; sleepq_lock(ident); - wakeup_swapper = sleepq_signal(ident, SLEEPQ_SLEEP, 0, 0); - sleepq_release(ident); + wakeup_swapper = sleepq_signal(ident, SLEEPQ_SLEEP | SLEEPQ_DROP, 0, 0); if (wakeup_swapper) kick_proc0(); } @@ -378,9 +377,8 @@ wakeup_any(const void *ident) int wakeup_swapper; sleepq_lock(ident); - wakeup_swapper = sleepq_signal(ident, SLEEPQ_SLEEP | SLEEPQ_UNFAIR, - 0, 0); - sleepq_release(ident); + wakeup_swapper = sleepq_signal(ident, SLEEPQ_SLEEP | SLEEPQ_UNFAIR | + SLEEPQ_DROP, 0, 0); if (wakeup_swapper) kick_proc0(); } diff --git a/sys/kern/subr_sleepqueue.c b/sys/kern/subr_sleepqueue.c index 0718f01fa48a..b146a978a60c 100644 --- a/sys/kern/subr_sleepqueue.c +++ b/sys/kern/subr_sleepqueue.c @@ -927,8 +927,11 @@ sleepq_signal(const void *wchan, int flags, int pri, int queue) KASSERT(wchan != NULL, ("%s: invalid NULL wait channel", __func__)); MPASS((queue >= 0) && (queue < NR_SLEEPQS)); sq = sleepq_lookup(wchan); - if (sq == NULL) + if (sq == NULL) { + if (flags & SLEEPQ_DROP) + sleepq_release(wchan); return (0); + } KASSERT(sq->sq_type == (flags & SLEEPQ_TYPE), ("%s: mismatch between sleep/wakeup and cv_*", __func__)); @@ -961,7 +964,8 @@ sleepq_signal(const void *wchan, int flags, int pri, int queue) } } MPASS(besttd != NULL); - wakeup_swapper = sleepq_resume_thread(sq, besttd, pri, SRQ_HOLD); + wakeup_swapper = sleepq_resume_thread(sq, besttd, pri, + (flags & SLEEPQ_DROP) ? 0 : SRQ_HOLD); return (wakeup_swapper); } diff --git a/sys/sys/sleepqueue.h b/sys/sys/sleepqueue.h index 18c7568777b6..6715894ed1f3 100644 --- a/sys/sys/sleepqueue.h +++ b/sys/sys/sleepqueue.h @@ -85,6 +85,7 @@ struct thread; #define SLEEPQ_LK 0x04 /* Used by a lockmgr. */ #define SLEEPQ_INTERRUPTIBLE 0x100 /* Sleep is interruptible. */ #define SLEEPQ_UNFAIR 0x200 /* Unfair wakeup order. */ +#define SLEEPQ_DROP 0x400 /* Return without lock held. */ void init_sleepqueues(void); int sleepq_abort(struct thread *td, int intrval); From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:24 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 222A965FAB4; Fri, 9 Jul 2021 13:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5N0WFpz3Jc4; Fri, 9 Jul 2021 13:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EE01E1572D; Fri, 9 Jul 2021 13:24:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DONQp063253; Fri, 9 Jul 2021 13:24:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DONuV063251; Fri, 9 Jul 2021 13:24:23 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:23 GMT Message-Id: <202107091324.169DONuV063251@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: fc4e7ccde0b6 - stable/12 - pf tests: Stress state retrieval MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: fc4e7ccde0b677d544a0f5ad42515b706071d093 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:24 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=fc4e7ccde0b677d544a0f5ad42515b706071d093 commit fc4e7ccde0b677d544a0f5ad42515b706071d093 Author: Kristof Provost AuthorDate: 2021-06-28 10:48:20 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:07:48 +0000 pf tests: Stress state retrieval Create and retrieve 20.000 states. There have been issues with nvlists causing very slow state retrieval. We don't impose a specific limit on the time required to retrieve the states, but do log it. In excessive cases the Kyua timeout will fail this test. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30943 (cherry picked from commit d8d43b2de1fa679179f7089cb3c31e6780ec82af) --- tests/sys/netpfil/common/Makefile | 2 + tests/sys/netpfil/common/pft_synflood.py | 62 ++++++++++++++++++++++++ tests/sys/netpfil/pf/Makefile | 1 + tests/sys/netpfil/pf/get_state.sh | 81 ++++++++++++++++++++++++++++++++ 4 files changed, 146 insertions(+) diff --git a/tests/sys/netpfil/common/Makefile b/tests/sys/netpfil/common/Makefile index 9e2fa132c84f..ce4026a5ed71 100644 --- a/tests/sys/netpfil/common/Makefile +++ b/tests/sys/netpfil/common/Makefile @@ -6,8 +6,10 @@ TESTSDIR= ${TESTSBASE}/sys/netpfil/common ${PACKAGE}FILES+= \ pft_ping.py \ + pft_synflood.py \ sniffer.py ${PACKAGE}FILESMODE_pft_ping.py= 0555 +${PACKAGE}FILESMODE_pft_synflood.py= 0555 .include diff --git a/tests/sys/netpfil/common/pft_synflood.py b/tests/sys/netpfil/common/pft_synflood.py new file mode 100644 index 000000000000..3d37724593e2 --- /dev/null +++ b/tests/sys/netpfil/common/pft_synflood.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +import argparse +import logging +logging.getLogger("scapy").setLevel(logging.CRITICAL) +import scapy.all as sp + +def syn_flood(args): + s = sp.conf.L2socket(iface=args.sendif[0]) + + # Set a src mac, to avoid doing lookups which really slow us down. + ether = sp.Ether(src='01:02:03:04:05') + ip = sp.IP(dst=args.to[0]) + for i in range(int(args.count[0])): + tcp = sp.TCP(flags='S', sport=1+i, dport=22, seq=500+i) + pkt = ether / ip / tcp + s.send(pkt) + +def main(): + parser = argparse.ArgumentParser("pft_synflood.py", + description="SYN flooding tool") + parser.add_argument('--sendif', nargs=1, + required=True, + help='The interface through which the packet(s) will be sent') + parser.add_argument('--to', nargs=1, + required=True, + help='The destination IP address for the SYN packets') + parser.add_argument('--count', nargs=1, + required=True, + help='The number of packets to send') + + args = parser.parse_args() + + syn_flood(args) + +if __name__ == '__main__': + main() diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Makefile index a8c5aac4af08..08174ad7ce19 100644 --- a/tests/sys/netpfil/pf/Makefile +++ b/tests/sys/netpfil/pf/Makefile @@ -10,6 +10,7 @@ ATF_TESTS_SH+= anchor \ checksum \ forward \ fragmentation \ + get_state \ icmp \ killstate \ map_e \ diff --git a/tests/sys/netpfil/pf/get_state.sh b/tests/sys/netpfil/pf/get_state.sh new file mode 100644 index 000000000000..5e6a9040e016 --- /dev/null +++ b/tests/sys/netpfil/pf/get_state.sh @@ -0,0 +1,81 @@ +# $FreeBSD$ +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +. $(atf_get_srcdir)/utils.subr + +common_dir=$(atf_get_srcdir)/../common + +atf_test_case "many" "cleanup" +many_head() +{ + atf_set descr 'Test retrieving many states' + atf_set require.user root + atf_set require.progs scapy +} + +many_body() +{ + pft_init + + epair=$(vnet_mkepair) + ifconfig ${epair}a 192.0.2.1/24 up + + vnet_mkjail alcatraz ${epair}b + jexec alcatraz ifconfig ${epair}b 192.0.2.2/24 up + jexec alcatraz pfctl -e + + pft_set_rules alcatraz "set timeout tcp.closed 60000" \ + "pass in proto icmp" \ + "pass in proto tcp" + + # Sanity check + atf_check -s exit:0 -o ignore ping -c 1 -W 1 192.0.2.2 + + # Now syn flood to create many states + ${common_dir}/pft_synflood.py \ + --sendif ${epair}a \ + --to 192.0.2.2 \ + --count 20000 + + count=$(time jexec alcatraz pfctl -ss | wc -l 2>time.txt) + echo "Found $count states in `cat time.txt`" + if [ $count -lt 20000 ]; + then + atf_fail "Fail to retrieve states" + fi +} + +many_cleanup() +{ + rm -f time.txt + pft_cleanup +} + +atf_init_test_cases() +{ + atf_add_test_case "many" +} From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:24 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A2E5865FDE9; Fri, 9 Jul 2021 13:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5N4Gj6z3Jgj; Fri, 9 Jul 2021 13:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7AB0814DFC; Fri, 9 Jul 2021 13:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOOf7063346; Fri, 9 Jul 2021 13:24:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOOav063345; Fri, 9 Jul 2021 13:24:24 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:24 GMT Message-Id: <202107091324.169DOOav063345@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 36c08263b74d - stable/13 - pf tests: Stress state retrieval MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 36c08263b74db894722c952612a22c6f21c23c8c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:24 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=36c08263b74db894722c952612a22c6f21c23c8c commit 36c08263b74db894722c952612a22c6f21c23c8c Author: Kristof Provost AuthorDate: 2021-06-28 10:48:20 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:07:10 +0000 pf tests: Stress state retrieval Create and retrieve 20.000 states. There have been issues with nvlists causing very slow state retrieval. We don't impose a specific limit on the time required to retrieve the states, but do log it. In excessive cases the Kyua timeout will fail this test. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30943 (cherry picked from commit d8d43b2de1fa679179f7089cb3c31e6780ec82af) --- tests/sys/netpfil/common/Makefile | 2 + tests/sys/netpfil/common/pft_synflood.py | 62 ++++++++++++++++++++++++ tests/sys/netpfil/pf/Makefile | 1 + tests/sys/netpfil/pf/get_state.sh | 81 ++++++++++++++++++++++++++++++++ 4 files changed, 146 insertions(+) diff --git a/tests/sys/netpfil/common/Makefile b/tests/sys/netpfil/common/Makefile index e63a23b59687..915c19ba590c 100644 --- a/tests/sys/netpfil/common/Makefile +++ b/tests/sys/netpfil/common/Makefile @@ -16,8 +16,10 @@ ${PACKAGE}FILES+= \ utils.subr \ runner.subr \ pft_ping.py \ + pft_synflood.py \ sniffer.py ${PACKAGE}FILESMODE_pft_ping.py= 0555 +${PACKAGE}FILESMODE_pft_synflood.py= 0555 .include diff --git a/tests/sys/netpfil/common/pft_synflood.py b/tests/sys/netpfil/common/pft_synflood.py new file mode 100644 index 000000000000..3d37724593e2 --- /dev/null +++ b/tests/sys/netpfil/common/pft_synflood.py @@ -0,0 +1,62 @@ +#!/usr/bin/env python3 +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +import argparse +import logging +logging.getLogger("scapy").setLevel(logging.CRITICAL) +import scapy.all as sp + +def syn_flood(args): + s = sp.conf.L2socket(iface=args.sendif[0]) + + # Set a src mac, to avoid doing lookups which really slow us down. + ether = sp.Ether(src='01:02:03:04:05') + ip = sp.IP(dst=args.to[0]) + for i in range(int(args.count[0])): + tcp = sp.TCP(flags='S', sport=1+i, dport=22, seq=500+i) + pkt = ether / ip / tcp + s.send(pkt) + +def main(): + parser = argparse.ArgumentParser("pft_synflood.py", + description="SYN flooding tool") + parser.add_argument('--sendif', nargs=1, + required=True, + help='The interface through which the packet(s) will be sent') + parser.add_argument('--to', nargs=1, + required=True, + help='The destination IP address for the SYN packets') + parser.add_argument('--count', nargs=1, + required=True, + help='The number of packets to send') + + args = parser.parse_args() + + syn_flood(args) + +if __name__ == '__main__': + main() diff --git a/tests/sys/netpfil/pf/Makefile b/tests/sys/netpfil/pf/Makefile index b693ca63adcb..e5139b4262e1 100644 --- a/tests/sys/netpfil/pf/Makefile +++ b/tests/sys/netpfil/pf/Makefile @@ -10,6 +10,7 @@ ATF_TESTS_SH+= anchor \ dup \ forward \ fragmentation \ + get_state \ icmp \ killstate \ map_e \ diff --git a/tests/sys/netpfil/pf/get_state.sh b/tests/sys/netpfil/pf/get_state.sh new file mode 100644 index 000000000000..5e6a9040e016 --- /dev/null +++ b/tests/sys/netpfil/pf/get_state.sh @@ -0,0 +1,81 @@ +# $FreeBSD$ +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +. $(atf_get_srcdir)/utils.subr + +common_dir=$(atf_get_srcdir)/../common + +atf_test_case "many" "cleanup" +many_head() +{ + atf_set descr 'Test retrieving many states' + atf_set require.user root + atf_set require.progs scapy +} + +many_body() +{ + pft_init + + epair=$(vnet_mkepair) + ifconfig ${epair}a 192.0.2.1/24 up + + vnet_mkjail alcatraz ${epair}b + jexec alcatraz ifconfig ${epair}b 192.0.2.2/24 up + jexec alcatraz pfctl -e + + pft_set_rules alcatraz "set timeout tcp.closed 60000" \ + "pass in proto icmp" \ + "pass in proto tcp" + + # Sanity check + atf_check -s exit:0 -o ignore ping -c 1 -W 1 192.0.2.2 + + # Now syn flood to create many states + ${common_dir}/pft_synflood.py \ + --sendif ${epair}a \ + --to 192.0.2.2 \ + --count 20000 + + count=$(time jexec alcatraz pfctl -ss | wc -l 2>time.txt) + echo "Found $count states in `cat time.txt`" + if [ $count -lt 20000 ]; + then + atf_fail "Fail to retrieve states" + fi +} + +many_cleanup() +{ + rm -f time.txt + pft_cleanup +} + +atf_init_test_cases() +{ + atf_add_test_case "many" +} From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:27 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 039A365F6B8; Fri, 9 Jul 2021 13:24:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5Q6PfGz3JW7; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C3227155D3; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOQ8X063448; Fri, 9 Jul 2021 13:24:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOQSw063447; Fri, 9 Jul 2021 13:24:26 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:26 GMT Message-Id: <202107091324.169DOQSw063447@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 1882bdd192bc - stable/13 - pf: getstates: avoid taking the hashrow lock if the row is empty MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 1882bdd192bc406459dda00dda7246bb2f56b66c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:27 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=1882bdd192bc406459dda00dda7246bb2f56b66c commit 1882bdd192bc406459dda00dda7246bb2f56b66c Author: Kristof Provost AuthorDate: 2021-06-29 09:34:49 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:08:32 +0000 pf: getstates: avoid taking the hashrow lock if the row is empty Reviewed by: mjg MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30946 (cherry picked from commit a19ff8ce9b58548a5f965db2c46eb03c38b15edb) --- sys/netpfil/pf/pf_ioctl.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 0097d8b0f6b6..2e332156a488 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -5055,6 +5055,10 @@ pf_getstates(struct pfioc_nv *nv) for (int i = 0; i < pf_hashmask; i++) { struct pf_idhash *ih = &V_pf_idhash[i]; + /* Avoid taking the lock if there are no states in the row. */ + if (LIST_EMPTY(&ih->states)) + continue; + PF_HASHROW_LOCK(ih); LIST_FOREACH(s, &ih->states, entry) { if (s->timeout == PFTM_UNLINKED) From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:26 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 433F665FABB; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5P6db1z3Jgk; Fri, 9 Jul 2021 13:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AAF6C1566E; Fri, 9 Jul 2021 13:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOPfP063394; Fri, 9 Jul 2021 13:24:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOPwo063393; Fri, 9 Jul 2021 13:24:25 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:25 GMT Message-Id: <202107091324.169DOPwo063393@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 9e5d987058fe - stable/13 - pf: Reduce the data returned in DIOCGETSTATESNV MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 9e5d987058fe79d46d9bc320b05630319bcca225 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:26 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=9e5d987058fe79d46d9bc320b05630319bcca225 commit 9e5d987058fe79d46d9bc320b05630319bcca225 Author: Kristof Provost AuthorDate: 2021-06-29 08:26:40 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:07:21 +0000 pf: Reduce the data returned in DIOCGETSTATESNV This call is particularly slow due to the large amount of data it returns. Remove all fields pfctl does not use. There is no functional impact to pfctl, but it somewhat speeds up the call. It might affect other (i.e. non-FreeBSD) code that uses the new interface, but this call is very new, so there's unlikely to be any. No releases contained the previous version, so we choose to live with the ABI modification. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30944 (cherry picked from commit 34285eefddc99c994c3e5374ba7836cc7cfc8e2e) --- lib/libpfctl/libpfctl.c | 22 ---------------------- lib/libpfctl/libpfctl.h | 12 ------------ sys/netpfil/pf/pf_nv.c | 34 +--------------------------------- 3 files changed, 1 insertion(+), 67 deletions(-) diff --git a/lib/libpfctl/libpfctl.c b/lib/libpfctl/libpfctl.c index 52cd0ed7f36c..524e2472238e 100644 --- a/lib/libpfctl/libpfctl.c +++ b/lib/libpfctl/libpfctl.c @@ -636,35 +636,15 @@ pfctl_nv_add_state_cmp(nvlist_t *nvl, const char *name, nvlist_destroy(nv); } -static void -pf_nvstate_scrub_to_state_scrub(const nvlist_t *nvl, - struct pfctl_state_scrub *scrub) -{ - bzero(scrub, sizeof(*scrub)); - - scrub->timestamp = nvlist_get_bool(nvl, "timestamp"); - scrub->ttl = nvlist_get_number(nvl, "ttl"); - scrub->ts_mod = nvlist_get_number(nvl, "ts_mod"); -} - static void pf_nvstate_peer_to_state_peer(const nvlist_t *nvl, struct pfctl_state_peer *peer) { bzero(peer, sizeof(*peer)); - if (nvlist_exists_nvlist(nvl, "scrub")) { - peer->scrub = malloc(sizeof(*peer->scrub)); - pf_nvstate_scrub_to_state_scrub( - nvlist_get_nvlist(nvl, "scrub"), - peer->scrub); - } - peer->seqlo = nvlist_get_number(nvl, "seqlo"); peer->seqhi = nvlist_get_number(nvl, "seqhi"); peer->seqdiff = nvlist_get_number(nvl, "seqdiff"); - peer->max_win = nvlist_get_number(nvl, "max_win"); - peer->mss = nvlist_get_number(nvl, "mss"); peer->state = nvlist_get_number(nvl, "state"); peer->wscale = nvlist_get_number(nvl, "wscale"); } @@ -721,9 +701,7 @@ pf_nvstate_to_state(const nvlist_t *nvl, struct pfctl_state *s) pf_nvuint_64_array(nvl, "packets", 2, s->packets, NULL); pf_nvuint_64_array(nvl, "bytes", 2, s->bytes, NULL); - s->log = nvlist_get_number(nvl, "log"); s->state_flags = nvlist_get_number(nvl, "state_flags"); - s->timeout = nvlist_get_number(nvl, "timeout"); s->sync_flags = nvlist_get_number(nvl, "sync_flags"); } diff --git a/lib/libpfctl/libpfctl.h b/lib/libpfctl/libpfctl.h index a54ee9db6ec7..62866e17f904 100644 --- a/lib/libpfctl/libpfctl.h +++ b/lib/libpfctl/libpfctl.h @@ -197,19 +197,10 @@ struct pfctl_kill { bool kill_match; }; -struct pfctl_state_scrub { - bool timestamp; - uint8_t ttl; - uint32_t ts_mod; -}; - struct pfctl_state_peer { - struct pfctl_state_scrub *scrub; uint32_t seqlo; uint32_t seqhi; uint32_t seqdiff; - uint16_t max_win; - uint16_t mss; uint8_t state; uint8_t wscale; }; @@ -243,10 +234,7 @@ struct pfctl_state { uint32_t creation; uint32_t expire; uint32_t pfsync_time; - uint16_t tag; - uint8_t log; uint8_t state_flags; - uint8_t timeout; uint32_t sync_flags; }; diff --git a/sys/netpfil/pf/pf_nv.c b/sys/netpfil/pf/pf_nv.c index 553290c88586..e52d52642eab 100644 --- a/sys/netpfil/pf/pf_nv.c +++ b/sys/netpfil/pf/pf_nv.c @@ -874,52 +874,22 @@ errout: return (NULL); } -static nvlist_t * -pf_state_scrub_to_nvstate_scrub(const struct pf_state_scrub *scrub) -{ - nvlist_t *nvl; - - nvl = nvlist_create(0); - if (nvl == NULL) - return (NULL); - - nvlist_add_bool(nvl, "timestamp", scrub->pfss_flags & PFSS_TIMESTAMP); - nvlist_add_number(nvl, "ttl", scrub->pfss_ttl); - nvlist_add_number(nvl, "ts_mod", scrub->pfss_ts_mod); - - return (nvl); -} - static nvlist_t * pf_state_peer_to_nvstate_peer(const struct pf_state_peer *peer) { - nvlist_t *nvl, *tmp; + nvlist_t *nvl; nvl = nvlist_create(0); if (nvl == NULL) return (NULL); - if (peer->scrub) { - tmp = pf_state_scrub_to_nvstate_scrub(peer->scrub); - if (tmp == NULL) - goto errout; - nvlist_add_nvlist(nvl, "scrub", tmp); - nvlist_destroy(tmp); - } - nvlist_add_number(nvl, "seqlo", peer->seqlo); nvlist_add_number(nvl, "seqhi", peer->seqhi); nvlist_add_number(nvl, "seqdiff", peer->seqdiff); - nvlist_add_number(nvl, "max_win", peer->max_win); - nvlist_add_number(nvl, "mss", peer->mss); nvlist_add_number(nvl, "state", peer->state); nvlist_add_number(nvl, "wscale", peer->wscale); return (nvl); - -errout: - nvlist_destroy(nvl); - return (NULL); } nvlist_t * @@ -989,9 +959,7 @@ pf_state_to_nvstate(const struct pf_state *s) nvlist_add_number(nvl, "creatorid", s->creatorid); nvlist_add_number(nvl, "direction", s->direction); - nvlist_add_number(nvl, "log", s->log); nvlist_add_number(nvl, "state_flags", s->state_flags); - nvlist_add_number(nvl, "timeout", s->timeout); if (s->src_node) flags |= PFSYNC_FLAG_SRCNODE; if (s->nat_src_node) From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:26 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9767265FEC5; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5Q3Y5zz3Jk3; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 36B1E155D2; Fri, 9 Jul 2021 13:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOQ03063420; Fri, 9 Jul 2021 13:24:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOQKH063419; Fri, 9 Jul 2021 13:24:26 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:26 GMT Message-Id: <202107091324.169DOQKH063419@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: b43db87c97dd - stable/12 - pf: getstates: avoid taking the hashrow lock if the row is empty MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: b43db87c97dd396da866919842715aef7b84daba Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:26 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=b43db87c97dd396da866919842715aef7b84daba commit b43db87c97dd396da866919842715aef7b84daba Author: Kristof Provost AuthorDate: 2021-06-29 09:34:49 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:08:32 +0000 pf: getstates: avoid taking the hashrow lock if the row is empty Reviewed by: mjg MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30946 (cherry picked from commit a19ff8ce9b58548a5f965db2c46eb03c38b15edb) --- sys/netpfil/pf/pf_ioctl.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sys/netpfil/pf/pf_ioctl.c b/sys/netpfil/pf/pf_ioctl.c index 0ce39db05c15..92b0cda8261e 100644 --- a/sys/netpfil/pf/pf_ioctl.c +++ b/sys/netpfil/pf/pf_ioctl.c @@ -5122,6 +5122,10 @@ pf_getstates(struct pfioc_nv *nv) for (int i = 0; i < pf_hashmask; i++) { struct pf_idhash *ih = &V_pf_idhash[i]; + /* Avoid taking the lock if there are no states in the row. */ + if (LIST_EMPTY(&ih->states)) + continue; + PF_HASHROW_LOCK(ih); LIST_FOREACH(s, &ih->states, entry) { if (s->timeout == PFTM_UNLINKED) From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:28 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 852E165FF52; Fri, 9 Jul 2021 13:24:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5R5mjjz3Jk6; Fri, 9 Jul 2021 13:24:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 58F141566F; Fri, 9 Jul 2021 13:24:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOR4J063473; Fri, 9 Jul 2021 13:24:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DORsm063472; Fri, 9 Jul 2021 13:24:27 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:27 GMT Message-Id: <202107091324.169DORsm063472@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: e553299fd630 - stable/12 - libpfctl: memory leak fix MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e553299fd630d181401f9179aacca62682e2f26c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:28 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=e553299fd630d181401f9179aacca62682e2f26c commit e553299fd630d181401f9179aacca62682e2f26c Author: Kristof Provost AuthorDate: 2021-06-30 11:02:35 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:08:39 +0000 libpfctl: memory leak fix We must remember to free the nvlist we create from the kernel's response to DIOCGETSTATESNV, on every iteration. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30957 (cherry picked from commit 0e9f1892ec739d7fbd854af699507167a0a5dde2) --- lib/libpfctl/libpfctl.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/libpfctl/libpfctl.c b/lib/libpfctl/libpfctl.c index 524e2472238e..bbb53edf8bc1 100644 --- a/lib/libpfctl/libpfctl.c +++ b/lib/libpfctl/libpfctl.c @@ -727,6 +727,7 @@ pfctl_get_states(int dev, struct pfctl_states *states) goto out; } + nvlist_destroy(nvl); nvl = nvlist_unpack(nv.data, nv.len, 0); if (nvl == NULL) { error = EIO; From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:25 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9597965FABA; Fri, 9 Jul 2021 13:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5P2KXnz3Jc9; Fri, 9 Jul 2021 13:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2EFA31566D; Fri, 9 Jul 2021 13:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOP8P063370; Fri, 9 Jul 2021 13:24:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOPmW063369; Fri, 9 Jul 2021 13:24:25 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:25 GMT Message-Id: <202107091324.169DOPmW063369@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 63a8dc255208 - stable/12 - pf: Reduce the data returned in DIOCGETSTATESNV MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 63a8dc255208692f022beff22f38ae16ea51565b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:26 -0000 The branch stable/12 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=63a8dc255208692f022beff22f38ae16ea51565b commit 63a8dc255208692f022beff22f38ae16ea51565b Author: Kristof Provost AuthorDate: 2021-06-29 08:26:40 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:07:52 +0000 pf: Reduce the data returned in DIOCGETSTATESNV This call is particularly slow due to the large amount of data it returns. Remove all fields pfctl does not use. There is no functional impact to pfctl, but it somewhat speeds up the call. It might affect other (i.e. non-FreeBSD) code that uses the new interface, but this call is very new, so there's unlikely to be any. No releases contained the previous version, so we choose to live with the ABI modification. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30944 (cherry picked from commit 34285eefddc99c994c3e5374ba7836cc7cfc8e2e) --- lib/libpfctl/libpfctl.c | 22 ---------------------- lib/libpfctl/libpfctl.h | 12 ------------ sys/netpfil/pf/pf_nv.c | 34 +--------------------------------- 3 files changed, 1 insertion(+), 67 deletions(-) diff --git a/lib/libpfctl/libpfctl.c b/lib/libpfctl/libpfctl.c index 52cd0ed7f36c..524e2472238e 100644 --- a/lib/libpfctl/libpfctl.c +++ b/lib/libpfctl/libpfctl.c @@ -636,35 +636,15 @@ pfctl_nv_add_state_cmp(nvlist_t *nvl, const char *name, nvlist_destroy(nv); } -static void -pf_nvstate_scrub_to_state_scrub(const nvlist_t *nvl, - struct pfctl_state_scrub *scrub) -{ - bzero(scrub, sizeof(*scrub)); - - scrub->timestamp = nvlist_get_bool(nvl, "timestamp"); - scrub->ttl = nvlist_get_number(nvl, "ttl"); - scrub->ts_mod = nvlist_get_number(nvl, "ts_mod"); -} - static void pf_nvstate_peer_to_state_peer(const nvlist_t *nvl, struct pfctl_state_peer *peer) { bzero(peer, sizeof(*peer)); - if (nvlist_exists_nvlist(nvl, "scrub")) { - peer->scrub = malloc(sizeof(*peer->scrub)); - pf_nvstate_scrub_to_state_scrub( - nvlist_get_nvlist(nvl, "scrub"), - peer->scrub); - } - peer->seqlo = nvlist_get_number(nvl, "seqlo"); peer->seqhi = nvlist_get_number(nvl, "seqhi"); peer->seqdiff = nvlist_get_number(nvl, "seqdiff"); - peer->max_win = nvlist_get_number(nvl, "max_win"); - peer->mss = nvlist_get_number(nvl, "mss"); peer->state = nvlist_get_number(nvl, "state"); peer->wscale = nvlist_get_number(nvl, "wscale"); } @@ -721,9 +701,7 @@ pf_nvstate_to_state(const nvlist_t *nvl, struct pfctl_state *s) pf_nvuint_64_array(nvl, "packets", 2, s->packets, NULL); pf_nvuint_64_array(nvl, "bytes", 2, s->bytes, NULL); - s->log = nvlist_get_number(nvl, "log"); s->state_flags = nvlist_get_number(nvl, "state_flags"); - s->timeout = nvlist_get_number(nvl, "timeout"); s->sync_flags = nvlist_get_number(nvl, "sync_flags"); } diff --git a/lib/libpfctl/libpfctl.h b/lib/libpfctl/libpfctl.h index a54ee9db6ec7..62866e17f904 100644 --- a/lib/libpfctl/libpfctl.h +++ b/lib/libpfctl/libpfctl.h @@ -197,19 +197,10 @@ struct pfctl_kill { bool kill_match; }; -struct pfctl_state_scrub { - bool timestamp; - uint8_t ttl; - uint32_t ts_mod; -}; - struct pfctl_state_peer { - struct pfctl_state_scrub *scrub; uint32_t seqlo; uint32_t seqhi; uint32_t seqdiff; - uint16_t max_win; - uint16_t mss; uint8_t state; uint8_t wscale; }; @@ -243,10 +234,7 @@ struct pfctl_state { uint32_t creation; uint32_t expire; uint32_t pfsync_time; - uint16_t tag; - uint8_t log; uint8_t state_flags; - uint8_t timeout; uint32_t sync_flags; }; diff --git a/sys/netpfil/pf/pf_nv.c b/sys/netpfil/pf/pf_nv.c index 553290c88586..e52d52642eab 100644 --- a/sys/netpfil/pf/pf_nv.c +++ b/sys/netpfil/pf/pf_nv.c @@ -874,52 +874,22 @@ errout: return (NULL); } -static nvlist_t * -pf_state_scrub_to_nvstate_scrub(const struct pf_state_scrub *scrub) -{ - nvlist_t *nvl; - - nvl = nvlist_create(0); - if (nvl == NULL) - return (NULL); - - nvlist_add_bool(nvl, "timestamp", scrub->pfss_flags & PFSS_TIMESTAMP); - nvlist_add_number(nvl, "ttl", scrub->pfss_ttl); - nvlist_add_number(nvl, "ts_mod", scrub->pfss_ts_mod); - - return (nvl); -} - static nvlist_t * pf_state_peer_to_nvstate_peer(const struct pf_state_peer *peer) { - nvlist_t *nvl, *tmp; + nvlist_t *nvl; nvl = nvlist_create(0); if (nvl == NULL) return (NULL); - if (peer->scrub) { - tmp = pf_state_scrub_to_nvstate_scrub(peer->scrub); - if (tmp == NULL) - goto errout; - nvlist_add_nvlist(nvl, "scrub", tmp); - nvlist_destroy(tmp); - } - nvlist_add_number(nvl, "seqlo", peer->seqlo); nvlist_add_number(nvl, "seqhi", peer->seqhi); nvlist_add_number(nvl, "seqdiff", peer->seqdiff); - nvlist_add_number(nvl, "max_win", peer->max_win); - nvlist_add_number(nvl, "mss", peer->mss); nvlist_add_number(nvl, "state", peer->state); nvlist_add_number(nvl, "wscale", peer->wscale); return (nvl); - -errout: - nvlist_destroy(nvl); - return (NULL); } nvlist_t * @@ -989,9 +959,7 @@ pf_state_to_nvstate(const struct pf_state *s) nvlist_add_number(nvl, "creatorid", s->creatorid); nvlist_add_number(nvl, "direction", s->direction); - nvlist_add_number(nvl, "log", s->log); nvlist_add_number(nvl, "state_flags", s->state_flags); - nvlist_add_number(nvl, "timeout", s->timeout); if (s->src_node) flags |= PFSYNC_FLAG_SRCNODE; if (s->nat_src_node) From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:28 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFDC565FF53; Fri, 9 Jul 2021 13:24:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5S2hvHz3Jk8; Fri, 9 Jul 2021 13:24:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E9817155D4; Fri, 9 Jul 2021 13:24:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOR2c063497; Fri, 9 Jul 2021 13:24:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DORsI063496; Fri, 9 Jul 2021 13:24:27 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:27 GMT Message-Id: <202107091324.169DORsI063496@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 4debb6a3a027 - stable/13 - libpfctl: memory leak fix MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 4debb6a3a027498aa657fcb79007d9936fc8b771 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:31 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=4debb6a3a027498aa657fcb79007d9936fc8b771 commit 4debb6a3a027498aa657fcb79007d9936fc8b771 Author: Kristof Provost AuthorDate: 2021-06-30 11:02:35 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:08:37 +0000 libpfctl: memory leak fix We must remember to free the nvlist we create from the kernel's response to DIOCGETSTATESNV, on every iteration. Reviewed by: donner MFC after: 1 week Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30957 (cherry picked from commit 0e9f1892ec739d7fbd854af699507167a0a5dde2) --- lib/libpfctl/libpfctl.c | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/libpfctl/libpfctl.c b/lib/libpfctl/libpfctl.c index 524e2472238e..bbb53edf8bc1 100644 --- a/lib/libpfctl/libpfctl.c +++ b/lib/libpfctl/libpfctl.c @@ -727,6 +727,7 @@ pfctl_get_states(int dev, struct pfctl_states *states) goto out; } + nvlist_destroy(nvl); nvl = nvlist_unpack(nv.data, nv.len, 0); if (nvl == NULL) { error = EIO; From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 13:24:30 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1419365F6C3; Fri, 9 Jul 2021 13:24:30 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLv5T52xfz3JfX; Fri, 9 Jul 2021 13:24:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1217815783; Fri, 9 Jul 2021 13:24:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169DOSv1063559; Fri, 9 Jul 2021 13:24:28 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169DOSRa063558; Fri, 9 Jul 2021 13:24:28 GMT (envelope-from git) Date: Fri, 9 Jul 2021 13:24:28 GMT Message-Id: <202107091324.169DOSRa063558@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kristof Provost Subject: git: 1f4a64039692 - stable/13 - netpfil tests: Basic dummynet pipe test MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 1f4a6403969271885d1c37fd642820d5a6562791 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 13:24:32 -0000 The branch stable/13 has been updated by kp: URL: https://cgit.FreeBSD.org/src/commit/?id=1f4a6403969271885d1c37fd642820d5a6562791 commit 1f4a6403969271885d1c37fd642820d5a6562791 Author: Kristof Provost AuthorDate: 2021-05-21 09:14:34 +0000 Commit: Kristof Provost CommitDate: 2021-07-09 08:09:34 +0000 netpfil tests: Basic dummynet pipe test Test dummynet pipes (i.e. bandwidth limitation) with ipfw. This is put in the common tests because we hope to add dummynet support to pf in the near future. MFC after: 2 weeks Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D30380 (cherry picked from commit ea3eca5cb6dbcb4deb7c7277a65c48911f0475d1) --- tests/sys/netpfil/common/Makefile | 1 + tests/sys/netpfil/common/dummynet.sh | 75 ++++++++++++++++++++++++++++++++++++ tests/sys/netpfil/common/utils.subr | 15 ++++++++ 3 files changed, 91 insertions(+) diff --git a/tests/sys/netpfil/common/Makefile b/tests/sys/netpfil/common/Makefile index 915c19ba590c..749e0dd99469 100644 --- a/tests/sys/netpfil/common/Makefile +++ b/tests/sys/netpfil/common/Makefile @@ -6,6 +6,7 @@ TESTSDIR= ${TESTSBASE}/sys/netpfil/common ATF_TESTS_SH+= \ + dummynet \ pass_block \ nat \ tos \ diff --git a/tests/sys/netpfil/common/dummynet.sh b/tests/sys/netpfil/common/dummynet.sh new file mode 100644 index 000000000000..82cd22b77fef --- /dev/null +++ b/tests/sys/netpfil/common/dummynet.sh @@ -0,0 +1,75 @@ +# $FreeBSD$ +# +# SPDX-License-Identifier: BSD-2-Clause-FreeBSD +# +# Copyright (c) 2021 Rubicon Communications, LLC (Netgate) +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +# FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +# DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +# OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +# HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +# OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +# SUCH DAMAGE. + +. $(atf_get_srcdir)/utils.subr +. $(atf_get_srcdir)/runner.subr + +pipe_head() +{ + atf_set descr 'Basic pipe test' + atf_set require.user root +} + +pipe_body() +{ + fw=$1 + firewall_init $fw + dummynet_init $fw + + epair=$(vnet_mkepair) + vnet_mkjail alcatraz ${epair}b + + ifconfig ${epair}a 192.0.2.1/24 up + jexec alcatraz ifconfig ${epair}b 192.0.2.2/24 up + + # Sanity check + atf_check -s exit:0 -o ignore ping -i .1 -c 3 -s 1200 192.0.2.2 + + jexec alcatraz dnctl pipe 1 config bw 30Byte/s + + firewall_config alcatraz ${fw} \ + "ipfw" \ + "ipfw add 1000 pipe 1 ip from any to any" + + # single ping succeeds just fine + atf_check -s exit:0 -o ignore ping -c 1 192.0.2.2 + + # Saturate the link + ping -i .1 -c 5 -s 1200 192.0.2.2 + + # We should now be hitting the limits and get this packet dropped. + atf_check -s exit:2 -o ignore ping -c 1 -s 1200 192.0.2.2 +} + +pipe_cleanup() +{ + firewall_cleanup $1 +} + +setup_tests \ + pipe \ + ipfw diff --git a/tests/sys/netpfil/common/utils.subr b/tests/sys/netpfil/common/utils.subr index 3f9d6b40183a..722271981af4 100644 --- a/tests/sys/netpfil/common/utils.subr +++ b/tests/sys/netpfil/common/utils.subr @@ -103,6 +103,21 @@ firewall_init() } +dummynet_init() +{ + firewall=$1 + + if ! kldstat -q -m dummynet; then + atf_skip "This test requires dummynet" + fi + + if [ ${firewall} == "ipfw" ]; then + # Nothing. This is okay. + else + atf_skip "${firewall} does not support dummynet" + fi +} + nat_init() { firewall=$1 From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 14:26:01 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E3B6660C18; Fri, 9 Jul 2021 14:26:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLwST088Cz3j67; Fri, 9 Jul 2021 14:26:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E0C7E162B7; Fri, 9 Jul 2021 14:26:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169EQ0xk043402; Fri, 9 Jul 2021 14:26:00 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169EQ0uW043401; Fri, 9 Jul 2021 14:26:00 GMT (envelope-from git) Date: Fri, 9 Jul 2021 14:26:00 GMT Message-Id: <202107091426.169EQ0uW043401@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Chuck Tuffli Subject: git: a7761d19dacd - stable/13 - bhyve: Fix NVMe iovec construction for large IOs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: chuck X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a7761d19dacd414c8b8269a6cf909ab4528783dc Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 14:26:01 -0000 The branch stable/13 has been updated by chuck: URL: https://cgit.FreeBSD.org/src/commit/?id=a7761d19dacd414c8b8269a6cf909ab4528783dc commit a7761d19dacd414c8b8269a6cf909ab4528783dc Author: Chuck Tuffli AuthorDate: 2021-06-27 22:14:52 +0000 Commit: Chuck Tuffli CommitDate: 2021-07-09 14:24:14 +0000 bhyve: Fix NVMe iovec construction for large IOs The UEFI driver included with Rocky Linux 8.4 uncovered an existing bug in the NVMe emulation's construction of iovec's. By default, NVMe data transfer operations use a scatter-gather list in which all entries point to a fixed size memory region. For example, if the Memory Page Size is 4KiB, a 2MiB IO requires 512 entries. Lists themselves are also fixed size (default is 512 entries). Because the list size is fixed, the last entry is special. If the IO requires more than 512 entries, the last entry in the list contains the address of the next list of entries. But if the IO requires exactly 512 entries, the last entry points to data. The NVMe emulation missed this logic and unconditionally treated the last entry as a pointer to the next list. Fix is to check if the remaining data is greater than the page size before using the last entry as a pointer to the next list. PR: 256422 Reported by: dave@syix.com Tested by: jason@tubnor.net Relnotes: yes (cherry picked from commit 91064841d72b285a146a3f1c32cb447251e062ea) --- usr.sbin/bhyve/pci_nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.sbin/bhyve/pci_nvme.c b/usr.sbin/bhyve/pci_nvme.c index 24f401630d6d..bd21819f0607 100644 --- a/usr.sbin/bhyve/pci_nvme.c +++ b/usr.sbin/bhyve/pci_nvme.c @@ -1976,7 +1976,7 @@ nvme_write_read_blockif(struct pci_nvme_softc *sc, /* PRP2 is pointer to a physical region page list */ while (bytes) { /* Last entry in list points to the next list */ - if (prp_list == last) { + if ((prp_list == last) && (bytes > PAGE_SIZE)) { uint64_t prp = *prp_list; prp_list = paddr_guest2host(vmctx, prp, From owner-dev-commits-src-branches@freebsd.org Fri Jul 9 14:27:34 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0BE0B660E88; Fri, 9 Jul 2021 14:27:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLwVF6YKFz3jMn; Fri, 9 Jul 2021 14:27:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C8AFC16233; Fri, 9 Jul 2021 14:27:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 169ERXGC043687; Fri, 9 Jul 2021 14:27:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 169ERXw8043686; Fri, 9 Jul 2021 14:27:33 GMT (envelope-from git) Date: Fri, 9 Jul 2021 14:27:33 GMT Message-Id: <202107091427.169ERXw8043686@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Chuck Tuffli Subject: git: cd8a5d2316a1 - stable/12 - bhyve: Fix NVMe iovec construction for large IOs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: chuck X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: cd8a5d2316a12a8abca458c31467dc9dcf8361ce Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Jul 2021 14:27:34 -0000 The branch stable/12 has been updated by chuck: URL: https://cgit.FreeBSD.org/src/commit/?id=cd8a5d2316a12a8abca458c31467dc9dcf8361ce commit cd8a5d2316a12a8abca458c31467dc9dcf8361ce Author: Chuck Tuffli AuthorDate: 2021-06-27 22:14:52 +0000 Commit: Chuck Tuffli CommitDate: 2021-07-09 14:25:45 +0000 bhyve: Fix NVMe iovec construction for large IOs The UEFI driver included with Rocky Linux 8.4 uncovered an existing bug in the NVMe emulation's construction of iovec's. By default, NVMe data transfer operations use a scatter-gather list in which all entries point to a fixed size memory region. For example, if the Memory Page Size is 4KiB, a 2MiB IO requires 512 entries. Lists themselves are also fixed size (default is 512 entries). Because the list size is fixed, the last entry is special. If the IO requires more than 512 entries, the last entry in the list contains the address of the next list of entries. But if the IO requires exactly 512 entries, the last entry points to data. The NVMe emulation missed this logic and unconditionally treated the last entry as a pointer to the next list. Fix is to check if the remaining data is greater than the page size before using the last entry as a pointer to the next list. PR: 256422 Reported by: dave@syix.com Tested by: jason@tubnor.net Relnotes: yes (cherry picked from commit 91064841d72b285a146a3f1c32cb447251e062ea) --- usr.sbin/bhyve/pci_nvme.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.sbin/bhyve/pci_nvme.c b/usr.sbin/bhyve/pci_nvme.c index 2679874d73fc..5f7a13d492f2 100644 --- a/usr.sbin/bhyve/pci_nvme.c +++ b/usr.sbin/bhyve/pci_nvme.c @@ -1973,7 +1973,7 @@ nvme_write_read_blockif(struct pci_nvme_softc *sc, /* PRP2 is pointer to a physical region page list */ while (bytes) { /* Last entry in list points to the next list */ - if (prp_list == last) { + if ((prp_list == last) && (bytes > PAGE_SIZE)) { uint64_t prp = *prp_list; prp_list = paddr_guest2host(vmctx, prp, From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 14:22:23 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EA4DD654765; Sat, 10 Jul 2021 14:22:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMXKq689Bz3Qsw; Sat, 10 Jul 2021 14:22:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AED6B1A62; Sat, 10 Jul 2021 14:22:23 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AEMNkd061205; Sat, 10 Jul 2021 14:22:23 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AEMNKo061204; Sat, 10 Jul 2021 14:22:23 GMT (envelope-from git) Date: Sat, 10 Jul 2021 14:22:23 GMT Message-Id: <202107101422.16AEMNKo061204@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Martin Matuska Subject: git: 804a06089158 - stable/13 - zfs: attach zpool_influxdb to build MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mm X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 804a0608915837eeb3f235a05f93324750770d19 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 14:22:24 -0000 The branch stable/13 has been updated by mm: URL: https://cgit.FreeBSD.org/src/commit/?id=804a0608915837eeb3f235a05f93324750770d19 commit 804a0608915837eeb3f235a05f93324750770d19 Author: Martin Matuska AuthorDate: 2021-07-07 17:45:52 +0000 Commit: Martin Matuska CommitDate: 2021-07-10 10:26:42 +0000 zfs: attach zpool_influxdb to build From the zpool_influxdb.8 manual page: zpool_influxdb produces InfluxDB-line-protocol-compatible metrics from zpools. Like the zpool command, zpool_influxdb reads the current pool status and statistics. Unlike the zpool command which is intended for humans, zpool_influxdb formats the output in the InfluxDB line protocol. The expected use is as a plugin to a metrics collector or aggregator, such as Telegraf. zpool_influxdb is installed into /usr/libexec/zfs/ Differential revision: https://reviews.freebsd.org/D31094 (cherry picked from commit 48b4fe0503282f03d25e23f44109c5cb6d450f7c) --- cddl/Makefile | 2 +- cddl/usr.libexec/Makefile | 13 +++++++++++ cddl/usr.libexec/zpool_influxdb/Makefile | 28 ++++++++++++++++++++++ cddl/usr.libexec/zpool_influxdb/Makefile.depend | 31 +++++++++++++++++++++++++ etc/mtree/BSD.debug.dist | 2 ++ etc/mtree/BSD.usr.dist | 2 ++ 6 files changed, 77 insertions(+), 1 deletion(-) diff --git a/cddl/Makefile b/cddl/Makefile index 7b9d13146afa..284a370c0167 100644 --- a/cddl/Makefile +++ b/cddl/Makefile @@ -3,7 +3,7 @@ .include SUBDIR= lib .WAIT \ - sbin share usr.bin usr.sbin + sbin share usr.bin usr.libexec usr.sbin SUBDIR.${MK_TESTS}+= tests diff --git a/cddl/usr.libexec/Makefile b/cddl/usr.libexec/Makefile new file mode 100644 index 000000000000..6280cbab5766 --- /dev/null +++ b/cddl/usr.libexec/Makefile @@ -0,0 +1,13 @@ +# $FreeBSD$ + +.include + +SUBDIR= ${_zpool_influxdb} + +.if ${MK_ZFS} != "no" +_zpool_influxdb= zpool_influxdb +.endif + +SUBDIR_PARALLEL= + +.include diff --git a/cddl/usr.libexec/zpool_influxdb/Makefile b/cddl/usr.libexec/zpool_influxdb/Makefile new file mode 100644 index 000000000000..2fb22f02ef32 --- /dev/null +++ b/cddl/usr.libexec/zpool_influxdb/Makefile @@ -0,0 +1,28 @@ +# $FreeBSD$ + +ZFSTOP= ${SRCTOP}/sys/contrib/openzfs + +.PATH: ${ZFSTOP}/cmd/zpool_influxdb +.PATH: ${ZFSTOP}/man/man8 + +PROG= zpool_influxdb +MAN= zpool_influxdb.8 +BINDIR?= /usr/libexec/zfs + +WARNS?= 0 +CSTD= c99 + +CFLAGS+= \ + -DIN_BASE \ + -I${ZFSTOP}/include \ + -I${ZFSTOP}/lib/libspl/include \ + -I${ZFSTOP}/lib/libspl/include/os/freebsd \ + -include ${ZFSTOP}/include/os/freebsd/spl/sys/ccompile.h \ + -DHAVE_ISSETUGID \ + -include ${SRCTOP}/sys/modules/zfs/zfs_config.h \ + -DSYSCONFDIR=\"/etc\" \ + -DPKGDATADIR=\"/usr/share/zfs\" + +LIBADD= nvpair spl zfs + +.include diff --git a/cddl/usr.libexec/zpool_influxdb/Makefile.depend b/cddl/usr.libexec/zpool_influxdb/Makefile.depend new file mode 100644 index 000000000000..53090da6e2fc --- /dev/null +++ b/cddl/usr.libexec/zpool_influxdb/Makefile.depend @@ -0,0 +1,31 @@ +# $FreeBSD$ +# Autogenerated - do NOT edit! + +DIRDEPS = \ + cddl/lib/libavl \ + cddl/lib/libnvpair \ + cddl/lib/libumem \ + cddl/lib/libuutil \ + cddl/lib/libzfs \ + cddl/lib/libzfs_core \ + gnu/lib/csu \ + include \ + include/xlocale \ + lib/${CSU_DIR} \ + lib/libc \ + lib/libcompiler_rt \ + lib/libexpat \ + lib/libgeom \ + lib/libmd \ + lib/libsbuf \ + lib/libthr \ + lib/libutil \ + lib/libz \ + lib/msun \ + + +.include + +.if ${DEP_RELDIR} == ${_DEP_RELDIR} +# local dependencies - needed for -jN in clean tree +.endif diff --git a/etc/mtree/BSD.debug.dist b/etc/mtree/BSD.debug.dist index 3ccdf50188c8..7df3113b183b 100644 --- a/etc/mtree/BSD.debug.dist +++ b/etc/mtree/BSD.debug.dist @@ -54,6 +54,8 @@ .. sm.bin .. + zfs + .. .. sbin .. diff --git a/etc/mtree/BSD.usr.dist b/etc/mtree/BSD.usr.dist index 9ae3acbe6eba..797bc06ab7df 100644 --- a/etc/mtree/BSD.usr.dist +++ b/etc/mtree/BSD.usr.dist @@ -156,6 +156,8 @@ .. sm.bin .. + zfs + .. .. local .. From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 15:58:01 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D4658656AD3; Sat, 10 Jul 2021 15:58:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMZS95Jc4z3qgN; Sat, 10 Jul 2021 15:58:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9E8BE26D2; Sat, 10 Jul 2021 15:58:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AFw1ZA080987; Sat, 10 Jul 2021 15:58:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AFw1tH080986; Sat, 10 Jul 2021 15:58:01 GMT (envelope-from git) Date: Sat, 10 Jul 2021 15:58:01 GMT Message-Id: <202107101558.16AFw1tH080986@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Martin Matuska Subject: git: a4c167674923 - stable/13 - zfs: update zfs_config.h to match current OpenZFS version (4f92fe0f5) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mm X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: a4c167674923394d985822fe281187779697ce2d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 15:58:01 -0000 The branch stable/13 has been updated by mm: URL: https://cgit.FreeBSD.org/src/commit/?id=a4c167674923394d985822fe281187779697ce2d commit a4c167674923394d985822fe281187779697ce2d Author: Martin Matuska AuthorDate: 2021-07-10 15:56:31 +0000 Commit: Martin Matuska CommitDate: 2021-07-10 15:56:31 +0000 zfs: update zfs_config.h to match current OpenZFS version (4f92fe0f5) TBD: fetch(3) support for keylocation=http(s):// (direct commit) --- sys/modules/zfs/zfs_config.h | 86 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 82 insertions(+), 4 deletions(-) diff --git a/sys/modules/zfs/zfs_config.h b/sys/modules/zfs/zfs_config.h index 92fb0571934e..c9d3a5feccaa 100644 --- a/sys/modules/zfs/zfs_config.h +++ b/sys/modules/zfs/zfs_config.h @@ -88,6 +88,9 @@ /* bdev_whole() is available */ /* #undef HAVE_BDEV_WHOLE */ +/* bio->bi_bdev->bd_disk exists */ +/* #undef HAVE_BIO_BDEV_DISK */ + /* bio->bi_opf is defined */ /* #undef HAVE_BIO_BI_OPF */ @@ -100,6 +103,9 @@ /* bio_*_io_acct() available */ /* #undef HAVE_BIO_IO_ACCT */ +/* bio_max_segs() is implemented */ +/* #undef HAVE_BIO_MAX_SEGS */ + /* bio_set_dev() is available */ /* #undef HAVE_BIO_SET_DEV */ @@ -151,6 +157,9 @@ /* blk_queue_write_cache() is GPL-only */ /* #undef HAVE_BLK_QUEUE_WRITE_CACHE_GPL_ONLY */ +/* Define if revalidate_disk() in block_device_operations */ +/* #undef HAVE_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK */ + /* Define to 1 if you have the Mac OS X function CFLocaleCopyCurrent in the CoreFoundation framework. */ /* #undef HAVE_CFLOCALECOPYCURRENT */ @@ -197,6 +206,9 @@ /* sops->dirty_inode() wants flags */ /* #undef HAVE_DIRTY_INODE_WITH_FLAGS */ +/* disk_*_io_acct() available */ +/* #undef HAVE_DISK_IO_ACCT */ + /* Define to 1 if you have the header file. */ #define HAVE_DLFCN_H 1 @@ -236,6 +248,9 @@ /* fops->fsync() without dentry */ /* #undef HAVE_FSYNC_WITHOUT_DENTRY */ +/* generic_fillattr requires struct user_namespace* */ +/* #undef HAVE_GENERIC_FILLATTR_USERNS */ + /* generic_*_io_acct() 3 arg available */ /* #undef HAVE_GENERIC_IO_ACCT_3ARG */ @@ -272,6 +287,12 @@ /* yes */ /* #undef HAVE_INODE_LOCK_SHARED */ +/* inode_owner_or_capable() exists */ +/* #undef HAVE_INODE_OWNER_OR_CAPABLE */ + +/* inode_owner_or_capable() takes user_ns */ +/* #undef HAVE_INODE_OWNER_OR_CAPABLE_IDMAPPED */ + /* inode_set_flags() exists */ /* #undef HAVE_INODE_SET_FLAGS */ @@ -290,6 +311,21 @@ /* in_compat_syscall() is available */ /* #undef HAVE_IN_COMPAT_SYSCALL */ +/* iops->create() takes struct user_namespace* */ +/* #undef HAVE_IOPS_CREATE_USERNS */ + +/* iops->mkdir() takes struct user_namespace* */ +/* #undef HAVE_IOPS_MKDIR_USERNS */ + +/* iops->mknod() takes struct user_namespace* */ +/* #undef HAVE_IOPS_MKNOD_USERNS */ + +/* iops->rename() takes struct user_namespace* */ +/* #undef HAVE_IOPS_RENAME_USERNS */ + +/* iops->symlink() takes struct user_namespace* */ +/* #undef HAVE_IOPS_SYMLINK_USERNS */ + /* iov_iter_advance() is available */ /* #undef HAVE_IOV_ITER_ADVANCE */ @@ -404,7 +440,7 @@ /* Define to 1 if you have the header file. */ #define HAVE_MEMORY_H 1 -/* iops->create()/mkdir()/mknod() take umode_t */ +/* iops->mkdir() takes umode_t */ /* #undef HAVE_MKDIR_UMODE_T */ /* Define to 1 if you have the `mlockall' function. */ @@ -425,6 +461,9 @@ /* Define if host toolchain supports PCLMULQDQ */ #define HAVE_PCLMULQDQ 1 +/* percpu_counter_add_batch() is defined */ +/* #undef HAVE_PERCPU_COUNTER_ADD_BATCH */ + /* percpu_counter_init() wants gfp_t */ /* #undef HAVE_PERCPU_COUNTER_INIT_WITH_GFP */ @@ -500,18 +539,33 @@ /* Define to 1 if you have the header file. */ #define HAVE_SECURITY_PAM_MODULES_H 1 -/* setattr_prepare() is available */ -/* #undef HAVE_SETATTR_PREPARE */ +/* setattr_prepare() is available, doesn't accept user_namespace */ +/* #undef HAVE_SETATTR_PREPARE_NO_USERNS */ -/* iops->set_acl() exists */ +/* setattr_prepare() accepts user_namespace */ +/* #undef HAVE_SETATTR_PREPARE_USERNS */ + +/* iops->set_acl() exists, takes 3 args */ /* #undef HAVE_SET_ACL */ +/* iops->set_acl() takes 4 args */ +/* #undef HAVE_SET_ACL_USERNS */ + /* set_cached_acl() is usable */ /* #undef HAVE_SET_CACHED_ACL_USABLE */ +/* set_special_state() exists */ +/* #undef HAVE_SET_SPECIAL_STATE */ + /* struct shrink_control exists */ /* #undef HAVE_SHRINK_CONTROL_STRUCT */ +/* kernel_siginfo_t exists */ +/* #undef HAVE_SIGINFO */ + +/* signal_stop() exists */ +/* #undef HAVE_SIGNAL_STOP */ + /* new shrinker callback wants 2 args */ /* #undef HAVE_SINGLE_SHRINKER_CALLBACK */ @@ -577,6 +631,9 @@ /* i_op->tmpfile() exists */ /* #undef HAVE_TMPFILE */ +/* i_op->tmpfile() has userns */ +/* #undef HAVE_TMPFILE_USERNS */ + /* totalhigh_pages() exists */ /* #undef HAVE_TOTALHIGH_PAGES */ @@ -592,6 +649,9 @@ /* Define to 1 if you have the header file. */ #define HAVE_UNISTD_H 1 +/* iops->getattr() takes struct user_namespace* */ +/* #undef HAVE_USERNS_IOPS_GETATTR */ + /* iops->getattr() takes a vfsmount */ /* #undef HAVE_VFSMOUNT_IOPS_GETATTR */ @@ -664,6 +724,9 @@ /* xattr_handler->set() wants xattr_handler */ /* #undef HAVE_XATTR_SET_HANDLER */ +/* xattr_handler->set() takes user_namespace */ +/* #undef HAVE_XATTR_SET_USERNS */ + /* Define if you have [z] */ #define HAVE_ZLIB 1 @@ -673,6 +736,21 @@ /* kernel exports FPU functions */ /* #undef KERNEL_EXPORTS_X86_FPU */ +/* TBD: fetch(3) support */ +#if 0 +/* whether the chosen libfetch is to be loaded at run-time */ +#define LIBFETCH_DYNAMIC 1 + +/* libfetch is fetch(3) */ +#define LIBFETCH_IS_FETCH 1 + +/* libfetch is libcurl */ +#define LIBFETCH_IS_LIBCURL 0 + +/* soname of chosen libfetch */ +#define LIBFETCH_SONAME "libfetch.so.6" +#endif + /* Define to the sub-directory where libtool stores uninstalled libraries. */ #define LT_OBJDIR ".libs/" From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 16:00:36 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78F33656B33; Sat, 10 Jul 2021 16:00:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMZW831fYz3r0s; Sat, 10 Jul 2021 16:00:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4FF542C96; Sat, 10 Jul 2021 16:00:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AG0aE0090297; Sat, 10 Jul 2021 16:00:36 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AG0aUH090296; Sat, 10 Jul 2021 16:00:36 GMT (envelope-from git) Date: Sat, 10 Jul 2021 16:00:36 GMT Message-Id: <202107101600.16AG0aUH090296@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Martin Matuska Subject: git: e1946a699d53 - stable/13 - zfs: merge openzfs/zfs@4f92fe0f5 (zfs-2.1-release) into stable/13 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mm X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e1946a699d530ede26a16598599c8aa6a2a70776 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 16:00:36 -0000 The branch stable/13 has been updated by mm: URL: https://cgit.FreeBSD.org/src/commit/?id=e1946a699d530ede26a16598599c8aa6a2a70776 commit e1946a699d530ede26a16598599c8aa6a2a70776 Merge: a4c167674923 4f92fe0f5c82 Author: Martin Matuska AuthorDate: 2021-07-10 15:58:35 +0000 Commit: Martin Matuska CommitDate: 2021-07-10 15:59:52 +0000 zfs: merge openzfs/zfs@4f92fe0f5 (zfs-2.1-release) into stable/13 OpenZFS release 2.1.0 Version bump only, no changes in code. Obtained from: OpenZFS OpenZFS commit: 4f92fe0f5c822f6802c6ec675809d7c112a46f2e OpenZFS tag: zfs-2.1.0 Relnotes: yes sys/contrib/openzfs/META | 2 +- sys/modules/zfs/zfs_config.h | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --cc sys/modules/zfs/zfs_config.h index c9d3a5feccaa,000000000000..ebad45d3def7 mode 100644,000000..100644 --- a/sys/modules/zfs/zfs_config.h +++ b/sys/modules/zfs/zfs_config.h @@@ -1,852 -1,0 +1,852 @@@ +/* + * $FreeBSD$ + */ + +/* zfs_config.h. Generated from zfs_config.h.in by configure. */ +/* zfs_config.h.in. Generated from configure.ac by autoheader. */ + +/* Define to 1 if translation of program messages to the user's native + language is requested. */ +/* #undef ENABLE_NLS */ + +/* bio_end_io_t wants 1 arg */ +/* #undef HAVE_1ARG_BIO_END_IO_T */ + +/* lookup_bdev() wants 1 arg */ +/* #undef HAVE_1ARG_LOOKUP_BDEV */ + +/* submit_bio() wants 1 arg */ +/* #undef HAVE_1ARG_SUBMIT_BIO */ + +/* bdi_setup_and_register() wants 2 args */ +/* #undef HAVE_2ARGS_BDI_SETUP_AND_REGISTER */ + +/* vfs_getattr wants 2 args */ +/* #undef HAVE_2ARGS_VFS_GETATTR */ + +/* zlib_deflate_workspacesize() wants 2 args */ +/* #undef HAVE_2ARGS_ZLIB_DEFLATE_WORKSPACESIZE */ + +/* bdi_setup_and_register() wants 3 args */ +/* #undef HAVE_3ARGS_BDI_SETUP_AND_REGISTER */ + +/* vfs_getattr wants 3 args */ +/* #undef HAVE_3ARGS_VFS_GETATTR */ + +/* vfs_getattr wants 4 args */ +/* #undef HAVE_4ARGS_VFS_GETATTR */ + +/* kernel has access_ok with 'type' parameter */ +/* #undef HAVE_ACCESS_OK_TYPE */ + +/* posix_acl has refcount_t */ +/* #undef HAVE_ACL_REFCOUNT */ + +/* Define if host toolchain supports AES */ +#define HAVE_AES 1 + +#ifdef __amd64__ +#ifndef RESCUE +/* Define if host toolchain supports AVX */ +#define HAVE_AVX 1 +#endif + +/* Define if host toolchain supports AVX2 */ +#define HAVE_AVX2 1 + +/* Define if host toolchain supports AVX512BW */ +#define HAVE_AVX512BW 1 + +/* Define if host toolchain supports AVX512CD */ +#define HAVE_AVX512CD 1 + +/* Define if host toolchain supports AVX512DQ */ +#define HAVE_AVX512DQ 1 + +/* Define if host toolchain supports AVX512ER */ +#define HAVE_AVX512ER 1 + +/* Define if host toolchain supports AVX512F */ +#define HAVE_AVX512F 1 + +/* Define if host toolchain supports AVX512IFMA */ +#define HAVE_AVX512IFMA 1 + +/* Define if host toolchain supports AVX512PF */ +#define HAVE_AVX512PF 1 + +/* Define if host toolchain supports AVX512VBMI */ +#define HAVE_AVX512VBMI 1 + +/* Define if host toolchain supports AVX512VL */ +#define HAVE_AVX512VL 1 +#endif + +/* bdev_check_media_change() exists */ +/* #undef HAVE_BDEV_CHECK_MEDIA_CHANGE */ + +/* bdev_whole() is available */ +/* #undef HAVE_BDEV_WHOLE */ + +/* bio->bi_bdev->bd_disk exists */ +/* #undef HAVE_BIO_BDEV_DISK */ + +/* bio->bi_opf is defined */ +/* #undef HAVE_BIO_BI_OPF */ + +/* bio->bi_status exists */ +/* #undef HAVE_BIO_BI_STATUS */ + +/* bio has bi_iter */ +/* #undef HAVE_BIO_BVEC_ITER */ + +/* bio_*_io_acct() available */ +/* #undef HAVE_BIO_IO_ACCT */ + +/* bio_max_segs() is implemented */ +/* #undef HAVE_BIO_MAX_SEGS */ + +/* bio_set_dev() is available */ +/* #undef HAVE_BIO_SET_DEV */ + +/* bio_set_dev() GPL-only */ +/* #undef HAVE_BIO_SET_DEV_GPL_ONLY */ + +/* bio_set_op_attrs is available */ +/* #undef HAVE_BIO_SET_OP_ATTRS */ + +/* blkdev_reread_part() exists */ +/* #undef HAVE_BLKDEV_REREAD_PART */ + +/* blkg_tryget() is available */ +/* #undef HAVE_BLKG_TRYGET */ + +/* blkg_tryget() GPL-only */ +/* #undef HAVE_BLKG_TRYGET_GPL_ONLY */ + +/* blk_alloc_queue() expects request function */ +/* #undef HAVE_BLK_ALLOC_QUEUE_REQUEST_FN */ + +/* blk_alloc_queue_rh() expects request function */ +/* #undef HAVE_BLK_ALLOC_QUEUE_REQUEST_FN_RH */ + +/* blk queue backing_dev_info is dynamic */ +/* #undef HAVE_BLK_QUEUE_BDI_DYNAMIC */ + +/* blk_queue_flag_clear() exists */ +/* #undef HAVE_BLK_QUEUE_FLAG_CLEAR */ + +/* blk_queue_flag_set() exists */ +/* #undef HAVE_BLK_QUEUE_FLAG_SET */ + +/* blk_queue_flush() is available */ +/* #undef HAVE_BLK_QUEUE_FLUSH */ + +/* blk_queue_flush() is GPL-only */ +/* #undef HAVE_BLK_QUEUE_FLUSH_GPL_ONLY */ + +/* blk_queue_secdiscard() is available */ +/* #undef HAVE_BLK_QUEUE_SECDISCARD */ + +/* blk_queue_secure_erase() is available */ +/* #undef HAVE_BLK_QUEUE_SECURE_ERASE */ + +/* blk_queue_write_cache() exists */ +/* #undef HAVE_BLK_QUEUE_WRITE_CACHE */ + +/* blk_queue_write_cache() is GPL-only */ +/* #undef HAVE_BLK_QUEUE_WRITE_CACHE_GPL_ONLY */ + +/* Define if revalidate_disk() in block_device_operations */ +/* #undef HAVE_BLOCK_DEVICE_OPERATIONS_REVALIDATE_DISK */ + +/* Define to 1 if you have the Mac OS X function CFLocaleCopyCurrent in the + CoreFoundation framework. */ +/* #undef HAVE_CFLOCALECOPYCURRENT */ + +/* Define to 1 if you have the Mac OS X function + CFLocaleCopyPreferredLanguages in the CoreFoundation framework. */ +/* #undef HAVE_CFLOCALECOPYPREFERREDLANGUAGES */ + +/* Define to 1 if you have the Mac OS X function CFPreferencesCopyAppValue in + the CoreFoundation framework. */ +/* #undef HAVE_CFPREFERENCESCOPYAPPVALUE */ + +/* check_disk_change() exists */ +/* #undef HAVE_CHECK_DISK_CHANGE */ + +/* clear_inode() is available */ +/* #undef HAVE_CLEAR_INODE */ + +/* dentry uses const struct dentry_operations */ +/* #undef HAVE_CONST_DENTRY_OPERATIONS */ + +/* copy_from_iter() is available */ +/* #undef HAVE_COPY_FROM_ITER */ + +/* copy_to_iter() is available */ +/* #undef HAVE_COPY_TO_ITER */ + +/* yes */ +/* #undef HAVE_CPU_HOTPLUG */ + +/* current_time() exists */ +/* #undef HAVE_CURRENT_TIME */ + +/* Define if the GNU dcgettext() function is already present or preinstalled. + */ +/* #undef HAVE_DCGETTEXT */ + +/* DECLARE_EVENT_CLASS() is available */ +/* #undef HAVE_DECLARE_EVENT_CLASS */ + +/* lookup_bdev() wants dev_t arg */ +/* #undef HAVE_DEVT_LOOKUP_BDEV */ + +/* sops->dirty_inode() wants flags */ +/* #undef HAVE_DIRTY_INODE_WITH_FLAGS */ + +/* disk_*_io_acct() available */ +/* #undef HAVE_DISK_IO_ACCT */ + +/* Define to 1 if you have the header file. */ +#define HAVE_DLFCN_H 1 + +/* d_make_root() is available */ +/* #undef HAVE_D_MAKE_ROOT */ + +/* d_prune_aliases() is available */ +/* #undef HAVE_D_PRUNE_ALIASES */ + +/* dops->d_revalidate() operation takes nameidata */ +/* #undef HAVE_D_REVALIDATE_NAMEIDATA */ + +/* eops->encode_fh() wants child and parent inodes */ +/* #undef HAVE_ENCODE_FH_WITH_INODE */ + +/* sops->evict_inode() exists */ +/* #undef HAVE_EVICT_INODE */ + +/* fops->aio_fsync() exists */ +/* #undef HAVE_FILE_AIO_FSYNC */ + +/* file_dentry() is available */ +/* #undef HAVE_FILE_DENTRY */ + +/* file_inode() is available */ +/* #undef HAVE_FILE_INODE */ + +/* iops->follow_link() cookie */ +/* #undef HAVE_FOLLOW_LINK_COOKIE */ + +/* iops->follow_link() nameidata */ +/* #undef HAVE_FOLLOW_LINK_NAMEIDATA */ + +/* fops->fsync() with range */ +/* #undef HAVE_FSYNC_RANGE */ + +/* fops->fsync() without dentry */ +/* #undef HAVE_FSYNC_WITHOUT_DENTRY */ + +/* generic_fillattr requires struct user_namespace* */ +/* #undef HAVE_GENERIC_FILLATTR_USERNS */ + +/* generic_*_io_acct() 3 arg available */ +/* #undef HAVE_GENERIC_IO_ACCT_3ARG */ + +/* generic_*_io_acct() 4 arg available */ +/* #undef HAVE_GENERIC_IO_ACCT_4ARG */ + +/* generic_readlink is global */ +/* #undef HAVE_GENERIC_READLINK */ + +/* generic_setxattr() exists */ +/* #undef HAVE_GENERIC_SETXATTR */ + +/* generic_write_checks() takes kiocb */ +/* #undef HAVE_GENERIC_WRITE_CHECKS_KIOCB */ + +/* Define if the GNU gettext() function is already present or preinstalled. */ +/* #undef HAVE_GETTEXT */ + +/* iops->get_link() cookie */ +/* #undef HAVE_GET_LINK_COOKIE */ + +/* iops->get_link() delayed */ +/* #undef HAVE_GET_LINK_DELAYED */ + +/* group_info->gid exists */ +/* #undef HAVE_GROUP_INFO_GID */ + +/* has_capability() is available */ +/* #undef HAVE_HAS_CAPABILITY */ + +/* Define if you have the iconv() function and it works. */ +#define HAVE_ICONV 1 + +/* yes */ +/* #undef HAVE_INODE_LOCK_SHARED */ + +/* inode_owner_or_capable() exists */ +/* #undef HAVE_INODE_OWNER_OR_CAPABLE */ + +/* inode_owner_or_capable() takes user_ns */ +/* #undef HAVE_INODE_OWNER_OR_CAPABLE_IDMAPPED */ + +/* inode_set_flags() exists */ +/* #undef HAVE_INODE_SET_FLAGS */ + +/* inode_set_iversion() exists */ +/* #undef HAVE_INODE_SET_IVERSION */ + +/* inode->i_*time's are timespec64 */ +/* #undef HAVE_INODE_TIMESPEC64_TIMES */ + +/* timestamp_truncate() exists */ +/* #undef HAVE_INODE_TIMESTAMP_TRUNCATE */ + +/* Define to 1 if you have the header file. */ +#define HAVE_INTTYPES_H 1 + +/* in_compat_syscall() is available */ +/* #undef HAVE_IN_COMPAT_SYSCALL */ + +/* iops->create() takes struct user_namespace* */ +/* #undef HAVE_IOPS_CREATE_USERNS */ + +/* iops->mkdir() takes struct user_namespace* */ +/* #undef HAVE_IOPS_MKDIR_USERNS */ + +/* iops->mknod() takes struct user_namespace* */ +/* #undef HAVE_IOPS_MKNOD_USERNS */ + +/* iops->rename() takes struct user_namespace* */ +/* #undef HAVE_IOPS_RENAME_USERNS */ + +/* iops->symlink() takes struct user_namespace* */ +/* #undef HAVE_IOPS_SYMLINK_USERNS */ + +/* iov_iter_advance() is available */ +/* #undef HAVE_IOV_ITER_ADVANCE */ + +/* iov_iter_count() is available */ +/* #undef HAVE_IOV_ITER_COUNT */ + +/* iov_iter_fault_in_readable() is available */ +/* #undef HAVE_IOV_ITER_FAULT_IN_READABLE */ + +/* iov_iter_init() is available */ +/* #undef HAVE_IOV_ITER_INIT */ + +/* iov_iter_init() is available */ +/* #undef HAVE_IOV_ITER_INIT_LEGACY */ + +/* iov_iter_revert() is available */ +/* #undef HAVE_IOV_ITER_REVERT */ + +/* iov_iter types are available */ +/* #undef HAVE_IOV_ITER_TYPES */ + +/* yes */ +/* #undef HAVE_IO_SCHEDULE_TIMEOUT */ + +/* Define to 1 if you have the `issetugid' function. */ +#define HAVE_ISSETUGID 1 + +/* kernel has kernel_fpu_* functions */ +/* #undef HAVE_KERNEL_FPU */ + +/* kernel has asm/fpu/api.h */ +/* #undef HAVE_KERNEL_FPU_API_HEADER */ + +/* kernel fpu internal */ +/* #undef HAVE_KERNEL_FPU_INTERNAL */ + +/* uncached_acl_sentinel() exists */ +/* #undef HAVE_KERNEL_GET_ACL_HANDLE_CACHE */ + +/* kernel does stack verification */ +/* #undef HAVE_KERNEL_OBJTOOL */ + +/* kernel has linux/objtool.h */ +/* #undef HAVE_KERNEL_OBJTOOL_HEADER */ + +/* kernel_read() take loff_t pointer */ +/* #undef HAVE_KERNEL_READ_PPOS */ + +/* timer_list.function gets a timer_list */ +/* #undef HAVE_KERNEL_TIMER_FUNCTION_TIMER_LIST */ + +/* struct timer_list has a flags member */ +/* #undef HAVE_KERNEL_TIMER_LIST_FLAGS */ + +/* timer_setup() is available */ +/* #undef HAVE_KERNEL_TIMER_SETUP */ + +/* kernel_write() take loff_t pointer */ +/* #undef HAVE_KERNEL_WRITE_PPOS */ + +/* kmem_cache_create_usercopy() exists */ +/* #undef HAVE_KMEM_CACHE_CREATE_USERCOPY */ + +/* kstrtoul() exists */ +/* #undef HAVE_KSTRTOUL */ + +/* ktime_get_coarse_real_ts64() exists */ +/* #undef HAVE_KTIME_GET_COARSE_REAL_TS64 */ + +/* ktime_get_raw_ts64() exists */ +/* #undef HAVE_KTIME_GET_RAW_TS64 */ + +/* kvmalloc exists */ +/* #undef HAVE_KVMALLOC */ + +/* kernel has large stacks */ +/* #undef HAVE_LARGE_STACKS */ + +/* Define if you have [aio] */ +/* #undef HAVE_LIBAIO */ + +/* Define if you have [blkid] */ +/* #undef HAVE_LIBBLKID */ + +/* Define if you have [crypto] */ +#define HAVE_LIBCRYPTO 1 + +/* Define if you have [tirpc] */ +/* #undef HAVE_LIBTIRPC */ + +/* Define if you have [udev] */ +/* #undef HAVE_LIBUDEV */ + +/* Define if you have [uuid] */ +/* #undef HAVE_LIBUUID */ + +/* lseek_execute() is available */ +/* #undef HAVE_LSEEK_EXECUTE */ + +/* makedev() is declared in sys/mkdev.h */ +/* #undef HAVE_MAKEDEV_IN_MKDEV */ + +/* makedev() is declared in sys/sysmacros.h */ +/* #undef HAVE_MAKEDEV_IN_SYSMACROS */ + +/* Noting that make_request_fn() returns blk_qc_t */ +/* #undef HAVE_MAKE_REQUEST_FN_RET_QC */ + +/* Noting that make_request_fn() returns void */ +/* #undef HAVE_MAKE_REQUEST_FN_RET_VOID */ + +/* Define to 1 if you have the header file. */ +#define HAVE_MEMORY_H 1 + +/* iops->mkdir() takes umode_t */ +/* #undef HAVE_MKDIR_UMODE_T */ + +/* Define to 1 if you have the `mlockall' function. */ +#define HAVE_MLOCKALL 1 + +/* lookup_bdev() wants mode arg */ +/* #undef HAVE_MODE_LOOKUP_BDEV */ + +/* Define if host toolchain supports MOVBE */ +#define HAVE_MOVBE 1 + +/* new_sync_read()/new_sync_write() are available */ +/* #undef HAVE_NEW_SYNC_READ */ + +/* iops->getattr() takes a path */ +/* #undef HAVE_PATH_IOPS_GETATTR */ + +/* Define if host toolchain supports PCLMULQDQ */ +#define HAVE_PCLMULQDQ 1 + +/* percpu_counter_add_batch() is defined */ +/* #undef HAVE_PERCPU_COUNTER_ADD_BATCH */ + +/* percpu_counter_init() wants gfp_t */ +/* #undef HAVE_PERCPU_COUNTER_INIT_WITH_GFP */ + +/* posix_acl_chmod() exists */ +/* #undef HAVE_POSIX_ACL_CHMOD */ + +/* posix_acl_from_xattr() needs user_ns */ +/* #undef HAVE_POSIX_ACL_FROM_XATTR_USERNS */ + +/* posix_acl_release() is available */ +/* #undef HAVE_POSIX_ACL_RELEASE */ + +/* posix_acl_release() is GPL-only */ +/* #undef HAVE_POSIX_ACL_RELEASE_GPL_ONLY */ + +/* posix_acl_valid() wants user namespace */ +/* #undef HAVE_POSIX_ACL_VALID_WITH_NS */ + +/* proc_ops structure exists */ +/* #undef HAVE_PROC_OPS_STRUCT */ + +/* iops->put_link() cookie */ +/* #undef HAVE_PUT_LINK_COOKIE */ + +/* iops->put_link() delayed */ +/* #undef HAVE_PUT_LINK_DELAYED */ + +/* iops->put_link() nameidata */ +/* #undef HAVE_PUT_LINK_NAMEIDATA */ + +/* If available, contains the Python version number currently in use. */ +#define HAVE_PYTHON "3.7" + +/* qat is enabled and existed */ +/* #undef HAVE_QAT */ + +/* iops->rename() wants flags */ +/* #undef HAVE_RENAME_WANTS_FLAGS */ + +/* REQ_DISCARD is defined */ +/* #undef HAVE_REQ_DISCARD */ + +/* REQ_FLUSH is defined */ +/* #undef HAVE_REQ_FLUSH */ + +/* REQ_OP_DISCARD is defined */ +/* #undef HAVE_REQ_OP_DISCARD */ + +/* REQ_OP_FLUSH is defined */ +/* #undef HAVE_REQ_OP_FLUSH */ + +/* REQ_OP_SECURE_ERASE is defined */ +/* #undef HAVE_REQ_OP_SECURE_ERASE */ + +/* REQ_PREFLUSH is defined */ +/* #undef HAVE_REQ_PREFLUSH */ + +/* revalidate_disk() is available */ +/* #undef HAVE_REVALIDATE_DISK */ + +/* revalidate_disk_size() is available */ +/* #undef HAVE_REVALIDATE_DISK_SIZE */ + +/* struct rw_semaphore has member activity */ +/* #undef HAVE_RWSEM_ACTIVITY */ + +/* struct rw_semaphore has atomic_long_t member count */ +/* #undef HAVE_RWSEM_ATOMIC_LONG_COUNT */ + +/* linux/sched/signal.h exists */ +/* #undef HAVE_SCHED_SIGNAL_HEADER */ + +/* Define to 1 if you have the header file. */ +#define HAVE_SECURITY_PAM_MODULES_H 1 + +/* setattr_prepare() is available, doesn't accept user_namespace */ +/* #undef HAVE_SETATTR_PREPARE_NO_USERNS */ + +/* setattr_prepare() accepts user_namespace */ +/* #undef HAVE_SETATTR_PREPARE_USERNS */ + +/* iops->set_acl() exists, takes 3 args */ +/* #undef HAVE_SET_ACL */ + +/* iops->set_acl() takes 4 args */ +/* #undef HAVE_SET_ACL_USERNS */ + +/* set_cached_acl() is usable */ +/* #undef HAVE_SET_CACHED_ACL_USABLE */ + +/* set_special_state() exists */ +/* #undef HAVE_SET_SPECIAL_STATE */ + +/* struct shrink_control exists */ +/* #undef HAVE_SHRINK_CONTROL_STRUCT */ + +/* kernel_siginfo_t exists */ +/* #undef HAVE_SIGINFO */ + +/* signal_stop() exists */ +/* #undef HAVE_SIGNAL_STOP */ + +/* new shrinker callback wants 2 args */ +/* #undef HAVE_SINGLE_SHRINKER_CALLBACK */ + +/* ->count_objects exists */ +/* #undef HAVE_SPLIT_SHRINKER_CALLBACK */ + +#if defined(__amd64__) || defined(__i386__) +/* Define if host toolchain supports SSE */ +#define HAVE_SSE 1 + +/* Define if host toolchain supports SSE2 */ +#define HAVE_SSE2 1 + +/* Define if host toolchain supports SSE3 */ +#define HAVE_SSE3 1 + +/* Define if host toolchain supports SSE4.1 */ +#define HAVE_SSE4_1 1 + +/* Define if host toolchain supports SSE4.2 */ +#define HAVE_SSE4_2 1 + +/* Define if host toolchain supports SSSE3 */ +#define HAVE_SSSE3 1 +#endif + +/* STACK_FRAME_NON_STANDARD is defined */ +/* #undef HAVE_STACK_FRAME_NON_STANDARD */ + +/* Define to 1 if you have the header file. */ +#define HAVE_STDINT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STDLIB_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRINGS_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_STRING_H 1 + +/* Define to 1 if you have the `strlcat' function. */ +#define HAVE_STRLCAT 1 + +/* Define to 1 if you have the `strlcpy' function. */ +#define HAVE_STRLCPY 1 + +/* submit_bio is member of struct block_device_operations */ +/* #undef HAVE_SUBMIT_BIO_IN_BLOCK_DEVICE_OPERATIONS */ + +/* super_setup_bdi_name() exits */ +/* #undef HAVE_SUPER_SETUP_BDI_NAME */ + +/* super_block->s_user_ns exists */ +/* #undef HAVE_SUPER_USER_NS */ + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_STAT_H 1 + +/* Define to 1 if you have the header file. */ +#define HAVE_SYS_TYPES_H 1 + +/* i_op->tmpfile() exists */ +/* #undef HAVE_TMPFILE */ + +/* i_op->tmpfile() has userns */ +/* #undef HAVE_TMPFILE_USERNS */ + +/* totalhigh_pages() exists */ +/* #undef HAVE_TOTALHIGH_PAGES */ + +/* kernel has totalram_pages() */ +/* #undef HAVE_TOTALRAM_PAGES_FUNC */ + +/* Define to 1 if you have the `udev_device_get_is_initialized' function. */ +/* #undef HAVE_UDEV_DEVICE_GET_IS_INITIALIZED */ + +/* kernel has __kernel_fpu_* functions */ +/* #undef HAVE_UNDERSCORE_KERNEL_FPU */ + +/* Define to 1 if you have the header file. */ +#define HAVE_UNISTD_H 1 + +/* iops->getattr() takes struct user_namespace* */ +/* #undef HAVE_USERNS_IOPS_GETATTR */ + +/* iops->getattr() takes a vfsmount */ +/* #undef HAVE_VFSMOUNT_IOPS_GETATTR */ + +/* aops->direct_IO() uses iovec */ +/* #undef HAVE_VFS_DIRECT_IO_IOVEC */ + +/* aops->direct_IO() uses iov_iter without rw */ +/* #undef HAVE_VFS_DIRECT_IO_ITER */ + +/* aops->direct_IO() uses iov_iter with offset */ +/* #undef HAVE_VFS_DIRECT_IO_ITER_OFFSET */ + +/* aops->direct_IO() uses iov_iter with rw and offset */ +/* #undef HAVE_VFS_DIRECT_IO_ITER_RW_OFFSET */ + +/* All required iov_iter interfaces are available */ +/* #undef HAVE_VFS_IOV_ITER */ + +/* fops->iterate() is available */ +/* #undef HAVE_VFS_ITERATE */ + +/* fops->iterate_shared() is available */ +/* #undef HAVE_VFS_ITERATE_SHARED */ + +/* fops->readdir() is available */ +/* #undef HAVE_VFS_READDIR */ + +/* fops->read/write_iter() are available */ +/* #undef HAVE_VFS_RW_ITERATE */ + +/* __vmalloc page flags exists */ +/* #undef HAVE_VMALLOC_PAGE_KERNEL */ + +/* yes */ +/* #undef HAVE_WAIT_ON_BIT_ACTION */ + +/* wait_queue_entry_t exists */ +/* #undef HAVE_WAIT_QUEUE_ENTRY_T */ + +/* wq_head->head and wq_entry->entry exist */ +/* #undef HAVE_WAIT_QUEUE_HEAD_ENTRY */ + +/* xattr_handler->get() wants dentry */ +/* #undef HAVE_XATTR_GET_DENTRY */ + +/* xattr_handler->get() wants both dentry and inode */ +/* #undef HAVE_XATTR_GET_DENTRY_INODE */ + +/* xattr_handler->get() wants xattr_handler */ +/* #undef HAVE_XATTR_GET_HANDLER */ + +/* xattr_handler has name */ +/* #undef HAVE_XATTR_HANDLER_NAME */ + +/* xattr_handler->list() wants dentry */ +/* #undef HAVE_XATTR_LIST_DENTRY */ + +/* xattr_handler->list() wants xattr_handler */ +/* #undef HAVE_XATTR_LIST_HANDLER */ + +/* xattr_handler->list() wants simple */ +/* #undef HAVE_XATTR_LIST_SIMPLE */ + +/* xattr_handler->set() wants dentry */ +/* #undef HAVE_XATTR_SET_DENTRY */ + +/* xattr_handler->set() wants both dentry and inode */ +/* #undef HAVE_XATTR_SET_DENTRY_INODE */ + +/* xattr_handler->set() wants xattr_handler */ +/* #undef HAVE_XATTR_SET_HANDLER */ + +/* xattr_handler->set() takes user_namespace */ +/* #undef HAVE_XATTR_SET_USERNS */ + +/* Define if you have [z] */ +#define HAVE_ZLIB 1 + +/* __posix_acl_chmod() exists */ +/* #undef HAVE___POSIX_ACL_CHMOD */ + +/* kernel exports FPU functions */ +/* #undef KERNEL_EXPORTS_X86_FPU */ + +/* TBD: fetch(3) support */ +#if 0 +/* whether the chosen libfetch is to be loaded at run-time */ +#define LIBFETCH_DYNAMIC 1 + +/* libfetch is fetch(3) */ +#define LIBFETCH_IS_FETCH 1 + +/* libfetch is libcurl */ +#define LIBFETCH_IS_LIBCURL 0 + +/* soname of chosen libfetch */ +#define LIBFETCH_SONAME "libfetch.so.6" +#endif + +/* Define to the sub-directory where libtool stores uninstalled libraries. */ +#define LT_OBJDIR ".libs/" + +/* make_request_fn() return type */ +/* #undef MAKE_REQUEST_FN_RET */ + +/* hardened module_param_call */ +/* #undef MODULE_PARAM_CALL_CONST */ + +/* struct shrink_control has nid */ +/* #undef SHRINK_CONTROL_HAS_NID */ + +/* Defined for legacy compatibility. */ +#define SPL_META_ALIAS ZFS_META_ALIAS + +/* Defined for legacy compatibility. */ +#define SPL_META_RELEASE ZFS_META_RELEASE + +/* Defined for legacy compatibility. */ +#define SPL_META_VERSION ZFS_META_VERSION + +/* True if ZFS is to be compiled for a FreeBSD system */ +#define SYSTEM_FREEBSD 1 + +/* True if ZFS is to be compiled for a Linux system */ +/* #undef SYSTEM_LINUX */ + +/* zfs debugging enabled */ +/* #undef ZFS_DEBUG */ + +/* /dev/zfs minor */ +/* #undef ZFS_DEVICE_MINOR */ + +/* enum node_stat_item contains NR_FILE_PAGES */ +/* #undef ZFS_ENUM_NODE_STAT_ITEM_NR_FILE_PAGES */ + +/* enum node_stat_item contains NR_INACTIVE_ANON */ +/* #undef ZFS_ENUM_NODE_STAT_ITEM_NR_INACTIVE_ANON */ + +/* enum node_stat_item contains NR_INACTIVE_FILE */ +/* #undef ZFS_ENUM_NODE_STAT_ITEM_NR_INACTIVE_FILE */ + +/* enum zone_stat_item contains NR_FILE_PAGES */ +/* #undef ZFS_ENUM_ZONE_STAT_ITEM_NR_FILE_PAGES */ + +/* enum zone_stat_item contains NR_INACTIVE_ANON */ +/* #undef ZFS_ENUM_ZONE_STAT_ITEM_NR_INACTIVE_ANON */ + +/* enum zone_stat_item contains NR_INACTIVE_FILE */ +/* #undef ZFS_ENUM_ZONE_STAT_ITEM_NR_INACTIVE_FILE */ + +/* global_node_page_state() exists */ +/* #undef ZFS_GLOBAL_NODE_PAGE_STATE */ + +/* global_zone_page_state() exists */ +/* #undef ZFS_GLOBAL_ZONE_PAGE_STATE */ + +/* Define to 1 if GPL-only symbols can be used */ +/* #undef ZFS_IS_GPL_COMPATIBLE */ + +/* Define the project alias string. */ - #define ZFS_META_ALIAS "zfs-2.1.0-FreeBSD_g508fff0e4" ++#define ZFS_META_ALIAS "zfs-2.1.0-FreeBSD_g4f92fe0f5" + +/* Define the project author. */ +#define ZFS_META_AUTHOR "OpenZFS" + +/* Define the project release date. */ +/* #undef ZFS_META_DATA */ + +/* Define the maximum compatible kernel version. */ +#define ZFS_META_KVER_MAX "5.13" + +/* Define the minimum compatible kernel version. */ +#define ZFS_META_KVER_MIN "3.10" + +/* Define the project license. */ +#define ZFS_META_LICENSE "CDDL" + +/* Define the libtool library 'age' version information. */ +/* #undef ZFS_META_LT_AGE */ + +/* Define the libtool library 'current' version information. */ +/* #undef ZFS_META_LT_CURRENT */ + +/* Define the libtool library 'revision' version information. */ +/* #undef ZFS_META_LT_REVISION */ + +/* Define the project name. */ +#define ZFS_META_NAME "zfs" + +/* Define the project release. */ - #define ZFS_META_RELEASE "FreeBSD_g508fff0e4" ++#define ZFS_META_RELEASE "FreeBSD_g4f92fe0f5" + +/* Define the project version. */ +#define ZFS_META_VERSION "2.1.0" + +/* count is located in percpu_ref.data */ +/* #undef ZFS_PERCPU_REF_COUNT_IN_DATA */ + From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:08:33 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 96BC66579B0; Sat, 10 Jul 2021 17:08:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc1Y3czmz4TxN; Sat, 10 Jul 2021 17:08:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 632563E59; Sat, 10 Jul 2021 17:08:33 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AH8Xl1075345; Sat, 10 Jul 2021 17:08:33 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AH8XTj075344; Sat, 10 Jul 2021 17:08:33 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:08:33 GMT Message-Id: <202107101708.16AH8XTj075344@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 666abb0888d2 - stable/13 - one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 666abb0888d277e82c6468851e015798e9a7629f Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:08:33 -0000 The branch stable/13 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=666abb0888d277e82c6468851e015798e9a7629f commit 666abb0888d277e82c6468851e015798e9a7629f Author: Warner Losh AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:07:26 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub PR: 143363, 143365, 143368, 143369, 143373, 143375, 214783 Sponsored by: Netflix (cherry picked from commit f39dd6a9784467f0db5886012b3f4b13899be6b8) --- contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-) diff --git a/contrib/one-true-awk/ChangeLog b/contrib/one-true-awk/ChangeLog index fd03b2bbca0b..dea4ed7e3187 100644 --- a/contrib/one-true-awk/ChangeLog +++ b/contrib/one-true-awk/ChangeLog @@ -1,3 +1,111 @@ +2020-07-30 Arnold D. Robbins + + By fiat, we use bison for $(YACC). Trying to accommodate + different versions didn't work. + + * makefile: Significant cleanup. Replace all ytab* references + with awkgram.tab.* and simplify definition of YACC. + * .gitignore: Remove ytab* references. + * b.c, lex.c, maketab.c, parse.c, run.c: Replace include of ytab.h + with awkgram.tab.h. + * lib.c, main.c, tran.c: Remove include of ytab.h, wasn't needed. + +2020-01-20 Arnold D. Robbins + + * run.c (openfile): Set the close-on-exec flag for file + and pipe redirections that aren't stdin/stdout/stderr. + +2020-01-06 Arnold D. Robbins + + Minor fixes. + * b.c (replace_repeat): Turn init_q back into an int. + * lex.c (string): Use \a instead of \007. + * tran.c (catstr): Use snprintf instead of sprintf. + +2020-01-01 Arnold D. Robbins + + * tran.c (syminit, arginit, envinit): Free sval member before + setting it. Thanks to valgrind. + * b.c: Small formatting cleanups in several routines. + +2019-12-27 Arnold D. Robbins + + * b.c (replace_repeat): Fix a bug whereby a{0,3} could match + four a's. Thanks to Anonymous AWK fan + for the report. Also, minor code formatting cleanups. + * testdir/T.int-expr: New file. + +2019-12-11 Arnold D. Robbins + + * README: Renamed to ... + * README.md: ... this. Cleaned up some as well, + including moving to Markdown. + +2019-11-08 Arnold D. Robbins + + * test/T.chem: Use $oldawk instead of hardwiring 'awk'. + * test/T.lilly: Remove gawk warnings from output, improves + portability. + +2019-10-17 Arnold D. Robbins + + Pull in systime() and strftime() from the NetBSD awk. + + * awk.1: Document the functions. + * run.c (bltin): Implement the functions. + * awk.h: Add defines for systime and strftime. + * lex.c: Add support for systime and strftime. + +2019-10-07 Arnold D. Robbins + + Integrate features from different *BSD versions of awk. + Gensub support from NetBSD. Bitwise functions from OpenBSD. + + * awk.h: Add defines for and, or, xor, compl, lshift and rshift. + * awkgram.y: Add support for gensub. + * maketab.c: Ditto. + * lex.c: Add support for gensub and bitwise functions. + * parse.c (node5, op5): New functions. + * proto.h (node5, op5): New declarations. + * run.c (bltin): Implement the bitwise functions. + (gensub): New function. + * awk.1: Document additional functions. + +2019-10-07 Arnold D. Robbins + + * b.c (fnematch): Change type of pbuf from unsigned char to char. + * proto.h (fnematch): Ditto. + +2019-10-06 Arnold D. Robbins + + * lib.c (readrec): Allow RS a regular expression. Imported + the code from the NetBSD awk. + * b.c (fnematch): New function for implementing the feature. + * awk.1: Updated. + * main.c (version): Updated. + +2019-06-24 Arnold D. Robbins + + * makefile: Revise to take into account there is no more awktest.tar, + add targets 'check' and 'test', and also 'testclean' to clean up + after test run. Have 'clean' and 'cleaner' depend upon 'testclean'. + +2019-06-23 Arnold D. Robbins + + * testdir: Extracted from awktest.tar and added to Git. + * awktest.tar: Removed. + +2019-06-06 Arnold D. Robbins + + * awk.1: Fix a typo, minor edits. + +2019-06-05 Arnold D. Robbins + + * b.c (relex): Count parentheses and treat umatched right paren + as a literal character. + * awktest.tar (testdir/T.re): Added a test case. + * main.c (version): Updated. + 2019-05-29 Arnold D. Robbins * lib.c (isclvar): Remove check for additional '=' after diff --git a/contrib/one-true-awk/FIXES b/contrib/one-true-awk/FIXES index 183eaedee47d..516458eee0c1 100644 --- a/contrib/one-true-awk/FIXES +++ b/contrib/one-true-awk/FIXES @@ -25,6 +25,229 @@ THIS SOFTWARE. This file lists all bug fixes, changes, etc., made since the AWK book was sent to the printers in August, 1987. +February 15, 2021: + Small fix so that awk will compile again with g++. Thanks to + Arnold Robbins. + +January 06, 2021: + Fix a decision bug with trailing stuff in lib.c:is_valid_number + after recent changes. Thanks to Ozan Yigit. + +December 18, 2020: + Fix problems converting inf and NaN values in lib.c:is_valid_number. + Enhance number to string conversion to do the right thing for + NaN and inf values. Things are now pretty much the same as in + gawk. (Found a gawk bug while we're at it.) Added a torture + test for these values. Thanks to Arnold Robbins. Allows closing + of PR #101. + +December 15, 2020: + Merge PR #99, which gets the right header for strcasecmp. + Thanks to GitHub user michaelforney. + +December 8, 2020: + Merge PR #98: Disallow hex data. Allow only +nan, -nan, + +inf, -inf (case independent) to give NaN and infinity values. + Improve things so that string to double conversion is only + done once, yielding something of a speedup. This obviate + PR #95. Thanks to Arnold Robbins. + +December 3, 2020: + Fix to argument parsing to avoid printing spurious newlines. + Thanks to Todd Miller. Merges PR #97. + +October 13, 2020: + Add casts before all the calls to malloc/calloc/realloc in order + to get it to compile with g++. Thanks to Arnold Robbins. + +August 16, 2020: + Additional fixes for DJGPP. Thanks to Eli Zaretskii for + the testing. + +August 7, 2020: + Merge PR #93, which adds casts to (void*) for debug prints + using the %p format specifier. Thanks to GitHub user YongHaoWu + ("Chris") for the fixes. + +August 4, 2020: + In run.c, use non-restartable multibyte routines to attain + portability to DJGPP. Should fix Issue 92. Thanks to Albert Wik + for the report and to Todd Miller for the suggested fix. + +July 30, 2020: + Merge PRs 88-91 which fix small bugs. Thanks to Todd Miller and + Tim van der Molen for the fixes. + + In order to make life easier, we move exclusively to bison + as the parser generator. + +July 2, 2020: + Merge PRs 85 and 86 which fix regressions. Thanks to + Tim van der Molen for the fixes. + +June 25, 2020: + Merge PRs 82 and 84. The latter fixes issue #83. Thanks to + Todd Miller and awkfan77. + +June 12, 2020: + Clear errno before calling errcheck to avoid any spurious errors + left over from previous calls that may have set it. Thanks to + Todd Miller for the fix, from PR #80. + + Fix Issue #78 by allowing \r to follow floating point numbers in + lib.c:is_number. Thanks to GitHub user ajcarr for the report + and to Arnold Robbins for the fix. + +June 5, 2020: + In fldbld(), make sure that inputFS is set before trying to + use it. Thanks to Steffen Nurpmeso + for the report. + +May 5, 2020: + Fix checks for compilers that can handle noreturn. Thanks to + GitHub user enh-google for pointing it out. Closes Issue #79. + +April 16, 2020: + Handle old compilers that don't support C11 (for noreturn). + Thanks to Arnold Robbins. + +April 5, 2020: + Use and noreturn instead of GCC attributes. + Thanks to GitHub user awkfan77. Closes PR #77. + +February 28, 2020: + More cleanups from Christos Zoulas: notably backslash continuation + inside strings removes the newline and a fix for RS = "^a". + Fix for address sanitizer-found problem. Thanks to GitHub user + enh-google. + +February 19, 2020: + More small cleanups from Christos Zoulas. + +February 18, 2020: + Additional cleanups from Christos Zoulas. It's no longer necessary + to use the -y flag to bison. + +February 6, 2020: + Additional small cleanups from Christos Zoulas. awk is now + a little more robust about reporting I/O errors upon exit. + +January 31, 2020: + Merge PR #70, which avoids use of variable length arrays. Thanks + to GitHub user michaelforney. Fix issue #60 ({0} in interval + expressions doesn't work). Also get all tests working again. + Thanks to Arnold Robbins. + +January 24, 2020: + A number of small cleanups from Christos Zoulas. Add the close + on exec flag to files/pipes opened for redirection; courtesy of + Arnold Robbins. + +January 19, 2020: + If POSIXLY_CORRECT is set in the environment, then sub and gsub + use POSIX rules for multiple backslashes. This fixes Issue #66, + while maintaining backwards compatibility. + +January 9, 2020: + Input/output errors on closing files are now fatal instead of + mere warnings. Thanks to Martijn Dekker . + +January 5, 2020: + Fix a bug in the concatentation of two string constants into + one done in the grammar. Fixes GitHub issue #61. Thanks + to GitHub user awkfan77 for pointing out the direction for + the fix. New test T.concat added to the test suite. + Fix a few memory leaks reported by valgrind, as well. + +December 27, 2019: + Fix a bug whereby a{0,3} could match four a's. Thanks to + "Anonymous AWK fan" for the report. + +December 11, 2019: + Further printf-related fixes for 32 bit systems. + Thanks again to Christos Zoulas. + +December 8, 2019: + Fix the return value of sprintf("%d") on 32 bit systems. + Thanks to Jim Lowe for the report and to Christos Zoulas + for the fix. + +November 10, 2019: + Convert a number of Boolean integer variables into + actual bools. Convert compile_time variable into an + enum and simplify some of the related code. Thanks + to Arnold Robbins. + +November 8, 2019: + Fix from Ori Bernstein to get UTF-8 characters instead of + bytes when FS = "". This is currently the only bit of + the One True Awk that understands multibyte characters. + From Arnold Robbins, apply some cleanups in the test suite. + +October 25, 2019: + More fixes and cleanups from NetBSD, courtesy of Christos + Zoulas. Merges PRs 54 and 55. + +October 24, 2019: + Import second round of code cleanups from NetBSD. Much thanks + to Christos Zoulas (GitHub user zoulasc). Merges PR 53. + Add an optimization for string concatenation, also from + Christos. + +October 17, 2019: + Import code cleanups from NetBSD. Much thanks to Christos + Zoulas (GitHub user zoulasc). Merges PR 51. + +October 6, 2019: + Import code from NetBSD awk that implements RS as a regular + expression. + +September 10, 2019: + Fixes for various array / memory overruns found via gcc's + -fsanitize=unknown. Thanks to Alexander Richardson (GitHub + user arichardson). Merges PRs 47 and 48. + +July 28, 2019: + Import grammar optimization from NetBSD: Two string constants + concatenated together get turned into a single string. + +July 26, 2019: + Support POSIX-specified C-style escape sequences "\a" (alarm) + and "\v" (vertical tab) in command line arguments and regular + expressions, further to the support for them in strings added on + Apr 9, 1989. These now no longer match as literal "a" and "v" + characters (as they don't on other awk implementations). + Thanks to Martijn Dekker. + +July 17, 2019: + Pull in a number of code cleanups and minor fixes from + Warner Losh's bsd-ota branch. The only user visible change + is the use of random(3) as the random number generator. + Thanks to Warner Losh for collecting all these fixes in + one easy place to get them from. + +July 16, 2019: + Fix field splitting to use FS value as of the time a record + was read or assigned to. Thanks to GitHub user Cody Mello (melloc) + for the fix. (Merged from his branch, via PR #42.) Updated + testdir/T.split per said PR as well. + +June 24, 2019: + Extract awktest.tar into testdir directory. Add some very + simple mechanics to the makefile for running the tests and + for cleaning up. No changes to awk itself. + +June 17, 2019: + Disallow deleting SYMTAB and its elements, which creates + use-after-free bugs. Thanks to GitHub user Cody Mello (melloc) + for the fix. (Merged from PR #43.) + +June 5, 2019: + Allow unmatched right parenthesis in a regular expression to + be treated literally. Fixes Issue #40. Thanks to GitHub user + Warner Losh (bsdimp) for the report. Thanks to Arnold Robbins + for the fix. + May 29,2019: Fix check for command line arguments to no longer require that first character after '=' not be another '='. Reverts change of @@ -34,7 +257,7 @@ May 29,2019: Apr 7, 2019: Update awktest.tar(p.50) to use modern options to sort. Needed for Android development. Thanks to GitHub user mohd-akram (Mohamed - Akram). From Comment #33. + Akram). From Issue #33. Mar 12, 2019: Added very simplistic support for cross-compiling in the @@ -54,7 +277,7 @@ Mar 3, 2019: #12: Avoid undefined behaviour when using ctype(3) functions in relex(). Thanks to GitHub user iamleot. #31: Make getline handle numeric strings, and update FIXES. Thanks - to GitHub user arnoldrobbins + to GitHub user arnoldrobbins. #32: maketab: support build systems with read-only source. Thanks to GitHub user enh. @@ -159,10 +382,10 @@ Jun 12, 2011: /pat/, \n /pat/ {...} is now legal, though bad style to use. added checks to new -v code that permits -vnospace; thanks to - ruslan ermilov for spotting this and providing the patch. + ruslan ermilov for spotting this and providing the patch. removed fixed limit on number of open files; thanks to aleksey - cheusov and christos zoulos. + cheusov and christos zoulos. fixed day 1 bug that resurrected deleted elements of ARGV when used as filenames (in lib.c). @@ -180,10 +403,10 @@ May 1, 2011: and arnold robbins, changed srand() to return the previous seed (which is 1 on the first call of srand). the seed is an Awkfloat internally though converted to unsigned int to - pass to the library srand(). thanks, everyone. + pass to the library srand(). thanks, everyone. fixed a subtle (and i hope low-probability) overflow error - in fldbld, by adding space for one extra \0. thanks to + in fldbld, by adding space for one extra \0. thanks to robert bassett for spotting this one and providing a fix. removed the files related to compilation on windows. i no @@ -220,7 +443,7 @@ Oct 8, 2008: Oct 23, 2007: minor fix in lib.c: increase inputFS to 100, change malloc - for fields to n+1. + for fields to n+1. fixed memory fault caused by out of order test in setsval. @@ -267,7 +490,7 @@ Jan 17, 2006: core dump on linux with BEGIN {nextfile}, now fixed. - removed some #ifdef's in run.c and lex.c that appear to no + removed some #ifdef's in run.c and lex.c that appear to no longer be necessary. Apr 24, 2005: @@ -281,8 +504,8 @@ Jan 14, 2005: rethinking it. Dec 31, 2004: - prevent overflow of -f array in main, head off potential error in - call of SYNTAX(), test malloc return in lib.c, all with thanks to + prevent overflow of -f array in main, head off potential error in + call of SYNTAX(), test malloc return in lib.c, all with thanks to todd miller. Dec 22, 2004: @@ -310,8 +533,8 @@ Nov 22, 2003: code known to man. fixed a storage leak in call() that appears to have been there since - 1983 or so -- a function without an explicit return that assigns a - string to a parameter leaked a Cell. thanks to moinak ghosh for + 1983 or so -- a function without an explicit return that assigns a + string to a parameter leaked a Cell. thanks to moinak ghosh for spotting this very subtle one. Jul 31, 2003: @@ -333,7 +556,7 @@ Jul 28, 2003: radix character in programs and command line arguments regardless of the locale; otherwise, the locale should prevail for input and output of numbers. so it's intended to work that way. - + i have rescinded the attempt to use strcoll in expanding shorthands in regular expressions (cclenter). its properties are much too surprising; for example [a-c] matches aAbBc in locale en_US but abBcC @@ -397,7 +620,7 @@ Nov 29, 2002: Jun 28, 2002: modified run/format() and tran/getsval() to do a slightly better job on using OFMT for output from print and CONVFMT for other - number->string conversions, as promised by posix and done by + number->string conversions, as promised by posix and done by gawk and mawk. there are still places where it doesn't work right if CONVFMT is changed; by then the STR attribute of the variable has been irrevocably set. thanks to arnold robbins for @@ -429,7 +652,7 @@ Feb 10, 2002: Jan 1, 2002: fflush() or fflush("") flushes all files and pipes. - length(arrayname) returns number of elements; thanks to + length(arrayname) returns number of elements; thanks to arnold robbins for suggestion. added a makefile.win to make it easier to build on windows. @@ -479,7 +702,7 @@ July 5, 2000: May 25, 2000: yet another attempt at making 8-bit input work, with another - band-aid in b.c (member()), and some (uschar) casts to head + band-aid in b.c (member()), and some (uschar) casts to head off potential errors in subscripts (like isdigit). also changed HAT to NCHARS-2. thanks again to santiago vila. @@ -526,7 +749,7 @@ Apr 21, 1999: the test case.) Apr 16, 1999: - with code kindly provided by Bruce Lilly, awk now parses + with code kindly provided by Bruce Lilly, awk now parses /=/ and similar constructs more sensibly in more places. Bruce also provided some helpful test cases. @@ -583,7 +806,7 @@ Jan 13, 1999: Oct 19, 1998: fixed a couple of bugs in getrec: could fail to update $0 - after a getline var; because inputFS wasn't initialized, + after a getline var; because inputFS wasn't initialized, could split $0 on every character, a misleading diversion. fixed caching bug in makedfa: LRU was actually removing @@ -731,7 +954,7 @@ May 2, 1996: input file. (thanks to arnold robbins for inspiration and code). small fixes to regexpr code: can now handle []], [[], and - variants; [] is now a syntax error, rather than matching + variants; [] is now a syntax error, rather than matching everything; [z-a] is now empty, not z. far from complete or correct, however. (thanks to jeffrey friedl for pointing out some awful behaviors.) diff --git a/contrib/one-true-awk/REGRESS b/contrib/one-true-awk/REGRESS index 7d3ded69d536..eb3b5d7ac70b 100755 --- a/contrib/one-true-awk/REGRESS +++ b/contrib/one-true-awk/REGRESS @@ -33,3 +33,7 @@ then fi REGRESS + +cd .. +cd bugs-fixed +REGRESS diff --git a/contrib/one-true-awk/awk.1 b/contrib/one-true-awk/awk.1 index b8e00cb60449..b3698eb07d1a 100644 --- a/contrib/one-true-awk/awk.1 +++ b/contrib/one-true-awk/awk.1 @@ -7,6 +7,10 @@ .fi .ft 1 .. +.de TF +.IP "" "\w'\fB\\$1\ \ \fP'u" +.PD 0 +.. .TH AWK 1 .CT 1 files prog_other .SH NAME @@ -48,7 +52,7 @@ matches the pattern. Each line is matched against the pattern portion of every pattern-action statement; the associated action is performed for each matched pattern. -The file name +The file name .B \- means the standard input. Any @@ -90,7 +94,7 @@ A pattern-action statement has the form: .IP .IB pattern " { " action " } .PP -A missing +A missing .BI { " action " } means print the line; a missing pattern always matches. @@ -209,7 +213,7 @@ or length of if no argument. .TP .B rand -random number on [0,1) +random number on [0,1). .TP .B srand sets seed for @@ -217,7 +221,7 @@ sets seed for and returns the previous seed. .TP .B int -truncates to an integer value +truncates to an integer value. .TP \fBsubstr(\fIs\fB, \fIm\fR [\fB, \fIn\^\fR]\fB)\fR the @@ -225,12 +229,11 @@ the substring of .I s that begins at position -.I m +.I m counted from 1. If no -.IR m , -use the rest of the string -.I +.IR n , +use the rest of the string. .TP .BI index( s , " t" ) the position in @@ -294,6 +297,25 @@ and .B gsub return the number of replacements. .TP +\fBgensub(\fIpat\fB, \fIrepl\fB, \fIhow\fR [\fB, \fItarget\fR]\fB)\fR +replaces instances of +.I pat +in +.I target +with +.IR repl . +If +.I how +is \fB"g"\fR or \fB"G"\fR, do so globally. Otherwise, +.I how +is a number indicating which occurrence to replace. If no +.IR target , +use +.BR $0 . +Return the resulting string; +.I target +is not modified. +.TP .BI sprintf( fmt , " expr" , " ...\fB) the string resulting from formatting .I expr ... @@ -302,13 +324,35 @@ according to the format .IR fmt . .TP +.B systime() +returns the current date and time as a standard +``seconds since the epoch'' value. +.TP +.BI strftime( fmt ", " timestamp\^ ) +formats +.I timestamp +(a value in seconds since the epoch) +according to +.IR fmt , +which is a format string as supported by +.IR strftime (3). +Both +.I timestamp +and +.I fmt +may be omitted; if no +.IR timestamp , +the current time of day is used, and if no +.IR fmt , +a default format of \fB"%a %b %e %H:%M:%S %Z %Y"\fR is used. +.TP .BI system( cmd ) executes .I cmd and returns its exit status. This will be \-1 upon error, .IR cmd 's exit status upon a normal exit, -256 + +256 + .I sig upon death-by-signal, where .I sig @@ -361,19 +405,26 @@ In all cases, returns 1 for a successful input, 0 for end of file, and \-1 for an error. .PP +The functions +.BR compl , +.BR and , +.BR or , +.BR xor , +.BR lshift , +and +.B rshift +peform the corresponding bitwise operations on their +operands, which are first truncated to integer. +.PP Patterns are arbitrary Boolean combinations (with .BR "! || &&" ) of regular expressions and relational expressions. Regular expressions are as in -.IR egrep (1) -except numeric quantifiers are not supported beyond the basic -.B + -and -.B ? -for quantities \&\f(CW"0 or 1"\fP and \&\f(CW"1 or more"\fP -respectively. +.IR egrep ; +see +.IR grep (1). Isolated regular expressions in a pattern apply to the entire line. Regular expressions may also occur in @@ -483,6 +534,11 @@ the length of a string matched by .TP .B RS input record separator (default newline). +If empty, blank lines separate records. +If more than one character long, +.B RS +is treated as a regular expression, and records are +separated by text matching the expression. .TP .B RSTART the start position of a string matched by @@ -502,6 +558,16 @@ functions may be called recursively. Parameters are local to the function; all other variables are global. Thus local variables may be created by providing excess parameters in the function definition. +.SH ENVIRONMENT VARIABLES +If +.B POSIXLY_CORRECT +is set in the environment, then +.I awk +follows the POSIX rules for +.B sub +and +.B gsub +with respect to consecutive backslashes and ampersands. .SH EXAMPLES .TP .EX @@ -546,8 +612,8 @@ BEGIN { # Simulate echo(1) .fi .EE .SH SEE ALSO -.IR grep (1), -.IR lex (1), +.IR grep (1), +.IR lex (1), .IR sed (1) .br A. V. Aho, B. W. Kernighan, P. J. Weinberger, @@ -558,8 +624,61 @@ There are no explicit conversions between numbers and strings. To force an expression to be treated as a number add 0 to it; to force it to be treated as a string concatenate \&\f(CW""\fP to it. -.br +.PP The scope rules for variables in functions are a botch; the syntax is worse. -.br +.PP Only eight-bit characters sets are handled correctly. +.SH UNUSUAL FLOATING-POINT VALUES +.I Awk +was designed before IEEE 754 arithmetic defined Not-A-Number (NaN) +and Infinity values, which are supported by all modern floating-point +hardware. +.PP +Because +.I awk +uses +.IR strtod (3) +and +.IR atof (3) +to convert string values to double-precision floating-point values, +modern C libraries also convert strings starting with +.B inf +and +.B nan +into infinity and NaN values respectively. This led to strange results, +with something like this: +.PP +.EX +.nf +echo nancy | awk '{ print $1 + 0 }' +.fi +.EE +.PP +printing +.B nan +instead of zero. +.PP +.I Awk +now follows GNU AWK, and prefilters string values before attempting +to convert them to numbers, as follows: +.TP +.I "Hexadecimal values" +Hexadecimal values (allowed since C99) convert to zero, as they did +prior to C99. +.TP +.I "NaN values" +The two strings +.B +nan +and +.B \-nan +(case independent) convert to NaN. No others do. +(NaNs can have signs.) +.TP +.I "Infinity values" +The two strings +.B +inf +and +.B \-inf +(case independent) convert to positive and negative infinity, respectively. +No others do. diff --git a/contrib/one-true-awk/awk.h b/contrib/one-true-awk/awk.h index 31d070aecddc..230eac41548c 100644 --- a/contrib/one-true-awk/awk.h +++ b/contrib/one-true-awk/awk.h @@ -23,6 +23,13 @@ THIS SOFTWARE. ****************************************************************/ #include +#include +#include +#if __STDC_VERSION__ <= 199901L +#define noreturn +#else +#include +#endif typedef double Awkfloat; @@ -30,24 +37,34 @@ typedef double Awkfloat; typedef unsigned char uschar; -#define xfree(a) { if ((a) != NULL) { free((void *) (a)); (a) = NULL; } } +#define xfree(a) { if ((a) != NULL) { free((void *)(intptr_t)(a)); (a) = NULL; } } +/* + * We sometimes cheat writing read-only pointers to NUL-terminate them + * and then put back the original value + */ +#define setptr(ptr, a) (*(char *)(intptr_t)(ptr)) = (a) -#define NN(p) ((p) ? (p) : "(null)") /* guaranteed non-null for dprintf +#define NN(p) ((p) ? (p) : "(null)") /* guaranteed non-null for DPRINTF */ #define DEBUG #ifdef DEBUG - /* uses have to be doubly parenthesized */ -# define dprintf(x) if (dbg) printf x +# define DPRINTF(...) if (dbg) printf(__VA_ARGS__) #else -# define dprintf(x) +# define DPRINTF(...) #endif -extern int compile_time; /* 1 if compiling, 0 if running */ -extern int safe; /* 0 => unsafe, 1 => safe */ +extern enum compile_states { + RUNNING, + COMPILING, + ERROR_PRINTING +} compile_time; + +extern bool safe; /* false => unsafe, true => safe */ #define RECSIZE (8 * 1024) /* sets limit on records, fields, etc., etc. */ extern int recsize; /* size of current record, orig RECSIZE */ +extern char EMPTY[]; /* this avoid -Wwritable-strings issues */ extern char **FS; extern char **RS; extern char **ORS; @@ -64,13 +81,11 @@ extern Awkfloat *RLENGTH; extern char *record; /* points to $0 */ extern int lineno; /* line number in awk program */ extern int errorflag; /* 1 if error has occurred */ -extern int donefld; /* 1 if record broken into fields */ -extern int donerec; /* 1 if record is valid (no fld has changed */ -extern char inputFS[]; /* FS at time of input, for field splitting */ - +extern bool donefld; /* true if record broken into fields */ +extern bool donerec; /* true if record is valid (no fld has changed */ extern int dbg; -extern char *patbeg; /* beginning of pattern matched */ +extern const char *patbeg; /* beginning of pattern matched */ extern int patlen; /* length of pattern matched. set in b.c */ /* Cell: all information about a variable or constant */ @@ -105,6 +120,7 @@ extern Cell *rsloc; /* RS */ extern Cell *rstartloc; /* RSTART */ extern Cell *rlengthloc; /* RLENGTH */ extern Cell *subseploc; /* SUBSEP */ +extern Cell *symtabloc; /* SYMTAB */ /* Cell.tval values: */ #define NUM 01 /* number value is valid */ @@ -134,12 +150,14 @@ extern Cell *subseploc; /* SUBSEP */ #define FTOUPPER 12 #define FTOLOWER 13 #define FFLUSH 14 -#define FAND 15 -#define FFOR 16 -#define FXOR 17 -#define FCOMPL 18 -#define FLSHIFT 19 -#define FRSHIFT 20 +#define FAND 15 +#define FFOR 16 +#define FXOR 17 +#define FCOMPL 18 +#define FLSHIFT 19 +#define FRSHIFT 20 +#define FSYSTIME 21 +#define FSTRFTIME 22 /* Node: parse tree is made of nodes, with Cell's at bottom */ @@ -167,7 +185,7 @@ extern Node *nullnode; #define CCOPY 6 #define CCON 5 #define CTEMP 4 -#define CNAME 3 +#define CNAME 3 #define CVAR 2 #define CFLD 1 #define CUNK 0 @@ -217,6 +235,7 @@ extern int pairstack[], paircnt; #define NCHARS (256+3) /* 256 handles 8-bit chars; 128 does 7-bit */ /* watch out in match(), etc. */ +#define HAT (NCHARS+2) /* matches ^ in regular expr */ #define NSTATES 32 #define HAT (NCHARS+2) /* matches ^ in regular expr */ /* NCHARS is 2**n */ @@ -232,16 +251,16 @@ typedef struct rrow { } rrow; typedef struct fa { - uschar gototab[NSTATES][HAT + 1]; - uschar out[NSTATES]; + unsigned int **gototab; + uschar *out; uschar *restr; - int *posns[NSTATES]; - int anchor; + int **posns; + int state_count; + bool anchor; int use; int initstat; int curstat; int accept; - int reset; struct rrow re[1]; /* variable: actual size set by calling malloc */ } fa; diff --git a/contrib/one-true-awk/awkgram.y b/contrib/one-true-awk/awkgram.y index e4abeeddcb6a..f37073d1f9ac 100644 --- a/contrib/one-true-awk/awkgram.y +++ b/contrib/one-true-awk/awkgram.y @@ -32,8 +32,8 @@ int yywrap(void) { return(1); } Node *beginloc = 0; Node *endloc = 0; -int infunc = 0; /* = 1 if in arglist or body of func */ *** 4848 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:47 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 810E76579E1; Sat, 10 Jul 2021 17:12:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6R1Z0Zz4V7J; Sat, 10 Jul 2021 17:12:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1E09A3AD5; Sat, 10 Jul 2021 17:12:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHClDS088195; Sat, 10 Jul 2021 17:12:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHClDo088194; Sat, 10 Jul 2021 17:12:47 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:47 GMT Message-Id: <202107101712.16AHClDo088194@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 421eebb4f8a3 - stable/12 - Update awk(1) manual to state an exception to egrep(1)-like RE syntax MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 421eebb4f8a346e7f91a5fde31e67b4193f289ff Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:47 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=421eebb4f8a346e7f91a5fde31e67b4193f289ff commit 421eebb4f8a346e7f91a5fde31e67b4193f289ff Author: Devin Teske AuthorDate: 2018-11-02 23:03:40 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:14 +0000 Update awk(1) manual to state an exception to egrep(1)-like RE syntax Reviewed by: imp, jmg MFC after: 3 days Sponsored by: Smule, Inc. Differential Revision: https://reviews.freebsd.org/D17739 (cherry picked from commit e0ff4751f0a9757f0bfc776b5da5421f6294daf8) --- contrib/one-true-awk/awk.1 | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/contrib/one-true-awk/awk.1 b/contrib/one-true-awk/awk.1 index b0d243b495cc..b2c36558fc83 100644 --- a/contrib/one-true-awk/awk.1 +++ b/contrib/one-true-awk/awk.1 @@ -352,9 +352,13 @@ Patterns are arbitrary Boolean combinations of regular expressions and relational expressions. Regular expressions are as in -.IR egrep ; -see -.IR grep (1). +.IR egrep (1) +except numeric quantifiers are not supported beyond the basic +.B + +and +.B ? +for quantities \&\f(CW"0 or 1"\fP and \&\f(CW"1 or more"\fP +respectively. Isolated regular expressions in a pattern apply to the entire line. Regular expressions may also occur in From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:48 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F991657E90; Sat, 10 Jul 2021 17:12:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6S2Ywkz4VH3; Sat, 10 Jul 2021 17:12:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3F2B24193; Sat, 10 Jul 2021 17:12:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCmVG088226; Sat, 10 Jul 2021 17:12:48 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCmDA088225; Sat, 10 Jul 2021 17:12:48 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:48 GMT Message-Id: <202107101712.16AHCmDA088225@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 330070826672 - stable/12 - Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 330070826672f3f63d17bc4e9ee1c81032d1669d Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:48 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=330070826672f3f63d17bc4e9ee1c81032d1669d commit 330070826672f3f63d17bc4e9ee1c81032d1669d Author: Warner Losh AuthorDate: 2019-06-02 16:25:07 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:14 +0000 Merge from upstream at 4189ef5d from https://github.com/onetrueawk/awk.git Note: this backs out a number of changes we've made to awk because they aren't upstream, but are on the vendor branch. Those will be reapplied. svn makes it needlessly difficult to know which ones, but at least r315426, r301289, and maybe r301691, though there may be others too. None of these are critical, so bisecting through this point is safe for all but awk regression tests :). (cherry picked from commit b5253557294400621041b8ce1dfbf11e124c1575) --- contrib/one-true-awk/ChangeLog | 245 +++++++++++++++++ contrib/one-true-awk/FIXES | 109 ++++++++ contrib/one-true-awk/LICENSE | 23 ++ contrib/one-true-awk/REGRESS | 35 +++ contrib/one-true-awk/awk.1 | 160 ++++++----- contrib/one-true-awk/awk.h | 10 +- contrib/one-true-awk/awkgram.y | 4 +- contrib/one-true-awk/b.c | 291 +++++++++++++++++++-- contrib/one-true-awk/bugs-fixed/README | 57 ++++ contrib/one-true-awk/bugs-fixed/a-format.awk | 3 + contrib/one-true-awk/bugs-fixed/a-format.bad | 3 + contrib/one-true-awk/bugs-fixed/a-format.ok | 1 + .../one-true-awk/bugs-fixed/concat-assign-same.awk | 4 + .../one-true-awk/bugs-fixed/concat-assign-same.bad | 2 + .../one-true-awk/bugs-fixed/concat-assign-same.ok | 2 + contrib/one-true-awk/bugs-fixed/decr-NF.awk | 11 + contrib/one-true-awk/bugs-fixed/decr-NF.bad | 5 + contrib/one-true-awk/bugs-fixed/decr-NF.ok | 5 + contrib/one-true-awk/bugs-fixed/fmt-overflow.awk | 1 + contrib/one-true-awk/bugs-fixed/fmt-overflow.ok | 1 + contrib/one-true-awk/bugs-fixed/fs-overflow.awk | 13 + .../one-true-awk/bugs-fixed/getline-numeric.awk | 6 + .../one-true-awk/bugs-fixed/getline-numeric.bad | 3 + contrib/one-true-awk/bugs-fixed/getline-numeric.in | 1 + contrib/one-true-awk/bugs-fixed/getline-numeric.ok | 3 + .../one-true-awk/bugs-fixed/missing-precision.awk | 1 + .../one-true-awk/bugs-fixed/missing-precision.ok | 2 + contrib/one-true-awk/bugs-fixed/negative-nf.awk | 1 + contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 + contrib/one-true-awk/bugs-fixed/nf-self-assign.awk | 6 + contrib/one-true-awk/bugs-fixed/nf-self-assign.bad | 1 + contrib/one-true-awk/bugs-fixed/nf-self-assign.ok | 1 + contrib/one-true-awk/bugs-fixed/numeric-fs.awk | 5 + contrib/one-true-awk/bugs-fixed/numeric-fs.ok | 3 + .../bugs-fixed/numeric-output-seps.awk | 8 + .../bugs-fixed/numeric-output-seps.bad | 2 + .../one-true-awk/bugs-fixed/numeric-output-seps.ok | 1 + contrib/one-true-awk/bugs-fixed/numeric-rs.awk | 6 + contrib/one-true-awk/bugs-fixed/numeric-rs.bad | 1 + contrib/one-true-awk/bugs-fixed/numeric-rs.ok | 4 + contrib/one-true-awk/bugs-fixed/numeric-subsep.awk | 5 + contrib/one-true-awk/bugs-fixed/numeric-subsep.bad | 1 + contrib/one-true-awk/bugs-fixed/numeric-subsep.ok | 1 + contrib/one-true-awk/bugs-fixed/ofs-rebuild.awk | 17 ++ contrib/one-true-awk/bugs-fixed/ofs-rebuild.bad | 1 + contrib/one-true-awk/bugs-fixed/ofs-rebuild.ok | 1 + contrib/one-true-awk/bugs-fixed/space.awk | 22 ++ contrib/one-true-awk/bugs-fixed/space.bad | 16 ++ contrib/one-true-awk/bugs-fixed/space.ok | 16 ++ .../bugs-fixed/split-fs-from-array.awk | 5 + .../one-true-awk/bugs-fixed/split-fs-from-array.ok | 1 + contrib/one-true-awk/bugs-fixed/string-conv.awk | 13 + contrib/one-true-awk/bugs-fixed/string-conv.bad | 4 + contrib/one-true-awk/bugs-fixed/string-conv.ok | 4 + .../one-true-awk/bugs-fixed/subsep-overflow.awk | 24 ++ contrib/one-true-awk/bugs-fixed/subsep-overflow.ok | 5 + contrib/one-true-awk/bugs-fixed/system-status.awk | 19 ++ contrib/one-true-awk/bugs-fixed/system-status.bad | 3 + contrib/one-true-awk/bugs-fixed/system-status.ok | 3 + contrib/one-true-awk/bugs-fixed/unary-plus.awk | 4 + contrib/one-true-awk/bugs-fixed/unary-plus.bad | 2 + contrib/one-true-awk/bugs-fixed/unary-plus.ok | 2 + contrib/one-true-awk/lex.c | 32 ++- contrib/one-true-awk/lib.c | 36 ++- contrib/one-true-awk/main.c | 13 +- contrib/one-true-awk/makefile | 53 ++-- contrib/one-true-awk/maketab.c | 13 +- contrib/one-true-awk/parse.c | 2 +- contrib/one-true-awk/proctab.c | 209 +++++++++++++++ contrib/one-true-awk/proto.h | 3 + contrib/one-true-awk/run.c | 187 ++++++++----- contrib/one-true-awk/tran.c | 183 +++++++++++-- 72 files changed, 1723 insertions(+), 219 deletions(-) diff --git a/contrib/one-true-awk/ChangeLog b/contrib/one-true-awk/ChangeLog new file mode 100644 index 000000000000..fd03b2bbca0b --- /dev/null +++ b/contrib/one-true-awk/ChangeLog @@ -0,0 +1,245 @@ +2019-05-29 Arnold D. Robbins + + * lib.c (isclvar): Remove check for additional '=' after + first one. No longer needed. + +2019-01-26 Arnold D. Robbins + + * main.c (version): Updated. + +2019-01-25 Arnold D. Robbins + + * run.c (awkgetline): Check for numeric value in all getline + variants. See the numeric-getline.* files in bugs-fixed directory. + +2018-08-29 Arnold D. Robbins + + * REGRESS: Check for existence of a.out. If not there, run + make. Enable core dumps for T.arnold system status test + to work on MacOS X. + +2018-08-22 Arnold D. Robbins + + * awktest.tar (testdir/T.expr): Fix test for unary plus. + +2018-08-22 Arnold D. Robbins + + * REGRESS: Extract tests if necessary, set PATH to include '.'. + * regdir/beebe.tar (Makefile): Fix longwrds test to prefix + sort with LC_ALL=C. + * awktest.tar: Updated from fixed test suite, directory + it extracts is now called 'testdir' to match what's in top-level + REGRESS script. + * regdir: Removed, as Brian wants to keep the test suite in + the tar file. + +2018-08-22 Arnold D. Robbins + + * FIXES, lib.c, run.c, makefile, main.c: Merge from Brian's tree. + * REGRESS: New file, from Brian. + * awktest.tar: Restored from Brian's tree. + +2018-08-22 Arnold D. Robbins + + * awkgram.y (UPLUS): New token. In the grammar, call op1() + with it. + * maketab.c (proc): Add entry for UPLUS. + * run.c (arith): Handle UPLUS. + * main.c (version): Updated. + * bugs-fixed/unary-plus.awk, bugs-fixed/unary-plus.bad, + bugs-fixed/unary-plus.ok: New files. + +2018-08-10 Arnold D. Robbins + + * TODO: Updated. + * awk.1: Improve use of macros, add some additional explanation + in a few places, alphabetize list of variables. + +2018-08-08 Arnold D. Robbins + + * awk.h (Cell): Add new field `fmt' to track xFMT value used + for a string conversion. + [CONVC, CONVO]: New flag macros. + * bugs-fixed/README: Updated. + * bugs-fixed/string-conv.awk, bugs-fixed/string-conv.bad, + bugs-fixed/string-conv.ok: New files. + * main.c (version): Updated. + * proto.h (flags2str): Add declaration. + * tran.c (setfval): Clear CONVC and CONVO flags and set vp->fmt + to NULL. + (setsval): Ditto. Add large comment and new code to manage + correct conversion of number to string based on various flags + and the value of vp->fmt. The idea is to not convert again + if xFMT is the same as before and we're doing the same conversion. + Otherwise, clear the old flags, set the new, and reconvert. + (flags2str): New function. For debug prints and for use from a debugger. + +2018-08-05 Arnold D. Robbins + + Fix filename conflicts in regdir where the only difference was + in letter case. This caused problems on Windows systems. + + * regdir/Compare.T1: Renamed from regdir/Compare.T. + * regdir/t.delete0: Renamed from regdir/t.delete. + * regdir/t.getline1: Renamed from regdir/t.getline. + * regdir/t.redir1: Renamed from regdir/t.redir. + * regdir/t.split1: Renamed from regdir/t.split. + * regdir/t.sub0: Renamed from regdir/t.sub. + * regdir/REGRESS: Adjusted. + +2018-08-04 Arnold D. Robbins + + With scalpel, tweasers, magnifying glass and bated breath, + borrow code from the NetBSD version of nawk to fix the years-old + bug whereby decrementing the value of NF did not change the + record. + + * lib.c (fldbld): Set donerec to 1 when done. + (setlastfld): New function. + * proto.h (setlastfld): Add declaration. + * run.c (copycell): Make code smarter about flags (from NetBSD code). + * tran.c (setfree): New function. + * tran.c (setfval): Normalize negative zero to positive zero. + If setting NF, clear donerec and call setlastfld(). + (setsval): Remove call to save_old_OFS(). If setting OFS, call + recbld(). If setting NF, clear donerec and call setlastfld(). + + As part of the process, revert OFS-related changes of 2018-05-22: + + * awk.h (saveOFS, saveOFSlen, save_old_OFS): Remove declarations. + * lib.c (recbld): Use *OFS instead of saveOFS. + * run.c (saveOFS, saveOFSlen, save_old_OFS): Remove. + * tran.c (syminit): Remove initialization of saveOFS and saveOFSlen. + + General stuff that goes along with all this: + + * bugs-fixed/README: Updated. + * bugs-fixed/decr-NF.awk, bugs-fixed/decr-NF.bad, + bugs-fixed/decr-NF.ok: New files. + * main.c (version): Updated. + * regdir/README.TESTS: Fix awk book title. + * regdir/T.misc: Revise test to match fixed code. + * run.c (format): Increase size of buffer used for %a test. (Unrelated + to NF or OFS, but fixes a compiler complaint.) + +2018-06-07 Arnold D. Robbins + + * regdir/beebe.tar: Fix longwrds.ok so that the test will pass. + The file was incorrectly sorted. + +2018-06-06 Arnold D. Robbins + + * regdir/T.lilly: Fix the bug again in the second instance + of the code. Thanks to BWK for pointing this out. + +2018-05-31 Arnold D. Robbins + + * regdir/T.lilly: Fix a syntax error and ordering bug + in creating the 'foo' file. + +2018-05-23 Arnold D. Robbins + + * awk.1: Remove standalone 'awk' at the top of file, it messed up + the formatting. Arrange built-in variable list in alphabetical + order. + +2018-05-23 Arnold D. Robbins + + * main.c (version): Add my email address and a date so that + users can tell this isn't straight BWK awk. + * README.md: Minor updates. + * TODO: Updated. + +2018-05-22 Arnold D. Robbins + + Add POSIX-required formats %a and %A. + + * run.c (format): Check for %a support in C library. If there, + allow %a and %A as valid formats. + * TODO: Updated. + * bugs-fixed/README: Updated. + * bugs-fixed/a-format.awk, bugs-fixed/a-format.bad, + bugs-fixed/a-format.ok: New files. + +2018-05-22 Arnold D. Robbins + + * FIXES: Restored a line from a much earlier version that + apparently got lost when the dates were reordered. + * TODO: Updated. + +2018-05-22 Arnold D. Robbins + + * README.md: New file. + +2018-05-22 Arnold D. Robbins + + * regdir/echo.c, regdir/time.c: Minor fixes to compile without + warning on current GCC / Linux. + +2018-05-22 Arnold D. Robbins + + * TODO: New file. + +2018-05-22 Arnold D. Robbins + + * makefile (gitadd, gitpush): Remove these targets. They + should not be automated and were incorrect for things that + would be done regularly. + +2018-05-22 Arnold D. Robbins + + Fix nawk so that [[:blank:]] only matches space and tab instead + of any whitespace character, originally made May 10, 2018. + See bugs-fixed/space.awk. + + This appears to have been a thinko on Brian's part. + + * b.c (charclasses): Use xisblank() function for [[:blank:]]. + * bugs-fixed/README: Updated. + * bugs-fixed/space.awk, bugs-fixed/space.bad, + bugs-fixed/space.ok: New files. + +2018-05-22 Arnold D. Robbins + + * .gitignore: New file. + +2018-05-22 Arnold D. Robbins + + Fix nawk to provide reasonable exit status for system(), + a la gawk, originally made March 12, 2016. See + bugs-fixed/system-status.awk. + + * run.c (bltin): For FSYSTEM, use the macros defined for wait(2) + to produce a reasonable exit value, instead of doing a floating-point + division by 256. + * awk.1: Document the return status values. + * bugs-fixed/README: Updated. + * bugs-fixed/system-status.awk, bugs-fixed/system-status.bad, + bugs-fixed/system-status.ok: New files. + +2018-05-22 Arnold D. Robbins + + Bug fix with respect to rebuilding a record, originally + made August 19, 2014. See bugs-fixed/ofs-rebuild.awk. + + * awk.h (saveOFS, saveOFSlen): Declare new variables. + * lib.c (recbld): Use them when rebuilding the record. + * run.c (saveOFS, saveOFSlen): Define new variables. + (save_old_OFS): New function to save OFS aside. + * tran.c (syminit): Initialize saveOFS and saveOFSlen. + (setsval): If setting a field, call save_old_OFS(). + * bugs-fixed/README, bugs-fixed/ofs-rebuild.awk, + bugs-fixed/ofs-rebuild.bad, bugs-fixed/ofs-rebuild.ok: New files. + +2018-05-22 Arnold D. Robbins + + * makefile (YACC): Use bison. + +2018-05-22 Arnold D. Robbins + + * ChangeLog: Created. + * regdir: Created. Based on contents of awktest.a. + * .gitattributes: Created, to preserve CR LF in regdir/t.crlf. + * awktest.a: Removed. + * regdir/T.gawk, regdir/T.latin1: Updated from awktest.tar. + * awktest.tar: Removed. diff --git a/contrib/one-true-awk/FIXES b/contrib/one-true-awk/FIXES index c78aabc511f3..183eaedee47d 100644 --- a/contrib/one-true-awk/FIXES +++ b/contrib/one-true-awk/FIXES @@ -25,6 +25,113 @@ THIS SOFTWARE. This file lists all bug fixes, changes, etc., made since the AWK book was sent to the printers in August, 1987. +May 29,2019: + Fix check for command line arguments to no longer require that + first character after '=' not be another '='. Reverts change of + August 11, 1989. Thanks to GitHub user Jamie Landeg Jones for + pointing out the issue; from Issue #38. + +Apr 7, 2019: + Update awktest.tar(p.50) to use modern options to sort. Needed + for Android development. Thanks to GitHub user mohd-akram (Mohamed + Akram). From Comment #33. + +Mar 12, 2019: + Added very simplistic support for cross-compiling in the + makefile. We are NOT going to go in the direction of the + autotools, though. Thanks to GitHub user nee-san for + the basic change. (Merged from PR #34.) + +Mar 5, 2019: + Added support for POSIX-standard interval expressions (a.k.a. + bounds, a.k.a. repetition expressions) in regular expressions, + backported (via NetBSD) from Apple awk-24 (20070501). + Thanks to Martijn Dekker for the port. + (Merged from PR #30.) + +Mar 3, 2019: + Merge PRs as follows: + #12: Avoid undefined behaviour when using ctype(3) functions in + relex(). Thanks to GitHub user iamleot. + #31: Make getline handle numeric strings, and update FIXES. Thanks + to GitHub user arnoldrobbins + #32: maketab: support build systems with read-only source. Thanks + to GitHub user enh. + +Jan 25, 2019: + Make getline handle numeric strings properly in all cases. + (Thanks, Arnold.) + +Jan 21, 2019: + Merged a number of small fixes from GitHub pull requests. + Thanks to GitHub users Arnold Robbins (arnoldrobbins), + Cody Mello (melloc) and Christoph Junghans (junghans). + PR numbers: 13-21, 23, 24, 27. + +Oct 25, 2018: + Added test in maketab.c to prevent generating a proctab entry + for YYSTYPE_IS_DEFINED. It was harmless but some gcc settings + generated a warning message. Thanks to Nan Xiao for report. + +Aug 27, 2018: + Disallow '$' in printf formats; arguments evaluated in order + and printed in order. + + Added some casts to silence warnings on debugging printfs. + (Thanks, Arnold.) + +Aug 23, 2018: + A long list of fixes courtesy of Arnold Robbins, + to whom profound thanks. + + 1. ofs-rebuild: OFS value used to rebuild the record was incorrect. + Fixed August 19, 2014. Revised fix August 2018. + + 2. system-status: Instead of a floating-point division by 256, use + the wait(2) macros to create a reasonable exit status. + Fixed March 12, 2016. + + 3. space: Use provided xisblank() function instead of ispace() for + matching [[:blank:]]. + + 4. a-format: Add POSIX standard %a and %A to supported formats. Check + at runtime that this format is available. + + 5. decr-NF: Decrementing NF did not change $0. This is a decades-old + bug. There are interactions with the old and new value of OFS as well. + Most of the fix came from the NetBSD awk. + + 6. string-conv: String conversions of scalars were sticky. Once a + conversion to string happened, even with OFMT, that value was used until + a new numeric value was assigned, even if OFMT differed from CONVFMT, + and also if CONVFMT changed. + + 7. unary-plus: Unary plus on a string constant returned the string. + Instead, it should convert the value to numeric and give that value. + + Also added Arnold's tests for these to awktest.tar as T.arnold. + +Aug 15, 2018: + fixed mangled awktest.tar (thanks, Arnold), posted all + current (very minor) fixes to github / onetrueawk + +Jun 7, 2018: + (yes, a long layoff) + Updated some broken tests (beebe.tar, T.lilly) + [thanks to Arnold Robbins] + +Mar 26, 2015: + buffer overflow in error reporting; thanks to tobias ulmer + and john-mark gurney for spotting it and the fix. + +Feb 4, 2013: + cleaned up a handful of tests that didn't seem to actually + test for correct behavior: T.latin1, T.gawk. + +Jan 5, 2013: + added ,NULL initializer to static Cells in run.c; not really + needed but cleaner. Thanks to Michael Bombardieri. + Dec 20, 2012: fiddled makefile to get correct yacc and bison flags. pick yacc (linux) or bison (mac) as necessary. @@ -493,6 +600,8 @@ May 12, 1998: Mar 12, 1998: added -V to print version number and die. +[notify dave kerns, dkerns@dacsoup.ih.lucent.com] + Feb 11, 1998: subtle silent bug in lex.c: if the program ended with a number longer than 1 digit, part of the input would be pushed back and diff --git a/contrib/one-true-awk/LICENSE b/contrib/one-true-awk/LICENSE new file mode 100644 index 000000000000..07dfd7b73b11 --- /dev/null +++ b/contrib/one-true-awk/LICENSE @@ -0,0 +1,23 @@ +/**************************************************************** +Copyright (C) Lucent Technologies 1997 +All Rights Reserved + +Permission to use, copy, modify, and distribute this software and +its documentation for any purpose and without fee is hereby +granted, provided that the above copyright notice appear in all +copies and that both that the copyright notice and this +permission notice and warranty disclaimer appear in supporting +documentation, and that the name Lucent Technologies or any of +its entities not be used in advertising or publicity pertaining +to distribution of the software without specific, written prior +permission. + +LUCENT DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, +INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. +IN NO EVENT SHALL LUCENT OR ANY OF ITS ENTITIES BE LIABLE FOR ANY +SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER +IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, +ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF +THIS SOFTWARE. +****************************************************************/ diff --git a/contrib/one-true-awk/REGRESS b/contrib/one-true-awk/REGRESS new file mode 100755 index 000000000000..7d3ded69d536 --- /dev/null +++ b/contrib/one-true-awk/REGRESS @@ -0,0 +1,35 @@ +#! /bin/sh + +case `uname` in +CYGWIN) EXE=a.exe ;; +*) EXE=a.out ;; +esac + +if [ ! -f $EXE ] +then + make || exit 1 +fi + +if [ -d testdir ] +then + true # do nothing +elif [ -f awktest.tar ] +then + echo extracting testdir + tar -xpf awktest.tar +else + echo $0: No testdir directory and no awktest.tar to extract it from! >&2 + exit 1 +fi + +cd testdir +pwd +PATH=.:$PATH +export PATH +if (ulimit -c unlimited > /dev/null 2>&1) +then + # Workaround broken default on MacOS X + ulimit -c unlimited +fi + +REGRESS diff --git a/contrib/one-true-awk/awk.1 b/contrib/one-true-awk/awk.1 index b2c36558fc83..b8e00cb60449 100644 --- a/contrib/one-true-awk/awk.1 +++ b/contrib/one-true-awk/awk.1 @@ -7,7 +7,6 @@ .fi .ft 1 .. -awk .TH AWK 1 .CT 1 files prog_other .SH NAME @@ -36,7 +35,7 @@ awk \- pattern-directed scanning and processing language scans each input .I file for lines that match any of a set of patterns specified literally in -.IR prog +.I prog or in one or more files specified as .B \-f @@ -53,7 +52,7 @@ The file name .B \- means the standard input. Any -.IR file +.I file of the form .I var=value is treated as an assignment, not a filename, @@ -70,12 +69,12 @@ any number of options may be present. The .B \-F -.IR fs +.I fs option defines the input field separator to be the regular expression -.IR fs. +.IR fs . .PP An input line is normally made up of fields separated by white space, -or by regular expression +or by the regular expression .BR FS . The fields are denoted .BR $1 , @@ -87,7 +86,7 @@ If .BR FS is null, the input line is split into one field per character. .PP -A pattern-action statement has the form +A pattern-action statement has the form: .IP .IB pattern " { " action " } .PP @@ -101,7 +100,7 @@ An action is a sequence of statements. A statement can be one of the following: .PP .EX -.ta \w'\f(CWdelete array[expression]'u +.ta \w'\f(CWdelete array[expression]\fR'u .RS .nf .ft CW @@ -145,7 +144,7 @@ The operators are also available in expressions. Variables may be scalars, array elements (denoted -.IB x [ i ] ) +.IB x [ i ] \fR) or fields. Variables are initialized to the null string. Array subscripts may be any string, @@ -161,11 +160,11 @@ The .B print statement prints its arguments on the standard output (or on a file if -.BI > file +.BI > " file or -.BI >> file +.BI >> " file is present or on a pipe if -.BI | cmd +.BI | " cmd is present), separated by the current output field separator, and terminated by the output record separator. .I file @@ -176,9 +175,10 @@ identical string values in different statements denote the same open file. The .B printf -statement formats its expression list according to the format +statement formats its expression list according to the +.I format (see -.IR printf (3)) . +.IR printf (3)). The built-in function .BI close( expr ) closes the file or pipe @@ -189,13 +189,13 @@ flushes any buffered output for the file or pipe .IR expr . .PP The mathematical functions +.BR atan2 , +.BR cos , .BR exp , .BR log , -.BR sqrt , .BR sin , -.BR cos , and -.BR atan2 +.B sqrt are built in. Other built-in functions: .TF length @@ -203,7 +203,8 @@ Other built-in functions: .B length the length of its argument taken as a string, -or of +number of elements in an array for an array argument, +or length of .B $0 if no argument. .TP @@ -218,14 +219,18 @@ and returns the previous seed. .B int truncates to an integer value .TP -.BI substr( s , " m" , " n\fB) +\fBsubstr(\fIs\fB, \fIm\fR [\fB, \fIn\^\fR]\fB)\fR the .IR n -character substring of .I s that begins at position -.IR m +.I m counted from 1. +If no +.IR m , +use the rest of the string +.I .TP .BI index( s , " t" ) the position in @@ -246,14 +251,14 @@ and .B RLENGTH are set to the position and length of the matched string. .TP -.BI split( s , " a" , " fs\fB) +\fBsplit(\fIs\fB, \fIa \fR[\fB, \fIfs\^\fR]\fB)\fR splits the string .I s into array elements -.IB a [1] , -.IB a [2] , +.IB a [1] \fR, +.IB a [2] \fR, \&..., -.IB a [ n ] , +.IB a [ n ] \fR, and returns .IR n . The separation is done with the regular expression @@ -266,7 +271,7 @@ is not given. An empty string as field separator splits the string into one array element per character. .TP -.BI sub( r , " t" , " s\fB) +\fBsub(\fIr\fB, \fIt \fR[, \fIs\^\fR]\fB) substitutes .I t for the first occurrence of the regular expression @@ -279,7 +284,7 @@ is not given, .B $0 is used. .TP -.B gsub +\fBgsub(\fIr\fB, \fIt \fR[, \fIs\^\fR]\fB) same as .B sub except that all occurrences of the regular expression @@ -289,18 +294,28 @@ and .B gsub return the number of replacements. .TP -.BI sprintf( fmt , " expr" , " ...\fB ) +.BI sprintf( fmt , " expr" , " ...\fB) the string resulting from formatting .I expr ... according to the .IR printf (3) format -.I fmt +.IR fmt . .TP .BI system( cmd ) executes .I cmd -and returns its exit status +and returns its exit status. This will be \-1 upon error, +.IR cmd 's +exit status upon a normal exit, +256 + +.I sig +upon death-by-signal, where +.I sig +is the number of the murdering signal, +or 512 + +.I sig +if there was a core dump. .TP .BI tolower( str ) returns a copy of @@ -321,7 +336,7 @@ sets .B $0 to the next input record from the current input file; .B getline -.BI < file +.BI < " file sets .B $0 to the next record from @@ -363,7 +378,7 @@ Isolated regular expressions in a pattern apply to the entire line. Regular expressions may also occur in relational expressions, using the operators -.BR ~ +.B ~ and .BR !~ . .BI / re / @@ -387,8 +402,12 @@ A relational expression is one of the following: .br .BI ( expr , expr,... ") in " array-name .PP -where a relop is any of the six relational operators in C, -and a matchop is either +where a +.I relop +is any of the six relational operators in C, +and a +.I matchop +is either .B ~ (matches) or @@ -409,57 +428,68 @@ and after the last. and .B END do not combine with other patterns. +They may appear multiple times in a program and execute +in the order they are read by +.IR awk . .PP Variable names with special meanings: .TF FILENAME .TP +.B ARGC +argument count, assignable. +.TP +.B ARGV +argument array, assignable; +non-null members are taken as filenames. +.TP .B CONVFMT conversion format used when converting numbers (default -.BR "%.6g" ) +.BR "%.6g" ). +.TP +.B ENVIRON +array of environment variables; subscripts are names. +.TP +.B FILENAME +the name of the current input file. +.TP +.B FNR +ordinal number of the current record in the current file. .TP .B FS regular expression used to separate fields; also settable by option -.BI \-F fs. +.BI \-F fs\fR. .TP .BR NF -number of fields in the current record +number of fields in the current record. .TP .B NR -ordinal number of the current record -.TP -.B FNR -ordinal number of the current record in the current file -.TP -.B FILENAME -the name of the current input file +ordinal number of the current record. .TP -.B RS -input record separator (default newline) +.B OFMT +output format for numbers (default +.BR "%.6g" ). .TP .B OFS -output field separator (default blank) +output field separator (default space). .TP .B ORS -output record separator (default newline) +output record separator (default newline). .TP -.B OFMT -output format for numbers (default -.BR "%.6g" ) -.TP -.B SUBSEP -separates multiple subscripts (default 034) +.B RLENGTH +the length of a string matched by +.BR match . .TP -.B ARGC -argument count, assignable +.B RS +input record separator (default newline). .TP -.B ARGV -argument array, assignable; -non-null members are taken as filenames +.B RSTART +the start position of a string matched by +.BR match . .TP -.B ENVIRON -array of environment variables; subscripts are names. +.B SUBSEP +separates multiple subscripts (default 034). .PD .PP Functions may be defined (at the position of a pattern-action statement) thus: @@ -490,7 +520,7 @@ BEGIN { FS = ",[ \et]*|[ \et]+" } .EE .ns .IP -Same, with input fields separated by comma and/or blanks and tabs. +Same, with input fields separated by comma and/or spaces and tabs. .PP .EX .nf @@ -516,13 +546,13 @@ BEGIN { # Simulate echo(1) .fi .EE .SH SEE ALSO +.IR grep (1), .IR lex (1), .IR sed (1) .br A. V. Aho, B. W. Kernighan, P. J. Weinberger, -.I -The AWK Programming Language, -Addison-Wesley, 1988. ISBN 0-201-07981-X +.IR "The AWK Programming Language" , +Addison-Wesley, 1988. ISBN 0-201-07981-X. .SH BUGS There are no explicit conversions between numbers and strings. To force an expression to be treated as a number add 0 to it; @@ -531,3 +561,5 @@ to force it to be treated as a string concatenate .br The scope rules for variables in functions are a botch; the syntax is worse. +.br +Only eight-bit characters sets are handled correctly. diff --git a/contrib/one-true-awk/awk.h b/contrib/one-true-awk/awk.h index 5564af7619a4..b16c2f36f828 100644 --- a/contrib/one-true-awk/awk.h +++ b/contrib/one-true-awk/awk.h @@ -81,7 +81,8 @@ typedef struct Cell { char *nval; /* name, for variables only */ char *sval; /* string value */ Awkfloat fval; /* value as number */ - int tval; /* type info: STR|NUM|ARR|FCN|FLD|CON|DONTFREE */ + int tval; /* type info: STR|NUM|ARR|FCN|FLD|CON|DONTFREE|CONVC|CONVO */ + char *fmt; /* CONVFMT/OFMT value used to convert from number */ struct Cell *cnext; /* ptr to next if chained */ } Cell; @@ -96,9 +97,14 @@ extern Array *symtab; extern Cell *nrloc; /* NR */ extern Cell *fnrloc; /* FNR */ +extern Cell *fsloc; /* FS */ extern Cell *nfloc; /* NF */ +extern Cell *ofsloc; /* OFS */ +extern Cell *orsloc; /* ORS */ +extern Cell *rsloc; /* RS */ extern Cell *rstartloc; /* RSTART */ extern Cell *rlengthloc; /* RLENGTH */ +extern Cell *subseploc; /* SUBSEP */ /* Cell.tval values: */ #define NUM 01 /* number value is valid */ @@ -109,6 +115,8 @@ extern Cell *rlengthloc; /* RLENGTH */ #define FCN 040 /* this is a function name */ #define FLD 0100 /* this is a field $1, $2, ... */ #define REC 0200 /* this is $0 */ +#define CONVC 0400 /* string was converted from number via CONVFMT */ +#define CONVO 01000 /* string was converted from number via OFMT */ /* function types */ diff --git a/contrib/one-true-awk/awkgram.y b/contrib/one-true-awk/awkgram.y index 5b5c461b3eed..e4abeeddcb6a 100644 --- a/contrib/one-true-awk/awkgram.y +++ b/contrib/one-true-awk/awkgram.y @@ -86,7 +86,7 @@ Node *arglist = 0; /* list of args for current function */ %left CAT %left '+' '-' %left '*' '/' '%' -%left NOT UMINUS +%left NOT UMINUS UPLUS %right POWER %right DECR INCR %left INDIRECT @@ -357,7 +357,7 @@ term: | term '%' term { $$ = op2(MOD, $1, $3); } | term POWER term { $$ = op2(POWER, $1, $3); } | '-' term %prec UMINUS { $$ = op1(UMINUS, $2); } - | '+' term %prec UMINUS { $$ = $2; } + | '+' term %prec UMINUS { $$ = op1(UPLUS, $2); } | NOT term %prec UMINUS { $$ = op1(NOT, notnull($2)); } | BLTIN '(' ')' { $$ = op2(BLTIN, itonp($1), rectonode()); } | BLTIN '(' patlist ')' { $$ = op2(BLTIN, itonp($1), $3); } diff --git a/contrib/one-true-awk/b.c b/contrib/one-true-awk/b.c index f9dae52a0a95..6bf473820488 100644 --- a/contrib/one-true-awk/b.c +++ b/contrib/one-true-awk/b.c @@ -30,6 +30,7 @@ __FBSDID("$FreeBSD$"); *** 2657 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:49 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EC60B657C18; Sat, 10 Jul 2021 17:12:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6T5ZPLz4VB9; Sat, 10 Jul 2021 17:12:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 61F1D42B2; Sat, 10 Jul 2021 17:12:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCndf088250; Sat, 10 Jul 2021 17:12:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCndw088249; Sat, 10 Jul 2021 17:12:49 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:49 GMT Message-Id: <202107101712.16AHCndw088249@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 46ee8211774c - stable/12 - Reapply r301289 by pfg: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 46ee8211774cc59d40e62dec883a42d075baad8b Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:50 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=46ee8211774cc59d40e62dec883a42d075baad8b commit 46ee8211774cc59d40e62dec883a42d075baad8b Author: Warner Losh AuthorDate: 2019-06-02 16:28:20 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Reapply r301289 by pfg: | MFV r300961: one-true-awk: replace 0 with NULL for pointers | Also remove a redundant semicolon. | Also had to rebase on upstream pull. (cherry picked from commit 10ce5b990fc0c874071c21e47d02194337732f27) --- contrib/one-true-awk/b.c | 20 ++++++++--------- contrib/one-true-awk/lex.c | 14 ++++++------ contrib/one-true-awk/maketab.c | 2 +- contrib/one-true-awk/parse.c | 2 +- contrib/one-true-awk/run.c | 49 ++++++++++++++++++++++-------------------- contrib/one-true-awk/tran.c | 2 +- 6 files changed, 46 insertions(+), 43 deletions(-) diff --git a/contrib/one-true-awk/b.c b/contrib/one-true-awk/b.c index 6bf473820488..24baa33d8db1 100644 --- a/contrib/one-true-awk/b.c +++ b/contrib/one-true-awk/b.c @@ -91,11 +91,11 @@ fa *makedfa(const char *s, int anchor) /* returns dfa for reg expr s */ fa *pfa; static int now = 1; - if (setvec == 0) { /* first time through any RE */ + if (setvec == NULL) { /* first time through any RE */ maxsetvec = MAXLIN; setvec = (int *) malloc(maxsetvec * sizeof(int)); tmpset = (int *) malloc(maxsetvec * sizeof(int)); - if (setvec == 0 || tmpset == 0) + if (setvec == NULL || tmpset == NULL) overflo("out of space initializing makedfa"); } @@ -317,11 +317,11 @@ char *cclenter(const char *argp) /* add a character class */ int j; uschar *p = (uschar *) argp; uschar *op, *bp; - static uschar *buf = 0; + static uschar *buf = NULL; static int bufsz = 100; op = p; - if (buf == 0 && (buf = (uschar *) malloc(bufsz)) == NULL) + if (buf == NULL && (buf = (uschar *) malloc(bufsz)) == NULL) FATAL("out of space for character class [%.10s...] 1", p); bp = buf; for (i = 0; (c = *p++) != 0; ) { @@ -380,7 +380,7 @@ void cfoll(fa *f, Node *v) /* enter follow set of each leaf of vertex v into lfo maxsetvec *= 4; setvec = (int *) realloc(setvec, maxsetvec * sizeof(int)); tmpset = (int *) realloc(tmpset, maxsetvec * sizeof(int)); - if (setvec == 0 || tmpset == 0) + if (setvec == NULL || tmpset == NULL) overflo("out of space in cfoll()"); } for (i = 0; i <= f->accept; i++) @@ -421,7 +421,7 @@ int first(Node *p) /* collects initially active leaves of p into setvec */ maxsetvec *= 4; setvec = (int *) realloc(setvec, maxsetvec * sizeof(int)); tmpset = (int *) realloc(tmpset, maxsetvec * sizeof(int)); - if (setvec == 0 || tmpset == 0) + if (setvec == NULL || tmpset == NULL) overflo("out of space in first()"); } if (type(p) == EMPTYRE) { @@ -923,7 +923,7 @@ int relex(void) /* lexical analyzer for reparse */ { int c, n; int cflag; - static uschar *buf = 0; + static uschar *buf = NULL; static int bufsz = 100; uschar *bp; struct charclass *cc; @@ -953,7 +953,7 @@ rescan: rlxval = c; return CHAR; case '[': - if (buf == 0 && (buf = (uschar *) malloc(bufsz)) == NULL) + if (buf == NULL && (buf = (uschar *) malloc(bufsz)) == NULL) FATAL("out of space in reg expr %.10s..", lastre); bp = buf; if (*prestr == '^') { @@ -1129,7 +1129,7 @@ int cgoto(fa *f, int s, int c) maxsetvec *= 4; setvec = (int *) realloc(setvec, maxsetvec * sizeof(int)); tmpset = (int *) realloc(tmpset, maxsetvec * sizeof(int)); - if (setvec == 0 || tmpset == 0) + if (setvec == NULL || tmpset == NULL) overflo("out of space in cgoto()"); } for (i = 0; i <= f->accept; i++) @@ -1151,7 +1151,7 @@ int cgoto(fa *f, int s, int c) maxsetvec *= 4; setvec = (int *) realloc(setvec, maxsetvec * sizeof(int)); tmpset = (int *) realloc(tmpset, maxsetvec * sizeof(int)); - if (setvec == 0 || tmpset == 0) + if (setvec == NULL || tmpset == NULL) overflo("cgoto overflow"); } if (setvec[q[j]] == 0) { diff --git a/contrib/one-true-awk/lex.c b/contrib/one-true-awk/lex.c index 18927f842b23..8e689f1a682a 100644 --- a/contrib/one-true-awk/lex.c +++ b/contrib/one-true-awk/lex.c @@ -176,10 +176,10 @@ int reg = 0; /* 1 => return a REGEXPR now */ int yylex(void) { int c; - static char *buf = 0; + static char *buf = NULL; static int bufsize = 5; /* BUG: setting this small causes core dump! */ - if (buf == 0 && (buf = (char *) malloc(bufsize)) == NULL) + if (buf == NULL && (buf = (char *) malloc(bufsize)) == NULL) FATAL( "out of space in yylex" ); if (sc) { sc = 0; @@ -366,10 +366,10 @@ int string(void) { int c, n; char *s, *bp; - static char *buf = 0; + static char *buf = NULL; static int bufsz = 500; - if (buf == 0 && (buf = (char *) malloc(bufsz)) == NULL) + if (buf == NULL && (buf = (char *) malloc(bufsz)) == NULL) FATAL("out of space for strings"); for (bp = buf; (c = input()) != '"'; ) { if (!adjbuf(&buf, &bufsz, bp-buf+2, 500, &bp, "string")) @@ -513,11 +513,11 @@ void startreg(void) /* next call to yylex will return a regular expression */ int regexpr(void) { int c; - static char *buf = 0; + static char *buf = NULL; static int bufsz = 500; char *bp; - if (buf == 0 && (buf = (char *) malloc(bufsz)) == NULL) + if (buf == NULL && (buf = (char *) malloc(bufsz)) == NULL) FATAL("out of space for rex expr"); bp = buf; for ( ; (c = input()) != '/' && c != 0; ) { @@ -549,7 +549,7 @@ char ebuf[300]; char *ep = ebuf; char yysbuf[100]; /* pushback buffer */ char *yysptr = yysbuf; -FILE *yyin = 0; +FILE *yyin = NULL; int input(void) /* get next lexical input character */ { diff --git a/contrib/one-true-awk/maketab.c b/contrib/one-true-awk/maketab.c index dbe3d241fcc8..9faed8414f10 100644 --- a/contrib/one-true-awk/maketab.c +++ b/contrib/one-true-awk/maketab.c @@ -156,7 +156,7 @@ int main(int argc, char *argv[]) table[p->token-FIRSTTOKEN] = p->name; printf("\nCell *(*proctab[%d])(Node **, int) = {\n", SIZE); for (i=0; innext, n++) + for (n = 0; p != NULL; p = p->nnext, n++) if (strcmp(((Cell *)(p->narg[0]))->nval, s) == 0) return n; return -1; diff --git a/contrib/one-true-awk/run.c b/contrib/one-true-awk/run.c index c3a3e5f4751a..d74b54ca3445 100644 --- a/contrib/one-true-awk/run.c +++ b/contrib/one-true-awk/run.c @@ -517,7 +517,7 @@ Cell *awkdelete(Node **a, int n) /* a[0] is symtab, a[1] is list of subscripts * x = execute(a[0]); /* Cell* for symbol table */ if (!isarr(x)) return True; - if (a[1] == 0) { /* delete the elements, not the table */ + if (a[1] == NULL) { /* delete the elements, not the table */ freesymtab(x); x->tval &= ~STR; x->tval |= ARR; @@ -603,7 +603,7 @@ Cell *matchop(Node **a, int n) /* ~ and match() */ } x = execute(a[1]); /* a[1] = target text */ s = getsval(x); - if (a[0] == 0) /* a[1] == 0: already-compiled reg expr */ + if (a[0] == NULL) /* a[1] == 0: already-compiled reg expr */ i = (*mf)((fa *) a[2], s); else { y = execute(a[2]); /* a[2] = regular expr */ @@ -719,7 +719,7 @@ Cell *gettemp(void) /* get a tempcell */ FATAL("out of space for temporaries"); for(i = 1; i < 100; i++) tmps[i-1].cnext = &tmps[i]; - tmps[i-1].cnext = 0; + tmps[i-1].cnext = NULL; } x = tmps; tmps = x->cnext; @@ -754,18 +754,18 @@ Cell *substr(Node **a, int nnn) /* substr(a[0], a[1], a[2]) */ int k, m, n; char *s; int temp; - Cell *x, *y, *z = 0; + Cell *x, *y, *z = NULL; x = execute(a[0]); y = execute(a[1]); - if (a[2] != 0) + if (a[2] != NULL) z = execute(a[2]); s = getsval(x); k = strlen(s) + 1; if (k <= 1) { tempfree(x); tempfree(y); - if (a[2] != 0) { + if (a[2] != NULL) { tempfree(z); } x = gettemp(); @@ -778,7 +778,7 @@ Cell *substr(Node **a, int nnn) /* substr(a[0], a[1], a[2]) */ else if (m > k) m = k; tempfree(y); - if (a[2] != 0) { + if (a[2] != NULL) { n = (int) getfval(z); tempfree(z); } else @@ -1219,7 +1219,7 @@ Cell *pastat(Node **a, int n) /* a[0] { a[1] } */ { Cell *x; - if (a[0] == 0) + if (a[0] == NULL) x = execute(a[1]); else { x = execute(a[0]); @@ -1256,9 +1256,9 @@ Cell *dopa2(Node **a, int n) /* a[0], a[1] { a[2] } */ Cell *split(Node **a, int nnn) /* split(a[0], a[1], a[2]); a[3] is type */ { - Cell *x = 0, *y, *ap; + Cell *x = NULL, *y, *ap; char *s, *origs; - char *fs, *origfs = NULL; + char *fs = NULL, *origfs = NULL; int sep; char *t, temp, num[50]; int n, tempstat, arg3type; @@ -1266,8 +1266,8 @@ Cell *split(Node **a, int nnn) /* split(a[0], a[1], a[2]); a[3] is type */ y = execute(a[0]); /* source string */ origs = s = strdup(getsval(y)); arg3type = ptoi(a[3]); - if (a[2] == 0) /* fs string */ - fs = getsval(fsloc); + if (a[2] == NULL) /* fs string */ + fs = *FS; else if (arg3type == STRING) { /* split(str,arr,"string") */ x = execute(a[2]); origfs = fs = strdup(getsval(x)); @@ -1387,6 +1387,9 @@ Cell *split(Node **a, int nnn) /* split(a[0], a[1], a[2]); a[3] is type */ tempfree(y); free(origs); free(origfs); + if (a[2] != NULL && arg3type == STRING) { + tempfree(x); + } x = gettemp(); x->tval = NUM; x->fval = n; @@ -1416,7 +1419,7 @@ Cell *ifstat(Node **a, int n) /* if (a[0]) a[1]; else a[2] */ if (istrue(x)) { tempfree(x); x = execute(a[1]); - } else if (a[2] != 0) { + } else if (a[2] != NULL) { tempfree(x); x = execute(a[2]); } @@ -1468,7 +1471,7 @@ Cell *forstat(Node **a, int n) /* for (a[0]; a[1]; a[2]) a[3] */ x = execute(a[0]); tempfree(x); for (;;) { - if (a[1]!=0) { + if (a[1]!=NULL) { x = execute(a[1]); if (!istrue(x)) return(x); else tempfree(x); @@ -1551,7 +1554,7 @@ Cell *bltin(Node **a, int n) /* builtin functions. a[0] is type, a[1] is arg lis case FCOS: u = cos(getfval(x)); break; case FATAN: - if (nextarg == 0) { + if (nextarg == NULL) { WARNING("atan2 requires two arguments; returning 1.0"); u = 1.0; } else { @@ -1697,7 +1700,7 @@ Cell *bltin(Node **a, int n) /* builtin functions. a[0] is type, a[1] is arg lis tempfree(x); x = gettemp(); setfval(x, u); - if (nextarg != 0) { + if (nextarg != NULL) { WARNING("warning: function has too many arguments"); for ( ; nextarg; nextarg = nextarg->nnext) execute(nextarg); @@ -1711,7 +1714,7 @@ Cell *printstat(Node **a, int n) /* print a[0] */ Cell *y; FILE *fp; - if (a[1] == 0) /* a[1] is redirection operator, a[2] is file */ + if (a[1] == NULL) /* a[1] is redirection operator, a[2] is file */ fp = stdout; else fp = redirect(ptoi(a[1]), a[2]); @@ -1724,7 +1727,7 @@ Cell *printstat(Node **a, int n) /* print a[0] */ else fputs(getsval(ofsloc), fp); } - if (a[1] != 0) + if (a[1] != NULL) fflush(fp); if (ferror(fp)) FATAL("write error on %s", filename(fp)); @@ -1781,7 +1784,7 @@ FILE *openfile(int a, const char *us) { const char *s = us; int i, m; - FILE *fp = 0; + FILE *fp = NULL; if (*s == '\0') FATAL("null file name in print or getline"); @@ -1796,7 +1799,7 @@ FILE *openfile(int a, const char *us) return NULL; for (i=0; i < nfiles; i++) - if (files[i].fp == 0) + if (files[i].fp == NULL) break; if (i >= nfiles) { struct files *nf; @@ -1912,7 +1915,7 @@ Cell *sub(Node **a, int nnn) /* substitute command */ FATAL("out of memory in sub"); x = execute(a[3]); /* target string */ t = getsval(x); - if (a[0] == 0) /* 0 => a[1] is already-compiled regexpr */ + if (a[0] == NULL) /* 0 => a[1] is already-compiled regexpr */ pfa = (fa *) a[1]; /* regular expression */ else { y = execute(a[1]); @@ -1952,7 +1955,7 @@ Cell *sub(Node **a, int nnn) /* substitute command */ if (pb > buf + bufsz) FATAL("sub result2 %.30s too big; can't happen", buf); setsval(x, buf); /* BUG: should be able to avoid copy */ - result = True;; + result = True; } tempfree(x); tempfree(y); @@ -1975,7 +1978,7 @@ Cell *gsub(Node **a, int nnn) /* global substitute */ num = 0; x = execute(a[3]); /* target string */ t = getsval(x); - if (a[0] == 0) /* 0 => a[1] is already-compiled regexpr */ + if (a[0] == NULL) /* 0 => a[1] is already-compiled regexpr */ pfa = (fa *) a[1]; /* regular expression */ else { y = execute(a[1]); diff --git a/contrib/one-true-awk/tran.c b/contrib/one-true-awk/tran.c index d1dfe2b2f176..8577a0b11d31 100644 --- a/contrib/one-true-awk/tran.c +++ b/contrib/one-true-awk/tran.c @@ -194,7 +194,7 @@ void freesymtab(Cell *ap) /* free a symbol table */ free(cp); tp->nelem--; } - tp->tab[i] = 0; + tp->tab[i] = NULL; } if (tp->nelem != 0) WARNING("can't happen: inconsistent element count freeing %s", ap->nval); From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:50 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C341E657C19; Sat, 10 Jul 2021 17:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6V4bJ9z4VHH; Sat, 10 Jul 2021 17:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 75A7F4111; Sat, 10 Jul 2021 17:12:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCo0k088274; Sat, 10 Jul 2021 17:12:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCotn088273; Sat, 10 Jul 2021 17:12:50 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:50 GMT Message-Id: <202107101712.16AHCotn088273@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 83f5846fc818 - stable/12 - Reapply r315426 by pfg: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 83f5846fc8180e17baff66480473846978a3f030 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:51 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=83f5846fc8180e17baff66480473846978a3f030 commit 83f5846fc8180e17baff66480473846978a3f030 Author: Warner Losh AuthorDate: 2019-06-02 16:30:53 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Reapply r315426 by pfg: | MFV r315425: one-true-awk: have calloc(3) do the multiplication. (cherry picked from commit 06d1e65393f95e0fd2d068fe118487f754e4248b) --- contrib/one-true-awk/b.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/contrib/one-true-awk/b.c b/contrib/one-true-awk/b.c index 24baa33d8db1..4de746fa087f 100644 --- a/contrib/one-true-awk/b.c +++ b/contrib/one-true-awk/b.c @@ -148,7 +148,7 @@ fa *mkdfa(const char *s, int anchor) /* does the real work of making a dfa */ f->accept = poscnt-1; /* penter has computed number of positions in re */ cfoll(f, p1); /* set up follow sets */ freetr(p1); - if ((f->posns[0] = (int *) calloc(1, *(f->re[0].lfollow)*sizeof(int))) == NULL) + if ((f->posns[0] = (int *) calloc(*(f->re[0].lfollow), sizeof(int))) == NULL) overflo("out of space in makedfa"); if ((f->posns[1] = (int *) calloc(1, sizeof(int))) == NULL) overflo("out of space in makedfa"); @@ -172,7 +172,7 @@ int makeinit(fa *f, int anchor) f->reset = 0; k = *(f->re[0].lfollow); xfree(f->posns[2]); - if ((f->posns[2] = (int *) calloc(1, (k+1)*sizeof(int))) == NULL) + if ((f->posns[2] = (int *) calloc(k+1, sizeof(int))) == NULL) overflo("out of space in makeinit"); for (i=0; i <= k; i++) { (f->posns[2])[i] = (f->re[0].lfollow)[i]; @@ -387,7 +387,7 @@ void cfoll(fa *f, Node *v) /* enter follow set of each leaf of vertex v into lfo setvec[i] = 0; setcnt = 0; follow(v); /* computes setvec and setcnt */ - if ((p = (int *) calloc(1, (setcnt+1)*sizeof(int))) == NULL) + if ((p = (int *) calloc(setcnt+1, sizeof(int))) == NULL) overflo("out of space building follow set"); f->re[info(v)].lfollow = p; *p = setcnt; @@ -561,7 +561,7 @@ int pmatch(fa *f, const char *p0) /* longest match, for sub */ for (i = 2; i <= f->curstat; i++) xfree(f->posns[i]); k = *f->posns[0]; - if ((f->posns[2] = (int *) calloc(1, (k+1)*sizeof(int))) == NULL) + if ((f->posns[2] = (int *) calloc(k+1, sizeof(int))) == NULL) overflo("out of space in pmatch"); for (i = 0; i <= k; i++) (f->posns[2])[i] = (f->posns[0])[i]; @@ -618,7 +618,7 @@ int nematch(fa *f, const char *p0) /* non-empty match, for sub */ for (i = 2; i <= f->curstat; i++) xfree(f->posns[i]); k = *f->posns[0]; - if ((f->posns[2] = (int *) calloc(1, (k+1)*sizeof(int))) == NULL) + if ((f->posns[2] = (int *) calloc(k+1, sizeof(int))) == NULL) overflo("out of state space"); for (i = 0; i <= k; i++) (f->posns[2])[i] = (f->posns[0])[i]; @@ -1194,7 +1194,7 @@ int cgoto(fa *f, int s, int c) for (i = 0; i < NCHARS; i++) f->gototab[f->curstat][i] = 0; xfree(f->posns[f->curstat]); - if ((p = (int *) calloc(1, (setcnt+1)*sizeof(int))) == NULL) + if ((p = (int *) calloc(setcnt+1, sizeof(int))) == NULL) overflo("out of space in cgoto"); f->posns[f->curstat] = p; From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:52 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 235FD657F89; Sat, 10 Jul 2021 17:12:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6W6Rh2z4V7V; Sat, 10 Jul 2021 17:12:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 981A44194; Sat, 10 Jul 2021 17:12:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCpZE088298; Sat, 10 Jul 2021 17:12:51 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCpFk088297; Sat, 10 Jul 2021 17:12:51 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:51 GMT Message-Id: <202107101712.16AHCpFk088297@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: cf150bbe650d - stable/12 - Reapply r301691: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: cf150bbe650db237f94de7331a25e486c0c497c5 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:52 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=cf150bbe650db237f94de7331a25e486c0c497c5 commit cf150bbe650db237f94de7331a25e486c0c497c5 Author: Warner Losh AuthorDate: 2019-06-02 20:47:15 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Reapply r301691: Revert r301689 - one-true-awk: Avoid a NULL dereference. I got this wrong and the coverity report doesn't match the NetBSD change, which was thought for a different version. The change wouldn't hurt but let's wait until upstream figures this out. (cherry picked from commit 2675e1b91dcfa744a414a42f5522f4fed958476c) --- contrib/one-true-awk/tran.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/one-true-awk/tran.c b/contrib/one-true-awk/tran.c index 8577a0b11d31..837f3742bc4f 100644 --- a/contrib/one-true-awk/tran.c +++ b/contrib/one-true-awk/tran.c @@ -366,7 +366,7 @@ char *setsval(Cell *vp, const char *s) /* set string val of a Cell */ if (donerec == 0) recbld(); } - t = s ? tostring(s) : tostring(""); /* in case it's self-assign */ + t = tostring(s); /* in case it's self-assign */ if (freeable(vp)) xfree(vp->sval); vp->tval &= ~(NUM|CONVC|CONVO); From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:53 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 78143657BCE; Sat, 10 Jul 2021 17:12:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6Y2LZSz4VBJ; Sat, 10 Jul 2021 17:12:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BAEB442B3; Sat, 10 Jul 2021 17:12:52 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCqTK088322; Sat, 10 Jul 2021 17:12:52 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCqd7088321; Sat, 10 Jul 2021 17:12:52 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:52 GMT Message-Id: <202107101712.16AHCqd7088321@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 086165376b02 - stable/12 - Revert r348518 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 086165376b029983658816a16a44e5ec0b4f0363 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:53 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=086165376b029983658816a16a44e5ec0b4f0363 commit 086165376b029983658816a16a44e5ec0b4f0363 Author: Warner Losh AuthorDate: 2019-06-02 20:52:21 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Revert r348518 It should not have happened. The change is actually in upstream and I misread the diffs. (cherry picked from commit adb46ac4c0a64c2cd1fbf46022f4aafb4e605cd3) --- contrib/one-true-awk/tran.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/one-true-awk/tran.c b/contrib/one-true-awk/tran.c index 837f3742bc4f..8577a0b11d31 100644 --- a/contrib/one-true-awk/tran.c +++ b/contrib/one-true-awk/tran.c @@ -366,7 +366,7 @@ char *setsval(Cell *vp, const char *s) /* set string val of a Cell */ if (donerec == 0) recbld(); } - t = tostring(s); /* in case it's self-assign */ + t = s ? tostring(s) : tostring(""); /* in case it's self-assign */ if (freeable(vp)) xfree(vp->sval); vp->tval &= ~(NUM|CONVC|CONVO); From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:54 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B5EE5657CA4; Sat, 10 Jul 2021 17:12:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6Z38nkz4VSP; Sat, 10 Jul 2021 17:12:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DD5FD3AD6; Sat, 10 Jul 2021 17:12:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCreb088352; Sat, 10 Jul 2021 17:12:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCrZo088351; Sat, 10 Jul 2021 17:12:53 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:53 GMT Message-Id: <202107101712.16AHCrZo088351@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 0ca88d564187 - stable/12 - Fix mismerge that crept into r301289. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 0ca88d56418731b16dd81d096bdbf3b88de90b68 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:54 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=0ca88d56418731b16dd81d096bdbf3b88de90b68 commit 0ca88d56418731b16dd81d096bdbf3b88de90b68 Author: Warner Losh AuthorDate: 2019-06-03 05:25:16 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Fix mismerge that crept into r301289. The conversion of 0 -> NULL required a rebase at some point, as noted in r301289 when pfg commited it. In that rebase, three lines remained that had been removed in a prior version of awk, and one of them had a 0 -> NULL change causing a conflict. The conflict should have been resolved by removing the three lines, but wasn't. This introduces a regression into f.split3 test which prior to this commit we were failing, but a pure onetrueawk wasn't. Remove the offending 3 lines. (cherry picked from commit 31d232c2a3c18fb05e5b2f1ec42872708cbfd27e) --- contrib/one-true-awk/run.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/contrib/one-true-awk/run.c b/contrib/one-true-awk/run.c index d74b54ca3445..1859263f7f4b 100644 --- a/contrib/one-true-awk/run.c +++ b/contrib/one-true-awk/run.c @@ -1387,9 +1387,6 @@ Cell *split(Node **a, int nnn) /* split(a[0], a[1], a[2]); a[3] is type */ tempfree(y); free(origs); free(origfs); - if (a[2] != NULL && arg3type == STRING) { - tempfree(x); - } x = gettemp(); x->tval = NUM; x->fval = n; From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:55 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C33DC657C2E; Sat, 10 Jul 2021 17:12:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6b3tGgz4V7f; Sat, 10 Jul 2021 17:12:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1BD5543CD; Sat, 10 Jul 2021 17:12:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCsoh088376; Sat, 10 Jul 2021 17:12:54 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCsAE088375; Sat, 10 Jul 2021 17:12:54 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:54 GMT Message-Id: <202107101712.16AHCsAE088375@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: ffc35739bac4 - stable/12 - Another partial revert of r301289. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: ffc35739bac49abe823182f61e36162d25e06fe4 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:56 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=ffc35739bac49abe823182f61e36162d25e06fe4 commit ffc35739bac49abe823182f61e36162d25e06fe4 Author: Warner Losh AuthorDate: 2019-06-03 05:25:22 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Another partial revert of r301289. In this case, a change was made in one-true-awk from *FS to getsval(fsloc) in a line just after one of the lines that had the 0 -> NULL change. It works both ways as far as I can tell. It looks like a bug fix, but I've not tried to track down which ancient version of one-true-awk it was in (github starts too late for tracking this down). Before and after the changes the regression suite is passes 100% relative to the un-modified one-true-awk. (cherry picked from commit d9e8cf281b12ca943d150b29ae5a847b0df888f3) --- contrib/one-true-awk/run.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contrib/one-true-awk/run.c b/contrib/one-true-awk/run.c index 1859263f7f4b..baea06ed2500 100644 --- a/contrib/one-true-awk/run.c +++ b/contrib/one-true-awk/run.c @@ -1267,7 +1267,7 @@ Cell *split(Node **a, int nnn) /* split(a[0], a[1], a[2]); a[3] is type */ origs = s = strdup(getsval(y)); arg3type = ptoi(a[3]); if (a[2] == NULL) /* fs string */ - fs = *FS; + fs = getsval(fsloc); else if (arg3type == STRING) { /* split(str,arr,"string") */ x = execute(a[2]); origfs = fs = strdup(getsval(x)); From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:57 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 16087657E99; Sat, 10 Jul 2021 17:12:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6c5JFVz4VSV; Sat, 10 Jul 2021 17:12:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 281D54112; Sat, 10 Jul 2021 17:12:56 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCuT4088400; Sat, 10 Jul 2021 17:12:56 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCuXP088399; Sat, 10 Jul 2021 17:12:56 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:56 GMT Message-Id: <202107101712.16AHCuXP088399@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: e32d04bed890 - stable/12 - awk: Fix subobject out-of-bounds access MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: e32d04bed89057c54147db931fa544bb601f48ac Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:57 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=e32d04bed89057c54147db931fa544bb601f48ac commit e32d04bed89057c54147db931fa544bb601f48ac Author: Alex Richardson AuthorDate: 2020-09-21 19:03:07 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 awk: Fix subobject out-of-bounds access When matching a regex with ^, it would attempt to access gototab[NSTATES][NCHARS+2], and therefore access the state for the \002 character instead. This change is required to run awk under CHERI (with sub-object bounds) and when running with UBSan instrumentation. This was committed upstream as https://github.com/onetrueawk/awk/commit/cbf924342b63a095a4c6842280c3085b1b63ae45 Found by: CHERI (with subobject bounds enabled) Obtained from: CheriBSD Reviewed By: imp Differential Revision: https://reviews.freebsd.org/D26509 (cherry picked from commit ae692c42cb46a5e72772070070840b15dd5d6bd8) --- contrib/one-true-awk/awk.h | 4 +++- contrib/one-true-awk/b.c | 2 -- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/contrib/one-true-awk/awk.h b/contrib/one-true-awk/awk.h index b16c2f36f828..31d070aecddc 100644 --- a/contrib/one-true-awk/awk.h +++ b/contrib/one-true-awk/awk.h @@ -218,6 +218,8 @@ extern int pairstack[], paircnt; #define NCHARS (256+3) /* 256 handles 8-bit chars; 128 does 7-bit */ /* watch out in match(), etc. */ #define NSTATES 32 +#define HAT (NCHARS+2) /* matches ^ in regular expr */ + /* NCHARS is 2**n */ typedef struct rrow { long ltype; /* long avoids pointer warnings on 64-bit */ @@ -230,7 +232,7 @@ typedef struct rrow { } rrow; typedef struct fa { - uschar gototab[NSTATES][NCHARS]; + uschar gototab[NSTATES][HAT + 1]; uschar out[NSTATES]; uschar *restr; int *posns[NSTATES]; diff --git a/contrib/one-true-awk/b.c b/contrib/one-true-awk/b.c index 4de746fa087f..0cdcf30a972e 100644 --- a/contrib/one-true-awk/b.c +++ b/contrib/one-true-awk/b.c @@ -37,8 +37,6 @@ __FBSDID("$FreeBSD$"); #include "awk.h" #include "ytab.h" -#define HAT (NCHARS+2) /* matches ^ in regular expr */ - /* NCHARS is 2**n */ #define MAXLIN 22 #define type(v) (v)->nobj /* badly overloaded here */ From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:58 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 65C8E657E67; Sat, 10 Jul 2021 17:12:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6d41dJz4VHk; Sat, 10 Jul 2021 17:12:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4F3333AD7; Sat, 10 Jul 2021 17:12:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCv1P088424; Sat, 10 Jul 2021 17:12:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCv3B088423; Sat, 10 Jul 2021 17:12:57 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:57 GMT Message-Id: <202107101712.16AHCv3B088423@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: a711add6ceff - stable/12 - Fix another UBSan error in awk MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: a711add6ceffc58dabb930a0fdd685ddbac7b83c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:58 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=a711add6ceffc58dabb930a0fdd685ddbac7b83c commit a711add6ceffc58dabb930a0fdd685ddbac7b83c Author: Alex Richardson AuthorDate: 2020-09-21 19:03:12 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 Fix another UBSan error in awk This applies my upstreamed fix: https://github.com/onetrueawk/awk/commit/ad9bd2f40a89ec9533b92254b86a756cf4f40fd4 Found By: UBSan (cherry picked from commit 1116946093d537569a29692a6a86de59cd49384e) --- contrib/one-true-awk/lex.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/contrib/one-true-awk/lex.c b/contrib/one-true-awk/lex.c index 8e689f1a682a..e537df4151da 100644 --- a/contrib/one-true-awk/lex.c +++ b/contrib/one-true-awk/lex.c @@ -467,9 +467,8 @@ int word(char *w) int c, n; n = binsearch(w, keywords, sizeof(keywords)/sizeof(keywords[0])); -/* BUG: this ought to be inside the if; in theory could fault (daniel barrett) */ - kp = keywords + n; if (n != -1) { /* found in table */ + kp = keywords + n; yylval.i = kp->sub; switch (kp->type) { /* special handling */ case BLTIN: From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 17:12:58 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E56DA657A69; Sat, 10 Jul 2021 17:12:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMc6f4pl3z4VHr; Sat, 10 Jul 2021 17:12:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 81A974195; Sat, 10 Jul 2021 17:12:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AHCwkR088448; Sat, 10 Jul 2021 17:12:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AHCwoE088447; Sat, 10 Jul 2021 17:12:58 GMT (envelope-from git) Date: Sat, 10 Jul 2021 17:12:58 GMT Message-Id: <202107101712.16AHCwoE088447@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 6edf5082bab7 - stable/12 - one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 6edf5082bab71cf923efff9f18e38efe5b83b0ec Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 17:12:59 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=6edf5082bab71cf923efff9f18e38efe5b83b0ec commit 6edf5082bab71cf923efff9f18e38efe5b83b0ec Author: Warner Losh AuthorDate: 2021-07-07 23:30:35 +0000 Commit: Warner Losh CommitDate: 2021-07-10 17:11:15 +0000 one-true-awk: import 20210221 (1e4bc42c53a1) which fixes a number of bugs Import the latest bsd-features branch of the one-true-awk upstream: o Move to bison for $YACC o Set close-on-exec flag for file and pipe redirects that aren't std* o lots of little fixes to modernize ocde base o free sval member before setting it o fix a bug where a{0,3} could match aaaa o pull in systime and strftime from NetBSD awk o pull in fixes from {Net,Free,Open}BSD (normalized our code with them) o add BSD extensions and, or, xor, compl, lsheift, rshift (mostly a nop) Also revert a few of the trivial FreeBSD changes that were done slightly differently in the upstreaming process. Also, our PR database may have been mined by upstream for these fixes, and Mikolaj Golub may deserve credit for some of the fixes in this update. Suggested by: Mikolaj Golub PR: 143363,143365,143368,143369,143373,143375,214782 Sponsored by: Netflix (cherry picked from commit f39dd6a9784467f0db5886012b3f4b13899be6b8) --- contrib/one-true-awk/ChangeLog | 108 +++ contrib/one-true-awk/FIXES | 261 +++++- contrib/one-true-awk/REGRESS | 4 + contrib/one-true-awk/awk.1 | 159 +++- contrib/one-true-awk/awk.h | 67 +- contrib/one-true-awk/awkgram.y | 48 +- contrib/one-true-awk/b.c | 438 +++++++--- .../one-true-awk/bugs-fixed/missing-precision.ok | 2 +- contrib/one-true-awk/bugs-fixed/negative-nf.ok | 2 +- contrib/one-true-awk/lex.c | 77 +- contrib/one-true-awk/lib.c | 327 +++++--- contrib/one-true-awk/main.c | 163 ++-- contrib/one-true-awk/makefile | 74 +- contrib/one-true-awk/maketab.c | 66 +- contrib/one-true-awk/parse.c | 29 +- contrib/one-true-awk/proctab.c | 202 ++--- contrib/one-true-awk/proto.h | 41 +- contrib/one-true-awk/run.c | 918 ++++++++++++++------- contrib/one-true-awk/tran.c | 164 ++-- usr.bin/awk/Makefile | 10 +- 20 files changed, 2192 insertions(+), 968 deletions(-) diff --git a/contrib/one-true-awk/ChangeLog b/contrib/one-true-awk/ChangeLog index fd03b2bbca0b..dea4ed7e3187 100644 --- a/contrib/one-true-awk/ChangeLog +++ b/contrib/one-true-awk/ChangeLog @@ -1,3 +1,111 @@ +2020-07-30 Arnold D. Robbins + + By fiat, we use bison for $(YACC). Trying to accommodate + different versions didn't work. + + * makefile: Significant cleanup. Replace all ytab* references + with awkgram.tab.* and simplify definition of YACC. + * .gitignore: Remove ytab* references. + * b.c, lex.c, maketab.c, parse.c, run.c: Replace include of ytab.h + with awkgram.tab.h. + * lib.c, main.c, tran.c: Remove include of ytab.h, wasn't needed. + +2020-01-20 Arnold D. Robbins + + * run.c (openfile): Set the close-on-exec flag for file + and pipe redirections that aren't stdin/stdout/stderr. + +2020-01-06 Arnold D. Robbins + + Minor fixes. + * b.c (replace_repeat): Turn init_q back into an int. + * lex.c (string): Use \a instead of \007. + * tran.c (catstr): Use snprintf instead of sprintf. + +2020-01-01 Arnold D. Robbins + + * tran.c (syminit, arginit, envinit): Free sval member before + setting it. Thanks to valgrind. + * b.c: Small formatting cleanups in several routines. + +2019-12-27 Arnold D. Robbins + + * b.c (replace_repeat): Fix a bug whereby a{0,3} could match + four a's. Thanks to Anonymous AWK fan + for the report. Also, minor code formatting cleanups. + * testdir/T.int-expr: New file. + +2019-12-11 Arnold D. Robbins + + * README: Renamed to ... + * README.md: ... this. Cleaned up some as well, + including moving to Markdown. + +2019-11-08 Arnold D. Robbins + + * test/T.chem: Use $oldawk instead of hardwiring 'awk'. + * test/T.lilly: Remove gawk warnings from output, improves + portability. + +2019-10-17 Arnold D. Robbins + + Pull in systime() and strftime() from the NetBSD awk. + + * awk.1: Document the functions. + * run.c (bltin): Implement the functions. + * awk.h: Add defines for systime and strftime. + * lex.c: Add support for systime and strftime. + +2019-10-07 Arnold D. Robbins + + Integrate features from different *BSD versions of awk. + Gensub support from NetBSD. Bitwise functions from OpenBSD. + + * awk.h: Add defines for and, or, xor, compl, lshift and rshift. + * awkgram.y: Add support for gensub. + * maketab.c: Ditto. + * lex.c: Add support for gensub and bitwise functions. + * parse.c (node5, op5): New functions. + * proto.h (node5, op5): New declarations. + * run.c (bltin): Implement the bitwise functions. + (gensub): New function. + * awk.1: Document additional functions. + +2019-10-07 Arnold D. Robbins + + * b.c (fnematch): Change type of pbuf from unsigned char to char. + * proto.h (fnematch): Ditto. + +2019-10-06 Arnold D. Robbins + + * lib.c (readrec): Allow RS a regular expression. Imported + the code from the NetBSD awk. + * b.c (fnematch): New function for implementing the feature. + * awk.1: Updated. + * main.c (version): Updated. + +2019-06-24 Arnold D. Robbins + + * makefile: Revise to take into account there is no more awktest.tar, + add targets 'check' and 'test', and also 'testclean' to clean up + after test run. Have 'clean' and 'cleaner' depend upon 'testclean'. + +2019-06-23 Arnold D. Robbins + + * testdir: Extracted from awktest.tar and added to Git. + * awktest.tar: Removed. + +2019-06-06 Arnold D. Robbins + + * awk.1: Fix a typo, minor edits. + +2019-06-05 Arnold D. Robbins + + * b.c (relex): Count parentheses and treat umatched right paren + as a literal character. + * awktest.tar (testdir/T.re): Added a test case. + * main.c (version): Updated. + 2019-05-29 Arnold D. Robbins * lib.c (isclvar): Remove check for additional '=' after diff --git a/contrib/one-true-awk/FIXES b/contrib/one-true-awk/FIXES index 183eaedee47d..516458eee0c1 100644 --- a/contrib/one-true-awk/FIXES +++ b/contrib/one-true-awk/FIXES @@ -25,6 +25,229 @@ THIS SOFTWARE. This file lists all bug fixes, changes, etc., made since the AWK book was sent to the printers in August, 1987. +February 15, 2021: + Small fix so that awk will compile again with g++. Thanks to + Arnold Robbins. + +January 06, 2021: + Fix a decision bug with trailing stuff in lib.c:is_valid_number + after recent changes. Thanks to Ozan Yigit. + +December 18, 2020: + Fix problems converting inf and NaN values in lib.c:is_valid_number. + Enhance number to string conversion to do the right thing for + NaN and inf values. Things are now pretty much the same as in + gawk. (Found a gawk bug while we're at it.) Added a torture + test for these values. Thanks to Arnold Robbins. Allows closing + of PR #101. + +December 15, 2020: + Merge PR #99, which gets the right header for strcasecmp. + Thanks to GitHub user michaelforney. + +December 8, 2020: + Merge PR #98: Disallow hex data. Allow only +nan, -nan, + +inf, -inf (case independent) to give NaN and infinity values. + Improve things so that string to double conversion is only + done once, yielding something of a speedup. This obviate + PR #95. Thanks to Arnold Robbins. + +December 3, 2020: + Fix to argument parsing to avoid printing spurious newlines. + Thanks to Todd Miller. Merges PR #97. + +October 13, 2020: + Add casts before all the calls to malloc/calloc/realloc in order + to get it to compile with g++. Thanks to Arnold Robbins. + +August 16, 2020: + Additional fixes for DJGPP. Thanks to Eli Zaretskii for + the testing. + +August 7, 2020: + Merge PR #93, which adds casts to (void*) for debug prints + using the %p format specifier. Thanks to GitHub user YongHaoWu + ("Chris") for the fixes. + +August 4, 2020: + In run.c, use non-restartable multibyte routines to attain + portability to DJGPP. Should fix Issue 92. Thanks to Albert Wik + for the report and to Todd Miller for the suggested fix. + +July 30, 2020: + Merge PRs 88-91 which fix small bugs. Thanks to Todd Miller and + Tim van der Molen for the fixes. + + In order to make life easier, we move exclusively to bison + as the parser generator. + +July 2, 2020: + Merge PRs 85 and 86 which fix regressions. Thanks to + Tim van der Molen for the fixes. + +June 25, 2020: + Merge PRs 82 and 84. The latter fixes issue #83. Thanks to + Todd Miller and awkfan77. + +June 12, 2020: + Clear errno before calling errcheck to avoid any spurious errors + left over from previous calls that may have set it. Thanks to + Todd Miller for the fix, from PR #80. + + Fix Issue #78 by allowing \r to follow floating point numbers in + lib.c:is_number. Thanks to GitHub user ajcarr for the report + and to Arnold Robbins for the fix. + +June 5, 2020: + In fldbld(), make sure that inputFS is set before trying to + use it. Thanks to Steffen Nurpmeso + for the report. + +May 5, 2020: + Fix checks for compilers that can handle noreturn. Thanks to + GitHub user enh-google for pointing it out. Closes Issue #79. + +April 16, 2020: + Handle old compilers that don't support C11 (for noreturn). + Thanks to Arnold Robbins. + +April 5, 2020: + Use and noreturn instead of GCC attributes. + Thanks to GitHub user awkfan77. Closes PR #77. + +February 28, 2020: + More cleanups from Christos Zoulas: notably backslash continuation + inside strings removes the newline and a fix for RS = "^a". + Fix for address sanitizer-found problem. Thanks to GitHub user + enh-google. + +February 19, 2020: + More small cleanups from Christos Zoulas. + +February 18, 2020: + Additional cleanups from Christos Zoulas. It's no longer necessary + to use the -y flag to bison. + +February 6, 2020: + Additional small cleanups from Christos Zoulas. awk is now + a little more robust about reporting I/O errors upon exit. + +January 31, 2020: + Merge PR #70, which avoids use of variable length arrays. Thanks + to GitHub user michaelforney. Fix issue #60 ({0} in interval + expressions doesn't work). Also get all tests working again. + Thanks to Arnold Robbins. + +January 24, 2020: + A number of small cleanups from Christos Zoulas. Add the close + on exec flag to files/pipes opened for redirection; courtesy of + Arnold Robbins. + +January 19, 2020: + If POSIXLY_CORRECT is set in the environment, then sub and gsub + use POSIX rules for multiple backslashes. This fixes Issue #66, + while maintaining backwards compatibility. + +January 9, 2020: + Input/output errors on closing files are now fatal instead of + mere warnings. Thanks to Martijn Dekker . + +January 5, 2020: + Fix a bug in the concatentation of two string constants into + one done in the grammar. Fixes GitHub issue #61. Thanks + to GitHub user awkfan77 for pointing out the direction for + the fix. New test T.concat added to the test suite. + Fix a few memory leaks reported by valgrind, as well. + +December 27, 2019: + Fix a bug whereby a{0,3} could match four a's. Thanks to + "Anonymous AWK fan" for the report. + +December 11, 2019: + Further printf-related fixes for 32 bit systems. + Thanks again to Christos Zoulas. + +December 8, 2019: + Fix the return value of sprintf("%d") on 32 bit systems. + Thanks to Jim Lowe for the report and to Christos Zoulas + for the fix. + +November 10, 2019: + Convert a number of Boolean integer variables into + actual bools. Convert compile_time variable into an + enum and simplify some of the related code. Thanks + to Arnold Robbins. + +November 8, 2019: + Fix from Ori Bernstein to get UTF-8 characters instead of + bytes when FS = "". This is currently the only bit of + the One True Awk that understands multibyte characters. + From Arnold Robbins, apply some cleanups in the test suite. + +October 25, 2019: + More fixes and cleanups from NetBSD, courtesy of Christos + Zoulas. Merges PRs 54 and 55. + +October 24, 2019: + Import second round of code cleanups from NetBSD. Much thanks + to Christos Zoulas (GitHub user zoulasc). Merges PR 53. + Add an optimization for string concatenation, also from + Christos. + +October 17, 2019: + Import code cleanups from NetBSD. Much thanks to Christos + Zoulas (GitHub user zoulasc). Merges PR 51. + +October 6, 2019: + Import code from NetBSD awk that implements RS as a regular + expression. + +September 10, 2019: + Fixes for various array / memory overruns found via gcc's + -fsanitize=unknown. Thanks to Alexander Richardson (GitHub + user arichardson). Merges PRs 47 and 48. + +July 28, 2019: + Import grammar optimization from NetBSD: Two string constants + concatenated together get turned into a single string. + +July 26, 2019: + Support POSIX-specified C-style escape sequences "\a" (alarm) + and "\v" (vertical tab) in command line arguments and regular + expressions, further to the support for them in strings added on + Apr 9, 1989. These now no longer match as literal "a" and "v" + characters (as they don't on other awk implementations). + Thanks to Martijn Dekker. + +July 17, 2019: + Pull in a number of code cleanups and minor fixes from + Warner Losh's bsd-ota branch. The only user visible change + is the use of random(3) as the random number generator. + Thanks to Warner Losh for collecting all these fixes in + one easy place to get them from. + +July 16, 2019: + Fix field splitting to use FS value as of the time a record + was read or assigned to. Thanks to GitHub user Cody Mello (melloc) + for the fix. (Merged from his branch, via PR #42.) Updated + testdir/T.split per said PR as well. + +June 24, 2019: + Extract awktest.tar into testdir directory. Add some very + simple mechanics to the makefile for running the tests and + for cleaning up. No changes to awk itself. + +June 17, 2019: + Disallow deleting SYMTAB and its elements, which creates + use-after-free bugs. Thanks to GitHub user Cody Mello (melloc) + for the fix. (Merged from PR #43.) + +June 5, 2019: + Allow unmatched right parenthesis in a regular expression to + be treated literally. Fixes Issue #40. Thanks to GitHub user + Warner Losh (bsdimp) for the report. Thanks to Arnold Robbins + for the fix. + May 29,2019: Fix check for command line arguments to no longer require that first character after '=' not be another '='. Reverts change of @@ -34,7 +257,7 @@ May 29,2019: Apr 7, 2019: Update awktest.tar(p.50) to use modern options to sort. Needed for Android development. Thanks to GitHub user mohd-akram (Mohamed - Akram). From Comment #33. + Akram). From Issue #33. Mar 12, 2019: Added very simplistic support for cross-compiling in the @@ -54,7 +277,7 @@ Mar 3, 2019: #12: Avoid undefined behaviour when using ctype(3) functions in relex(). Thanks to GitHub user iamleot. #31: Make getline handle numeric strings, and update FIXES. Thanks - to GitHub user arnoldrobbins + to GitHub user arnoldrobbins. #32: maketab: support build systems with read-only source. Thanks to GitHub user enh. @@ -159,10 +382,10 @@ Jun 12, 2011: /pat/, \n /pat/ {...} is now legal, though bad style to use. added checks to new -v code that permits -vnospace; thanks to - ruslan ermilov for spotting this and providing the patch. + ruslan ermilov for spotting this and providing the patch. removed fixed limit on number of open files; thanks to aleksey - cheusov and christos zoulos. + cheusov and christos zoulos. fixed day 1 bug that resurrected deleted elements of ARGV when used as filenames (in lib.c). @@ -180,10 +403,10 @@ May 1, 2011: and arnold robbins, changed srand() to return the previous seed (which is 1 on the first call of srand). the seed is an Awkfloat internally though converted to unsigned int to - pass to the library srand(). thanks, everyone. + pass to the library srand(). thanks, everyone. fixed a subtle (and i hope low-probability) overflow error - in fldbld, by adding space for one extra \0. thanks to + in fldbld, by adding space for one extra \0. thanks to robert bassett for spotting this one and providing a fix. removed the files related to compilation on windows. i no @@ -220,7 +443,7 @@ Oct 8, 2008: Oct 23, 2007: minor fix in lib.c: increase inputFS to 100, change malloc - for fields to n+1. + for fields to n+1. fixed memory fault caused by out of order test in setsval. @@ -267,7 +490,7 @@ Jan 17, 2006: core dump on linux with BEGIN {nextfile}, now fixed. - removed some #ifdef's in run.c and lex.c that appear to no + removed some #ifdef's in run.c and lex.c that appear to no longer be necessary. Apr 24, 2005: @@ -281,8 +504,8 @@ Jan 14, 2005: rethinking it. Dec 31, 2004: - prevent overflow of -f array in main, head off potential error in - call of SYNTAX(), test malloc return in lib.c, all with thanks to + prevent overflow of -f array in main, head off potential error in + call of SYNTAX(), test malloc return in lib.c, all with thanks to todd miller. Dec 22, 2004: @@ -310,8 +533,8 @@ Nov 22, 2003: code known to man. fixed a storage leak in call() that appears to have been there since - 1983 or so -- a function without an explicit return that assigns a - string to a parameter leaked a Cell. thanks to moinak ghosh for + 1983 or so -- a function without an explicit return that assigns a + string to a parameter leaked a Cell. thanks to moinak ghosh for spotting this very subtle one. Jul 31, 2003: @@ -333,7 +556,7 @@ Jul 28, 2003: radix character in programs and command line arguments regardless of the locale; otherwise, the locale should prevail for input and output of numbers. so it's intended to work that way. - + i have rescinded the attempt to use strcoll in expanding shorthands in regular expressions (cclenter). its properties are much too surprising; for example [a-c] matches aAbBc in locale en_US but abBcC @@ -397,7 +620,7 @@ Nov 29, 2002: Jun 28, 2002: modified run/format() and tran/getsval() to do a slightly better job on using OFMT for output from print and CONVFMT for other - number->string conversions, as promised by posix and done by + number->string conversions, as promised by posix and done by gawk and mawk. there are still places where it doesn't work right if CONVFMT is changed; by then the STR attribute of the variable has been irrevocably set. thanks to arnold robbins for @@ -429,7 +652,7 @@ Feb 10, 2002: Jan 1, 2002: fflush() or fflush("") flushes all files and pipes. - length(arrayname) returns number of elements; thanks to + length(arrayname) returns number of elements; thanks to arnold robbins for suggestion. added a makefile.win to make it easier to build on windows. @@ -479,7 +702,7 @@ July 5, 2000: May 25, 2000: yet another attempt at making 8-bit input work, with another - band-aid in b.c (member()), and some (uschar) casts to head + band-aid in b.c (member()), and some (uschar) casts to head off potential errors in subscripts (like isdigit). also changed HAT to NCHARS-2. thanks again to santiago vila. @@ -526,7 +749,7 @@ Apr 21, 1999: the test case.) Apr 16, 1999: - with code kindly provided by Bruce Lilly, awk now parses + with code kindly provided by Bruce Lilly, awk now parses /=/ and similar constructs more sensibly in more places. Bruce also provided some helpful test cases. @@ -583,7 +806,7 @@ Jan 13, 1999: Oct 19, 1998: fixed a couple of bugs in getrec: could fail to update $0 - after a getline var; because inputFS wasn't initialized, + after a getline var; because inputFS wasn't initialized, could split $0 on every character, a misleading diversion. fixed caching bug in makedfa: LRU was actually removing @@ -731,7 +954,7 @@ May 2, 1996: input file. (thanks to arnold robbins for inspiration and code). small fixes to regexpr code: can now handle []], [[], and - variants; [] is now a syntax error, rather than matching + variants; [] is now a syntax error, rather than matching everything; [z-a] is now empty, not z. far from complete or correct, however. (thanks to jeffrey friedl for pointing out some awful behaviors.) diff --git a/contrib/one-true-awk/REGRESS b/contrib/one-true-awk/REGRESS index 7d3ded69d536..eb3b5d7ac70b 100755 --- a/contrib/one-true-awk/REGRESS +++ b/contrib/one-true-awk/REGRESS @@ -33,3 +33,7 @@ then fi REGRESS + +cd .. +cd bugs-fixed +REGRESS diff --git a/contrib/one-true-awk/awk.1 b/contrib/one-true-awk/awk.1 index b8e00cb60449..b3698eb07d1a 100644 --- a/contrib/one-true-awk/awk.1 +++ b/contrib/one-true-awk/awk.1 @@ -7,6 +7,10 @@ .fi .ft 1 .. +.de TF +.IP "" "\w'\fB\\$1\ \ \fP'u" +.PD 0 +.. .TH AWK 1 .CT 1 files prog_other .SH NAME @@ -48,7 +52,7 @@ matches the pattern. Each line is matched against the pattern portion of every pattern-action statement; the associated action is performed for each matched pattern. -The file name +The file name .B \- means the standard input. Any @@ -90,7 +94,7 @@ A pattern-action statement has the form: .IP .IB pattern " { " action " } .PP -A missing +A missing .BI { " action " } means print the line; a missing pattern always matches. @@ -209,7 +213,7 @@ or length of if no argument. .TP .B rand -random number on [0,1) +random number on [0,1). .TP .B srand sets seed for @@ -217,7 +221,7 @@ sets seed for and returns the previous seed. .TP .B int -truncates to an integer value +truncates to an integer value. .TP \fBsubstr(\fIs\fB, \fIm\fR [\fB, \fIn\^\fR]\fB)\fR the @@ -225,12 +229,11 @@ the substring of .I s that begins at position -.I m +.I m counted from 1. If no -.IR m , -use the rest of the string -.I +.IR n , +use the rest of the string. .TP .BI index( s , " t" ) the position in @@ -294,6 +297,25 @@ and .B gsub return the number of replacements. .TP +\fBgensub(\fIpat\fB, \fIrepl\fB, \fIhow\fR [\fB, \fItarget\fR]\fB)\fR +replaces instances of +.I pat +in +.I target +with +.IR repl . +If +.I how +is \fB"g"\fR or \fB"G"\fR, do so globally. Otherwise, +.I how +is a number indicating which occurrence to replace. If no +.IR target , +use +.BR $0 . +Return the resulting string; +.I target +is not modified. +.TP .BI sprintf( fmt , " expr" , " ...\fB) the string resulting from formatting .I expr ... @@ -302,13 +324,35 @@ according to the format .IR fmt . .TP +.B systime() +returns the current date and time as a standard +``seconds since the epoch'' value. +.TP +.BI strftime( fmt ", " timestamp\^ ) +formats +.I timestamp +(a value in seconds since the epoch) +according to +.IR fmt , +which is a format string as supported by +.IR strftime (3). +Both +.I timestamp +and +.I fmt +may be omitted; if no +.IR timestamp , +the current time of day is used, and if no +.IR fmt , +a default format of \fB"%a %b %e %H:%M:%S %Z %Y"\fR is used. +.TP .BI system( cmd ) executes .I cmd and returns its exit status. This will be \-1 upon error, .IR cmd 's exit status upon a normal exit, -256 + +256 + .I sig upon death-by-signal, where .I sig @@ -361,19 +405,26 @@ In all cases, returns 1 for a successful input, 0 for end of file, and \-1 for an error. .PP +The functions +.BR compl , +.BR and , +.BR or , +.BR xor , +.BR lshift , +and +.B rshift +peform the corresponding bitwise operations on their +operands, which are first truncated to integer. +.PP Patterns are arbitrary Boolean combinations (with .BR "! || &&" ) of regular expressions and relational expressions. Regular expressions are as in -.IR egrep (1) -except numeric quantifiers are not supported beyond the basic -.B + -and -.B ? -for quantities \&\f(CW"0 or 1"\fP and \&\f(CW"1 or more"\fP -respectively. +.IR egrep ; +see +.IR grep (1). Isolated regular expressions in a pattern apply to the entire line. Regular expressions may also occur in @@ -483,6 +534,11 @@ the length of a string matched by .TP .B RS input record separator (default newline). +If empty, blank lines separate records. +If more than one character long, +.B RS +is treated as a regular expression, and records are +separated by text matching the expression. .TP .B RSTART the start position of a string matched by @@ -502,6 +558,16 @@ functions may be called recursively. Parameters are local to the function; all other variables are global. Thus local variables may be created by providing excess parameters in the function definition. +.SH ENVIRONMENT VARIABLES +If +.B POSIXLY_CORRECT +is set in the environment, then +.I awk +follows the POSIX rules for +.B sub +and +.B gsub +with respect to consecutive backslashes and ampersands. .SH EXAMPLES .TP .EX @@ -546,8 +612,8 @@ BEGIN { # Simulate echo(1) .fi .EE .SH SEE ALSO -.IR grep (1), -.IR lex (1), +.IR grep (1), +.IR lex (1), .IR sed (1) .br A. V. Aho, B. W. Kernighan, P. J. Weinberger, @@ -558,8 +624,61 @@ There are no explicit conversions between numbers and strings. To force an expression to be treated as a number add 0 to it; to force it to be treated as a string concatenate \&\f(CW""\fP to it. -.br +.PP The scope rules for variables in functions are a botch; the syntax is worse. -.br +.PP Only eight-bit characters sets are handled correctly. +.SH UNUSUAL FLOATING-POINT VALUES +.I Awk +was designed before IEEE 754 arithmetic defined Not-A-Number (NaN) +and Infinity values, which are supported by all modern floating-point +hardware. +.PP +Because +.I awk +uses +.IR strtod (3) +and +.IR atof (3) +to convert string values to double-precision floating-point values, +modern C libraries also convert strings starting with +.B inf +and +.B nan +into infinity and NaN values respectively. This led to strange results, +with something like this: +.PP +.EX +.nf +echo nancy | awk '{ print $1 + 0 }' +.fi +.EE +.PP +printing +.B nan +instead of zero. +.PP +.I Awk +now follows GNU AWK, and prefilters string values before attempting +to convert them to numbers, as follows: +.TP +.I "Hexadecimal values" +Hexadecimal values (allowed since C99) convert to zero, as they did +prior to C99. +.TP +.I "NaN values" +The two strings +.B +nan +and +.B \-nan +(case independent) convert to NaN. No others do. +(NaNs can have signs.) +.TP +.I "Infinity values" +The two strings +.B +inf +and +.B \-inf +(case independent) convert to positive and negative infinity, respectively. +No others do. diff --git a/contrib/one-true-awk/awk.h b/contrib/one-true-awk/awk.h index 31d070aecddc..230eac41548c 100644 --- a/contrib/one-true-awk/awk.h +++ b/contrib/one-true-awk/awk.h @@ -23,6 +23,13 @@ THIS SOFTWARE. ****************************************************************/ #include +#include +#include +#if __STDC_VERSION__ <= 199901L +#define noreturn +#else +#include +#endif typedef double Awkfloat; @@ -30,24 +37,34 @@ typedef double Awkfloat; typedef unsigned char uschar; -#define xfree(a) { if ((a) != NULL) { free((void *) (a)); (a) = NULL; } } +#define xfree(a) { if ((a) != NULL) { free((void *)(intptr_t)(a)); (a) = NULL; } } +/* + * We sometimes cheat writing read-only pointers to NUL-terminate them + * and then put back the original value + */ +#define setptr(ptr, a) (*(char *)(intptr_t)(ptr)) = (a) -#define NN(p) ((p) ? (p) : "(null)") /* guaranteed non-null for dprintf +#define NN(p) ((p) ? (p) : "(null)") /* guaranteed non-null for DPRINTF */ #define DEBUG #ifdef DEBUG - /* uses have to be doubly parenthesized */ -# define dprintf(x) if (dbg) printf x +# define DPRINTF(...) if (dbg) printf(__VA_ARGS__) #else -# define dprintf(x) +# define DPRINTF(...) #endif -extern int compile_time; /* 1 if compiling, 0 if running */ -extern int safe; /* 0 => unsafe, 1 => safe */ +extern enum compile_states { + RUNNING, + COMPILING, + ERROR_PRINTING +} compile_time; + +extern bool safe; /* false => unsafe, true => safe */ #define RECSIZE (8 * 1024) /* sets limit on records, fields, etc., etc. */ extern int recsize; /* size of current record, orig RECSIZE */ +extern char EMPTY[]; /* this avoid -Wwritable-strings issues */ extern char **FS; extern char **RS; extern char **ORS; @@ -64,13 +81,11 @@ extern Awkfloat *RLENGTH; extern char *record; /* points to $0 */ extern int lineno; /* line number in awk program */ extern int errorflag; /* 1 if error has occurred */ -extern int donefld; /* 1 if record broken into fields */ -extern int donerec; /* 1 if record is valid (no fld has changed */ -extern char inputFS[]; /* FS at time of input, for field splitting */ - +extern bool donefld; /* true if record broken into fields */ +extern bool donerec; /* true if record is valid (no fld has changed */ extern int dbg; -extern char *patbeg; /* beginning of pattern matched */ +extern const char *patbeg; /* beginning of pattern matched */ extern int patlen; /* length of pattern matched. set in b.c */ /* Cell: all information about a variable or constant */ @@ -105,6 +120,7 @@ extern Cell *rsloc; /* RS */ extern Cell *rstartloc; /* RSTART */ extern Cell *rlengthloc; /* RLENGTH */ extern Cell *subseploc; /* SUBSEP */ +extern Cell *symtabloc; /* SYMTAB */ /* Cell.tval values: */ #define NUM 01 /* number value is valid */ @@ -134,12 +150,14 @@ extern Cell *subseploc; /* SUBSEP */ #define FTOUPPER 12 #define FTOLOWER 13 #define FFLUSH 14 -#define FAND 15 -#define FFOR 16 -#define FXOR 17 -#define FCOMPL 18 -#define FLSHIFT 19 -#define FRSHIFT 20 +#define FAND 15 +#define FFOR 16 +#define FXOR 17 +#define FCOMPL 18 +#define FLSHIFT 19 +#define FRSHIFT 20 +#define FSYSTIME 21 +#define FSTRFTIME 22 /* Node: parse tree is made of nodes, with Cell's at bottom */ @@ -167,7 +185,7 @@ extern Node *nullnode; #define CCOPY 6 #define CCON 5 #define CTEMP 4 -#define CNAME 3 +#define CNAME 3 #define CVAR 2 #define CFLD 1 #define CUNK 0 @@ -217,6 +235,7 @@ extern int pairstack[], paircnt; #define NCHARS (256+3) /* 256 handles 8-bit chars; 128 does 7-bit */ /* watch out in match(), etc. */ +#define HAT (NCHARS+2) /* matches ^ in regular expr */ #define NSTATES 32 #define HAT (NCHARS+2) /* matches ^ in regular expr */ /* NCHARS is 2**n */ @@ -232,16 +251,16 @@ typedef struct rrow { } rrow; typedef struct fa { - uschar gototab[NSTATES][HAT + 1]; - uschar out[NSTATES]; + unsigned int **gototab; + uschar *out; uschar *restr; - int *posns[NSTATES]; - int anchor; + int **posns; + int state_count; + bool anchor; int use; int initstat; int curstat; int accept; - int reset; struct rrow re[1]; /* variable: actual size set by calling malloc */ } fa; diff --git a/contrib/one-true-awk/awkgram.y b/contrib/one-true-awk/awkgram.y index e4abeeddcb6a..f37073d1f9ac 100644 --- a/contrib/one-true-awk/awkgram.y +++ b/contrib/one-true-awk/awkgram.y @@ -32,8 +32,8 @@ int yywrap(void) { return(1); } Node *beginloc = 0; Node *endloc = 0; -int infunc = 0; /* = 1 if in arglist or body of func */ *** 4847 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 18:27:06 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 30593658F38; Sat, 10 Jul 2021 18:27:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMdmB0fl5z4dgD; Sat, 10 Jul 2021 18:27:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F304E518C; Sat, 10 Jul 2021 18:27:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AIR5Yl081320; Sat, 10 Jul 2021 18:27:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AIR5no081319; Sat, 10 Jul 2021 18:27:05 GMT (envelope-from git) Date: Sat, 10 Jul 2021 18:27:05 GMT Message-Id: <202107101827.16AIR5no081319@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: f2712c6c073c - stable/12 - This should have been committed in r348511 with the awk update. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: f2712c6c073c58d64863e452e7a75b413834a80a Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 18:27:06 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=f2712c6c073c58d64863e452e7a75b413834a80a commit f2712c6c073c58d64863e452e7a75b413834a80a Author: Warner Losh AuthorDate: 2019-06-02 16:44:50 +0000 Commit: Warner Losh CommitDate: 2021-07-10 18:24:20 +0000 This should have been committed in r348511 with the awk update. It was in my tree, the build worked, but I committed from contrib/one-true-awk rather than the top level, so was omitted. (cherry picked from commit e9b659753c07b539df17441012b0bfa5ebdbe0ea) --- usr.bin/awk/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/usr.bin/awk/Makefile b/usr.bin/awk/Makefile index fc032f66459b..d31e811bbec5 100644 --- a/usr.bin/awk/Makefile +++ b/usr.bin/awk/Makefile @@ -23,7 +23,7 @@ awkgram.tab.h: awkgram.h ln -sf ${.ALLSRC:M*.h} ${.TARGET} proctab.c: maketab - ${BTOOLSPATH:U.}/maketab > proctab.c + ${BTOOLSPATH:U.}/maketab awkgram.h > proctab.c build-tools: maketab maketab: awkgram.tab.h maketab.c ${BUILD_TOOLS_META} From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:10:40 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5B889659DEF; Sat, 10 Jul 2021 19:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMfkS29rHz4jtp; Sat, 10 Jul 2021 19:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 27E855932; Sat, 10 Jul 2021 19:10:40 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJAeBp046580; Sat, 10 Jul 2021 19:10:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJAe4d046579; Sat, 10 Jul 2021 19:10:40 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:10:40 GMT Message-Id: <202107101910.16AJAe4d046579@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: e50fd6778435 - stable/13 - Improve handling of USB device re-open in the LibUSB v1.x API. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e50fd67784355030aedca8704223456c056fdeb4 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:10:40 -0000 The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=e50fd67784355030aedca8704223456c056fdeb4 commit e50fd67784355030aedca8704223456c056fdeb4 Author: Hans Petter Selasky AuthorDate: 2021-06-11 15:06:10 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:10:10 +0000 Improve handling of USB device re-open in the LibUSB v1.x API. Make sure the "device_is_gone" flag is cleared after every successful open, so that the "device_is_gone" flag doesn't persist forever. Found by: sergii.dmytruk@3mdeb.com PR: 256296 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit 6847ea50196f1a685be408a24f01cb8d407da19c) --- lib/libusb/libusb10.c | 9 +++++++++ lib/libusb/libusb10_io.c | 12 ++++++++++-- lib/libusb/libusb20.c | 5 +++++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/lib/libusb/libusb10.c b/lib/libusb/libusb10.c index 1c45b87d8f0b..ffe0cf3f366a 100644 --- a/lib/libusb/libusb10.c +++ b/lib/libusb/libusb10.c @@ -529,6 +529,15 @@ libusb_open(libusb_device *dev, libusb_device_handle **devh) libusb_unref_device(dev); return (LIBUSB_ERROR_NO_MEM); } + + /* + * Clear the device gone flag, in case the device was opened + * after a re-attach, to allow new transaction: + */ + CTX_LOCK(ctx); + dev->device_is_gone = 0; + CTX_UNLOCK(ctx); + libusb10_add_pollfd(ctx, &dev->dev_poll, pdev, libusb20_dev_get_fd(pdev), POLLIN | POLLOUT | POLLRDNORM | POLLWRNORM); diff --git a/lib/libusb/libusb10_io.c b/lib/libusb/libusb10_io.c index 53f5b040436d..0e32fc31c8e0 100644 --- a/lib/libusb/libusb10_io.c +++ b/lib/libusb/libusb10_io.c @@ -165,8 +165,16 @@ libusb10_handle_events_sub(struct libusb_context *ctx, struct timeval *tv) err = libusb20_dev_process(ppdev[i]); if (err) { - /* set device is gone */ - dev->device_is_gone = 1; + /* + * When the device is opened + * set the "device_is_gone" + * flag. This prevents the + * client from submitting new + * USB transfers to a detached + * device. + */ + if (ppdev[i]->is_opened) + dev->device_is_gone = 1; /* remove USB device from polling loop */ libusb10_remove_pollfd(dev->ctx, &dev->dev_poll); diff --git a/lib/libusb/libusb20.c b/lib/libusb/libusb20.c index 6c2bf721bea9..4323552b83ad 100644 --- a/lib/libusb/libusb20.c +++ b/lib/libusb/libusb20.c @@ -608,6 +608,11 @@ libusb20_dev_close(struct libusb20_device *pdev) pdev->is_opened = 0; + /* + * Make sure libusb20_tr_get_pointer() fails: + */ + pdev->nTransfer = 0; + /* * The following variable is only used by the libusb v0.1 * compat layer: From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:14:17 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 217D0659F0D; Sat, 10 Jul 2021 19:14:17 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMfpc754tz4kct; Sat, 10 Jul 2021 19:14:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CFE4F5D84; Sat, 10 Jul 2021 19:14:16 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJEG8n048616; Sat, 10 Jul 2021 19:14:16 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJEGOi048615; Sat, 10 Jul 2021 19:14:16 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:14:16 GMT Message-Id: <202107101914.16AJEGOi048615@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 1e68639f62f9 - stable/12 - Improve handling of USB device re-open in the LibUSB v1.x API. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 1e68639f62f9d965cd46e6f5ca9c3dde0941d704 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:14:17 -0000 The branch stable/12 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=1e68639f62f9d965cd46e6f5ca9c3dde0941d704 commit 1e68639f62f9d965cd46e6f5ca9c3dde0941d704 Author: Hans Petter Selasky AuthorDate: 2021-06-11 15:06:10 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:13:51 +0000 Improve handling of USB device re-open in the LibUSB v1.x API. Make sure the "device_is_gone" flag is cleared after every successful open, so that the "device_is_gone" flag doesn't persist forever. Found by: sergii.dmytruk@3mdeb.com PR: 256296 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit 6847ea50196f1a685be408a24f01cb8d407da19c) --- lib/libusb/libusb10.c | 9 +++++++++ lib/libusb/libusb10_io.c | 12 ++++++++++-- lib/libusb/libusb20.c | 5 +++++ 3 files changed, 24 insertions(+), 2 deletions(-) diff --git a/lib/libusb/libusb10.c b/lib/libusb/libusb10.c index 1c45b87d8f0b..ffe0cf3f366a 100644 --- a/lib/libusb/libusb10.c +++ b/lib/libusb/libusb10.c @@ -529,6 +529,15 @@ libusb_open(libusb_device *dev, libusb_device_handle **devh) libusb_unref_device(dev); return (LIBUSB_ERROR_NO_MEM); } + + /* + * Clear the device gone flag, in case the device was opened + * after a re-attach, to allow new transaction: + */ + CTX_LOCK(ctx); + dev->device_is_gone = 0; + CTX_UNLOCK(ctx); + libusb10_add_pollfd(ctx, &dev->dev_poll, pdev, libusb20_dev_get_fd(pdev), POLLIN | POLLOUT | POLLRDNORM | POLLWRNORM); diff --git a/lib/libusb/libusb10_io.c b/lib/libusb/libusb10_io.c index 53f5b040436d..0e32fc31c8e0 100644 --- a/lib/libusb/libusb10_io.c +++ b/lib/libusb/libusb10_io.c @@ -165,8 +165,16 @@ libusb10_handle_events_sub(struct libusb_context *ctx, struct timeval *tv) err = libusb20_dev_process(ppdev[i]); if (err) { - /* set device is gone */ - dev->device_is_gone = 1; + /* + * When the device is opened + * set the "device_is_gone" + * flag. This prevents the + * client from submitting new + * USB transfers to a detached + * device. + */ + if (ppdev[i]->is_opened) + dev->device_is_gone = 1; /* remove USB device from polling loop */ libusb10_remove_pollfd(dev->ctx, &dev->dev_poll); diff --git a/lib/libusb/libusb20.c b/lib/libusb/libusb20.c index 6c2bf721bea9..4323552b83ad 100644 --- a/lib/libusb/libusb20.c +++ b/lib/libusb/libusb20.c @@ -608,6 +608,11 @@ libusb20_dev_close(struct libusb20_device *pdev) pdev->is_opened = 0; + /* + * Make sure libusb20_tr_get_pointer() fails: + */ + pdev->nTransfer = 0; + /* * The following variable is only used by the libusb v0.1 * compat layer: From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:18:59 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5A1E659FB3; Sat, 10 Jul 2021 19:18:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMfw35bk6z4lC9; Sat, 10 Jul 2021 19:18:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A48D75B9E; Sat, 10 Jul 2021 19:18:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJIxD6049094; Sat, 10 Jul 2021 19:18:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJIxob049093; Sat, 10 Jul 2021 19:18:59 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:18:59 GMT Message-Id: <202107101918.16AJIxob049093@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 5e0a56256cb0 - stable/13 - ibstat: Include prototype for sysctlbyname(). MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 5e0a56256cb0390c7b7f89e8dcf5398d1a0f1577 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:18:59 -0000 The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=5e0a56256cb0390c7b7f89e8dcf5398d1a0f1577 commit 5e0a56256cb0390c7b7f89e8dcf5398d1a0f1577 Author: Hans Petter Selasky AuthorDate: 2021-05-27 07:39:49 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:17:24 +0000 ibstat: Include prototype for sysctlbyname(). Fixes the following compile warning: implicit declaration of function 'sysctlbyname' is invalid in C99 [-Wimplicit-function-declaration] Found by: J87 Differential Revision: https://reviews.freebsd.org/D30484 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit 16fa3dcba027d13dcda9ee78e6057e3e5a79f80c) --- contrib/ofed/infiniband-diags/src/ibstat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/ofed/infiniband-diags/src/ibstat.c b/contrib/ofed/infiniband-diags/src/ibstat.c index 6e08fad409ca..e4f3608dfd76 100644 --- a/contrib/ofed/infiniband-diags/src/ibstat.c +++ b/contrib/ofed/infiniband-diags/src/ibstat.c @@ -48,6 +48,7 @@ #include #include #include +#include #include From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:19:00 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E607C65A16A; Sat, 10 Jul 2021 19:19:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMfw466Yxz4lHT; Sat, 10 Jul 2021 19:19:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BAA4F58DC; Sat, 10 Jul 2021 19:19:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJJ0Tl049125; Sat, 10 Jul 2021 19:19:00 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJJ0bx049124; Sat, 10 Jul 2021 19:19:00 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:19:00 GMT Message-Id: <202107101919.16AJJ0bx049124@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 77f7133bd55c - stable/13 - Add support for RTL8153B, RTL8156 and RTL8156B to if_ure(4). MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 77f7133bd55c3f6a73b6428ceaa8c266940a55f3 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:19:01 -0000 The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=77f7133bd55c3f6a73b6428ceaa8c266940a55f3 commit 77f7133bd55c3f6a73b6428ceaa8c266940a55f3 Author: Hans Petter Selasky AuthorDate: 2021-06-04 08:28:58 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:17:26 +0000 Add support for RTL8153B, RTL8156 and RTL8156B to if_ure(4). Submitted by: fbbz@synack.eu PR: 253374 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit d4cf41a99b405c73288aea81e3c4580d1de18435) --- sys/dev/usb/net/if_ure.c | 965 ++++++++++++++++++++++++++++++++------------ sys/dev/usb/net/if_urereg.h | 200 ++++++++- 2 files changed, 890 insertions(+), 275 deletions(-) diff --git a/sys/dev/usb/net/if_ure.c b/sys/dev/usb/net/if_ure.c index 30fcee59cce3..2e4053a0e0c6 100644 --- a/sys/dev/usb/net/if_ure.c +++ b/sys/dev/usb/net/if_ure.c @@ -96,16 +96,17 @@ SYSCTL_INT(_hw_usb_ure, OID_AUTO, debug, CTLFLAG_RWTUN, &ure_debug, 0, */ static const STRUCT_USB_HOST_ID ure_devs[] = { #define URE_DEV(v,p,i) { USB_VPI(USB_VENDOR_##v, USB_PRODUCT_##v##_##p, i) } - URE_DEV(LENOVO, RTL8153, 0), + URE_DEV(LENOVO, RTL8153, URE_FLAG_8153), URE_DEV(LENOVO, TBT3LAN, 0), URE_DEV(LENOVO, TBT3LANGEN2, 0), URE_DEV(LENOVO, ONELINK, 0), URE_DEV(LENOVO, USBCLAN, 0), URE_DEV(LENOVO, USBCLANGEN2, 0), - URE_DEV(NVIDIA, RTL8153, 0), + URE_DEV(NVIDIA, RTL8153, URE_FLAG_8153), URE_DEV(REALTEK, RTL8152, URE_FLAG_8152), - URE_DEV(REALTEK, RTL8153, 0), - URE_DEV(TPLINK, RTL8153, 0), + URE_DEV(REALTEK, RTL8153, URE_FLAG_8153), + URE_DEV(TPLINK, RTL8153, URE_FLAG_8153), + URE_DEV(REALTEK, RTL8156, URE_FLAG_8156), #undef URE_DEV }; @@ -141,6 +142,7 @@ static int ure_write_2(struct ure_softc *, uint16_t, uint16_t, uint32_t); static int ure_write_4(struct ure_softc *, uint16_t, uint16_t, uint32_t); static uint16_t ure_ocp_reg_read(struct ure_softc *, uint16_t); static void ure_ocp_reg_write(struct ure_softc *, uint16_t, uint16_t); +static void ure_sram_write(struct ure_softc *, uint16_t, uint16_t); static int ure_sysctl_chipver(SYSCTL_HANDLER_ARGS); @@ -149,96 +151,21 @@ static int ure_attach_post_sub(struct usb_ether *); static void ure_reset(struct ure_softc *); static int ure_ifmedia_upd(struct ifnet *); static void ure_ifmedia_sts(struct ifnet *, struct ifmediareq *); -static int ure_ioctl(struct ifnet *, u_long, caddr_t); +static void ure_add_media_types(struct ure_softc *); +static void ure_link_state(struct ure_softc *sc); +static int ure_get_link_status(struct ure_softc *); +static int ure_ioctl(struct ifnet *, u_long, caddr_t); static void ure_rtl8152_init(struct ure_softc *); +static void ure_rtl8152_nic_reset(struct ure_softc *); static void ure_rtl8153_init(struct ure_softc *); +static void ure_rtl8153b_init(struct ure_softc *); +static void ure_rtl8153b_nic_reset(struct ure_softc *); static void ure_disable_teredo(struct ure_softc *); -static void ure_init_fifo(struct ure_softc *); +static void ure_enable_aldps(struct ure_softc *, bool); +static uint16_t ure_phy_status(struct ure_softc *, uint16_t); static void ure_rxcsum(int capenb, struct ure_rxpkt *rp, struct mbuf *m); static int ure_txcsum(struct mbuf *m, int caps, uint32_t *regout); -static const struct usb_config ure_config_rx[URE_N_TRANSFER] = { - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_IN, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.short_xfer_ok = 1,}, - .callback = ure_bulk_read_callback, - .timeout = 0, /* no timeout */ - }, - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_IN, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.short_xfer_ok = 1,}, - .callback = ure_bulk_read_callback, - .timeout = 0, /* no timeout */ - }, -#if URE_N_TRANSFER == 4 - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_IN, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.short_xfer_ok = 1,}, - .callback = ure_bulk_read_callback, - .timeout = 0, /* no timeout */ - }, - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_IN, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.short_xfer_ok = 1,}, - .callback = ure_bulk_read_callback, - .timeout = 0, /* no timeout */ - }, -#endif -}; - -static const struct usb_config ure_config_tx[URE_N_TRANSFER] = { - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_OUT, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.force_short_xfer = 1,}, - .callback = ure_bulk_write_callback, - .timeout = 10000, /* 10 seconds */ - }, - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_OUT, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.force_short_xfer = 1,}, - .callback = ure_bulk_write_callback, - .timeout = 10000, /* 10 seconds */ - }, -#if URE_N_TRANSFER == 4 - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_OUT, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.force_short_xfer = 1,}, - .callback = ure_bulk_write_callback, - .timeout = 10000, /* 10 seconds */ - }, - { - .type = UE_BULK, - .endpoint = UE_ADDR_ANY, - .direction = UE_DIR_OUT, - .bufsize = URE_TRANSFER_SIZE, - .flags = {.pipe_bof = 1,.force_short_xfer = 1,}, - .callback = ure_bulk_write_callback, - .timeout = 10000, /* 10 seconds */ - }, -#endif -}; - static device_method_t ure_methods[] = { /* Device interface. */ DEVMETHOD(device_probe, ure_probe), @@ -283,6 +210,20 @@ static const struct usb_ether_methods ure_ue_methods = { .ue_mii_sts = ure_ifmedia_sts, }; +#define URE_SETBIT_1(sc, reg, index, x) \ + ure_write_1(sc, reg, index, ure_read_1(sc, reg, index) | (x)) +#define URE_SETBIT_2(sc, reg, index, x) \ + ure_write_2(sc, reg, index, ure_read_2(sc, reg, index) | (x)) +#define URE_SETBIT_4(sc, reg, index, x) \ + ure_write_4(sc, reg, index, ure_read_4(sc, reg, index) | (x)) + +#define URE_CLRBIT_1(sc, reg, index, x) \ + ure_write_1(sc, reg, index, ure_read_1(sc, reg, index) & ~(x)) +#define URE_CLRBIT_2(sc, reg, index, x) \ + ure_write_2(sc, reg, index, ure_read_2(sc, reg, index) & ~(x)) +#define URE_CLRBIT_4(sc, reg, index, x) \ + ure_write_4(sc, reg, index, ure_read_4(sc, reg, index) & ~(x)) + static int ure_ctl(struct ure_softc *sc, uint8_t rw, uint16_t val, uint16_t index, void *buf, int len) @@ -435,6 +376,13 @@ ure_ocp_reg_write(struct ure_softc *sc, uint16_t addr, uint16_t data) ure_write_2(sc, reg, URE_MCU_TYPE_PLA, data); } +static void +ure_sram_write(struct ure_softc *sc, uint16_t addr, uint16_t data) +{ + ure_ocp_reg_write(sc, URE_OCP_SRAM_ADDR, addr); + ure_ocp_reg_write(sc, URE_OCP_SRAM_DATA, data); +} + static int ure_miibus_readreg(device_t dev, int phy, int reg) { @@ -558,25 +506,58 @@ ure_attach(device_t dev) struct usb_attach_arg *uaa = device_get_ivars(dev); struct ure_softc *sc = device_get_softc(dev); struct usb_ether *ue = &sc->sc_ue; + struct usb_config ure_config_rx[URE_MAX_RX]; + struct usb_config ure_config_tx[URE_MAX_TX]; uint8_t iface_index; int error; + int i; sc->sc_flags = USB_GET_DRIVER_INFO(uaa); device_set_usb_desc(dev); mtx_init(&sc->sc_mtx, device_get_nameunit(dev), NULL, MTX_DEF); iface_index = URE_IFACE_IDX; + + if (sc->sc_flags & (URE_FLAG_8153 | URE_FLAG_8153B)) + sc->sc_rxbufsz = URE_8153_RX_BUFSZ; + else if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) + sc->sc_rxbufsz = URE_8156_RX_BUFSZ; + else + sc->sc_rxbufsz = URE_8152_RX_BUFSZ; + + for (i = 0; i < URE_MAX_RX; i++) { + ure_config_rx[i] = (struct usb_config) { + .type = UE_BULK, + .endpoint = UE_ADDR_ANY, + .direction = UE_DIR_IN, + .bufsize = sc->sc_rxbufsz, + .flags = {.pipe_bof = 1,.short_xfer_ok = 1,}, + .callback = ure_bulk_read_callback, + .timeout = 0, /* no timeout */ + }; + } error = usbd_transfer_setup(uaa->device, &iface_index, sc->sc_rx_xfer, - ure_config_rx, URE_N_TRANSFER, sc, &sc->sc_mtx); + ure_config_rx, URE_MAX_RX, sc, &sc->sc_mtx); if (error != 0) { device_printf(dev, "allocating USB RX transfers failed\n"); goto detach; } + for (i = 0; i < URE_MAX_TX; i++) { + ure_config_tx[i] = (struct usb_config) { + .type = UE_BULK, + .endpoint = UE_ADDR_ANY, + .direction = UE_DIR_OUT, + .bufsize = URE_TX_BUFSZ, + .flags = {.pipe_bof = 1,.force_short_xfer = 1,}, + .callback = ure_bulk_write_callback, + .timeout = 10000, /* 10 seconds */ + }; + } error = usbd_transfer_setup(uaa->device, &iface_index, sc->sc_tx_xfer, - ure_config_tx, URE_N_TRANSFER, sc, &sc->sc_mtx); + ure_config_tx, URE_MAX_TX, sc, &sc->sc_mtx); if (error != 0) { - usbd_transfer_unsetup(sc->sc_rx_xfer, URE_N_TRANSFER); + usbd_transfer_unsetup(sc->sc_rx_xfer, URE_MAX_RX); device_printf(dev, "allocating USB TX transfers failed\n"); goto detach; } @@ -605,8 +586,8 @@ ure_detach(device_t dev) struct ure_softc *sc = device_get_softc(dev); struct usb_ether *ue = &sc->sc_ue; - usbd_transfer_unsetup(sc->sc_tx_xfer, URE_N_TRANSFER); - usbd_transfer_unsetup(sc->sc_rx_xfer, URE_N_TRANSFER); + usbd_transfer_unsetup(sc->sc_tx_xfer, URE_MAX_TX); + usbd_transfer_unsetup(sc->sc_rx_xfer, URE_MAX_RX); uether_ifdetach(ue); mtx_destroy(&sc->sc_mtx); @@ -785,7 +766,7 @@ tr_setup: caps = if_getcapenable(ifp); pos = 0; - rem = URE_TRANSFER_SIZE; + rem = URE_TX_BUFSZ; while (rem > sizeof(txpkt)) { IFQ_DRV_DEQUEUE(&ifp->if_snd, m); if (m == NULL) @@ -895,21 +876,51 @@ ure_read_chipver(struct ure_softc *sc) switch (ver) { case 0x4c00: sc->sc_chip |= URE_CHIP_VER_4C00; + sc->sc_flags = URE_FLAG_8152; break; case 0x4c10: sc->sc_chip |= URE_CHIP_VER_4C10; + sc->sc_flags = URE_FLAG_8152; break; case 0x5c00: sc->sc_chip |= URE_CHIP_VER_5C00; + sc->sc_flags = URE_FLAG_8153; break; case 0x5c10: sc->sc_chip |= URE_CHIP_VER_5C10; + sc->sc_flags = URE_FLAG_8153; break; case 0x5c20: sc->sc_chip |= URE_CHIP_VER_5C20; + sc->sc_flags = URE_FLAG_8153; break; case 0x5c30: sc->sc_chip |= URE_CHIP_VER_5C30; + sc->sc_flags = URE_FLAG_8153; + break; + case 0x6000: + sc->sc_flags = URE_FLAG_8153B; + sc->sc_chip |= URE_CHIP_VER_6000; + break; + case 0x6010: + sc->sc_flags = URE_FLAG_8153B; + sc->sc_chip |= URE_CHIP_VER_6010; + break; + case 0x7020: + sc->sc_flags = URE_FLAG_8156; + sc->sc_chip |= URE_CHIP_VER_7020; + break; + case 0x7030: + sc->sc_flags = URE_FLAG_8156; + sc->sc_chip |= URE_CHIP_VER_7030; + break; + case 0x7400: + sc->sc_flags = URE_FLAG_8156B; + sc->sc_chip |= URE_CHIP_VER_7400; + break; + case 0x7410: + sc->sc_flags = URE_FLAG_8156B; + sc->sc_chip |= URE_CHIP_VER_7410; break; default: device_printf(sc->sc_ue.ue_dev, @@ -949,6 +960,8 @@ ure_attach_post(struct usb_ether *ue) /* Initialize controller and get station address. */ if (sc->sc_flags & URE_FLAG_8152) ure_rtl8152_init(sc); + else if (sc->sc_flags & (URE_FLAG_8153B | URE_FLAG_8156 | URE_FLAG_8156B)) + ure_rtl8153b_init(sc); else ure_rtl8153_init(sc); @@ -972,7 +985,7 @@ static int ure_attach_post_sub(struct usb_ether *ue) { struct sysctl_ctx_list *sctx; - struct sysctl_oid *soid; + struct sysctl_oid *soid; struct ure_softc *sc; struct ifnet *ifp; int error; @@ -1001,9 +1014,19 @@ ure_attach_post_sub(struct usb_ether *ue) if_setcapenable(ifp, if_getcapabilities(ifp)); mtx_lock(&Giant); - error = mii_attach(ue->ue_dev, &ue->ue_miibus, ifp, - uether_ifmedia_upd, ue->ue_methods->ue_mii_sts, - BMSR_DEFCAPMASK, sc->sc_phyno, MII_OFFSET_ANY, 0); + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + ifmedia_init(&sc->sc_ifmedia, IFM_IMASK, ure_ifmedia_upd, + ure_ifmedia_sts); + ure_add_media_types(sc); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_AUTO, 0, NULL); + ifmedia_set(&sc->sc_ifmedia, IFM_ETHER | IFM_AUTO); + sc->sc_ifmedia.ifm_media = IFM_ETHER | IFM_AUTO; + error = 0; + } else { + error = mii_attach(ue->ue_dev, &ue->ue_miibus, ifp, + uether_ifmedia_upd, ue->ue_methods->ue_mii_sts, + BMSR_DEFCAPMASK, sc->sc_phyno, MII_OFFSET_ANY, 0); + } mtx_unlock(&Giant); sctx = device_get_sysctl_ctx(sc->sc_ue.ue_dev); @@ -1022,6 +1045,7 @@ ure_init(struct usb_ether *ue) struct ure_softc *sc = uether_getsc(ue); struct ifnet *ifp = uether_getifp(ue); uint16_t cpcr; + uint32_t reg; URE_LOCK_ASSERT(sc, MA_OWNED); @@ -1031,7 +1055,10 @@ ure_init(struct usb_ether *ue) /* Cancel pending I/O. */ ure_stop(ue); - ure_reset(sc); + if (sc->sc_flags & (URE_FLAG_8153B | URE_FLAG_8156 | URE_FLAG_8156B)) + ure_rtl8153b_nic_reset(sc); + else + ure_reset(sc); /* Set MAC address. */ ure_write_1(sc, URE_PLA_CRWECR, URE_MCU_TYPE_PLA, URE_CRWECR_CONFIG); @@ -1039,13 +1066,50 @@ ure_init(struct usb_ether *ue) IF_LLADDR(ifp), 8); ure_write_1(sc, URE_PLA_CRWECR, URE_MCU_TYPE_PLA, URE_CRWECR_NORAML); + /* Set RX EARLY timeout and size */ + if (sc->sc_flags & URE_FLAG_8153) { + switch (usbd_get_speed(sc->sc_ue.ue_udev)) { + case USB_SPEED_SUPER: + reg = URE_COALESCE_SUPER / 8; + break; + case USB_SPEED_HIGH: + reg = URE_COALESCE_HIGH / 8; + break; + default: + reg = URE_COALESCE_SLOW / 8; + break; + } + ure_write_2(sc, URE_USB_RX_EARLY_AGG, URE_MCU_TYPE_USB, reg); + reg = URE_8153_RX_BUFSZ - (URE_FRAMELEN(if_getmtu(ifp)) + + sizeof(struct ure_rxpkt) + URE_RXPKT_ALIGN); + ure_write_2(sc, URE_USB_RX_EARLY_SIZE, URE_MCU_TYPE_USB, reg / 4); + } else if (sc->sc_flags & URE_FLAG_8153B) { + ure_write_2(sc, URE_USB_RX_EARLY_AGG, URE_MCU_TYPE_USB, 158); + ure_write_2(sc, URE_USB_RX_EXTRA_AGG_TMR, URE_MCU_TYPE_USB, 1875); + reg = URE_8153_RX_BUFSZ - (URE_FRAMELEN(if_getmtu(ifp)) + + sizeof(struct ure_rxpkt) + URE_RXPKT_ALIGN); + ure_write_2(sc, URE_USB_RX_EARLY_SIZE, URE_MCU_TYPE_USB, reg / 8); + ure_write_1(sc, URE_USB_UPT_RXDMA_OWN, URE_MCU_TYPE_USB, + URE_OWN_UPDATE | URE_OWN_CLEAR); + } else if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + ure_write_2(sc, URE_USB_RX_EARLY_AGG, URE_MCU_TYPE_USB, 80); + ure_write_2(sc, URE_USB_RX_EXTRA_AGG_TMR, URE_MCU_TYPE_USB, 1875); + reg = URE_8156_RX_BUFSZ - (URE_FRAMELEN(if_getmtu(ifp)) + + sizeof(struct ure_rxpkt) + URE_RXPKT_ALIGN); + ure_write_2(sc, URE_USB_RX_EARLY_SIZE, URE_MCU_TYPE_USB, reg / 8); + ure_write_1(sc, URE_USB_UPT_RXDMA_OWN, URE_MCU_TYPE_USB, + URE_OWN_UPDATE | URE_OWN_CLEAR); + } + + if (sc->sc_flags & URE_FLAG_8156B) { + URE_CLRBIT_2(sc, URE_USB_FW_TASK, URE_MCU_TYPE_USB, URE_FC_PATCH_TASK); + uether_pause(&sc->sc_ue, hz / 500); + URE_SETBIT_2(sc, URE_USB_FW_TASK, URE_MCU_TYPE_USB, URE_FC_PATCH_TASK); + } + /* Reset the packet filter. */ - ure_write_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA) & - ~URE_FMC_FCR_MCU_EN); - ure_write_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA) | - URE_FMC_FCR_MCU_EN); + URE_CLRBIT_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA, URE_FMC_FCR_MCU_EN); + URE_SETBIT_2(sc, URE_PLA_FMC, URE_MCU_TYPE_PLA, URE_FMC_FCR_MCU_EN); /* Enable RX VLANs if enabled */ cpcr = ure_read_2(sc, URE_PLA_CPCR, URE_MCU_TYPE_PLA); @@ -1059,13 +1123,9 @@ ure_init(struct usb_ether *ue) ure_write_2(sc, URE_PLA_CPCR, URE_MCU_TYPE_PLA, cpcr); /* Enable transmit and receive. */ - ure_write_1(sc, URE_PLA_CR, URE_MCU_TYPE_PLA, - ure_read_1(sc, URE_PLA_CR, URE_MCU_TYPE_PLA) | URE_CR_RE | - URE_CR_TE); + URE_SETBIT_1(sc, URE_PLA_CR, URE_MCU_TYPE_PLA, URE_CR_RE | URE_CR_TE); - ure_write_2(sc, URE_PLA_MISC_1, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_MISC_1, URE_MCU_TYPE_PLA) & - ~URE_RXDY_GATED_EN); + URE_CLRBIT_2(sc, URE_PLA_MISC_1, URE_MCU_TYPE_PLA, URE_RXDY_GATED_EN); /* Configure RX filters. */ ure_rxfilter(ue); @@ -1084,26 +1144,31 @@ ure_tick(struct usb_ether *ue) { struct ure_softc *sc = uether_getsc(ue); struct ifnet *ifp = uether_getifp(ue); - struct mii_data *mii = GET_MII(sc); + struct mii_data *mii; URE_LOCK_ASSERT(sc, MA_OWNED); (void)ifp; - for (int i = 0; i < URE_N_TRANSFER; i++) + for (int i = 0; i < URE_MAX_RX; i++) DEVPRINTFN(13, sc->sc_ue.ue_dev, "rx[%d] = %d\n", i, USB_GET_STATE(sc->sc_rx_xfer[i])); - for (int i = 0; i < URE_N_TRANSFER; i++) + for (int i = 0; i < URE_MAX_TX; i++) DEVPRINTFN(13, sc->sc_ue.ue_dev, "tx[%d] = %d\n", i, USB_GET_STATE(sc->sc_tx_xfer[i])); - mii_tick(mii); - if ((sc->sc_flags & URE_FLAG_LINK) == 0 - && mii->mii_media_status & IFM_ACTIVE && - IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { - sc->sc_flags |= URE_FLAG_LINK; - sc->sc_rxstarted = 0; - ure_start(ue); + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + ure_link_state(sc); + } else { + mii = GET_MII(sc); + mii_tick(mii); + if ((sc->sc_flags & URE_FLAG_LINK) == 0 + && mii->mii_media_status & IFM_ACTIVE && + IFM_SUBTYPE(mii->mii_media_active) != IFM_NONE) { + sc->sc_flags |= URE_FLAG_LINK; + sc->sc_rxstarted = 0; + ure_start(ue); + } } } @@ -1171,11 +1236,11 @@ ure_start(struct usb_ether *ue) if (!sc->sc_rxstarted) { sc->sc_rxstarted = 1; - for (i = 0; i != URE_N_TRANSFER; i++) + for (i = 0; i != URE_MAX_RX; i++) usbd_transfer_start(sc->sc_rx_xfer[i]); } - for (i = 0; i != URE_N_TRANSFER; i++) + for (i = 0; i != URE_MAX_TX; i++) usbd_transfer_start(sc->sc_tx_xfer[i]); } @@ -1203,12 +1268,73 @@ static int ure_ifmedia_upd(struct ifnet *ifp) { struct ure_softc *sc = ifp->if_softc; - struct mii_data *mii = GET_MII(sc); + struct ifmedia *ifm; + struct mii_data *mii; struct mii_softc *miisc; + int gig; + int reg; + int anar; + int locked; int error; - URE_LOCK_ASSERT(sc, MA_OWNED); + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + ifm = &sc->sc_ifmedia; + if (IFM_TYPE(ifm->ifm_media) != IFM_ETHER) + return (EINVAL); + + locked = mtx_owned(&sc->sc_mtx); + if (!locked) + URE_LOCK(sc); + reg = ure_ocp_reg_read(sc, 0xa5d4); + reg &= ~URE_ADV_2500TFDX; + + anar = gig = 0; + switch (IFM_SUBTYPE(ifm->ifm_media)) { + case IFM_AUTO: + anar |= ANAR_TX_FD | ANAR_TX | ANAR_10_FD | ANAR_10; + gig |= GTCR_ADV_1000TFDX | GTCR_ADV_1000THDX; + reg |= URE_ADV_2500TFDX; + break; + case IFM_2500_T: + anar |= ANAR_TX_FD | ANAR_TX | ANAR_10_FD | ANAR_10; + gig |= GTCR_ADV_1000TFDX | GTCR_ADV_1000THDX; + reg |= URE_ADV_2500TFDX; + ifp->if_baudrate = IF_Mbps(2500); + break; + case IFM_1000_T: + anar |= ANAR_TX_FD | ANAR_TX | ANAR_10_FD | ANAR_10; + gig |= GTCR_ADV_1000TFDX | GTCR_ADV_1000THDX; + ifp->if_baudrate = IF_Gbps(1); + break; + case IFM_100_TX: + anar |= ANAR_TX | ANAR_TX_FD; + ifp->if_baudrate = IF_Mbps(100); + break; + case IFM_10_T: + anar |= ANAR_10 | ANAR_10_FD; + ifp->if_baudrate = IF_Mbps(10); + break; + default: + device_printf(sc->sc_ue.ue_dev, "unsupported media type\n"); + if (!locked) + URE_UNLOCK(sc); + return (EINVAL); + } + + ure_ocp_reg_write(sc, URE_OCP_BASE_MII + MII_ANAR * 2, + anar | ANAR_PAUSE_ASYM | ANAR_FC); + ure_ocp_reg_write(sc, URE_OCP_BASE_MII + MII_100T2CR * 2, gig); + ure_ocp_reg_write(sc, 0xa5d4, reg); + ure_ocp_reg_write(sc, URE_OCP_BASE_MII + MII_BMCR, + BMCR_AUTOEN | BMCR_STARTNEG); + if (!locked) + URE_UNLOCK(sc); + return (0); + } + mii = GET_MII(sc); + + URE_LOCK_ASSERT(sc, MA_OWNED); LIST_FOREACH(miisc, &mii->mii_phys, mii_list) PHY_RESET(miisc); error = mii_mediachg(mii); @@ -1223,8 +1349,34 @@ ure_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr) { struct ure_softc *sc; struct mii_data *mii; + uint16_t status; sc = ifp->if_softc; + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + URE_LOCK(sc); + ifmr->ifm_status = IFM_AVALID; + if (ure_get_link_status(sc)) { + ifmr->ifm_status |= IFM_ACTIVE; + status = ure_read_2(sc, URE_PLA_PHYSTATUS, + URE_MCU_TYPE_PLA); + if ((status & URE_PHYSTATUS_FDX) || + (status & URE_PHYSTATUS_2500MBPS)) + ifmr->ifm_active |= IFM_FDX; + else + ifmr->ifm_active |= IFM_HDX; + if (status & URE_PHYSTATUS_10MBPS) + ifmr->ifm_active |= IFM_10_T; + else if (status & URE_PHYSTATUS_100MBPS) + ifmr->ifm_active |= IFM_100_TX; + else if (status & URE_PHYSTATUS_1000MBPS) + ifmr->ifm_active |= IFM_1000_T; + else if (status & URE_PHYSTATUS_2500MBPS) + ifmr->ifm_active |= IFM_2500_T; + } + URE_UNLOCK(sc); + return; + } + mii = GET_MII(sc); URE_LOCK(sc); @@ -1234,6 +1386,54 @@ ure_ifmedia_sts(struct ifnet *ifp, struct ifmediareq *ifmr) URE_UNLOCK(sc); } +static void +ure_add_media_types(struct ure_softc *sc) +{ + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_10_T, 0, NULL); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_10_T | IFM_FDX, 0, NULL); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_100_TX, 0, NULL); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_100_TX | IFM_FDX, 0, NULL); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_1000_T | IFM_FDX, 0, NULL); + ifmedia_add(&sc->sc_ifmedia, IFM_ETHER | IFM_2500_T | IFM_FDX, 0, NULL); +} + +static void +ure_link_state(struct ure_softc *sc) +{ + struct ifnet *ifp = uether_getifp(&sc->sc_ue); + + if (ure_get_link_status(sc)) { + if (ifp->if_link_state != LINK_STATE_UP) { + if_link_state_change(ifp, LINK_STATE_UP); + /* Enable transmit and receive. */ + URE_SETBIT_1(sc, URE_PLA_CR, URE_MCU_TYPE_PLA, URE_CR_RE | URE_CR_TE); + + if (ure_read_2(sc, URE_PLA_PHYSTATUS, URE_MCU_TYPE_PLA) & + URE_PHYSTATUS_2500MBPS) + URE_CLRBIT_2(sc, URE_PLA_MAC_PWR_CTRL4, URE_MCU_TYPE_PLA, 0x40); + else + URE_SETBIT_2(sc, URE_PLA_MAC_PWR_CTRL4, URE_MCU_TYPE_PLA, 0x40); + } + } else { + if (ifp->if_link_state != LINK_STATE_DOWN) { + if_link_state_change(ifp, LINK_STATE_DOWN); + } + } +} + +static int +ure_get_link_status(struct ure_softc *sc) +{ + if (ure_read_2(sc, URE_PLA_PHYSTATUS, URE_MCU_TYPE_PLA) & + URE_PHYSTATUS_LINK) { + sc->sc_flags |= URE_FLAG_LINK; + return (1); + } else { + sc->sc_flags &= ~URE_FLAG_LINK; + return (0); + } +} + static int ure_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) { @@ -1298,8 +1498,17 @@ ure_ioctl(struct ifnet *ifp, u_long cmd, caddr_t data) URE_UNLOCK(sc); break; + case SIOCGIFMEDIA: + case SIOCSIFMEDIA: + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) + error = ifmedia_ioctl(ifp, ifr, &sc->sc_ifmedia, cmd); + else + error = uether_ioctl(ifp, cmd, data); + break; + default: error = uether_ioctl(ifp, cmd, data); + break; } return (error); @@ -1310,27 +1519,18 @@ ure_rtl8152_init(struct ure_softc *sc) { uint32_t pwrctrl; - /* Disable ALDPS. */ - ure_ocp_reg_write(sc, URE_OCP_ALDPS_CONFIG, URE_ENPDNPS | URE_LINKENA | - URE_DIS_SDSAVE); - uether_pause(&sc->sc_ue, hz / 50); + ure_enable_aldps(sc, false); if (sc->sc_chip & URE_CHIP_VER_4C00) { - ure_write_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA) & - ~URE_LED_MODE_MASK); + URE_CLRBIT_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA, URE_LED_MODE_MASK); } - ure_write_2(sc, URE_USB_UPS_CTRL, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_UPS_CTRL, URE_MCU_TYPE_USB) & - ~URE_POWER_CUT); - ure_write_2(sc, URE_USB_PM_CTRL_STATUS, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_PM_CTRL_STATUS, URE_MCU_TYPE_USB) & - ~URE_RESUME_INDICATE); + URE_CLRBIT_2(sc, URE_USB_UPS_CTRL, URE_MCU_TYPE_USB, URE_POWER_CUT); + + URE_CLRBIT_2(sc, URE_USB_PM_CTRL_STATUS, URE_MCU_TYPE_USB, URE_RESUME_INDICATE); + + URE_SETBIT_2(sc, URE_PLA_PHY_PWR, URE_MCU_TYPE_PLA, URE_TX_10M_IDLE_EN | URE_PFM_PWM_SWITCH); - ure_write_2(sc, URE_PLA_PHY_PWR, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_PHY_PWR, URE_MCU_TYPE_PLA) | - URE_TX_10M_IDLE_EN | URE_PFM_PWM_SWITCH); pwrctrl = ure_read_4(sc, URE_PLA_MAC_PWR_CTRL, URE_MCU_TYPE_PLA); pwrctrl &= ~URE_MCU_CLK_RATIO_MASK; pwrctrl |= URE_MCU_CLK_RATIO | URE_D3_CLK_GATED_EN; @@ -1340,16 +1540,11 @@ ure_rtl8152_init(struct ure_softc *sc) URE_SPDWN_LINKCHG_MSK); /* Enable Rx aggregation. */ - ure_write_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB) & - ~URE_RX_AGG_DISABLE); + URE_CLRBIT_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB, URE_RX_AGG_DISABLE | URE_RX_ZERO_EN); - /* Disable ALDPS. */ - ure_ocp_reg_write(sc, URE_OCP_ALDPS_CONFIG, URE_ENPDNPS | URE_LINKENA | - URE_DIS_SDSAVE); - uether_pause(&sc->sc_ue, hz / 50); + ure_enable_aldps(sc, false); - ure_init_fifo(sc); + ure_rtl8152_nic_reset(sc); ure_write_1(sc, URE_USB_TX_AGG, URE_MCU_TYPE_USB, URE_TX_AGG_MAX_THRESHOLD); @@ -1365,10 +1560,7 @@ ure_rtl8153_init(struct ure_softc *sc) uint8_t u1u2[8]; int i; - /* Disable ALDPS. */ - ure_ocp_reg_write(sc, URE_OCP_POWER_CFG, - ure_ocp_reg_read(sc, URE_OCP_POWER_CFG) & ~URE_EN_ALDPS); - uether_pause(&sc->sc_ue, hz / 50); + ure_enable_aldps(sc, false); memset(u1u2, 0x00, sizeof(u1u2)); ure_write_mem(sc, URE_USB_TOLERANCE, @@ -1395,9 +1587,7 @@ ure_rtl8153_init(struct ure_softc *sc) device_printf(sc->sc_ue.ue_dev, "timeout waiting for phy to stabilize\n"); - ure_write_2(sc, URE_USB_U2P3_CTRL, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_U2P3_CTRL, URE_MCU_TYPE_USB) & - ~URE_U2P3_ENABLE); + URE_CLRBIT_2(sc, URE_USB_U2P3_CTRL, URE_MCU_TYPE_USB, URE_U2P3_ENABLE); if (sc->sc_chip & URE_CHIP_VER_5C10) { val = ure_read_2(sc, URE_USB_SSPHYLINK2, URE_MCU_TYPE_USB); @@ -1405,14 +1595,10 @@ ure_rtl8153_init(struct ure_softc *sc) val |= URE_PWD_DN_SCALE(96); ure_write_2(sc, URE_USB_SSPHYLINK2, URE_MCU_TYPE_USB, val); - ure_write_1(sc, URE_USB_USB2PHY, URE_MCU_TYPE_USB, - ure_read_1(sc, URE_USB_USB2PHY, URE_MCU_TYPE_USB) | - URE_USB2PHY_L1 | URE_USB2PHY_SUSPEND); - } else if (sc->sc_chip & URE_CHIP_VER_5C20) { - ure_write_1(sc, URE_PLA_DMY_REG0, URE_MCU_TYPE_PLA, - ure_read_1(sc, URE_PLA_DMY_REG0, URE_MCU_TYPE_PLA) & - ~URE_ECM_ALDPS); - } + URE_SETBIT_1(sc, URE_USB_USB2PHY, URE_MCU_TYPE_USB, URE_USB2PHY_L1 | URE_USB2PHY_SUSPEND); + } else if (sc->sc_chip & URE_CHIP_VER_5C20) + URE_CLRBIT_1(sc, URE_PLA_DMY_REG0, URE_MCU_TYPE_PLA, URE_ECM_ALDPS); + if (sc->sc_chip & (URE_CHIP_VER_5C20 | URE_CHIP_VER_5C30)) { val = ure_read_1(sc, URE_USB_CSR_DUMMY1, URE_MCU_TYPE_USB); if (ure_read_2(sc, URE_USB_BURST_SIZE, URE_MCU_TYPE_USB) == @@ -1423,17 +1609,11 @@ ure_rtl8153_init(struct ure_softc *sc) ure_write_1(sc, URE_USB_CSR_DUMMY1, URE_MCU_TYPE_USB, val); } - ure_write_1(sc, URE_USB_CSR_DUMMY2, URE_MCU_TYPE_USB, - ure_read_1(sc, URE_USB_CSR_DUMMY2, URE_MCU_TYPE_USB) | - URE_EP4_FULL_FC); + URE_SETBIT_1(sc, URE_USB_CSR_DUMMY2, URE_MCU_TYPE_USB, URE_EP4_FULL_FC); - ure_write_2(sc, URE_USB_WDT11_CTRL, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_WDT11_CTRL, URE_MCU_TYPE_USB) & - ~URE_TIMER11_EN); + URE_CLRBIT_2(sc, URE_USB_WDT11_CTRL, URE_MCU_TYPE_USB, URE_TIMER11_EN); - ure_write_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA, - ure_read_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA) & - ~URE_LED_MODE_MASK); + URE_CLRBIT_2(sc, URE_PLA_LED_FEATURE, URE_MCU_TYPE_PLA, URE_LED_MODE_MASK); if ((sc->sc_chip & URE_CHIP_VER_5C10) && usbd_get_speed(sc->sc_ue.ue_udev) != USB_SPEED_SUPER) @@ -1450,12 +1630,9 @@ ure_rtl8153_init(struct ure_softc *sc) ure_write_2(sc, URE_USB_CONNECT_TIMER, URE_MCU_TYPE_USB, 0x0001); - ure_write_2(sc, URE_USB_POWER_CUT, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_POWER_CUT, URE_MCU_TYPE_USB) & - ~(URE_PWR_EN | URE_PHASE2_EN)); - ure_write_2(sc, URE_USB_MISC_0, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_MISC_0, URE_MCU_TYPE_USB) & - ~URE_PCUT_STATUS); + URE_CLRBIT_2(sc, URE_USB_POWER_CUT, URE_MCU_TYPE_USB, URE_PWR_EN | URE_PHASE2_EN); + + URE_CLRBIT_2(sc, URE_USB_MISC_0, URE_MCU_TYPE_USB, URE_PCUT_STATUS); memset(u1u2, 0xff, sizeof(u1u2)); ure_write_mem(sc, URE_USB_TOLERANCE, @@ -1484,17 +1661,41 @@ ure_rtl8153_init(struct ure_softc *sc) ure_write_mem(sc, URE_USB_TOLERANCE, URE_MCU_TYPE_USB | URE_BYTE_EN_SIX_BYTES, u1u2, sizeof(u1u2)); - /* Disable ALDPS. */ + ure_enable_aldps(sc, false); + + if (sc->sc_chip & (URE_CHIP_VER_5C00 | URE_CHIP_VER_5C10 | + URE_CHIP_VER_5C20)) { + ure_ocp_reg_write(sc, URE_OCP_ADC_CFG, + URE_CKADSEL_L | URE_ADC_EN | URE_EN_EMI_L); + } + if (sc->sc_chip & URE_CHIP_VER_5C00) { + ure_ocp_reg_write(sc, URE_OCP_EEE_CFG, + ure_ocp_reg_read(sc, URE_OCP_EEE_CFG) & + ~URE_CTAP_SHORT_EN); + } ure_ocp_reg_write(sc, URE_OCP_POWER_CFG, - ure_ocp_reg_read(sc, URE_OCP_POWER_CFG) & ~URE_EN_ALDPS); - uether_pause(&sc->sc_ue, hz / 50); + ure_ocp_reg_read(sc, URE_OCP_POWER_CFG) | + URE_EEE_CLKDIV_EN); + ure_ocp_reg_write(sc, URE_OCP_DOWN_SPEED, + ure_ocp_reg_read(sc, URE_OCP_DOWN_SPEED) | + URE_EN_10M_BGOFF); + ure_ocp_reg_write(sc, URE_OCP_POWER_CFG, + ure_ocp_reg_read(sc, URE_OCP_POWER_CFG) | + URE_EN_10M_PLLOFF); + ure_sram_write(sc, URE_SRAM_IMPEDANCE, 0x0b13); + URE_SETBIT_2(sc, URE_PLA_PHY_PWR, URE_MCU_TYPE_PLA, URE_PFM_PWM_SWITCH); + + /* Enable LPF corner auto tune. */ + ure_sram_write(sc, URE_SRAM_LPF_CFG, 0xf70f); - ure_init_fifo(sc); + /* Adjust 10M amplitude. */ + ure_sram_write(sc, URE_SRAM_10M_AMP1, 0x00af); + ure_sram_write(sc, URE_SRAM_10M_AMP2, 0x0208); + + ure_rtl8152_nic_reset(sc); /* Enable Rx aggregation. */ - ure_write_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB, - ure_read_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB) & - ~URE_RX_AGG_DISABLE); + URE_CLRBIT_2(sc, URE_USB_USB_CTRL, URE_MCU_TYPE_USB, URE_RX_AGG_DISABLE | URE_RX_ZERO_EN); val = ure_read_2(sc, URE_USB_U2P3_CTRL, URE_MCU_TYPE_USB); if (!(sc->sc_chip & (URE_CHIP_VER_5C00 | URE_CHIP_VER_5C10))) @@ -1508,6 +1709,266 @@ ure_rtl8153_init(struct ure_softc *sc) URE_MCU_TYPE_USB | URE_BYTE_EN_SIX_BYTES, u1u2, sizeof(u1u2)); } +static void +ure_rtl8153b_init(struct ure_softc *sc) +{ + uint16_t val; + int i; + + if (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B)) { + URE_CLRBIT_1(sc, 0xd26b, URE_MCU_TYPE_USB, 0x01); + ure_write_2(sc, 0xd32a, URE_MCU_TYPE_USB, 0); + URE_SETBIT_2(sc, 0xcfee, URE_MCU_TYPE_USB, 0x0020); + } + + if (sc->sc_flags & URE_FLAG_8156B) { + URE_SETBIT_2(sc, 0xb460, URE_MCU_TYPE_USB, 0x08); + } + + ure_enable_aldps(sc, false); + + /* Disable U1U2 */ + URE_CLRBIT_2(sc, URE_USB_LPM_CONFIG, URE_MCU_TYPE_USB, URE_LPM_U1U2_EN); + + /* Wait loading flash */ + if (sc->sc_chip == URE_CHIP_VER_7410) { + if ((ure_read_2(sc, 0xd3ae, URE_MCU_TYPE_PLA) & 0x0002) && + !(ure_read_2(sc, 0xd284, URE_MCU_TYPE_USB) & 0x0020)) { + for (i=0; i < 100; i++) { + if (ure_read_2(sc, 0xd284, URE_MCU_TYPE_USB) & 0x0004) + break; + uether_pause(&sc->sc_ue, hz / 1000); + } + } + } + + for (i = 0; i < URE_TIMEOUT; i++) { + if (ure_read_2(sc, URE_PLA_BOOT_CTRL, URE_MCU_TYPE_PLA) & + URE_AUTOLOAD_DONE) + break; + uether_pause(&sc->sc_ue, hz / 100); + } + if (i == URE_TIMEOUT) + device_printf(sc->sc_ue.ue_dev, + "timeout waiting for chip autoload\n"); + + val = ure_phy_status(sc, 0); + if ((val == URE_PHY_STAT_EXT_INIT) & + (sc->sc_flags & (URE_FLAG_8156 | URE_FLAG_8156B))) { + ure_ocp_reg_write(sc, 0xa468, + ure_ocp_reg_read(sc, 0xa468) & ~0x0a); + if (sc->sc_flags & URE_FLAG_8156B) + ure_ocp_reg_write(sc, 0xa466, + ure_ocp_reg_read(sc, 0xa466) & ~0x01); + } + + val = ure_ocp_reg_read(sc, URE_OCP_BASE_MII + MII_BMCR); + if (val & BMCR_PDOWN) { + val &= ~BMCR_PDOWN; + ure_ocp_reg_write(sc, URE_OCP_BASE_MII + MII_BMCR, val); + } + + ure_phy_status(sc, URE_PHY_STAT_LAN_ON); + + /* Disable U2P3 */ + URE_CLRBIT_2(sc, URE_USB_U2P3_CTRL, URE_MCU_TYPE_USB, URE_U2P3_ENABLE); + + /* MSC timer, 32760 ms. */ + ure_write_2(sc, URE_USB_MSC_TIMER, URE_MCU_TYPE_USB, 0x0fff); + + /* U1/U2/L1 idle timer, 500 us. */ + ure_write_2(sc, URE_USB_U1U2_TIMER, URE_MCU_TYPE_USB, 500); + + /* Disable power cut */ + URE_CLRBIT_2(sc, URE_USB_POWER_CUT, URE_MCU_TYPE_USB, URE_PWR_EN); + URE_CLRBIT_2(sc, URE_USB_MISC_0, URE_MCU_TYPE_USB, URE_PCUT_STATUS); + + /* Disable ups */ + URE_CLRBIT_1(sc, URE_USB_POWER_CUT, URE_MCU_TYPE_USB, URE_UPS_EN | URE_USP_PREWAKE); + URE_CLRBIT_1(sc, 0xcfff, URE_MCU_TYPE_USB, 0x01); + + /* Disable queue wake */ + URE_CLRBIT_1(sc, URE_PLA_INDICATE_FALG, URE_MCU_TYPE_USB, URE_UPCOMING_RUNTIME_D3); + URE_CLRBIT_1(sc, URE_PLA_SUSPEND_FLAG, URE_MCU_TYPE_USB, URE_LINK_CHG_EVENT); + URE_CLRBIT_2(sc, URE_PLA_EXTRA_STATUS, URE_MCU_TYPE_USB, URE_LINK_CHANGE_FLAG); + + /* Disable runtime suspend */ + ure_write_1(sc, URE_PLA_CRWECR, URE_MCU_TYPE_PLA, URE_CRWECR_CONFIG); + URE_CLRBIT_2(sc, URE_PLA_CONFIG34, URE_MCU_TYPE_USB, URE_LINK_OFF_WAKE_EN); + ure_write_1(sc, URE_PLA_CRWECR, URE_MCU_TYPE_PLA, URE_CRWECR_NORAML); + + /* Enable U1U2 */ + if (usbd_get_speed(sc->sc_ue.ue_udev) == USB_SPEED_SUPER) + URE_SETBIT_2(sc, URE_USB_LPM_CONFIG, URE_MCU_TYPE_USB, URE_LPM_U1U2_EN); + + if (sc->sc_flags & URE_FLAG_8156B) { + URE_CLRBIT_2(sc, 0xc010, URE_MCU_TYPE_PLA, 0x0800); + URE_SETBIT_2(sc, 0xe854, URE_MCU_TYPE_PLA, 0x0001); *** 740 LINES SKIPPED *** From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:19:02 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4D4B1659F2C; Sat, 10 Jul 2021 19:19:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMfw6068Hz4lHY; Sat, 10 Jul 2021 19:19:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DD6BB5B4D; Sat, 10 Jul 2021 19:19:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJJ1Yn049149; Sat, 10 Jul 2021 19:19:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJJ1BN049148; Sat, 10 Jul 2021 19:19:01 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:19:01 GMT Message-Id: <202107101919.16AJJ1BN049148@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 47c5e288eee9 - stable/13 - Narrow down the probe range for if_ure(4) compatible devices to only match the first vendor specific interface, if any. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 47c5e288eee94c67369443d0ed7c2ab21235c924 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:19:02 -0000 The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=47c5e288eee94c67369443d0ed7c2ab21235c924 commit 47c5e288eee94c67369443d0ed7c2ab21235c924 Author: Hans Petter Selasky AuthorDate: 2021-06-04 13:48:15 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:17:29 +0000 Narrow down the probe range for if_ure(4) compatible devices to only match the first vendor specific interface, if any. PR: 253374 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit dab84426a68d43efaede62ccf86ca3ef852f8ae3) --- sys/dev/usb/net/if_ure.c | 7 ++++--- sys/dev/usb/usb.h | 1 + 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/sys/dev/usb/net/if_ure.c b/sys/dev/usb/net/if_ure.c index 2e4053a0e0c6..6439a0bfd71d 100644 --- a/sys/dev/usb/net/if_ure.c +++ b/sys/dev/usb/net/if_ure.c @@ -95,7 +95,10 @@ SYSCTL_INT(_hw_usb_ure, OID_AUTO, debug, CTLFLAG_RWTUN, &ure_debug, 0, * Various supported device vendors/products. */ static const STRUCT_USB_HOST_ID ure_devs[] = { -#define URE_DEV(v,p,i) { USB_VPI(USB_VENDOR_##v, USB_PRODUCT_##v##_##p, i) } +#define URE_DEV(v,p,i) { \ + USB_VPI(USB_VENDOR_##v, USB_PRODUCT_##v##_##p, i), \ + USB_IFACE_CLASS(UICLASS_VENDOR), \ + USB_IFACE_SUBCLASS(UISUBCLASS_VENDOR) } URE_DEV(LENOVO, RTL8153, URE_FLAG_8153), URE_DEV(LENOVO, TBT3LAN, 0), URE_DEV(LENOVO, TBT3LANGEN2, 0), @@ -488,8 +491,6 @@ ure_probe(device_t dev) uaa = device_get_ivars(dev); if (uaa->usb_mode != USB_MODE_HOST) return (ENXIO); - if (uaa->info.bConfigIndex != URE_CONFIG_IDX) - return (ENXIO); if (uaa->info.bIfaceIndex != URE_IFACE_IDX) return (ENXIO); diff --git a/sys/dev/usb/usb.h b/sys/dev/usb/usb.h index dcdb62114d63..ad0381366f98 100644 --- a/sys/dev/usb/usb.h +++ b/sys/dev/usb/usb.h @@ -520,6 +520,7 @@ typedef struct usb_interface_assoc_descriptor usb_interface_assoc_descriptor_t; #define UICLASS_VENDOR 0xff #define UISUBCLASS_XBOX360_CONTROLLER 0x5d +#define UISUBCLASS_VENDOR 0xff #define UIPROTO_XBOX360_GAMEPAD 0x01 struct usb_endpoint_descriptor { From owner-dev-commits-src-branches@freebsd.org Sat Jul 10 19:24:26 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AE81265A58A; Sat, 10 Jul 2021 19:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GMg2L4Zt2z4lhq; Sat, 10 Jul 2021 19:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 809115F27; Sat, 10 Jul 2021 19:24:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16AJOQjx062011; Sat, 10 Jul 2021 19:24:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16AJOQe1062010; Sat, 10 Jul 2021 19:24:26 GMT (envelope-from git) Date: Sat, 10 Jul 2021 19:24:26 GMT Message-Id: <202107101924.16AJOQe1062010@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 0f82fd0714a9 - stable/12 - ibstat: Include prototype for sysctlbyname(). MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 0f82fd0714a9e6faabb0fead2c3b1e42444c1400 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jul 2021 19:24:26 -0000 The branch stable/12 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=0f82fd0714a9e6faabb0fead2c3b1e42444c1400 commit 0f82fd0714a9e6faabb0fead2c3b1e42444c1400 Author: Hans Petter Selasky AuthorDate: 2021-05-27 07:39:49 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-10 19:24:05 +0000 ibstat: Include prototype for sysctlbyname(). Fixes the following compile warning: implicit declaration of function 'sysctlbyname' is invalid in C99 [-Wimplicit-function-declaration] Found by: J87 Differential Revision: https://reviews.freebsd.org/D30484 Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit 16fa3dcba027d13dcda9ee78e6057e3e5a79f80c) --- contrib/ofed/infiniband-diags/src/ibstat.c | 1 + 1 file changed, 1 insertion(+) diff --git a/contrib/ofed/infiniband-diags/src/ibstat.c b/contrib/ofed/infiniband-diags/src/ibstat.c index 6e08fad409ca..e4f3608dfd76 100644 --- a/contrib/ofed/infiniband-diags/src/ibstat.c +++ b/contrib/ofed/infiniband-diags/src/ibstat.c @@ -48,6 +48,7 @@ #include #include #include +#include #include From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 22:30:19 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFC1666FC76; Sun, 11 Jul 2021 22:30:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNM6M4tK2z3vDY; Sun, 11 Jul 2021 22:30:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8FC4523A06; Sun, 11 Jul 2021 22:30:19 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BMUJsW025715; Sun, 11 Jul 2021 22:30:19 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BMUJen025708; Sun, 11 Jul 2021 22:30:19 GMT (envelope-from git) Date: Sun, 11 Jul 2021 22:30:19 GMT Message-Id: <202107112230.16BMUJen025708@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 8a04edfdcbd2 - stable/13 - nfscl: Change the default minor version for NFSv4 mounts MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 8a04edfdcbd2267ce8d8485374101bba580f5d83 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 22:30:19 -0000 The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=8a04edfdcbd2267ce8d8485374101bba580f5d83 commit 8a04edfdcbd2267ce8d8485374101bba580f5d83 Author: Rick Macklem AuthorDate: 2021-06-25 01:52:23 +0000 Commit: Rick Macklem CommitDate: 2021-07-11 22:26:55 +0000 nfscl: Change the default minor version for NFSv4 mounts When NFSv4.1 support was added to the client, the implementation was still experimental and, as such, the default minor version was set to 0. Since the NFSv4.1 client implementation is now believed to be solid and the NFSv4.1/4.2 protocol is significantly better than NFSv4.0, I beieve that NFSv4.1/4.2 should be used where possible. This patch changes the default minor version for NFSv4 to be the highest minor version supported by the NFSv4 server. If a specific minor version is desired, the "minorversion" mount option can be used to override this default. This is compatible with the Linux NFSv4 client behaviour. This was discussed on freebsd-current@ in mid-May 2021 under the subject "changing the default NFSv4 minor version" and the consensus seemed to be support for this change. It also appeared that changing this for FreeBSD 13.1 was not considered a POLA violation, so long as UPDATING and RELNOTES entries were made for it. (cherry picked from commit a145cf3f73c7d0f6071a6bddbe8a50a280285900) --- sys/fs/nfsclient/nfs_clstate.c | 9 ++++++++- sys/fs/nfsclient/nfs_clvfsops.c | 14 ++++++++++---- 2 files changed, 18 insertions(+), 5 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clstate.c b/sys/fs/nfsclient/nfs_clstate.c index fb90c80c69e7..8ea5d77d2053 100644 --- a/sys/fs/nfsclient/nfs_clstate.c +++ b/sys/fs/nfsclient/nfs_clstate.c @@ -959,11 +959,18 @@ nfscl_getcl(struct mount *mp, struct ucred *cred, NFSPROC_T *p, error == NFSERR_BADSESSION || error == NFSERR_CLIDINUSE) { (void) nfs_catnap(PZERO, error, "nfs_setcl"); + } else if (error == NFSERR_MINORVERMISMATCH && + tryminvers) { + if (nmp->nm_minorvers > 0) + nmp->nm_minorvers--; + else + tryminvers = false; } } while (((error == NFSERR_STALECLIENTID || error == NFSERR_BADSESSION || error == NFSERR_STALEDONTRECOVER) && --trystalecnt > 0) || - (error == NFSERR_CLIDINUSE && --clidinusedelay > 0)); + (error == NFSERR_CLIDINUSE && --clidinusedelay > 0) || + (error == NFSERR_MINORVERMISMATCH && tryminvers)); if (error) { NFSLOCKCLSTATE(); nfsv4_unlock(&clp->nfsc_lock, 0); diff --git a/sys/fs/nfsclient/nfs_clvfsops.c b/sys/fs/nfsclient/nfs_clvfsops.c index 5abd41dee1a1..974bc684bf5b 100644 --- a/sys/fs/nfsclient/nfs_clvfsops.c +++ b/sys/fs/nfsclient/nfs_clvfsops.c @@ -895,7 +895,7 @@ nfs_mount(struct mount *mp) char *cp, *opt, *name, *secname, *tlscertname; int nametimeo = NFS_DEFAULT_NAMETIMEO; int negnametimeo = NFS_DEFAULT_NEGNAMETIMEO; - int minvers = 0; + int minvers = -1; int dirlen, has_nfs_args_opt, has_nfs_from_opt, krbnamelen, srvkrbnamelen; size_t hstlen; @@ -1417,6 +1417,7 @@ mountnfs(struct nfs_args *argp, struct mount *mp, struct sockaddr *nam, struct nfsclclient *clp; struct nfsclds *dsp, *tdsp; uint32_t lease; + bool tryminvers; static u_int64_t clval = 0; #ifdef KERN_TLS u_int maxlen; @@ -1521,9 +1522,14 @@ mountnfs(struct nfs_args *argp, struct mount *mp, struct sockaddr *nam, nmp->nm_wcommitsize *= 2; nmp->nm_wcommitsize *= 256; - if ((argp->flags & NFSMNT_NFSV4) != 0) + tryminvers = false; + if ((argp->flags & NFSMNT_NFSV4) != 0) { + if (minvers < 0) { + tryminvers = true; + minvers = NFSV42_MINORVERSION; + } nmp->nm_minorvers = minvers; - else + } else nmp->nm_minorvers = 0; nfs_decode_args(mp, nmp, argp, hst, cred, td); @@ -1574,7 +1580,7 @@ mountnfs(struct nfs_args *argp, struct mount *mp, struct sockaddr *nam, /* For NFSv4, get the clientid now. */ if ((argp->flags & NFSMNT_NFSV4) != 0) { NFSCL_DEBUG(3, "at getcl\n"); - error = nfscl_getcl(mp, cred, td, false, &clp); + error = nfscl_getcl(mp, cred, td, tryminvers, &clp); NFSCL_DEBUG(3, "aft getcl=%d\n", error); if (error != 0) goto bad; From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 22:33:09 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1D6CC66FCE2; Sun, 11 Jul 2021 22:33:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNM9c52Xfz3vpj; Sun, 11 Jul 2021 22:33:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 938282391D; Sun, 11 Jul 2021 22:33:08 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BMX8Xr032148; Sun, 11 Jul 2021 22:33:08 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BMX8Jq032147; Sun, 11 Jul 2021 22:33:08 GMT (envelope-from git) Date: Sun, 11 Jul 2021 22:33:08 GMT Message-Id: <202107112233.16BMX8Jq032147@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 50f2705d2674 - stable/13 - mount_nfs.8: Update the man page for commit a145cf3f73c7 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 50f2705d267491b27376d37ee25986f33f92271e Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 22:33:09 -0000 The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=50f2705d267491b27376d37ee25986f33f92271e commit 50f2705d267491b27376d37ee25986f33f92271e Author: Rick Macklem AuthorDate: 2021-06-26 21:09:28 +0000 Commit: Rick Macklem CommitDate: 2021-07-11 22:30:01 +0000 mount_nfs.8: Update the man page for commit a145cf3f73c7 The NFSv4 client now uses the highest minor version of NFSv4 by default instead of minor version 0, for NFSv4 mounts. The "minorversion" mount option may be used to override this default. This patch updates the man page to reflect this change. While here, fix nfsstat(8) to be nfsstat(1). (cherry picked from commit b413b03597db10dbee514141b10f7c7ef236abe6) --- sbin/mount_nfs/mount_nfs.8 | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/sbin/mount_nfs/mount_nfs.8 b/sbin/mount_nfs/mount_nfs.8 index 76e009817d8b..36a3c180b1e3 100644 --- a/sbin/mount_nfs/mount_nfs.8 +++ b/sbin/mount_nfs/mount_nfs.8 @@ -28,7 +28,7 @@ .\" @(#)mount_nfs.8 8.3 (Berkeley) 3/29/95 .\" $FreeBSD$ .\" -.Dd January 17, 2021 +.Dd June 26, 2021 .Dt MOUNT_NFS 8 .Os .Sh NAME @@ -206,9 +206,15 @@ Use the NFS Version 3 protocol. Use the NFS Version 4 protocol. This option will force the mount to use TCP transport. +By default, the highest minor version of NFS Version 4 that is +supported by the NFS Version 4 server will be used. +See the +.Cm minorversion +option. .It Cm minorversion Ns = Ns Aq Ar value -Override the default of 0 for the minor version of the NFS Version 4 protocol. -The minor versions other than 0 currently supported are 1 and 2. +Use the specified minor version for a NFS Version 4 mount, +overriding the default. +The minor versions supported are 0, 1, and 2. This option is only meaningful when used with the .Cm nfsv4 option. @@ -221,7 +227,7 @@ It may be required when an accumulation of NFS version 4 Opens occurs, as indicated by the .Dq Opens count displayed by -.Xr nfsstat 8 +.Xr nfsstat 1 with the .Fl c and @@ -231,22 +237,16 @@ A common case for an accumulation of Opens is a shared library within the NFS mount that is used by several processes, where at least one of these processes is always running. This option cannot be used for an NFS Version 4, minor version 0 mount. -As such, this option requires the -.Cm minorversion -option be specified with a value of 1 for AmazonEFS, because AmazonEFS does -not support minor version 2 at this time. It may not work correctly when Delegations are being issued by a server, but note that the AmazonEFS server does not issued delegations at this time. This option is only meaningful when used with the .Cm nfsv4 -and -.Cm minorversion -options. +option. .It Cm pnfs Enable support for parallel NFS (pNFS) for minor version 1 or 2 of the NFS Version 4 protocol. This option is only meaningful when used with the -.Cm minorversion +.Cm nfsv4 option. .It Cm noac Disable attribute caching. From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 22:42:03 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A7BB864948E; Sun, 11 Jul 2021 22:42:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNMMv4QyQz3wRb; Sun, 11 Jul 2021 22:42:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8069B23CD7; Sun, 11 Jul 2021 22:42:03 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BMg3m0042718; Sun, 11 Jul 2021 22:42:03 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BMg3Po042717; Sun, 11 Jul 2021 22:42:03 GMT (envelope-from git) Date: Sun, 11 Jul 2021 22:42:03 GMT Message-Id: <202107112242.16BMg3Po042717@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: e5c2f39823d1 - stable/13 - UPDATING: Add an entry for commit 8a04edfdcbd2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: e5c2f39823d103007e3a4d5492b36835c70dc4ef Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 22:42:03 -0000 The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=e5c2f39823d103007e3a4d5492b36835c70dc4ef commit e5c2f39823d103007e3a4d5492b36835c70dc4ef Author: Rick Macklem AuthorDate: 2021-07-11 22:39:23 +0000 Commit: Rick Macklem CommitDate: 2021-07-11 22:39:23 +0000 UPDATING: Add an entry for commit 8a04edfdcbd2 This is a direct commit. --- UPDATING | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/UPDATING b/UPDATING index 8dd158f0a05c..073e9b03867a 100644 --- a/UPDATING +++ b/UPDATING @@ -12,6 +12,15 @@ Items affecting the ports and packages system can be found in /usr/ports/UPDATING. Please read that file before updating system packages and/or ports. +20210711: + Commit 8a04edfdcbd2 changes the default minor version + used for NFSv4 mounts to the highest minor version supported + by the NFSv4 server. This default can be overridden by using + the "minorversion" mount option. + The consensus of a discussion on freebsd-current@ indicated + that this would be acceptable to do and would not be considered + a POLA violation. + 20210630: Commit ca179c4d74f2/632e3f2f3a66 changed the package in which the OpenSSL libraries and utilities are packaged. From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 22:47:42 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B3F516495C9; Sun, 11 Jul 2021 22:47:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNMVQ4m3Fz4RFr; Sun, 11 Jul 2021 22:47:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 8739F23D1E; Sun, 11 Jul 2021 22:47:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BMlgig045740; Sun, 11 Jul 2021 22:47:42 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BMlgLE045739; Sun, 11 Jul 2021 22:47:42 GMT (envelope-from git) Date: Sun, 11 Jul 2021 22:47:42 GMT Message-Id: <202107112247.16BMlgLE045739@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: ac0343b88ddc - stable/13 - RELNOTES: Add an entry for commit 8a04edfdcbd2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: ac0343b88ddc439437874ce3e4c0f6deac81d9ae Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 22:47:42 -0000 The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=ac0343b88ddc439437874ce3e4c0f6deac81d9ae commit ac0343b88ddc439437874ce3e4c0f6deac81d9ae Author: Rick Macklem AuthorDate: 2021-07-11 22:45:00 +0000 Commit: Rick Macklem CommitDate: 2021-07-11 22:45:00 +0000 RELNOTES: Add an entry for commit 8a04edfdcbd2 This is a direct commit. --- RELNOTES | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/RELNOTES b/RELNOTES index 320d947c24e7..1e8c23f98703 100644 --- a/RELNOTES +++ b/RELNOTES @@ -10,6 +10,12 @@ newline. Entries should be separated by a newline. Changes to this file should not be MFCed. +8a04edfdcbd2: + Change the default minor version used for an NFSv4 mount + to the highest minor version supported by the NFSv4 server. + This default can be overridden by using the "minorversion" + mount option. + 2c76eebca71b, 59f6f5e23c1a: Add two daemons rpc.tlsclntd(8) and rpc.tlsservd(8) that provide support for NFS-over-TLS as described in the Internet Draft titled From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 23:29:18 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BE10464A313; Sun, 11 Jul 2021 23:29:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNNQQ537xz4XWw; Sun, 11 Jul 2021 23:29:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 956B723F6F; Sun, 11 Jul 2021 23:29:18 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BNTIBY099114; Sun, 11 Jul 2021 23:29:18 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BNTIf2099113; Sun, 11 Jul 2021 23:29:18 GMT (envelope-from git) Date: Sun, 11 Jul 2021 23:29:18 GMT Message-Id: <202107112329.16BNTIf2099113@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ed Maste Subject: git: 2e96f2635591 - stable/13 - clang: stop linking _p libs for -pg as of FreeBSD 14 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 2e96f2635591abb839c92566046e3a9b6ee5ff35 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 23:29:18 -0000 The branch stable/13 has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=2e96f2635591abb839c92566046e3a9b6ee5ff35 commit 2e96f2635591abb839c92566046e3a9b6ee5ff35 Author: Ed Maste AuthorDate: 2021-06-26 23:58:16 +0000 Commit: Ed Maste CommitDate: 2021-07-11 23:27:47 +0000 clang: stop linking _p libs for -pg as of FreeBSD 14 In FreeBSD 14 we will stop providing _p libraries (compiled with -pg). [Note this is controlled by the target version. There is no change for FreeBSD <= 13.] Reviewed by: dim (upstream) Obtained from: LLVM 699d47472c3f MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D30861 (cherry picked from commit b762974cf4b9ea77f1decf4a6d829372f0a97f75) --- .../clang/lib/Driver/ToolChains/FreeBSD.cpp | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp b/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp index baefb1607619..c14f34b6107c 100644 --- a/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp +++ b/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp @@ -286,6 +286,8 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, addLinkerCompressDebugSectionsOption(ToolChain, Args, CmdArgs); AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs, JA); + bool Profiling = Args.hasArg(options::OPT_pg) && + ToolChain.getTriple().getOSMajorVersion() < 14; if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) { // Use the static OpenMP runtime with -static-openmp bool StaticOpenMP = Args.hasArg(options::OPT_static_openmp) && @@ -295,7 +297,7 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, if (D.CCCIsCXX()) { if (ToolChain.ShouldLinkCXXStdlib(Args)) ToolChain.AddCXXStdlibLibArgs(Args, CmdArgs); - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lm_p"); else CmdArgs.push_back("-lm"); @@ -306,13 +308,13 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, linkXRayRuntimeDeps(ToolChain, CmdArgs); // FIXME: For some reason GCC passes -lgcc and -lgcc_s before adding // the default system libraries. Just mimic this for now. - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lgcc_p"); else CmdArgs.push_back("-lgcc"); if (Args.hasArg(options::OPT_static)) { CmdArgs.push_back("-lgcc_eh"); - } else if (Args.hasArg(options::OPT_pg)) { + } else if (Profiling) { CmdArgs.push_back("-lgcc_eh_p"); } else { CmdArgs.push_back("--as-needed"); @@ -321,13 +323,13 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, } if (Args.hasArg(options::OPT_pthread)) { - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lpthread_p"); else CmdArgs.push_back("-lpthread"); } - if (Args.hasArg(options::OPT_pg)) { + if (Profiling) { if (Args.hasArg(options::OPT_shared)) CmdArgs.push_back("-lc"); else @@ -340,7 +342,7 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, if (Args.hasArg(options::OPT_static)) { CmdArgs.push_back("-lgcc_eh"); - } else if (Args.hasArg(options::OPT_pg)) { + } else if (Profiling) { CmdArgs.push_back("-lgcc_eh_p"); } else { CmdArgs.push_back("--as-needed"); @@ -408,7 +410,8 @@ void FreeBSD::addLibStdCxxIncludePaths( void FreeBSD::AddCXXStdlibLibArgs(const ArgList &Args, ArgStringList &CmdArgs) const { CXXStdlibType Type = GetCXXStdlibType(Args); - bool Profiling = Args.hasArg(options::OPT_pg); + bool Profiling = + Args.hasArg(options::OPT_pg) && getTriple().getOSMajorVersion() < 14; switch (Type) { case ToolChain::CST_Libcxx: From owner-dev-commits-src-branches@freebsd.org Sun Jul 11 23:30:34 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A8DBD64A14D; Sun, 11 Jul 2021 23:30:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GNNRt4SrQz4XZX; Sun, 11 Jul 2021 23:30:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 81FDC23F71; Sun, 11 Jul 2021 23:30:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16BNUY9t007619; Sun, 11 Jul 2021 23:30:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16BNUYp5007618; Sun, 11 Jul 2021 23:30:34 GMT (envelope-from git) Date: Sun, 11 Jul 2021 23:30:34 GMT Message-Id: <202107112330.16BNUYp5007618@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Ed Maste Subject: git: d004f3428f95 - stable/12 - clang: stop linking _p libs for -pg as of FreeBSD 14 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: emaste X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: d004f3428f9532837275b0b8e2458dc56f2aab7c Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jul 2021 23:30:34 -0000 The branch stable/12 has been updated by emaste: URL: https://cgit.FreeBSD.org/src/commit/?id=d004f3428f9532837275b0b8e2458dc56f2aab7c commit d004f3428f9532837275b0b8e2458dc56f2aab7c Author: Ed Maste AuthorDate: 2021-06-26 23:58:16 +0000 Commit: Ed Maste CommitDate: 2021-07-11 23:29:51 +0000 clang: stop linking _p libs for -pg as of FreeBSD 14 In FreeBSD 14 we will stop providing _p libraries (compiled with -pg). [Note this is controlled by the target version. There is no change for FreeBSD <= 13.] Reviewed by: dim (upstream) Obtained from: LLVM 699d47472c3f MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D30861 (cherry picked from commit b762974cf4b9ea77f1decf4a6d829372f0a97f75) --- .../clang/lib/Driver/ToolChains/FreeBSD.cpp | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp b/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp index 6fb4ddd7f501..37ea8bed0745 100644 --- a/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp +++ b/contrib/llvm-project/clang/lib/Driver/ToolChains/FreeBSD.cpp @@ -284,6 +284,8 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, bool NeedsXRayDeps = addXRayRuntime(ToolChain, Args, CmdArgs); AddLinkerInputs(ToolChain, Inputs, Args, CmdArgs, JA); + bool Profiling = Args.hasArg(options::OPT_pg) && + ToolChain.getTriple().getOSMajorVersion() < 14; if (!Args.hasArg(options::OPT_nostdlib, options::OPT_nodefaultlibs)) { // Use the static OpenMP runtime with -static-openmp bool StaticOpenMP = Args.hasArg(options::OPT_static_openmp) && @@ -293,7 +295,7 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, if (D.CCCIsCXX()) { if (ToolChain.ShouldLinkCXXStdlib(Args)) ToolChain.AddCXXStdlibLibArgs(Args, CmdArgs); - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lm_p"); else CmdArgs.push_back("-lm"); @@ -304,13 +306,13 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, linkXRayRuntimeDeps(ToolChain, CmdArgs); // FIXME: For some reason GCC passes -lgcc and -lgcc_s before adding // the default system libraries. Just mimic this for now. - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lgcc_p"); else CmdArgs.push_back("-lgcc"); if (Args.hasArg(options::OPT_static)) { CmdArgs.push_back("-lgcc_eh"); - } else if (Args.hasArg(options::OPT_pg)) { + } else if (Profiling) { CmdArgs.push_back("-lgcc_eh_p"); } else { CmdArgs.push_back("--as-needed"); @@ -319,13 +321,13 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, } if (Args.hasArg(options::OPT_pthread)) { - if (Args.hasArg(options::OPT_pg)) + if (Profiling) CmdArgs.push_back("-lpthread_p"); else CmdArgs.push_back("-lpthread"); } - if (Args.hasArg(options::OPT_pg)) { + if (Profiling) { if (Args.hasArg(options::OPT_shared)) CmdArgs.push_back("-lc"); else @@ -338,7 +340,7 @@ void freebsd::Linker::ConstructJob(Compilation &C, const JobAction &JA, if (Args.hasArg(options::OPT_static)) { CmdArgs.push_back("-lgcc_eh"); - } else if (Args.hasArg(options::OPT_pg)) { + } else if (Profiling) { CmdArgs.push_back("-lgcc_eh_p"); } else { CmdArgs.push_back("--as-needed"); @@ -405,7 +407,8 @@ void FreeBSD::addLibStdCxxIncludePaths( void FreeBSD::AddCXXStdlibLibArgs(const ArgList &Args, ArgStringList &CmdArgs) const { CXXStdlibType Type = GetCXXStdlibType(Args); - bool Profiling = Args.hasArg(options::OPT_pg); + bool Profiling = + Args.hasArg(options::OPT_pg) && getTriple().getOSMajorVersion() < 14; switch (Type) { case ToolChain::CST_Libcxx: