From owner-dev-commits-src-branches@freebsd.org Thu Sep 16 11:21:11 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A46CE678F68; Thu, 16 Sep 2021 11:21:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H9F5M49tbz3J5D; Thu, 16 Sep 2021 11:21:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6CDF820A01; Thu, 16 Sep 2021 11:21:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 18GBLBGE024805; Thu, 16 Sep 2021 11:21:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 18GBLBNw024804; Thu, 16 Sep 2021 11:21:11 GMT (envelope-from git) Date: Thu, 16 Sep 2021 11:21:11 GMT Message-Id: <202109161121.18GBLBNw024804@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mateusz Guzik Subject: git: bef0c20ec74e - stable/12 - Convert cryptostats to a counter_u64 array. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mjg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: bef0c20ec74e0aaa4f08bc879efa1a5ee8fed0f1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Sep 2021 11:21:11 -0000 The branch stable/12 has been updated by mjg: URL: https://cgit.FreeBSD.org/src/commit/?id=bef0c20ec74e0aaa4f08bc879efa1a5ee8fed0f1 commit bef0c20ec74e0aaa4f08bc879efa1a5ee8fed0f1 Author: Mark Johnston AuthorDate: 2020-06-30 22:01:21 +0000 Commit: Mateusz Guzik CommitDate: 2021-09-16 11:20:54 +0000 Convert cryptostats to a counter_u64 array. The global counters were not SMP-friendly. Use per-CPU counters instead. Reviewed by: jhb Sponsored by: Rubicon Communications, LLC (Netgate) Differential Revision: https://reviews.freebsd.org/D25466 (cherry picked from commit 7290cb47fced28b5d0654cd1de7cc721e214ea32) --- sys/opencrypto/crypto.c | 46 ++++++++++++++++++++++++++++++++++------------ sys/opencrypto/cryptodev.h | 16 ++++++++-------- 2 files changed, 42 insertions(+), 20 deletions(-) diff --git a/sys/opencrypto/crypto.c b/sys/opencrypto/crypto.c index 01283807a670..1e11a7ead376 100644 --- a/sys/opencrypto/crypto.c +++ b/sys/opencrypto/crypto.c @@ -58,7 +58,7 @@ __FBSDID("$FreeBSD$"); #include #include -#include +#include #include #include #include @@ -217,9 +217,31 @@ static void crypto_remove(struct cryptocap *cap); static void crypto_task_invoke(void *ctx, int pending); static void crypto_batch_enqueue(struct cryptop *crp); -static struct cryptostats cryptostats; -SYSCTL_STRUCT(_kern, OID_AUTO, crypto_stats, CTLFLAG_RW, &cryptostats, - cryptostats, "Crypto system statistics"); +static counter_u64_t cryptostats[sizeof(struct cryptostats) / sizeof(uint64_t)]; +SYSCTL_COUNTER_U64_ARRAY(_kern_crypto, OID_AUTO, stats, CTLFLAG_RW, + cryptostats, nitems(cryptostats), + "Crypto system statistics"); + +#define CRYPTOSTAT_INC(stat) do { \ + counter_u64_add( \ + cryptostats[offsetof(struct cryptostats, stat) / sizeof(uint64_t)],\ + 1); \ +} while (0) + +static void +cryptostats_init(void *arg __unused) +{ + COUNTER_ARRAY_ALLOC(cryptostats, nitems(cryptostats), M_WAITOK); +} +SYSINIT(cryptostats_init, SI_SUB_COUNTER, SI_ORDER_ANY, cryptostats_init, NULL); + +static void +cryptostats_fini(void *arg __unused) +{ + COUNTER_ARRAY_FREE(cryptostats, nitems(cryptostats)); +} +SYSUNINIT(cryptostats_fini, SI_SUB_COUNTER, SI_ORDER_ANY, cryptostats_fini, + NULL); /* Try to avoid directly exposing the key buffer as a symbol */ static struct keybuf *keybuf; @@ -1004,7 +1026,7 @@ crypto_dispatch(struct cryptop *crp) u_int32_t hid; int result; - cryptostats.cs_ops++; + CRYPTOSTAT_INC(cs_ops); crp->crp_retw_id = ((uintptr_t)crp->crp_session) % crypto_workers_num; @@ -1069,7 +1091,7 @@ crypto_kdispatch(struct cryptkop *krp) { int error; - cryptostats.cs_kops++; + CRYPTOSTAT_INC(cs_kops); error = crypto_kinvoke(krp, krp->krp_crid); if (error == ERESTART) { @@ -1350,7 +1372,7 @@ crypto_done(struct cryptop *crp) ("crypto_done: op already done, flags 0x%x", crp->crp_flags)); crp->crp_flags |= CRYPTO_F_DONE; if (crp->crp_etype != 0) - cryptostats.cs_errs++; + CRYPTOSTAT_INC(cs_errs); /* * CBIMM means unconditionally do the callback immediately; @@ -1422,7 +1444,7 @@ crypto_kdone(struct cryptkop *krp) struct cryptocap *cap; if (krp->krp_status != 0) - cryptostats.cs_kerrs++; + CRYPTOSTAT_INC(cs_kerrs); CRYPTO_DRIVER_LOCK(); /* XXX: What if driver is loaded in the meantime? */ if (krp->krp_hid < crypto_drivers_num) { @@ -1564,7 +1586,7 @@ crypto_proc(void) /* XXX validate sid again? */ crypto_drivers[crypto_ses2hid(submit->crp_session)].cc_qblocked = 1; TAILQ_INSERT_HEAD(&crp_q, submit, crp_next); - cryptostats.cs_blocks++; + CRYPTOSTAT_INC(cs_blocks); } } @@ -1604,7 +1626,7 @@ crypto_proc(void) /* XXX validate sid again? */ crypto_drivers[krp->krp_hid].cc_kqblocked = 1; TAILQ_INSERT_HEAD(&crp_kq, krp, krp_next); - cryptostats.cs_kblocks++; + CRYPTOSTAT_INC(cs_kblocks); } } @@ -1626,7 +1648,7 @@ crypto_proc(void) crp_sleep = 0; if (cryptoproc == NULL) break; - cryptostats.cs_intrs++; + CRYPTOSTAT_INC(cs_intrs); } } CRYPTO_Q_UNLOCK(); @@ -1687,7 +1709,7 @@ crypto_ret_proc(struct crypto_ret_worker *ret_worker) "crypto_ret_wait", 0); if (ret_worker->cryptoretproc == NULL) break; - cryptostats.cs_rets++; + CRYPTOSTAT_INC(cs_rets); } } CRYPTO_RETW_UNLOCK(ret_worker); diff --git a/sys/opencrypto/cryptodev.h b/sys/opencrypto/cryptodev.h index 64020be843bc..2e61ee3ed90f 100644 --- a/sys/opencrypto/cryptodev.h +++ b/sys/opencrypto/cryptodev.h @@ -349,14 +349,14 @@ struct crypt_kop { #define CIOCCRYPTAEAD _IOWR('c', 109, struct crypt_aead) struct cryptostats { - u_int32_t cs_ops; /* symmetric crypto ops submitted */ - u_int32_t cs_errs; /* symmetric crypto ops that failed */ - u_int32_t cs_kops; /* asymetric/key ops submitted */ - u_int32_t cs_kerrs; /* asymetric/key ops that failed */ - u_int32_t cs_intrs; /* crypto swi thread activations */ - u_int32_t cs_rets; /* crypto return thread activations */ - u_int32_t cs_blocks; /* symmetric op driver block */ - u_int32_t cs_kblocks; /* symmetric op driver block */ + uint64_t cs_ops; /* symmetric crypto ops submitted */ + uint64_t cs_errs; /* symmetric crypto ops that failed */ + uint64_t cs_kops; /* asymetric/key ops submitted */ + uint64_t cs_kerrs; /* asymetric/key ops that failed */ + uint64_t cs_intrs; /* crypto swi thread activations */ + uint64_t cs_rets; /* crypto return thread activations */ + uint64_t cs_blocks; /* symmetric op driver block */ + uint64_t cs_kblocks; /* symmetric op driver block */ }; #ifdef _KERNEL