From owner-dev-commits-src-branches@freebsd.org Mon Jul 26 16:14:00 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E8A366F8B4; Mon, 26 Jul 2021 16:14:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GYQ3C46V3z3pgY; Mon, 26 Jul 2021 16:13:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 240911A44F; Mon, 26 Jul 2021 16:13:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16QGDwvM006141; Mon, 26 Jul 2021 16:13:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16QGDwsK006140; Mon, 26 Jul 2021 16:13:58 GMT (envelope-from git) Date: Mon, 26 Jul 2021 16:13:58 GMT Message-Id: <202107261613.16QGDwsK006140@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Hans Petter Selasky Subject: git: 857966b357ee - stable/13 - mlx5en: Allocate per-channel doorbells. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: hselasky X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 857966b357eed16c5e79fd1a40cb389980373100 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jul 2021 16:14:01 -0000 The branch stable/13 has been updated by hselasky: URL: https://cgit.FreeBSD.org/src/commit/?id=857966b357eed16c5e79fd1a40cb389980373100 commit 857966b357eed16c5e79fd1a40cb389980373100 Author: Hans Petter Selasky AuthorDate: 2021-06-16 13:01:59 +0000 Commit: Hans Petter Selasky CommitDate: 2021-07-26 16:04:33 +0000 mlx5en: Allocate per-channel doorbells. To avoid congestion on the same PCI memory register space when traffic consists mostly of small packets. Reviewed by: kib Sponsored by: Mellanox Technologies // NVIDIA Networking (cherry picked from commit 9dfa21486e1db730305abd63df449bcc1e76127b) --- sys/dev/mlx5/mlx5_en/en.h | 6 ++-- sys/dev/mlx5/mlx5_en/mlx5_en_main.c | 61 ++++++++++++++++++++++++------------- sys/dev/mlx5/mlx5_en/mlx5_en_rl.c | 5 +-- 3 files changed, 44 insertions(+), 28 deletions(-) diff --git a/sys/dev/mlx5/mlx5_en/en.h b/sys/dev/mlx5/mlx5_en/en.h index b249a82d30ef..c84e2af237b2 100644 --- a/sys/dev/mlx5/mlx5_en/en.h +++ b/sys/dev/mlx5/mlx5_en/en.h @@ -870,6 +870,7 @@ mlx5e_sq_queue_level(struct mlx5e_sq *sq) struct mlx5e_channel { struct mlx5e_rq rq; struct m_snd_tag tag; + struct mlx5_sq_bfreg bfreg; struct mlx5e_sq sq[MLX5E_MAX_TX_NUM_TC]; struct mlx5e_priv *priv; struct completion completion; @@ -1073,8 +1074,6 @@ struct mlx5e_priv { struct mlx5e_dcbx dcbx; bool sw_is_port_buf_owner; - struct mlx5_sq_bfreg bfreg; - struct pfil_head *pfil; struct mlx5e_channel channel[]; }; @@ -1201,7 +1200,8 @@ int mlx5e_open_cq(struct mlx5e_priv *, struct mlx5e_cq_param *, void mlx5e_close_cq(struct mlx5e_cq *); void mlx5e_free_sq_db(struct mlx5e_sq *); int mlx5e_alloc_sq_db(struct mlx5e_sq *); -int mlx5e_enable_sq(struct mlx5e_sq *, struct mlx5e_sq_param *, int tis_num); +int mlx5e_enable_sq(struct mlx5e_sq *, struct mlx5e_sq_param *, + const struct mlx5_sq_bfreg *, int tis_num); int mlx5e_modify_sq(struct mlx5e_sq *, int curr_state, int next_state); void mlx5e_disable_sq(struct mlx5e_sq *); void mlx5e_drain_sq(struct mlx5e_sq *); diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c index d9860c24114b..b67f382522e7 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_main.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_main.c @@ -1619,8 +1619,6 @@ mlx5e_create_sq(struct mlx5e_channel *c, &sq->dma_tag))) goto done; - sq->uar_map = priv->bfreg.map; - err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, &sq->wq, &sq->wq_ctrl); if (err) @@ -1668,7 +1666,7 @@ mlx5e_destroy_sq(struct mlx5e_sq *sq) int mlx5e_enable_sq(struct mlx5e_sq *sq, struct mlx5e_sq_param *param, - int tis_num) + const struct mlx5_sq_bfreg *bfreg, int tis_num) { void *in; void *sqc; @@ -1683,6 +1681,8 @@ mlx5e_enable_sq(struct mlx5e_sq *sq, struct mlx5e_sq_param *param, if (in == NULL) return (-ENOMEM); + sq->uar_map = bfreg->map; + ts_format = mlx5_get_sq_default_ts(sq->priv->mdev); sqc = MLX5_ADDR_OF(create_sq_in, in, ctx); wq = MLX5_ADDR_OF(sqc, sqc, wq); @@ -1698,7 +1698,7 @@ mlx5e_enable_sq(struct mlx5e_sq *sq, struct mlx5e_sq_param *param, MLX5_SET(sqc, sqc, allow_swp, 1); MLX5_SET(wq, wq, wq_type, MLX5_WQ_TYPE_CYCLIC); - MLX5_SET(wq, wq, uar_page, sq->priv->bfreg.index); + MLX5_SET(wq, wq, uar_page, bfreg->index); MLX5_SET(wq, wq, log_wq_pg_sz, sq->wq_ctrl.buf.page_shift - PAGE_SHIFT); MLX5_SET64(wq, wq, dbr_addr, sq->wq_ctrl.db.dma); @@ -1764,7 +1764,7 @@ mlx5e_open_sq(struct mlx5e_channel *c, if (err) return (err); - err = mlx5e_enable_sq(sq, param, c->priv->tisn[tc]); + err = mlx5e_enable_sq(sq, param, &c->bfreg, c->priv->tisn[tc]); if (err) goto err_destroy_sq; @@ -3763,10 +3763,12 @@ static const char *mlx5e_pport_stats_desc[] = { MLX5E_PPORT_STATS(MLX5E_STATS_DESC) }; -static void -mlx5e_priv_static_init(struct mlx5e_priv *priv, const uint32_t channels) +static int +mlx5e_priv_static_init(struct mlx5e_priv *priv, struct mlx5_core_dev *mdev, + const uint32_t channels) { uint32_t x; + int err; mtx_init(&priv->async_events_mtx, "mlx5async", MTX_NETWORK_LOCK, MTX_DEF); sx_init(&priv->state_lock, "mlx5state"); @@ -3774,13 +3776,34 @@ mlx5e_priv_static_init(struct mlx5e_priv *priv, const uint32_t channels) MLX5_INIT_DOORBELL_LOCK(&priv->doorbell_lock); for (x = 0; x != channels; x++) mlx5e_chan_static_init(priv, &priv->channel[x], x); + + for (x = 0; x != channels; x++) { + err = mlx5_alloc_bfreg(mdev, &priv->channel[x].bfreg, false, false); + if (err) + goto err_alloc_bfreg; + } + return (0); + +err_alloc_bfreg: + while (x--) + mlx5_free_bfreg(mdev, &priv->channel[x].bfreg); + + for (x = 0; x != channels; x++) + mlx5e_chan_static_destroy(&priv->channel[x]); + callout_drain(&priv->watchdog); + mtx_destroy(&priv->async_events_mtx); + sx_destroy(&priv->state_lock); + return (err); } static void -mlx5e_priv_static_destroy(struct mlx5e_priv *priv, const uint32_t channels) +mlx5e_priv_static_destroy(struct mlx5e_priv *priv, struct mlx5_core_dev *mdev, + const uint32_t channels) { uint32_t x; + for (x = 0; x != channels; x++) + mlx5_free_bfreg(mdev, &priv->channel[x].bfreg); for (x = 0; x != channels; x++) mlx5e_chan_static_destroy(&priv->channel[x]); callout_drain(&priv->watchdog); @@ -4397,7 +4420,10 @@ mlx5e_create_ifp(struct mlx5_core_dev *mdev) goto err_free_priv; } /* setup all static fields */ - mlx5e_priv_static_init(priv, mdev->priv.eq_table.num_comp_vectors); + if (mlx5e_priv_static_init(priv, mdev, mdev->priv.eq_table.num_comp_vectors)) { + mlx5_core_err(mdev, "mlx5e_priv_static_init() failed\n"); + goto err_free_ifp; + } ifp->if_softc = priv; if_initname(ifp, "mce", device_get_unit(mdev->pdev->dev.bsddev)); @@ -4508,11 +4534,6 @@ mlx5e_create_ifp(struct mlx5_core_dev *mdev) mlx5_en_err(ifp, "mlx5e_create_mkey failed, %d\n", err); goto err_dealloc_transport_domain; } - err = mlx5_alloc_bfreg(mdev, &priv->bfreg, false, false); - if (err) { - mlx5_en_err(ifp, "alloc bfreg failed, %d\n", err); - goto err_create_mkey; - } mlx5_query_nic_vport_mac_address(priv->mdev, 0, dev_addr); /* check if we should generate a random MAC address */ @@ -4525,7 +4546,7 @@ mlx5e_create_ifp(struct mlx5_core_dev *mdev) err = mlx5e_rl_init(priv); if (err) { mlx5_en_err(ifp, "mlx5e_rl_init failed, %d\n", err); - goto err_alloc_bfreg; + goto err_create_mkey; } err = mlx5e_tls_init(priv); @@ -4664,9 +4685,6 @@ mlx5e_create_ifp(struct mlx5_core_dev *mdev) err_rl_init: mlx5e_rl_cleanup(priv); -err_alloc_bfreg: - mlx5_free_bfreg(mdev, &priv->bfreg); - err_create_mkey: mlx5_core_destroy_mkey(priv->mdev, &priv->mr); @@ -4683,7 +4701,9 @@ err_free_sysctl: sysctl_ctx_free(&priv->sysctl_ctx); if (priv->sysctl_debug) sysctl_ctx_free(&priv->stats.port_stats_debug.ctx); - mlx5e_priv_static_destroy(priv, mdev->priv.eq_table.num_comp_vectors); + mlx5e_priv_static_destroy(priv, mdev, mdev->priv.eq_table.num_comp_vectors); + +err_free_ifp: if_free(ifp); err_free_priv: @@ -4767,13 +4787,12 @@ mlx5e_destroy_ifp(struct mlx5_core_dev *mdev, void *vpriv) sysctl_ctx_free(&priv->stats.port_stats_debug.ctx); sysctl_ctx_free(&priv->sysctl_ctx); - mlx5_free_bfreg(priv->mdev, &priv->bfreg); mlx5_core_destroy_mkey(priv->mdev, &priv->mr); mlx5_dealloc_transport_domain(priv->mdev, priv->tdn); mlx5_core_dealloc_pd(priv->mdev, priv->pdn); mlx5e_disable_async_events(priv); flush_workqueue(priv->wq); - mlx5e_priv_static_destroy(priv, mdev->priv.eq_table.num_comp_vectors); + mlx5e_priv_static_destroy(priv, mdev, mdev->priv.eq_table.num_comp_vectors); if_free(ifp); free(priv, M_MLX5EN); } diff --git a/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c b/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c index 33b93b1b5ced..bd66b8ecd725 100644 --- a/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c +++ b/sys/dev/mlx5/mlx5_en/mlx5_en_rl.c @@ -116,9 +116,6 @@ mlx5e_rl_create_sq(struct mlx5e_priv *priv, struct mlx5e_sq *sq, &sq->dma_tag))) goto done; - /* use shared UAR */ - sq->uar_map = priv->bfreg.map; - err = mlx5_wq_cyc_create(mdev, ¶m->wq, sqc_wq, &sq->wq, &sq->wq_ctrl); if (err) @@ -165,7 +162,7 @@ mlx5e_rl_open_sq(struct mlx5e_priv *priv, struct mlx5e_sq *sq, if (err) return (err); - err = mlx5e_enable_sq(sq, param, priv->rl.tisn); + err = mlx5e_enable_sq(sq, param, &priv->channel[ix].bfreg, priv->rl.tisn); if (err) goto err_destroy_sq;