From owner-svn-src-head@freebsd.org Mon Nov 7 11:39:46 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A857AC34D5C; Mon, 7 Nov 2016 11:39:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50B5BAFA; Mon, 7 Nov 2016 11:39:46 +0000 (UTC) (envelope-from hselasky@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id uA7BdjY2003587; Mon, 7 Nov 2016 11:39:45 GMT (envelope-from hselasky@FreeBSD.org) Received: (from hselasky@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id uA7BdjlP003584; Mon, 7 Nov 2016 11:39:45 GMT (envelope-from hselasky@FreeBSD.org) Message-Id: <201611071139.uA7BdjlP003584@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: hselasky set sender to hselasky@FreeBSD.org using -f From: Hans Petter Selasky Date: Mon, 7 Nov 2016 11:39:45 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r308416 - head/sys/dev/mlx5/mlx5_en X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2016 11:39:46 -0000 Author: hselasky Date: Mon Nov 7 11:39:45 2016 New Revision: 308416 URL: https://svnweb.freebsd.org/changeset/base/308416 Log: Add timer to watch the RQ when we are out of mbufs. The firmware/hardware does not generate additional completion events unless we post new buffers. Use a timer to try to post more buffers in case we are temporarily out of mbufs. Else the receive schedule completely stops. Sponsored by: Mellanox Technologies MFC after: 1 week Modified: head/sys/dev/mlx5/mlx5_en/en.h head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c head/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Modified: head/sys/dev/mlx5/mlx5_en/en.h ============================================================================== --- head/sys/dev/mlx5/mlx5_en/en.h Mon Nov 7 11:35:14 2016 (r308415) +++ head/sys/dev/mlx5/mlx5_en/en.h Mon Nov 7 11:39:45 2016 (r308416) @@ -478,6 +478,7 @@ struct mlx5e_rq { struct mlx5_wq_ctrl wq_ctrl; u32 rqn; struct mlx5e_channel *channel; + struct callout watchdog; } __aligned(MLX5E_CACHELINE_SIZE); struct mlx5e_sq_mbuf { Modified: head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c ============================================================================== --- head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Mon Nov 7 11:35:14 2016 (r308415) +++ head/sys/dev/mlx5/mlx5_en/mlx5_en_main.c Mon Nov 7 11:39:45 2016 (r308416) @@ -855,7 +855,13 @@ err_destroy_rq: static void mlx5e_close_rq(struct mlx5e_rq *rq) { + mtx_lock(&rq->mtx); rq->enabled = 0; + callout_stop(&rq->watchdog); + mtx_unlock(&rq->mtx); + + callout_drain(&rq->watchdog); + mlx5e_modify_rq(rq, MLX5_RQC_STATE_RDY, MLX5_RQC_STATE_ERR); } @@ -1439,6 +1445,8 @@ mlx5e_chan_mtx_init(struct mlx5e_channel mtx_init(&c->rq.mtx, "mlx5rx", MTX_NETWORK_LOCK, MTX_DEF); + callout_init_mtx(&c->rq.watchdog, &c->rq.mtx, 0); + for (tc = 0; tc < c->num_tc; tc++) { struct mlx5e_sq *sq = c->sq + tc; Modified: head/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c ============================================================================== --- head/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Mon Nov 7 11:35:14 2016 (r308415) +++ head/sys/dev/mlx5/mlx5_en/mlx5_en_rx.c Mon Nov 7 11:39:45 2016 (r308416) @@ -82,9 +82,10 @@ mlx5e_post_rx_wqes(struct mlx5e_rq *rq) while (!mlx5_wq_ll_is_full(&rq->wq)) { struct mlx5e_rx_wqe *wqe = mlx5_wq_ll_get_wqe(&rq->wq, rq->wq.head); - if (unlikely(mlx5e_alloc_rx_wqe(rq, wqe, rq->wq.head))) + if (unlikely(mlx5e_alloc_rx_wqe(rq, wqe, rq->wq.head))) { + callout_reset_curcpu(&rq->watchdog, 1, (void *)&mlx5e_post_rx_wqes, rq); break; - + } mlx5_wq_ll_push(&rq->wq, be16_to_cpu(wqe->next.next_wqe_index)); }