From owner-dev-commits-src-all@freebsd.org Fri Oct 1 16:56:22 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 76B8F6B282A; Fri, 1 Oct 2021 16:56:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HLbqB2cs1z4kF1; Fri, 1 Oct 2021 16:56:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2F268230AD; Fri, 1 Oct 2021 16:56:22 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 191GuMZj004034; Fri, 1 Oct 2021 16:56:22 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 191GuMD0004033; Fri, 1 Oct 2021 16:56:22 GMT (envelope-from git) Date: Fri, 1 Oct 2021 16:56:22 GMT Message-Id: <202110011656.191GuMD0004033@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: d5fca1dc1d7d - main - nvme_ctrlr_enable: Remove unnecessary 5ms delays MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d5fca1dc1d7de15695b65374d6457abd29a747ee Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2021 16:56:22 -0000 The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=d5fca1dc1d7de15695b65374d6457abd29a747ee commit d5fca1dc1d7de15695b65374d6457abd29a747ee Author: Warner Losh AuthorDate: 2021-10-01 16:47:27 +0000 Commit: Warner Losh CommitDate: 2021-10-01 16:56:10 +0000 nvme_ctrlr_enable: Remove unnecessary 5ms delays Remove the 5ms delays after writing the administrative queue registers. These delays are from the very earliest days of the driver (they are in the first commit) and were most likely vestiges of the Chatham NVMe prototype card that was used to create this driver. Many of the workarounds necessary for it aren't necessary for standards compliant cards. The original driver had other areas marked for Chatham, but these were not. They are unneeded. There's three lines of supporting evidence. First, the NVMe standards make no mention of a delay time after these registers are written. Second, the Linux driver doesn't have them, even as an option. Third, all my nvme cards work w/o them. To be safe, add a write barrier between setting up the admin queue and enabling the controller. Sponsored by: Netflix Reviewed by: mav Differential Revision: https://reviews.freebsd.org/D32247 --- sys/dev/nvme/nvme_ctrlr.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index 39cde3f065dd..897d832047ce 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -52,6 +52,12 @@ __FBSDID("$FreeBSD$"); static void nvme_ctrlr_construct_and_submit_aer(struct nvme_controller *ctrlr, struct nvme_async_event_request *aer); +static void +nvme_ctrlr_barrier(struct nvme_controller *ctrlr, int flags) +{ + bus_barrier(ctrlr->resource, 0, rman_get_size(ctrlr->resource), flags); +} + static void nvme_ctrlr_devctl_log(struct nvme_controller *ctrlr, const char *type, const char *msg, ...) { @@ -356,9 +362,7 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) } nvme_mmio_write_8(ctrlr, asq, ctrlr->adminq.cmd_bus_addr); - DELAY(5000); nvme_mmio_write_8(ctrlr, acq, ctrlr->adminq.cpl_bus_addr); - DELAY(5000); /* acqs and asqs are 0-based. */ qsize = ctrlr->adminq.num_entries - 1; @@ -367,7 +371,6 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) aqa = (qsize & NVME_AQA_REG_ACQS_MASK) << NVME_AQA_REG_ACQS_SHIFT; aqa |= (qsize & NVME_AQA_REG_ASQS_MASK) << NVME_AQA_REG_ASQS_SHIFT; nvme_mmio_write_4(ctrlr, aqa, aqa); - DELAY(5000); /* Initialization values for CC */ cc = 0; @@ -381,6 +384,7 @@ nvme_ctrlr_enable(struct nvme_controller *ctrlr) /* This evaluates to 0, which is according to spec. */ cc |= (PAGE_SIZE >> 13) << NVME_CC_REG_MPS_SHIFT; + nvme_ctrlr_barrier(ctrlr, BUS_SPACE_BARRIER_WRITE); nvme_mmio_write_4(ctrlr, cc, cc); return (nvme_ctrlr_wait_for_ready(ctrlr, 1));