From owner-freebsd-scsi@freebsd.org Tue Apr 4 18:13:30 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FB36D2EFF2 for ; Tue, 4 Apr 2017 18:13:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 01906EDF for ; Tue, 4 Apr 2017 18:13:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x233.google.com with SMTP id b140so101247807iof.1 for ; Tue, 04 Apr 2017 11:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=KSQZPBxG/XENWQ/CmbSxRQXXWatbx3vc411yVsjYes4=; b=HdBifWxNweMf3PQHM167fmr1SZYJ6V/gB6dC5oAtAI8XJRGqzBPdcOTyMbVuBKCfiw qtF8vHAoXjXzZ1aFddB6zJYInWihq1aY+iDcCfH+0FOlv5E81aVt+LlUz/VfHin4FUnk 1c9wsFmMNoS76c/J1QHMcoOhStna4OnYJzJ+hwpH91teqI1krB+bSWOWZrYU9uY80bDW Ccz7Bczu1t5D9WPYxem6NYsDHT5xXYSSaR26YfzNbkh4HdOtT3ckAOXYxVUmLkbvqkCR B2F/MZXofCc1NENMzMyU987TRcGSfrvXu/6XrxKfnpbQrLqFsTt7ZrBlXqtBS33OE38U GN4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=KSQZPBxG/XENWQ/CmbSxRQXXWatbx3vc411yVsjYes4=; b=bzHZq6XNbCfB6rUFo/LN+mwnCgXKPdLXd9MVdlTZL0+DM8myV9zBgez2vly3ozZV1b i4nur+qyL15mL8Kp1Et/gtuNRMKCNeD/uSNE9epacZcVMxGIpnkcuPFH6KUVsjocYg4z ov1YtyPWy6oUa+oXYLr0wxKFqCj6ICwuLw1C+WGibXQalJPxxXTtMzw6Dqy2zKG/7hdz Qn7F7CoWm1s2GQ/TPdMeekQh146818YOvpa9+/Ank5IkNq8kNprA3oyccF2bKAiAHf1J PhtM29MivJTNYlDm3pQPcbqc6c4mw/I1xpkn5CZLmOEFDTT7vdmshmsMS/EVu3Pb55zp YMPg== X-Gm-Message-State: AFeK/H2XYGRN4Agc2MfLQgKitbwDx/LEYAweEhM4C8+jPy7f3mH2cIyM3GHRQZFLZsbyrF4t9MVZUHPdjMvklA== X-Received: by 10.107.134.94 with SMTP id i91mr23579428iod.0.1491329609072; Tue, 04 Apr 2017 11:13:29 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Tue, 4 Apr 2017 11:13:28 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: From: Warner Losh Date: Tue, 4 Apr 2017 12:13:28 -0600 X-Google-Sender-Auth: ZqEXfQyO3jnBOhV1biOyhhPIfKs Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Chuck Tuffli Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 18:13:30 -0000 On Tue, Apr 4, 2017 at 11:21 AM, Chuck Tuffli wrote: > On Tue, Apr 4, 2017 at 7:54 AM, Warner Losh wrote: >> On Tue, Apr 4, 2017 at 8:30 AM, Chuck Tuffli wrote: >>> Hi >>> >>> I posted a patch on Phabricator[1] which adds IO pass-through support >>> for NVMe and would like some feedback on it. The bulk of the change is >>> pretty straight forward, but there was one part I'm not sure is the >>> best approach. The driver needs to know what type of command the >>> application is submitting in order to place it on the correct type of >>> NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of >>> the ccb header for this purpose. xflags was convenient to use, but >>> since other drivers don't use it, my guess is this is "the wrong way". >>> If it is OK to use xflags for this, cool, otherwise I'm open to >>> suggestions. >> >> Yea, one of the comments I posted to review... At the very least, it >> needs nvmecontrol uses different devices to distinguish between the >> different queues. I find this somewhat unsatisfying, but it works. >> nda, however, doesn't publish a transport level or sim level device >> that one could easily latch on to. The issue becomes one of security: >> If I can open nda0/pass0, can I control namespace operations for the >> drive? Download new firmware? etc. However, you can do all these >> things with other classes of devices today, so maybe that's not the >> biggest issue. >> >>> As background, NVMe has two categories of commands: administrative and >>> NVM (sometimes referred to as "IO"). I don't think it is possible for >>> the driver to guess the command category based on the cmd bytes as: >>> - the command opcode values alias. E.g. Flush (NVM) and Delete >>> Submission queue (Admin) use opcode 0x0 >>> - both categories have commands which use a non-zero Namespace ID >>> (like a LUN). so filtering on NSID wouldn't work >> >> In theory, you could use 0xffff as the NSID, since that's not a valid NSID. > > Hmm, I think any value greater than zero is valid according to the > spec (note that 0xffffffff has a special meaning). Certainly, devices > won't support NSID values above their published Identify Controller.NN > (Number of Namespaces) value, but hijacking the NSID value to specify > the NVMe queue seems problematic as there are Admin commands which > require a valid NSID. Fair Enough. I'd thought 0xffff was the magic number :). However, you raise a good point. Grep tells me all the xflags are actually unused. So we could use it, but after chatting with Scott Long, I'm not sure that we should. However, I think Jim's idea of having a separate command for commands for the I/O queue and commands for the admin queue might be the better part of valor here. I'd initially read Jim's mail as use #defines for the xflags values, but that's not at all what he was saying. The code change would be a bit bigger, but not by a lot. It's super easy to add new XPT_ function code. >> The current support of name spaces is a bit weak in nvme/nvd too. And >> it's even weaker in nda since I didn't have a multi-namespace drive >> until lately. >> >> So what are your goals with this work? I'm aware of another effort to >> improve the name space support in nvmecontrol which isn't going >> through nda. I'm also working on getting the nvmecontrol functionality >> moved over to camcontrol. > > I'm working on a tool to gather SMART information and have been using > CAM pass for ATA and SCIS. Having pass support for NVMe too would > simplify the code and seemed like any easy addition seeing as how > you'd already done all the heavy lifting. Cool. I'd like to get all the functionality in nvmecontrol into camcontrol, but haven't had a decent notion of how to do that for this new transport that's so different than ATA and SCSI. Though we have camcontrol identify and camcontrol inquiry for ATA and SCSI respectively. There's no way to specify the protocol to send down for things that could be generic, or base what we do based on the protocol field of the passX device we wind up opening. You might want to look at nvmecontrol. It already prints out all the standard SMART log pages for NVMe as well as all the vendor-specific ones I could find w/o NDA (and a few I did under NDA that aren't in the tree due to vendor hesitance at approving their release). I had hoped to use the same code for camcontrol when I got it working there, but haven't worked to that point yet. I haven't done the SMART error reporting that NVMe can do yet. Warner