From owner-freebsd-scsi@freebsd.org Tue Apr 4 14:30:35 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68281D2E1EF for ; Tue, 4 Apr 2017 14:30:35 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-yw0-x22a.google.com (mail-yw0-x22a.google.com [IPv6:2607:f8b0:4002:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3187A810 for ; Tue, 4 Apr 2017 14:30:35 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-yw0-x22a.google.com with SMTP id p77so62148925ywg.1 for ; Tue, 04 Apr 2017 07:30:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to:cc; bh=jk9myweU69EMFkULlHSEEld2JeurnHL4tuOoMyv2up0=; b=1J+yIbwTxm2+/SlU2UKkoL4CDMnCB0Knnzz+vxPP2ttB8gPq1mGQvoGD9JzbSnNkCu Ln1HEXiSugneyAKeNMdCxC2NucrqS29t5oe15TF+oxX3TuPzncZOtVwSUxUyRXnPeq+c rjCDpApOECASa6SPuEm+h1AbEZQp12+Z9dT05ZI9C6QoIqOHE19iMpLDwEUZ4DWQi5LJ MWEn92ol9TEqhsvwHN87uLWoUxfR+vFd0diIZptQZD2INoybLLZyX+nHnpTVAfUd5xkq igNvH4n7LXtZaMe/YuYDkkaSQ+BOSQzKTHuhnPEjdLlK28rsaOK7773+cRrNagDo79Dt 1KRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=jk9myweU69EMFkULlHSEEld2JeurnHL4tuOoMyv2up0=; b=l60YcOAUfyMOXbnEvbRwhSCzPA3NASBa/7YvMA2O/+sFGFvVxhAQVmBCIfH6lEUwDj pe/GfEbLe6W7jn+ay7cN93uSnA/N0y4x/U9ahL3Zd+/iP60kAODSsJ72mEZLRX2G9vfp eSIj+sV9sCnH1RQaQuW/0oKEspj7unQfUuyzr8aj1KtJB0udBz1jWTKzh6Y/EF0eCXVB 0OBtdPbtwqo07+lf+IeQVmwWadC5RG0AI1q1vpfv5EwNxAuVDLXMSu2c20Y/E8rmqhDI nb5BlKcZnMPvJDtlPhPA9tz9RPg5PjJCY0SiS05ltutj/0v/zLBClVHFJzOWAOsPvtHk +gMA== X-Gm-Message-State: AFeK/H24PFSVp8PkX44JFP6oqya7BPyBgCkBOK3V4VZ2LRcDuKzRCtgH4rLzsz1Ghp4nj2A0ttfbekS0Nlrd2Q== X-Received: by 10.129.84.68 with SMTP id i65mr15486164ywb.42.1491316231063; Tue, 04 Apr 2017 07:30:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.167.65 with HTTP; Tue, 4 Apr 2017 07:30:30 -0700 (PDT) From: Chuck Tuffli Date: Tue, 4 Apr 2017 07:30:30 -0700 Message-ID: Subject: [RFC] CAM pass(4) patch for NVMe To: freebsd-scsi Cc: Warner Losh Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 14:30:35 -0000 Hi I posted a patch on Phabricator[1] which adds IO pass-through support for NVMe and would like some feedback on it. The bulk of the change is pretty straight forward, but there was one part I'm not sure is the best approach. The driver needs to know what type of command the application is submitting in order to place it on the correct type of NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of the ccb header for this purpose. xflags was convenient to use, but since other drivers don't use it, my guess is this is "the wrong way". If it is OK to use xflags for this, cool, otherwise I'm open to suggestions. As background, NVMe has two categories of commands: administrative and NVM (sometimes referred to as "IO"). I don't think it is possible for the driver to guess the command category based on the cmd bytes as: - the command opcode values alias. E.g. Flush (NVM) and Delete Submission queue (Admin) use opcode 0x0 - both categories have commands which use a non-zero Namespace ID (like a LUN). so filtering on NSID wouldn't work --chuck [1] https://reviews.freebsd.org/D10247 From owner-freebsd-scsi@freebsd.org Tue Apr 4 14:54:12 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A442AD2EEB0 for ; Tue, 4 Apr 2017 14:54:12 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7667DC1F for ; Tue, 4 Apr 2017 14:54:12 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22d.google.com with SMTP id y18so61752646itc.1 for ; Tue, 04 Apr 2017 07:54:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=CPaAVFzMOSG3fA5gaC0Qe38+Jd/ItaRNVKdfRaUAySA=; b=KujLMP7E2XoN7YLPl0WPJobVVjl/lG6bS5Kp+6ynOjxWkFTE+Y0zXaDISFrip53mhf F1Me0Jw6BcDE/0or8mHxSds2fn6ireghHF7aQnn+T74Kn1/eycbitjXLYj5qyrtY0hcI 4AXIW7v41TvxDD0/PEFeSllG0aiubKz05bp07d3BryrGTcyU5HmlFIokKQLW2jKbgIRc ZkdFHKUIJ+Ar1saeOkc6YNLeQOgYY28qmn7ke2u1bAvUFrfZ0deHqjcsoP2bO5cDpZTm SUWPsh58l6o27Ui068FWzl90txJONI8kby4J2C5SyAERSsVb2Xy3iVUF785bvFgOKJnJ ZesQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=CPaAVFzMOSG3fA5gaC0Qe38+Jd/ItaRNVKdfRaUAySA=; b=BPc1nOXoBKlWdRT+a1CrzfRJXNFJ5CygKRF9cRwyFDOLFIJws1BbFVSErdTeiC8wBX OI3QVYuwBvMQtRCnAIrCzg96w9TVhjXpjFuxx6GChFLGv7eac0MWEr/BInOD+gPJliBT OCNOecC/GiT7OwMKrel+i5OnN549kkleUBSM1PJrQrkubx7Z8lVv8X1r62F6p3KLkGNm We6VGL+MyiEI2MF+Xb5HEk3QVYkC5Qu0gqPlSQZ/vcXW4sLQIEDqc74kKz8pK5ggueRU Nw7aDnl45K9m3Va5LykdJfRYY5Rz5VCwDKabtxduyZCTUVSW1LxWfdPQvwvOUEn4ZwQ0 Dxtg== X-Gm-Message-State: AFeK/H2Wk2g9iD0ONLUXFrhm1OE1GDb2TAqAdKT33J+KYljIp7QGRXW1thoActh0cwVimWXdWrDxZT8hGnd7jg== X-Received: by 10.36.147.71 with SMTP id y68mr15806414itd.85.1491317651692; Tue, 04 Apr 2017 07:54:11 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Tue, 4 Apr 2017 07:54:11 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: From: Warner Losh Date: Tue, 4 Apr 2017 08:54:11 -0600 X-Google-Sender-Auth: QuLCL8-Gf5RR4MD07wlgbXGFrq4 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Chuck Tuffli Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 14:54:12 -0000 On Tue, Apr 4, 2017 at 8:30 AM, Chuck Tuffli wrote: > Hi > > I posted a patch on Phabricator[1] which adds IO pass-through support > for NVMe and would like some feedback on it. The bulk of the change is > pretty straight forward, but there was one part I'm not sure is the > best approach. The driver needs to know what type of command the > application is submitting in order to place it on the correct type of > NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of > the ccb header for this purpose. xflags was convenient to use, but > since other drivers don't use it, my guess is this is "the wrong way". > If it is OK to use xflags for this, cool, otherwise I'm open to > suggestions. Yea, one of the comments I posted to review... At the very least, it needs nvmecontrol uses different devices to distinguish between the different queues. I find this somewhat unsatisfying, but it works. nda, however, doesn't publish a transport level or sim level device that one could easily latch on to. The issue becomes one of security: If I can open nda0/pass0, can I control namespace operations for the drive? Download new firmware? etc. However, you can do all these things with other classes of devices today, so maybe that's not the biggest issue. > As background, NVMe has two categories of commands: administrative and > NVM (sometimes referred to as "IO"). I don't think it is possible for > the driver to guess the command category based on the cmd bytes as: > - the command opcode values alias. E.g. Flush (NVM) and Delete > Submission queue (Admin) use opcode 0x0 > - both categories have commands which use a non-zero Namespace ID > (like a LUN). so filtering on NSID wouldn't work In theory, you could use 0xffff as the NSID, since that's not a valid NSID. The current support of name spaces is a bit weak in nvme/nvd too. And it's even weaker in nda since I didn't have a multi-namespace drive until lately. So what are your goals with this work? I'm aware of another effort to improve the name space support in nvmecontrol which isn't going through nda. I'm also working on getting the nvmecontrol functionality moved over to camcontrol. Warner From owner-freebsd-scsi@freebsd.org Tue Apr 4 16:01:03 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80574D2EE6A for ; Tue, 4 Apr 2017 16:01:03 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-io0-x231.google.com (mail-io0-x231.google.com [IPv6:2607:f8b0:4001:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A7C7E8F for ; Tue, 4 Apr 2017 16:01:03 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by mail-io0-x231.google.com with SMTP id f84so98341120ioj.0 for ; Tue, 04 Apr 2017 09:01:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xFPRLtd8u/dBW6SPlNyFoRPZ2M1FUahwoWK7ZWIJld0=; b=VJYf8aggp76pPM929r21O0d9u54v7P8t/xYgid5W0JkgI2wHF+bBd4E4gMcc4GTIIO FelBKu80j+nhacmqPe7vWR3Y60kSLvijIv9zd9/9+3esbDSTpFTGTNgO8OAkaGp2/cvm UgMinVsO3hGlWFjZiTLDx+QH8YpXq+XlKRZMVDMVXBuXIOcqjpeAWWEXA8EwTRsCH9qK Aw1es1h7ydzVlMR8KByarfSa0WKVhPYR4170I779snyqfgwMPQAnrpV7I+mhynFZEE+9 abFm8+V3UUrdzhOPkHWSQGiTbvZuDXbTsqBWBu0duxkg+P9NtgjMwpoz5KC7b+wRmNkq Damg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xFPRLtd8u/dBW6SPlNyFoRPZ2M1FUahwoWK7ZWIJld0=; b=MbvmsrlL+4uw9rytJLH3r8RCQGm22qxFFO5tD33eZD2Bwz4MqIehHM4Cz1QPKNd3I8 8wyy5SZa4V68gTwLhjISryuvkP1yIknZv7GLakjvDzixvIvMiV0iq2Qyrym352vKB1h5 UAWR4yoBaJ1pVlzLdtVoVCRkhhEPoKJOH9MBNfhYZ83RQT6btG5kaEBmY+hICNS07V1o qSd+7QzUOxlXI0jgbGq7eVRsb33Kw2zYHxgJwors2f7d3QcMoPfGZqz81bkmCONff4x5 yfkYeAC5y+mm/Ti+gMNuwyOlXWdYxx+tzMgP3/PStSeeSkmJQ06/GVvU1BUH5iZ9WDpI +eWw== X-Gm-Message-State: AFeK/H0x7k5iW96KhA4MSu/qRX/52svzQ6AN7rJyoQKktEZp5yyKuMc686PN8l3MVVqW7r98DWk+kNfCiEkHug== X-Received: by 10.107.19.222 with SMTP id 91mr24632071iot.211.1491321662306; Tue, 04 Apr 2017 09:01:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.145.206 with HTTP; Tue, 4 Apr 2017 09:01:01 -0700 (PDT) In-Reply-To: References: From: Jim Harris Date: Tue, 4 Apr 2017 09:01:01 -0700 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Warner Losh Cc: Chuck Tuffli , freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 16:01:03 -0000 On Tue, Apr 4, 2017 at 7:54 AM, Warner Losh wrote: > On Tue, Apr 4, 2017 at 8:30 AM, Chuck Tuffli wrote: >> Hi >> >> I posted a patch on Phabricator[1] which adds IO pass-through support >> for NVMe and would like some feedback on it. The bulk of the change is >> pretty straight forward, but there was one part I'm not sure is the >> best approach. The driver needs to know what type of command the >> application is submitting in order to place it on the correct type of >> NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of >> the ccb header for this purpose. xflags was convenient to use, but >> since other drivers don't use it, my guess is this is "the wrong way". >> If it is OK to use xflags for this, cool, otherwise I'm open to >> suggestions. > > Yea, one of the comments I posted to review... At the very least, it > needs nvmecontrol uses different devices to distinguish between the > different queues. I find this somewhat unsatisfying, but it works. > nda, however, doesn't publish a transport level or sim level device > that one could easily latch on to. The issue becomes one of security: > If I can open nda0/pass0, can I control namespace operations for the > drive? Download new firmware? etc. However, you can do all these > things with other classes of devices today, so maybe that's not the > biggest issue. Would separate XPT_NVME_IO and XPT_NVME_ADMIN values work here instead? > >> As background, NVMe has two categories of commands: administrative and >> NVM (sometimes referred to as "IO"). I don't think it is possible for >> the driver to guess the command category based on the cmd bytes as: >> - the command opcode values alias. E.g. Flush (NVM) and Delete >> Submission queue (Admin) use opcode 0x0 >> - both categories have commands which use a non-zero Namespace ID >> (like a LUN). so filtering on NSID wouldn't work > > In theory, you could use 0xffff as the NSID, since that's not a valid NSID. > > The current support of name spaces is a bit weak in nvme/nvd too. And > it's even weaker in nda since I didn't have a multi-namespace drive > until lately. > > So what are your goals with this work? I'm aware of another effort to > improve the name space support in nvmecontrol which isn't going > through nda. I'm also working on getting the nvmecontrol functionality > moved over to camcontrol. > > Warner > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Tue Apr 4 17:21:31 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30DE9D2E77A for ; Tue, 4 Apr 2017 17:21:31 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-yb0-x22d.google.com (mail-yb0-x22d.google.com [IPv6:2607:f8b0:4002:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 071106DC for ; Tue, 4 Apr 2017 17:21:30 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-yb0-x22d.google.com with SMTP id f204so55942487ybc.2 for ; Tue, 04 Apr 2017 10:21:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=VjmmR+83dmMwhYK/hcucbhdCYEi5dtQYuSUX572oGFA=; b=SRmycR/Tal19LNM1KUEm3Hh68eoQahSBJAZ/1kjH2IbH9BG0y/E0HEzzdrG9gu/RZ/ ruwdMDiV3XrXkTs0GBOQkAwfyQW/CdKvYTUZlZvsS/TIe91v/gAS61ZRxUHM+dUfIP+7 Pn701zG6PS3JNRyyC/EgxxEXEHk+gkyCfPDFc7LlqOfoZ7ICabm0mkUF5PcIKo8jBYrD 74CPGCzo2/fI1Fyl/Nj6RffuxHSP4mdd/fwHAn9rg18FCtNjA9FGrMEA6GustuGz+wdZ hTWefwEcDsd/ZwJEms5vKnJENUCb3VR46P4xV3023MgWOfK6ZqnR/pN34IcwOxTPNHgP 0GGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=VjmmR+83dmMwhYK/hcucbhdCYEi5dtQYuSUX572oGFA=; b=FnJ41mMyNXzytVFnweTHz2krhrBAyCqGgFLmeNJlonrrx447xxa3KP6icKLA+MuVS7 WzMrkKxQr7TM+kK6ogZosmJwIzeYLcw5+h16AHx7ek0KHH4GVyf8DZJk47cVbGAzQCES yfz0m/M4Sy/RZMdGy7IjdKVdKTMLAD0uOUlIV8fI9CV4ITHpfNbNs6zg4fECLbcaw9nM QZTupGuOMvJ+Z6BJVCFpkQys9bVa3KFYBAlOZ7AclcwJdjGHDXY8YC4vmKDr9/dKoQ4g esd7mwAwKJSMoL9Gb5n/4Tu5QSSLJAUdGnpnsa3sgGL0JNy1HnQJ6nMUzHF+H2rJoypB imjw== X-Gm-Message-State: AFeK/H3WcYGjZeYc/ntJM8NcMVZfm3wpHK0sKF3uXrjoFLlc3ATUSwfE8EbTu0+UrLfvRf3JT3bH14JkMbTIWw== X-Received: by 10.37.195.129 with SMTP id t123mr14323990ybf.176.1491326489984; Tue, 04 Apr 2017 10:21:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.167.65 with HTTP; Tue, 4 Apr 2017 10:21:29 -0700 (PDT) In-Reply-To: References: From: Chuck Tuffli Date: Tue, 4 Apr 2017 10:21:29 -0700 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Warner Losh Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 17:21:31 -0000 On Tue, Apr 4, 2017 at 7:54 AM, Warner Losh wrote: > On Tue, Apr 4, 2017 at 8:30 AM, Chuck Tuffli wrote: >> Hi >> >> I posted a patch on Phabricator[1] which adds IO pass-through support >> for NVMe and would like some feedback on it. The bulk of the change is >> pretty straight forward, but there was one part I'm not sure is the >> best approach. The driver needs to know what type of command the >> application is submitting in order to place it on the correct type of >> NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of >> the ccb header for this purpose. xflags was convenient to use, but >> since other drivers don't use it, my guess is this is "the wrong way". >> If it is OK to use xflags for this, cool, otherwise I'm open to >> suggestions. > > Yea, one of the comments I posted to review... At the very least, it > needs nvmecontrol uses different devices to distinguish between the > different queues. I find this somewhat unsatisfying, but it works. > nda, however, doesn't publish a transport level or sim level device > that one could easily latch on to. The issue becomes one of security: > If I can open nda0/pass0, can I control namespace operations for the > drive? Download new firmware? etc. However, you can do all these > things with other classes of devices today, so maybe that's not the > biggest issue. > >> As background, NVMe has two categories of commands: administrative and >> NVM (sometimes referred to as "IO"). I don't think it is possible for >> the driver to guess the command category based on the cmd bytes as: >> - the command opcode values alias. E.g. Flush (NVM) and Delete >> Submission queue (Admin) use opcode 0x0 >> - both categories have commands which use a non-zero Namespace ID >> (like a LUN). so filtering on NSID wouldn't work > > In theory, you could use 0xffff as the NSID, since that's not a valid NSID. Hmm, I think any value greater than zero is valid according to the spec (note that 0xffffffff has a special meaning). Certainly, devices won't support NSID values above their published Identify Controller.NN (Number of Namespaces) value, but hijacking the NSID value to specify the NVMe queue seems problematic as there are Admin commands which require a valid NSID. > The current support of name spaces is a bit weak in nvme/nvd too. And > it's even weaker in nda since I didn't have a multi-namespace drive > until lately. > > So what are your goals with this work? I'm aware of another effort to > improve the name space support in nvmecontrol which isn't going > through nda. I'm also working on getting the nvmecontrol functionality > moved over to camcontrol. I'm working on a tool to gather SMART information and have been using CAM pass for ATA and SCIS. Having pass support for NVMe too would simplify the code and seemed like any easy addition seeing as how you'd already done all the heavy lifting. --chuck From owner-freebsd-scsi@freebsd.org Tue Apr 4 18:13:30 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2FB36D2EFF2 for ; Tue, 4 Apr 2017 18:13:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 01906EDF for ; Tue, 4 Apr 2017 18:13:29 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x233.google.com with SMTP id b140so101247807iof.1 for ; Tue, 04 Apr 2017 11:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=KSQZPBxG/XENWQ/CmbSxRQXXWatbx3vc411yVsjYes4=; b=HdBifWxNweMf3PQHM167fmr1SZYJ6V/gB6dC5oAtAI8XJRGqzBPdcOTyMbVuBKCfiw qtF8vHAoXjXzZ1aFddB6zJYInWihq1aY+iDcCfH+0FOlv5E81aVt+LlUz/VfHin4FUnk 1c9wsFmMNoS76c/J1QHMcoOhStna4OnYJzJ+hwpH91teqI1krB+bSWOWZrYU9uY80bDW Ccz7Bczu1t5D9WPYxem6NYsDHT5xXYSSaR26YfzNbkh4HdOtT3ckAOXYxVUmLkbvqkCR B2F/MZXofCc1NENMzMyU987TRcGSfrvXu/6XrxKfnpbQrLqFsTt7ZrBlXqtBS33OE38U GN4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=KSQZPBxG/XENWQ/CmbSxRQXXWatbx3vc411yVsjYes4=; b=bzHZq6XNbCfB6rUFo/LN+mwnCgXKPdLXd9MVdlTZL0+DM8myV9zBgez2vly3ozZV1b i4nur+qyL15mL8Kp1Et/gtuNRMKCNeD/uSNE9epacZcVMxGIpnkcuPFH6KUVsjocYg4z ov1YtyPWy6oUa+oXYLr0wxKFqCj6ICwuLw1C+WGibXQalJPxxXTtMzw6Dqy2zKG/7hdz Qn7F7CoWm1s2GQ/TPdMeekQh146818YOvpa9+/Ank5IkNq8kNprA3oyccF2bKAiAHf1J PhtM29MivJTNYlDm3pQPcbqc6c4mw/I1xpkn5CZLmOEFDTT7vdmshmsMS/EVu3Pb55zp YMPg== X-Gm-Message-State: AFeK/H2XYGRN4Agc2MfLQgKitbwDx/LEYAweEhM4C8+jPy7f3mH2cIyM3GHRQZFLZsbyrF4t9MVZUHPdjMvklA== X-Received: by 10.107.134.94 with SMTP id i91mr23579428iod.0.1491329609072; Tue, 04 Apr 2017 11:13:29 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Tue, 4 Apr 2017 11:13:28 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: From: Warner Losh Date: Tue, 4 Apr 2017 12:13:28 -0600 X-Google-Sender-Auth: ZqEXfQyO3jnBOhV1biOyhhPIfKs Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Chuck Tuffli Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 18:13:30 -0000 On Tue, Apr 4, 2017 at 11:21 AM, Chuck Tuffli wrote: > On Tue, Apr 4, 2017 at 7:54 AM, Warner Losh wrote: >> On Tue, Apr 4, 2017 at 8:30 AM, Chuck Tuffli wrote: >>> Hi >>> >>> I posted a patch on Phabricator[1] which adds IO pass-through support >>> for NVMe and would like some feedback on it. The bulk of the change is >>> pretty straight forward, but there was one part I'm not sure is the >>> best approach. The driver needs to know what type of command the >>> application is submitting in order to place it on the correct type of >>> NVMe queue. The patch uses bit 5 (i.e. 0x10) in the xflags variable of >>> the ccb header for this purpose. xflags was convenient to use, but >>> since other drivers don't use it, my guess is this is "the wrong way". >>> If it is OK to use xflags for this, cool, otherwise I'm open to >>> suggestions. >> >> Yea, one of the comments I posted to review... At the very least, it >> needs nvmecontrol uses different devices to distinguish between the >> different queues. I find this somewhat unsatisfying, but it works. >> nda, however, doesn't publish a transport level or sim level device >> that one could easily latch on to. The issue becomes one of security: >> If I can open nda0/pass0, can I control namespace operations for the >> drive? Download new firmware? etc. However, you can do all these >> things with other classes of devices today, so maybe that's not the >> biggest issue. >> >>> As background, NVMe has two categories of commands: administrative and >>> NVM (sometimes referred to as "IO"). I don't think it is possible for >>> the driver to guess the command category based on the cmd bytes as: >>> - the command opcode values alias. E.g. Flush (NVM) and Delete >>> Submission queue (Admin) use opcode 0x0 >>> - both categories have commands which use a non-zero Namespace ID >>> (like a LUN). so filtering on NSID wouldn't work >> >> In theory, you could use 0xffff as the NSID, since that's not a valid NSID. > > Hmm, I think any value greater than zero is valid according to the > spec (note that 0xffffffff has a special meaning). Certainly, devices > won't support NSID values above their published Identify Controller.NN > (Number of Namespaces) value, but hijacking the NSID value to specify > the NVMe queue seems problematic as there are Admin commands which > require a valid NSID. Fair Enough. I'd thought 0xffff was the magic number :). However, you raise a good point. Grep tells me all the xflags are actually unused. So we could use it, but after chatting with Scott Long, I'm not sure that we should. However, I think Jim's idea of having a separate command for commands for the I/O queue and commands for the admin queue might be the better part of valor here. I'd initially read Jim's mail as use #defines for the xflags values, but that's not at all what he was saying. The code change would be a bit bigger, but not by a lot. It's super easy to add new XPT_ function code. >> The current support of name spaces is a bit weak in nvme/nvd too. And >> it's even weaker in nda since I didn't have a multi-namespace drive >> until lately. >> >> So what are your goals with this work? I'm aware of another effort to >> improve the name space support in nvmecontrol which isn't going >> through nda. I'm also working on getting the nvmecontrol functionality >> moved over to camcontrol. > > I'm working on a tool to gather SMART information and have been using > CAM pass for ATA and SCIS. Having pass support for NVMe too would > simplify the code and seemed like any easy addition seeing as how > you'd already done all the heavy lifting. Cool. I'd like to get all the functionality in nvmecontrol into camcontrol, but haven't had a decent notion of how to do that for this new transport that's so different than ATA and SCSI. Though we have camcontrol identify and camcontrol inquiry for ATA and SCSI respectively. There's no way to specify the protocol to send down for things that could be generic, or base what we do based on the protocol field of the passX device we wind up opening. You might want to look at nvmecontrol. It already prints out all the standard SMART log pages for NVMe as well as all the vendor-specific ones I could find w/o NDA (and a few I did under NDA that aren't in the tree due to vendor hesitance at approving their release). I had hoped to use the same code for camcontrol when I got it working there, but haven't worked to that point yet. I haven't done the SMART error reporting that NVMe can do yet. Warner From owner-freebsd-scsi@freebsd.org Tue Apr 4 20:48:02 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC209D2F2F9 for ; Tue, 4 Apr 2017 20:48:02 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-yw0-x22b.google.com (mail-yw0-x22b.google.com [IPv6:2607:f8b0:4002:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 94704BEE for ; Tue, 4 Apr 2017 20:48:02 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-yw0-x22b.google.com with SMTP id i203so67684344ywc.3 for ; Tue, 04 Apr 2017 13:48:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=yRcnwILIwttSwRVU08tSiqijk2PGDDBVG5hcyvzje98=; b=q+NQWOrrpXgRPeKT3/K3UnnByevQWXCZ/7eLM28s5X+84EZPtNWxVV3UyhyquieBbJ 0LoCytTsSzI78t8uujcPIeYHuwIrDoaexLDalhm+xopTPQ21pGxMi6T0zLHwgg0J7K+R nibFxydyImBqzrENBaV2bHr6hh1Jtvx0Mz+J8M7c21fRzfFMuqcU8StKV0HhxNdhD2g3 +wMNHAtXTalX6JVwBh/AESN+x6JNHRX8bbNRmaHu276vBZgUKgl5ug5H9FIlo/OtFtNB 8HcxqlGKezzIgDi9dmOCPdTDp+mXQrH9bUpx8Gtllu4SARnmOFDXdfzD8tPwD8W9NNJ1 zMxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yRcnwILIwttSwRVU08tSiqijk2PGDDBVG5hcyvzje98=; b=NtZ6kgcE5bOyxDb0bkqFBLrVmFhZz2j+UhRvS2QDZwgj2lMlgZh98Z6VofXKb3ObXR 8WHjAB0sJTVfQ1HVLYKdAGZiSDzlD6dHDtlafJb6BD1Ho2IITbs2ewy8XY7U2hWfQ9eY 3nH91LvB7HKmF9ZRhS5WG7Qaosh7LMDdItvNO3myVMm23Q31KNakzubJjhOrUjzCzh+h 51ign42h0bSaz6WqM2QsHrh7fX247h2rnwAw44UTsbLqDWlXMttIJ6bqWshgKBeUtYav dPRUGjPEiYCQw6EqT5zuEwZL7KmELuoZQmunXac/L2JIRvjhz8bzToi8M7mZNIFl+UQc dj6Q== X-Gm-Message-State: AFeK/H3E52fwuG3SwKbmftJi4yYfC3YeiOMq0Qnm0psG6vnA+SpDb3Lm5iq1sY9SkIK8Pje73pXkbYMhMYGR0Q== X-Received: by 10.129.172.79 with SMTP id z15mr15441362ywj.255.1491338881403; Tue, 04 Apr 2017 13:48:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.167.65 with HTTP; Tue, 4 Apr 2017 13:48:00 -0700 (PDT) In-Reply-To: References: From: Chuck Tuffli Date: Tue, 4 Apr 2017 13:48:00 -0700 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Warner Losh Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 20:48:02 -0000 On Tue, Apr 4, 2017 at 11:13 AM, Warner Losh wrote: ... > Fair Enough. I'd thought 0xffff was the magic number :). However, you > raise a good point. > > Grep tells me all the xflags are actually unused. So we could use it, > but after chatting with Scott Long, I'm not sure that we should. > > However, I think Jim's idea of having a separate command for commands > for the I/O queue and commands for the admin queue might be the better > part of valor here. I'd initially read Jim's mail as use #defines for > the xflags values, but that's not at all what he was saying. > > The code change would be a bit bigger, but not by a lot. It's super > easy to add new XPT_ function code. OK, I'll head down that path and add a new XPT opcode XPT_NVME_ADMIN and helper macro cam_fill_nvmeadmin() which would be used for Admin commands. The existing XPT_NVME_IO would be used for NVM/IO commands. Both opcodes would use the ccb_nvmeio structure unless there are objections .... ? --chuck From owner-freebsd-scsi@freebsd.org Tue Apr 4 20:50:06 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C0244D2F3D9 for ; Tue, 4 Apr 2017 20:50:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8DA6DC95 for ; Tue, 4 Apr 2017 20:50:06 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id z13so103978262iof.2 for ; Tue, 04 Apr 2017 13:50:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=ZU15d2gwpvDjRMvFQtKWiuHZBoAQxw7i4837mcsKkSc=; b=vQxrNzLFPAKg+TAS+g3m17L3YEYrD9CsI65uLGkRoyLt3TiZEmjB6dRekbX+81NQTZ AHakk9a84PPDqIIlGoi6k18YEdlRHcZCI7WEVCOkRTXE/QCubD9Xtkkn8IVin2Gwt9SL t1ThRKhAV3YqpUV2+kmRkelmbakn1h81NZPY7rwYauNhENEq/UgmOnXniThrlEZ+EeCa zIAABKcMODMriEyoFbm6NjGXGq6rUjONaDW7IwLm634Py/dXfb7eeyrVFuIoEJKKzfEG CBHzFygDbVAav5Ku1bUnEVKqaAdJSUDwt20aYUWv++tJViTjnTvsCIpF3oI7/COPX1bV Lrfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=ZU15d2gwpvDjRMvFQtKWiuHZBoAQxw7i4837mcsKkSc=; b=GkhAJx9aQ9v3zs6XsRCrC+DM8NtEckf6lRn9+bdlN7SRM2Pga2NES2gAhzUM9isd6e lYeDUc+KMHwDHBrsJQeGvgFz/88EgMJ8FBRRi4eRaXibGeg0mp1rZ/a9yXFqDrPcMPhO 8RibRIPVWTT7vGjTM4yCVgfPa/75g0OTtLEG2NNG7IGTHu8JM26xB/BgdEdFIcXM8jfi eOKxuPRr2p1egf++GvybhrMkqD5e5K4UuUhwi2m20pvO+qs3Ym573H6U38rRreO6/a9L FQWg9V7aIetTz87dlIingjaagaGZAfrG+k6jC5EDRXjHgNBynIZRaznpvufcj2QP8WcN 1+JA== X-Gm-Message-State: AFeK/H0MY0xwEjQxBkxzhOWM6PBqdOMl/CYmRh1PjTew8JhMwP790EFdL4szoUu4zLV1jIX2o7iIGayfJ3BEXg== X-Received: by 10.107.174.220 with SMTP id n89mr25949159ioo.166.1491339005679; Tue, 04 Apr 2017 13:50:05 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.146.24 with HTTP; Tue, 4 Apr 2017 13:50:05 -0700 (PDT) X-Originating-IP: [2607:fb10:7021:1::9a56] In-Reply-To: References: From: Warner Losh Date: Tue, 4 Apr 2017 14:50:05 -0600 X-Google-Sender-Auth: Sx_3GTXekijQ7QuTG2sVQOdOJls Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Chuck Tuffli Cc: freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 20:50:06 -0000 On Tue, Apr 4, 2017 at 2:48 PM, Chuck Tuffli wrote: > On Tue, Apr 4, 2017 at 11:13 AM, Warner Losh wrote: > ... >> Fair Enough. I'd thought 0xffff was the magic number :). However, you >> raise a good point. >> >> Grep tells me all the xflags are actually unused. So we could use it, >> but after chatting with Scott Long, I'm not sure that we should. >> >> However, I think Jim's idea of having a separate command for commands >> for the I/O queue and commands for the admin queue might be the better >> part of valor here. I'd initially read Jim's mail as use #defines for >> the xflags values, but that's not at all what he was saying. >> >> The code change would be a bit bigger, but not by a lot. It's super >> easy to add new XPT_ function code. > > OK, I'll head down that path and add a new XPT opcode XPT_NVME_ADMIN > and helper macro cam_fill_nvmeadmin() which would be used for Admin > commands. The existing XPT_NVME_IO would be used for NVM/IO commands. > Both opcodes would use the ccb_nvmeio structure unless there are > objections .... ? Seems reasonable to me. Warner From owner-freebsd-scsi@freebsd.org Tue Apr 4 21:04:08 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B716FD2F9CB for ; Tue, 4 Apr 2017 21:04:08 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E38E9D4 for ; Tue, 4 Apr 2017 21:04:08 +0000 (UTC) (envelope-from jim.harris@gmail.com) Received: by mail-io0-x22f.google.com with SMTP id l7so103762196ioe.3 for ; Tue, 04 Apr 2017 14:04:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=vSYPYhdkUrUl4iUenYSfppUReAy+lQrfmUC8MwlvgmI=; b=ua2Tl/lvdK8QJTINy/Kx9yeVs4pTvuo7eLGbk76n9botZaV8osc98UxVPGtJmy1jWR kuXqjgGgM4f6yw5lD2vzlaszk+kRG8KSIN0HRxpYu/8TOvHkjJMJddhQRgL8/bX34n1s 0GJOSCpPXvOzAF404LFie64agOKPPj6El8ebRhqOucV2S7bEuD5QUrclS0qwk3MrAwwk +y3uNEvKfBx9NBsy/EjjMsHFsosJ1WuFd90UvbT8I273gsVhgTJoa8ZcSXpAgkS70QsJ 4ZVHvEcP1xu0w/ZGLVjfGP9iJXFzv4QWOMd9ejBBl9ttfU1IpGrKhFkewSvQe4rNljhr vbSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=vSYPYhdkUrUl4iUenYSfppUReAy+lQrfmUC8MwlvgmI=; b=qYqQ5ouVnAJJry+beiH5lQIimv7OH2DOS8zlKvhGI6JNIRVHiQ4SWyvor2cpmVpB3s GozW5T11K2VmUgEbPnWr1oNgS7OLmMJT/jtQZWc1EC/XnAT57FoC95BQawzstiduF7DM cSaIu2Xis+H/mzB1AAL8WU9cKyWdm/QknlAHViJaeNLFi2e9nwc8Vk8brGS/t87KMvAx fTTwA+BOpLChH//VMDm4KPnA12FavrJ1+ECqb7wz542/GslvcGpUrRTMIPOH52AHsy0v 2EDtqV05I/yE9VCdGWRYWLGessctN1rNBe0wMTYmTQLI9szSckCnHHsN1aNKWIPEl61K u3ZQ== X-Gm-Message-State: AFeK/H3e88WUYkBJBI22hIZuVizJ+79k5kMMWbGMcHs1XXxl0DURlbDtiaD2nhnn5KvScGkMIm4+7azHRVVVPw== X-Received: by 10.107.149.7 with SMTP id x7mr23730682iod.167.1491339847873; Tue, 04 Apr 2017 14:04:07 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.145.206 with HTTP; Tue, 4 Apr 2017 14:04:07 -0700 (PDT) In-Reply-To: References: From: Jim Harris Date: Tue, 4 Apr 2017 14:04:07 -0700 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Warner Losh Cc: Chuck Tuffli , freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Apr 2017 21:04:08 -0000 On Tue, Apr 4, 2017 at 1:50 PM, Warner Losh wrote: > On Tue, Apr 4, 2017 at 2:48 PM, Chuck Tuffli wrote: >> On Tue, Apr 4, 2017 at 11:13 AM, Warner Losh wrote: >> ... >>> Fair Enough. I'd thought 0xffff was the magic number :). However, you >>> raise a good point. >>> >>> Grep tells me all the xflags are actually unused. So we could use it, >>> but after chatting with Scott Long, I'm not sure that we should. >>> >>> However, I think Jim's idea of having a separate command for commands >>> for the I/O queue and commands for the admin queue might be the better >>> part of valor here. I'd initially read Jim's mail as use #defines for >>> the xflags values, but that's not at all what he was saying. >>> >>> The code change would be a bit bigger, but not by a lot. It's super >>> easy to add new XPT_ function code. >> >> OK, I'll head down that path and add a new XPT opcode XPT_NVME_ADMIN >> and helper macro cam_fill_nvmeadmin() which would be used for Admin >> commands. The existing XPT_NVME_IO would be used for NVM/IO commands. >> Both opcodes would use the ccb_nvmeio structure unless there are >> objections .... ? > > Seems reasonable to me. Me too. From owner-freebsd-scsi@freebsd.org Fri Apr 7 21:04:47 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A327D33555 for ; Fri, 7 Apr 2017 21:04:47 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-yw0-x22b.google.com (mail-yw0-x22b.google.com [IPv6:2607:f8b0:4002:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6DB45DA9 for ; Fri, 7 Apr 2017 21:04:47 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-yw0-x22b.google.com with SMTP id l189so2354539ywb.0 for ; Fri, 07 Apr 2017 14:04:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=eTpCYWqQ0LDhuPFR/Vej3eDlGT51GzscDXMUAwAu5gc=; b=UFzFUUfUJ/YjzzZ+A0Fq1dRrwEbCuuwb0GhZ9dFj0aedFD74D0Xl4t8Y7M3S2+sFc3 uEZd3fJmJlVzH3Pl3GOQQUbQgMUfj6bRJJmBpZavtCzBgitJtxbbfS+qyVeaCachXb7P XfUTdaHmvL2FAY4d+HfXVEJI+eWIgGWF9RZ/PcajPI3N3seBt6LwgP7SzZlPxLSYFEr9 XovLsvWT74ocUyUrPN7M3f1/WgnNMTBRi43voDc/07GyMx4OPnuxzHUiaTLgmeCikrHI kfF8ttU0HZLv8bm+3fP0fKTxYwSBkk8kYWUUT7NYdk20vxp9EC/FYNSNIffkROWD2QOw aVRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=eTpCYWqQ0LDhuPFR/Vej3eDlGT51GzscDXMUAwAu5gc=; b=rf4kU+hmDTbVaphtUJTuXniNhUNANmA1kSEYFSo+fHD2xWtUG1PZGQhINp6Wej61ke qp0+GIhN/W7Q/pABd3wdFVfZUHEEhJCUWtEihuXOK2xuPTUMT71s7BefRGfw6MhNeO0q Wav/k8VtmcfQYWdwUGS2/C8KvU2Jz7kQjifbOBp6bjI0JhAvmmqSIdBFqcsnH3rLvAjY q3CVrLbxgq1EtfF7OThBSTM+VZcdDfmwnEoZ9G8j8K3UOjeXe2SWaAUW+V26UHypX5s4 OpkCTO/caOYHXFVWCfzHAczXQZLj5SH6co0C7+Or6SZlt2ODmhNWkS0DvX2k54DITEMj Pc6Q== X-Gm-Message-State: AFeK/H3sDllErMbgv7Gd9OOufvTq9Iyi9oYyNqqlNK3+vi4ORXUQGd/kjEFDG7TduWQpuCBBr7J4HR8MC0SoMg== X-Received: by 10.129.84.68 with SMTP id i65mr29382514ywb.42.1491599086345; Fri, 07 Apr 2017 14:04:46 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.167.65 with HTTP; Fri, 7 Apr 2017 14:04:45 -0700 (PDT) In-Reply-To: References: From: Chuck Tuffli Date: Fri, 7 Apr 2017 14:04:45 -0700 Message-ID: Subject: Re: [RFC] CAM pass(4) patch for NVMe To: Jim Harris Cc: Warner Losh , freebsd-scsi Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Apr 2017 21:04:47 -0000 On Tue, Apr 4, 2017 at 2:04 PM, Jim Harris wrote: > On Tue, Apr 4, 2017 at 1:50 PM, Warner Losh wrote: >> On Tue, Apr 4, 2017 at 2:48 PM, Chuck Tuffli wrote: >>> On Tue, Apr 4, 2017 at 11:13 AM, Warner Losh wrote: >>> ... >>>> Fair Enough. I'd thought 0xffff was the magic number :). However, you >>>> raise a good point. >>>> >>>> Grep tells me all the xflags are actually unused. So we could use it, >>>> but after chatting with Scott Long, I'm not sure that we should. >>>> >>>> However, I think Jim's idea of having a separate command for commands >>>> for the I/O queue and commands for the admin queue might be the better >>>> part of valor here. I'd initially read Jim's mail as use #defines for >>>> the xflags values, but that's not at all what he was saying. >>>> >>>> The code change would be a bit bigger, but not by a lot. It's super >>>> easy to add new XPT_ function code. >>> >>> OK, I'll head down that path and add a new XPT opcode XPT_NVME_ADMIN >>> and helper macro cam_fill_nvmeadmin() which would be used for Admin >>> commands. The existing XPT_NVME_IO would be used for NVM/IO commands. >>> Both opcodes would use the ccb_nvmeio structure unless there are >>> objections .... ? >> >> Seems reasonable to me. > > Me too. OK, v2 of the patch is up using the new XPT op-code for admin commands instead of the original xflags hack. I tried a couple of passthru commands and it seems to work. --chuck