From owner-freebsd-arch@FreeBSD.ORG Mon Mar 11 18:25:02 2013 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 44EEFA92 for ; Mon, 11 Mar 2013 18:25:02 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 97D3675F for ; Mon, 11 Mar 2013 18:25:01 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.6/8.14.6) with ESMTP id r2BIOtAi004101; Mon, 11 Mar 2013 20:24:55 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.0 kib.kiev.ua r2BIOtAi004101 Received: (from kostik@localhost) by tom.home (8.14.6/8.14.6/Submit) id r2BIOsc8004100; Mon, 11 Mar 2013 20:24:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 11 Mar 2013 20:24:54 +0200 From: Konstantin Belousov To: Dag-Erling Sm??rgrav Subject: Re: Unmapped buffers: to be merged in several days Message-ID: <20130311182454.GX3794@kib.kiev.ua> References: <20130311091852.GR3794@kib.kiev.ua> <86k3pe1cl3.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="oMQ/0Eds4GFUZehv" Content-Disposition: inline In-Reply-To: <86k3pe1cl3.fsf@ds4.des.no> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 11 Mar 2013 18:25:02 -0000 --oMQ/0Eds4GFUZehv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 11, 2013 at 10:59:36AM +0100, Dag-Erling Sm??rgrav wrote: > Konstantin Belousov writes: > > The ahci(4) and md(4) is converted to accept unmapped BIO requests. > > Other drivers and geom classes get the compat mapped BIOs, the > > transient mapping is established by the geom down thread. The KVA > > for the transient maping the carved from the buffer map, up to 10% > > of which is repurposed to the transient bio KVA. The hope is that > > the rest of drivers and geom classes will be converted to accept > > unmapped i/o shortly, making the transient map unused. >=20 > Could you briefly summarize what needs to be done to convert a driver? There are different kind of drivers, each kind requires its own approach, and got different ground work done to ease the transition. For the geom classes, if the provider does not require access to the bio data, the conversion is as easy as setting the G_PF_ACCEPT_UNMAPPED in the flags. Look for the example in the sys/geom/part/g_part.c, which marks all gpart providers as accepting unmapped bios. If the class does need to access the bio data, to be marked unmapped-processing, the class should be rewritten. Now, the class should verify is the bio passed is mapped or not, and process the pages passed in the bio_ma array instead of bio_data. The involved example is sys/dev/md/md.c. For the disk(9) drivers, the flag to set is DISKFLAG_UNMAPPED_BIO. Again, the driver should be inspected before the flag is set. For the CAM-backed HBAs, there is a support provided to make the convertion trivial. If the hardware uses dma to perform the i/o and the driver uses busdma(9) to prepare the dma-able buffers for the dma engine, all the driver author needs is to ensure that bus_dmamap_load_ccb(9) function is used to handle the buffer load, and set the PIM_UNMAPPED flag. Most of the drivers were converted already to use load_ccb() during the Jeff' physbio work. For an examples of the trivial conversion, look at the sys/dev/ahci/ahci.c and sys/dev/siis/siis.c. I was conservative on setting the PIM_UNMAPPED for the untested drivers, but the only work that is needed is testing. Quite non-trivial driver to convert is ad(4), either CAMified or not. The issue is that even the drive that uses DMA could fall back to the PIO mode at runtime, requiring driver access to the potentially unmapped data. There are some preparations done for providing easy to use transient mapping KPI for such drivers, but it is not in the state where it could be usefully applied to the task, yet. --oMQ/0Eds4GFUZehv Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iQIcBAEBAgAGBQJRPiF2AAoJEJDCuSvBvK1B4LcP/1NjEO+XgYIHaTwAHcJvDr9J 3RZZtno75xpp0W/GeGynwT0S6XS/+NM+5YM2AShwzlzf41pk4aQ1Kq2iWMBmsl8u xyfGBSj0LwbqZpioTLwc3p77xdqH1esnrnDebTThuMy2FFtgxJWpARKALceyLcji WYdZPw0Rl0Oo03yB6qButBYLOO6kTG0V0kgJmUx83xMMSLD/0vhk0OJuL+iMh33h 5IA7KUOilJcQFxqM82E0Q08kBsLfxt+7AqhcfGa0B6Y5ihZUWReLGf6h0/Ayv/MZ Bd3MjNm8auZx6Ou4AGNDiZfIOee9cwjH3ODt2Pc2ztJrKOnNiUzdAwYW4pHcyxQH tuY0NVoxSDxfxOMMdZjYs3kxJ99q3BcAIB4z7Fqpz0k2mzbhdJ8DdPW5pE1xqLK+ a1niidq49okJHcyZpiCY9RxpJaOi2fjY6coXH0Fdof3MXlc7+hn18Ldf/d3GqGkW I9Ux+VtMEWathZOSuj0q8HtQnZI70ffwq/Ik6AeOp1ivuusI9tSmoHLx9hQujoMm iKUY8ksR9jYDhcoEcEmq/LONowcRtnhcPJQ47LSTgQJpatdVHMktchoeJTj5nNYs RUodo50TuLaSIpjI5tWchqP1/0nGuO57xL3oC+7hdZOSvNovGq7ks5Nn+167/x4Y d0PHAyw8N/Tsy/tyPdCK =p/xK -----END PGP SIGNATURE----- --oMQ/0Eds4GFUZehv--