From owner-freebsd-arch@FreeBSD.ORG Wed Apr 29 10:22:20 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 144A7707; Wed, 29 Apr 2015 10:22:20 +0000 (UTC) Received: from mail-ie0-x231.google.com (mail-ie0-x231.google.com [IPv6:2607:f8b0:4001:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D20941AE8; Wed, 29 Apr 2015 10:22:19 +0000 (UTC) Received: by iejt8 with SMTP id t8so38512954iej.2; Wed, 29 Apr 2015 03:22:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=tko/ed6toqS5d2a1mxw+D0TWHs0226KwrmCSduVh5yQ=; b=QDguw+9rIztsXOeWENfTx7OFZKjrgH+Z84JzkeuUxUPRqtxJ4N+Mz0XJB0byqrWpur 3eRsbDqVke1zzi6ZmMnVsUcEG73j8LL63+TakdDLbAzmbb/Ehw4VPIuaMEHXqLNBfpwr VpLs7kffRCIR/rh9dLGLeQJoTAA5p3rxLT3/gZ1SfpTIitpp6W0OL0l7DpekFPeZvryV 4EGAxplO7DQjuUhgyRyULurQhnFGNAfmNGAwQI9vlnbCu3Vc4c13nrtiaSdSkVjdhXeZ aFTG7u5fDacAkBN6VARrPCLmL8zZsHaIyeJDpDQBnzlUkepd4jdsEAXLrncLXp/hY3Ww zIwg== MIME-Version: 1.0 X-Received: by 10.50.77.13 with SMTP id o13mr26742856igw.39.1430302939208; Wed, 29 Apr 2015 03:22:19 -0700 (PDT) Received: by 10.64.13.81 with HTTP; Wed, 29 Apr 2015 03:22:19 -0700 (PDT) In-Reply-To: <1761247.Bq816CMB8v@ralph.baldwin.cx> References: <38574E63-2D74-4ECB-8D68-09AC76DFB30C@bsdimp.com> <1761247.Bq816CMB8v@ralph.baldwin.cx> Date: Wed, 29 Apr 2015 12:22:19 +0200 Message-ID: Subject: Re: bus_dmamap_sync() for bounced client buffers from user address space From: Svatopluk Kraus To: John Baldwin Cc: Adrian Chadd , Warner Losh , Konstantin Belousov , Jason Harmening , freebsd-arch Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Apr 2015 10:22:20 -0000 On Wed, Apr 29, 2015 at 12:27 AM, John Baldwin wrote: > On Tuesday, April 28, 2015 12:10:43 PM Adrian Chadd wrote: >> On 28 April 2015 at 09:19, Warner Losh wrote: >> > >> >> On Apr 28, 2015, at 7:40 AM, John Baldwin wrote: >> >> >> >>> I believe UIO_USERSPACE is almost unused, it might be there for some >> >>> obscure (and buggy) driver. >> >> >> >> I believe it was added (and only ever used) in crypto drivers, and th= at they >> >> all did bus_dma operations in the context of the thread that passed i= n the >> >> uio. I definitely think it is fragile and should be replaced with so= mething >> >> more reliable. >> > >> > Fusion I/O=E2=80=99s SDK used this trick to allow mapping of userspace= buffers down >> > into the block layer after doing the requisite locking / pinning / etc= of the buffers >> > into memory. That=E2=80=99s if memory serves correctly (the SDK did th= ese things, I can=E2=80=99t >> > easily check on that detail since I=E2=80=99m no longer at FIO). >> >> This is a long-standing trick. physio() does it too, >> aio_read/aio_write does it for direct block accesses. Now that pbufs >> aren't involved anymore, it should scale rather well. >> >> So I'd like to see more of it in the kernel and disk/net APIs and driver= s. > > aio_read/write jump through gross hacks to create dedicated kthreads that > "borrow" the address space of the requester. The fact is that we want to > make unmapped I/O work in the general case and the same solutions for > temporary mappings for that can be reused to temporarily map the wired > pages backing a user request when needed. Reusing user mappings directly > in the kernel isn't really the way forward. > If using unmapped buffers is the way we will take to play with user space buffers, then: (1) DMA clients, which support DMA for user space buffers, must use some variant of _bus_dmamap_load_phys(). They must wire physical pages in system anyway. (2) Maybe some better way how to temporarily allocate KVA for unmapped buffers should be implemented. (3) DMA clients which already use _bus_dmamap_load_uio() with UIO_USERSPACE must be reimplemented or made obsolete. (4) UIO_USERSPACE must be off limit in _bus_dmamap_load_uio() and man page should be changed according to it. (5) And pmap can be deleted from struct bus_dmamap and all functions which use it as argument. Only kernel pmap will be used in DMA framework. Did I miss out something? > -- > John Baldwin