From owner-freebsd-drivers@FreeBSD.ORG Sun Jun 8 05:07:24 2008 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BA221065673 for ; Sun, 8 Jun 2008 05:07:24 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.30]) by mx1.freebsd.org (Postfix) with ESMTP id 541EA8FC12 for ; Sun, 8 Jun 2008 05:07:24 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so846528ywe.13 for ; Sat, 07 Jun 2008 22:07:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:mime-version:content-type:content-transfer-encoding :content-disposition:message-id; bh=RE7NIta3ZfJiTYWhwTmfI/rLOThqfn76jNeD+2IexQ0=; b=hMVUs98dZOXE7UFRfurxDDelB2UVp9d/gwcT9WuDUbODXMWIZ8YujT/micr9YZw9Ht vYWTO7i3JOHUNhuE9eGwCsiF85nv0yKuxurBJ+pRnLNNRUp3FTK+fjlruzytWss8BMQV Rtm4fxzxDWFsSsoXCOOK0Yw2oeLtimVXKWSm4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:mime-version:content-type :content-transfer-encoding:content-disposition:message-id; b=LzXFRhIwtpI4oPE3Ioh/NazjCG7Wt5fBpWruOUy4VVcWkr49VH53JF+c9pE+M+k94Q VkDL+VtkgsQupNCnxMtD5beOveCC+tNOpJcnkfFTTx0Q0MZl81pZCOGjvPbD22V5s++0 07njcsoIHPQugOkvhocr7FOSd+K29woZt7SCI= Received: by 10.150.202.8 with SMTP id z8mr3555612ybf.5.1212899926875; Sat, 07 Jun 2008 21:38:46 -0700 (PDT) Received: from ?192.168.0.137? ( [24.174.5.175]) by mx.google.com with ESMTPS id u25sm8147428ele.5.2008.06.07.21.38.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sat, 07 Jun 2008 21:38:46 -0700 (PDT) From: Jason Harmening To: freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org Date: Sat, 7 Jun 2008 23:41:16 -0500 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806072341.17041.jason.harmening@gmail.com> Cc: Subject: bus_dmamem_alloc X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2008 05:07:24 -0000 Hello, I've written a FreeBSD driver for Conexant CX2388x-based PCI TV capture cards. Of course the driver uses busdma to be as machine-independent as possible. One problem I've encountered is that bus_dmamem_alloc is inadequate for my needs. The CX2388x only understands 32-bit physical addreses, and the driver has two separate use cases for busdma: 1) Data buffers: These buffers may be relatively large (a 640x480 RGB32 video frame is ~1.2M), and therefore it is desirable that these buffers not be physically contiguous. 2) DMA program buffers: The DMA engine on the CX2388x is controlled by special-purpose RISC instructions, usually stored in host memory, that provide information on, among other things, the physical layout of the data buffers, which enables handling of non-contiguous data buffers. These programs are rarely more than a few pages in size, so for the sake of simplicity it is desirable that DMA program buffers be physically contiguous. For case 1), I malloc(9) the buffers and then feed them to busdma, since on most machines bus_dmamem_alloc just calls contigmalloc. Use of malloc(9) is suboptimal as it may result in bounce buffering for non-IOMMU machines with large amounts of RAM. For case 2), I contigmalloc the DMA program buffers in the 32-bit physical address range and then feed them to busdma. I don't use bus_dmamem_alloc here because it always allocates the maximum size specified in the bus_dma_tag_t. Since the driver supports dynamic data buffer allocation and DMA program generation, DMA program sizes may vary significantly. I therefore just create the bus_dma_tag_t with the maximum possible size for a DMA program buffer since I'd prefer not to have to re-create the tag every time the DMA program size changes. But always allocating the maximum buffer size would be a huge waste of contiguous memory, so bus_dmamem_alloc is out of the question here too. At the same time, use of contigmalloc is suboptimal as it may not be necessary to restrict the allocation to 32-bit physical addresses on IOMMU-equipped machines. This is something that bus_dmamem_alloc could take care of, if only it supported a size parameter (as I believe the NetBSD version does). So I have 3 questions: 1) Would it be possible to provide a bus_dmamem_alloc overload that takes a size parameter? We could call it bus_dmamem_alloc_size and have bus_dmamem_alloc just call bus_dmamem_alloc_size with dmat->maxsize to preserve source-level compatibility with existing drivers. 2) Are there currently any serious plans to have bus_dmamem_alloc perform multi-segment allocations on non-IOMMU machines? It looks like NetBSD does this by reserving the physical segments and then stitching them together into a virtually contiguous range. Is something like this feasible for FreeBSD? 3) Are there currently any serious plans to support IOMMUs on anything besides Sun machines? The AMD AGP GART, PowerPC 970 DART, and Intel VT-d and AMD IOMMU all come to mind. If any of these ideas sound feasible, I'd be more than willing to help research/implement/test them. Thanks, Jason From owner-freebsd-drivers@FreeBSD.ORG Sun Jun 8 13:18:01 2008 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A525106567D for ; Sun, 8 Jun 2008 13:18:01 +0000 (UTC) (envelope-from ken@mthelicon.com) Received: from hercules.mthelicon.com (hercules.mthelicon.com [66.90.103.179]) by mx1.freebsd.org (Postfix) with ESMTP id 4C1048FC16 for ; Sun, 8 Jun 2008 13:18:01 +0000 (UTC) (envelope-from ken@mthelicon.com) Received: from [192.168.0.192] (host86-166-57-83.range86-166.btcentralplus.com [86.166.57.83]) (authenticated bits=0) by hercules.mthelicon.com (8.14.2/8.14.2) with ESMTP id m58CeZSm022655 (version=TLSv1/SSLv3 cipher=DHE-DSS-AES256-SHA bits=256 verify=NO) for ; Sun, 8 Jun 2008 12:40:37 GMT (envelope-from ken@mthelicon.com) From: Pegasus Mc cleaft Organization: Feathers To: freebsd-drivers@freebsd.org Date: Sun, 8 Jun 2008 13:40:30 +0100 User-Agent: KMail/1.9.7 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806081340.30132.ken@mthelicon.com> Subject: no -mno-sse3 in kernel build X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 08 Jun 2008 13:18:01 -0000 Hello everyone, Firstly, let me apologise if this message is not in the correct area, but I coulden't find a specific kernel section, so I thought this might be the best place. On my amd64 platform, I currently have built the world with -msse3 extensions enabled in my CFLAGS within /etc/make.conf. (I know the benefits of this are debatable, but I wanted to try and see what happened) I noticed that while the kernel was building (after the world build), there are many options sent that disable the mmx, sse and 3dnow extensions, but they only seem to override up to sse2.. Should there be a -mno-sse3 inserted in there as well? As it stands, I am running on a kernel built with sse3 extensions presumably enabled and nothing has gone bang yet, but that could be just a matter of time with the right load and applications running ;> Ta Ken From owner-freebsd-drivers@FreeBSD.ORG Wed Jun 11 18:49:32 2008 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F4D6106566C; Wed, 11 Jun 2008 18:49:32 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (gate.funkthat.com [69.17.45.168]) by mx1.freebsd.org (Postfix) with ESMTP id C35D98FC14; Wed, 11 Jun 2008 18:49:31 +0000 (UTC) (envelope-from jmg@hydrogen.funkthat.com) Received: from hydrogen.funkthat.com (lrckrpa6uyv4eshi@localhost.funkthat.com [127.0.0.1]) by hydrogen.funkthat.com (8.13.6/8.13.3) with ESMTP id m5BIQXPf062705; Wed, 11 Jun 2008 11:26:33 -0700 (PDT) (envelope-from jmg@hydrogen.funkthat.com) Received: (from jmg@localhost) by hydrogen.funkthat.com (8.13.6/8.13.3/Submit) id m5BIQWLg062704; Wed, 11 Jun 2008 11:26:32 -0700 (PDT) (envelope-from jmg) Date: Wed, 11 Jun 2008 11:26:32 -0700 From: John-Mark Gurney To: Jason Harmening Message-ID: <20080611182632.GT3767@funkthat.com> Mail-Followup-To: Jason Harmening , freebsd-drivers@freebsd.org, freebsd-hackers@freebsd.org References: <200806072341.17041.jason.harmening@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200806072341.17041.jason.harmening@gmail.com> User-Agent: Mutt/1.4.2.1i X-Operating-System: FreeBSD 5.4-RELEASE-p6 i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (hydrogen.funkthat.com [127.0.0.1]); Wed, 11 Jun 2008 11:26:33 -0700 (PDT) Cc: freebsd-hackers@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: bus_dmamem_alloc X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2008 18:49:32 -0000 Jason Harmening wrote this message on Sat, Jun 07, 2008 at 23:41 -0500: > I've written a FreeBSD driver for Conexant CX2388x-based PCI TV capture cards. I have a partial one in P4 that seems to handle data transers fine, but as ATI never gave me the docs for programming the ATSC demodulator, I haven't worked on it in a long time.. > Of course the driver uses busdma to be as machine-independent as possible. > One problem I've encountered is that bus_dmamem_alloc is inadequate for my > needs. The CX2388x only understands 32-bit physical addreses, and the driver This restriction is set up by the dma tag... > has two separate use cases for busdma: > > 1) Data buffers: These buffers may be relatively large (a 640x480 RGB32 video > frame is ~1.2M), and therefore it is desirable that these buffers not be > physically contiguous. > > 2) DMA program buffers: The DMA engine on the CX2388x is controlled by > special-purpose RISC instructions, usually stored in host memory, that > provide information on, among other things, the physical layout of the data > buffers, which enables handling of non-contiguous data buffers. These > programs are rarely more than a few pages in size, so for the sake of > simplicity it is desirable that DMA program buffers be physically contiguous. Why not use the SRAM for this? That's what my driver does... w/ 32k SRAM, it's more than enough for more programs... > For case 1), I malloc(9) the buffers and then feed them to busdma, since on > most machines bus_dmamem_alloc just calls contigmalloc. Use of malloc(9) is > suboptimal as it may result in bounce buffering for non-IOMMU machines with > large amounts of RAM. I prefer to do direct to use DMA as it saves on allocating a buffer in the kernel, and then coping the data from that buffer... > For case 2), I contigmalloc the DMA program buffers in the 32-bit physical > address range and then feed them to busdma. I don't use bus_dmamem_alloc > here because it always allocates the maximum size specified in the > bus_dma_tag_t. Since the driver supports dynamic data buffer allocation and > DMA program generation, DMA program sizes may vary significantly. I > therefore just create the bus_dma_tag_t with the maximum possible size for a > DMA program buffer since I'd prefer not to have to re-create the tag every > time the DMA program size changes. But always allocating the maximum buffer > size would be a huge waste of contiguous memory, so bus_dmamem_alloc is out > of the question here too. At the same time, use of contigmalloc is > suboptimal as it may not be necessary to restrict the allocation to 32-bit > physical addresses on IOMMU-equipped machines. This is something that > bus_dmamem_alloc could take care of, if only it supported a size parameter > (as I believe the NetBSD version does). > > So I have 3 questions: > > 1) Would it be possible to provide a bus_dmamem_alloc overload that takes a > size parameter? We could call it bus_dmamem_alloc_size and have > bus_dmamem_alloc just call bus_dmamem_alloc_size with dmat->maxsize to > preserve source-level compatibility with existing drivers. It would be nice, but hasn't been something someone has gotten around to implementing yet... > 2) Are there currently any serious plans to have bus_dmamem_alloc perform > multi-segment allocations on non-IOMMU machines? It looks like NetBSD does > this by reserving the physical segments and then stitching them together into > a virtually contiguous range. Is something like this feasible for FreeBSD? This would be useful for large allocations, but for now our code works, and most IO isn't that large so it hasn't been a bit issue.. It would be nice though.. :) > 3) Are there currently any serious plans to support IOMMUs on anything besides > Sun machines? The AMD AGP GART, PowerPC 970 DART, and Intel VT-d and AMD > IOMMU all come to mind. I know that one person recently was working on Intel's VT IOMMU and I thought it was close to being committed, but I haven't been following the work... > If any of these ideas sound feasible, I'd be more than willing to help > research/implement/test them. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-drivers@FreeBSD.ORG Thu Jun 12 01:01:27 2008 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8607B1065678 for ; Thu, 12 Jun 2008 01:01:27 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.29]) by mx1.freebsd.org (Postfix) with ESMTP id 35CAA8FC1C for ; Thu, 12 Jun 2008 01:01:26 +0000 (UTC) (envelope-from jason.harmening@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so1701857ywe.13 for ; Wed, 11 Jun 2008 18:01:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:subject:date :user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:content-disposition:message-id; bh=u5lxJbYpV/5kAUPT7Te8CVXo+JMqFlELKbVS4a0qxqo=; b=G3Jw3gHEFETnRyNZkbiJUyBmrSTWRon0LuSkqixrDFPeVS61Nti3rJdrn7BOVjvi8b tdzhfLAYdVIT6ZskEL9EvaAGjKA6iIPXM9ZeFkAwhiZBPt6x08LocC4+56gJBuQAqKxq U1yL0cyk082ZIzmljZT1uXPokGiRMcOl7taUw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:subject:date:user-agent:cc:references:in-reply-to :mime-version:content-type:content-transfer-encoding :content-disposition:message-id; b=E0aiHjqpFoiwWInQPTyNX7ahBB30bxbUtv/9wBqhS2ARNYCT2cb6R5+n9dsUxenafJ iEhDPW9yIYp3/euU4NO9zhy6M2ohEL8iNZLK9668QcCvF6HrGj7wJRR017rZZb3YGjDh cDury74c05emUFBD1vN4ZWn5LCvIVm7BWt1FI= Received: by 10.150.202.3 with SMTP id z3mr1232219ybf.224.1213232482565; Wed, 11 Jun 2008 18:01:22 -0700 (PDT) Received: from ?192.168.0.137? ( [24.174.5.175]) by mx.google.com with ESMTPS id i27sm1051562elf.11.2008.06.11.18.01.14 (version=TLSv1/SSLv3 cipher=RC4-MD5); Wed, 11 Jun 2008 18:01:15 -0700 (PDT) From: Jason Harmening To: John-Mark Gurney Date: Wed, 11 Jun 2008 20:03:42 -0500 User-Agent: KMail/1.9.7 References: <200806072341.17041.jason.harmening@gmail.com> <20080611182632.GT3767@funkthat.com> In-Reply-To: <20080611182632.GT3767@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200806112003.43326.jason.harmening@gmail.com> Cc: freebsd-hackers@freebsd.org, freebsd-drivers@freebsd.org Subject: Re: bus_dmamem_alloc X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Jun 2008 01:01:27 -0000 On Wednesday 11 June 2008, John-Mark Gurney wrote: > > Why not use the SRAM for this? That's what my driver does... w/ 32k > SRAM, it's more than enough for more programs... The DMA programs (or at least a significant chunk of them) will get stored in the SRAM, if there's enough room. That's often the case with just MPEG TS, but once you add analog video there's usually not enough room. > > > For case 1), I malloc(9) the buffers and then feed them to busdma, since > > on most machines bus_dmamem_alloc just calls contigmalloc. Use of > > malloc(9) is suboptimal as it may result in bounce buffering for > > non-IOMMU machines with large amounts of RAM. > > I prefer to do direct to use DMA as it saves on allocating a buffer in > the kernel, and then coping the data from that buffer... The driver actually supports both. Kernel-mode buffers are mmap'ed, so there shouldn't be a copy. Of course user-mode buffers can still bounce... > > 1) Would it be possible to provide a bus_dmamem_alloc overload that > > takes a size parameter? We could call it bus_dmamem_alloc_size and have > > bus_dmamem_alloc just call bus_dmamem_alloc_size with dmat->maxsize to > > preserve source-level compatibility with existing drivers. > > It would be nice, but hasn't been something someone has gotten around to > implementing yet... That's probably my only real complaint here--it seems like dmat->maxsize should just be a restriction, not an allocation unit. I don't want to sound ungrateful--busdma, at an API level at least, is great compared to what other OSes have to offer. > > I know that one person recently was working on Intel's VT IOMMU and I > thought it was close to being committed, but I haven't been following > the work... That'll be awesome when it's ready.