From owner-freebsd-hackers Mon Jan 22 12:26:53 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from mass.dis.org (mass.dis.org [216.240.45.41]) by hub.freebsd.org (Postfix) with ESMTP id 95C6A37B400 for ; Mon, 22 Jan 2001 12:26:34 -0800 (PST) Received: from mass.dis.org (localhost [127.0.0.1]) by mass.dis.org (8.11.1/8.11.1) with ESMTP id f0MKfY001170; Mon, 22 Jan 2001 12:41:34 -0800 (PST) (envelope-from msmith@mass.dis.org) Message-Id: <200101222041.f0MKfY001170@mass.dis.org> X-Mailer: exmh version 2.1.1 10/15/1999 To: Robert Lipe Cc: Alfred Perlstein , freebsd-hackers@FreeBSD.ORG Subject: Re: contigmalloc, M_WAITOK, & leaks. In-reply-to: Your message of "Mon, 22 Jan 2001 13:26:47 CST." <20010122132647.I10504@rjlhome.sco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 22 Jan 2001 12:41:34 -0800 From: Mike Smith Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Aaaah. There's a hint. Yes, by the time we get into trouble, I've > allocated (and freed) several thousand chunks that are < PAGE_SIZE/2. > This is happening in periods of a few dozen. (I don't know the number, > but I think it's 48 or 64.) The test program allocates a few dozen tx > buffers, fills them up, then the tx completion handler releases them. > There are never very many of these outstanding at any time. All the > allocs (in this specific case) are always the same size, and there are > no intervening calls to contigmalloc. This should stack the deck pretty > well for fragmentation. Several things to note. - Calling contigmalloc is wrong. Use bus_dmamem_alloc() - Calling an allocator is generally wrong; keep a freelist and just recycle your buffers. If you're not sure about how many you need, then by all means error out in the "freelist empty" case and allocate a few more (up to some sane limit). > So if I sandbagged the allocs to be *larger* the KMA would behave more > consistently with what I'd expect becuase it would then reclaim? I > didn't see that one coming. :-) No, this isn't going to help much. It's possible that contigfree is actually doing the wrong thing, but regardless, what you're doing is a recipie for frustration. If your allocations are always guaranteed to be much smaller than PAGE_SIZE, making an individual allocation per object is also wasteful (since contigmalloc will have to give you a whole page...). You should definitely be using bus_dmamem_alloc, and allocate a cluster of objects. eg. if your object is (say) 20 bytes in size, you should round up to 32, then allocate (PAGE_SIZE / 32) objects and push them all onto your freelist. (This requires tracking the base addresses of your clusters, but it's much more efficient.) > Is there an s/g memory interface in FreeBSD? This was my first choice, > but since I couldn't find a set of functions to build a list of buffers > that satisfied a set of constraints, I fell back to contigmalloc to get > things off the ground. I'd be delighted to use an interface to get me > an s/g list with a given set of constraints that pointed to a list of > buffers with a given set of constraints. Yes; this is where the busdma stuff comes in. Use bus_dma_tag_create to create a DMA tag which defines the costraints applicable to your DMA-able memory. Then you can either use bus_dmamem_alloc/bus_dmamem_free to allocate conforming memory, or just take memory in the kernel's address space. In either case, you use bus_dmamap_load to build the scatter/ gather list (and possibly perform other setup work), bus_dmamap_unload to tear down any of the setup work, and bus_dmamem_sync to perform possibly-required bounce buffering, bus scatter/gather control, etc. > UDI (www.projectudi.org) allows a driver writer to specify a very rich > set of constraints on DMA objects. I'm implementing the piece of the > UDI environment that provides this to the drivers. So while I know > that the test program and driver I'm using right now don't *really* > need contigmalloc, we'll eventually need the ability to ensure that DMA > buffers for 32-bit PCI cards are in the bottom 32 bits, (or perhaps > even not the bottom if you have intervening bus bridges) that stupid > hardware providing 24 bits of addressing but bolted to a PCI controller > gets allocated in the bottom 16Mb, etc. All this is already provided for with the above. I hope this helps, feel free to ask more questions if you need more answers. 8) -- ... every activity meets with opposition, everyone who acts has his rivals and unfortunately opponents also. But not because people want to be opponents, rather because the tasks and relationships force people to take different points of view. [Dr. Fritz Todt] V I C T O R Y N O T V E N G E A N C E To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message