Date: Wed, 26 Feb 2003 10:41:04 -0700 From: "Justin T. Gibbs" <gibbs@scsiguy.com> To: Thomas Moestl <tmoestl@gmx.net> Cc: Maxime Henrion <mux@freebsd.org>, src-committers@freebsd.org, cvs-src@freebsd.org, cvs-all@freebsd.org Subject: Re: cvs commit: src/sys/alpha/alpha busdma_machdep.c src/sys/alpha/include bus.h src/sys/i386/i386 busdma_machdep.c src/sys/i386/include bus_at386.h bus_pc98.h src/sys/ia64/ia64 busdma_machdep.c src/sys/ia64/include bus.h src/sys/powerpc/include bus.h ... Message-ID: <718290000.1046281263@aslan.btc.adaptec.com> In-Reply-To: <20030226165116.GC631@crow.dom2ip.de> References: <200302260216.h1Q2G6UK033030@repoman.freebsd.org> <20030226124632.GA631@crow.dom2ip.de> <508522704.1046275118@aslan.scsiguy.com> <20030226165116.GC631@crow.dom2ip.de>
next in thread | previous in thread | raw e-mail | index | archive | help
> Yes, that was what I was meaning to say. However if a parent tag is > meant to impose no restriction above the default at all, the maximum > supported size may as well be used. I disagree. The tags for bridges, etc. should document the capabilities of the hardware, not the underlying implementation. For example, if the hardware can support a 32bit count it should say so. The implementer of the tag shouldn't need to know if the limit they are imposing is greater or less than the implementation's limit. That is why using BUS_SPACE_MAXSIZE is rarely correct if used to describe a tag for real hardware. > Maximum sizes specified for parent tags seem not to be honored by any > implementation anyway. That, just like the BUS_SPACE_MAXSIZE issue, is a bug in the implementation. Don't use bugs to justify other bugs. >> I would expect that DVMA allocation has to occur (up to some limit) >> with every bus dmamap that is allocated against a tag. Is there some >> reason that all allocations cannot occur in this fashion? > > Preallocation at map creation time is done, and usually suffices to > not have to do any reallocations at load time at all (which is quite > important for decent performance). Okay. > However, many busdma users specify > tag sizes that are much larger than any map that they are going to > load, or need many maps which may be as large as the maximum size > indicates, but usually are not, so we are running out of DVMA too > quickly. Therefore, preallocation is clamped to a certain amount of > DVMA, and failures are ignored silently at map creation time. When the > map is to be loaded and an insufficient amount of DVMA is allocated > for the map, reallocation is tried, stealing DVMA from currently > unloaded maps if necessary (and using the current request size as a > better approximation of how much is needed). Unloaded maps keep their > DVMA (unless it's stolen) until they are destroyed. > This seems to work quite well in practice, reallocations at load time > are very rare under normal load; if it proved to be a problem at some > point, additional tuning would likely make it acceptable again. This scheme seems somewhat similar to the x86 bounce scheme with a few, perhaps unecessary?, complications. Why keep DVMA associated with inactive maps at all? Why not just always allocate the DVMA from a common pool at load time and avoid the complicated "steeling scheme"? If you run out of DVMA for a particular load operation, you just queue up in line for the pool and satisfy, in order, the requestors as mappings are unloaded. If you determine that you are constantly running out of space, you might allocate more DVMA and add it to the pool. The only wrinkle in this is that the implementation should always reserve enough resources to handle one mapping against each tag concurrently so as to avoid deadlock in situations where an operation requires more than one mapping to occur to complete a transaction and then release resources. The x86 bounce scheme uses a bounded pool of bounce buffers and blocks if necessary. As you say, the amount of resources required to get decent performance is orders of magnitude less than that required by a brute force approach, but it does require either the ability to block or callback in the case of temporary resource shortage. This was the main reason I diverged from the NetBSD API. With only 512K of bounce buffers in a 64MB system, the system could handle 4 ISA SCSI controllers rarely blocking. The NetBSD implementation allocated all space up front which, assuming a MAXPHYS of 64K meant about 1MB of bounce space per-controller. -- Justin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe cvs-all" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?718290000.1046281263>