Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 26 Feb 2003 17:51:16 +0100
From:      Thomas Moestl <tmoestl@gmx.net>
To:        "Justin T. Gibbs" <gibbs@scsiguy.com>
Cc:        Maxime Henrion <mux@FreeBSD.org>, src-committers@FreeBSD.org, cvs-src@FreeBSD.org, cvs-all@FreeBSD.org
Subject:   Re: cvs commit: src/sys/alpha/alpha busdma_machdep.c src/sys/alpha/include bus.h src/sys/i386/i386 busdma_machdep.c src/sys/i386/include bus_at386.h bus_pc98.h src/sys/ia64/ia64 busdma_machdep.c src/sys/ia64/include bus.h src/sys/powerpc/include bus.h ...
Message-ID:  <20030226165116.GC631@crow.dom2ip.de>
In-Reply-To: <508522704.1046275118@aslan.scsiguy.com>
References:  <200302260216.h1Q2G6UK033030@repoman.freebsd.org> <20030226124632.GA631@crow.dom2ip.de> <508522704.1046275118@aslan.scsiguy.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 2003/02/26 at 08:58:38 -0700, Justin T. Gibbs wrote:
> > Users of the busdma interface should use realistic sizes and for the
> > maxsize argument to bus_dma_tag_create(), since tag creation can in
> > fact result in bounce buffer allocation (or e.g. DVMA allocation), so
> > excessive sizes  will just suck up resources, and this constant was a
> > useful hint (while the new one might be used for maxsegsz like the
> > old one could be too, but should not be for maxsize).
> 
> Allocating backing store in bus_dma_tag_create() is extremely unwise
> unless the ALLOC_NOW flag is provided.  Consider a tag that is simply
> a placeholder in the hierarchy of tags from root bus to the end device.
> This tag may be for a bridge that can handle any size up to MAXSIZE_32BIT,
> but will never allocate any dmamaps or memory directly against its own
> tag.  Using a large size is, in effect, indicating "no restriction" for
> the size parameter due to this particular device.  If the implementation
> has some other limitation, place it in the root bus's tag and allow
> inheritance to do the rest.
> 
> Tags that have maps created for them should have sane sizes specified
> for them.

Yes, that was what I was meaning to say. However if a parent tag is
meant to impose no restriction above the default at all, the maximum
supported size may as well be used. Maximum sizes specified for parent
tags seem not to be honored by any implementation anyway.

> > This change also breaks the sparc64 IOMMU code, which relied on the
> > old semantics for it's preallocation clamping code.
> 
> I would expect that DVMA allocation has to occur (up to some limit)
> with every bus dmamap that is allocated against a tag.  Is there some
> reason that all allocations cannot occur in this fashion?

Preallocation at map creation time is done, and usually suffices to
not have to do any reallocations at load time at all (which is quite
important for decent performance). However, many busdma users specify
tag sizes that are much larger than any map that they are going to
load, or need many maps which may be as large as the maximum size
indicates, but usually are not, so we are running out of DVMA too
quickly. Therefore, preallocation is clamped to a certain amount of
DVMA, and failures are ignored silently at map creation time. When the
map is to be loaded and an insufficient amount of DVMA is allocated
for the map, reallocation is tried, stealing DVMA from currently
unloaded maps if necessary (and using the current request size as a
better approximation of how much is needed). Unloaded maps keep their
DVMA (unless it's stolen) until they are destroyed.
This seems to work quite well in practice, reallocations at load time
are very rare under normal load; if it proved to be a problem at some
point, additional tuning would likely make it acceptable again.

> Does the Sparc64 implementation even allow blocking for resources
> and doing  a deferred callback so you don't have to allocate for all
> requests?

It does not, currently; this could be added quite easily, though.
The amount of DVMA space that is provided is 128 MB now (this could be
increased at the expense of using more memory for the IOTSB). The
problem is usually over-preallocation as described above, which is
mostly solved by the current implementation , not being unable to hold
all requests active at a time.

	- Thomas

-- 
Thomas Moestl <tmoestl@gmx.net>	http://www.tu-bs.de/~y0015675/
              <tmm@FreeBSD.org>	http://people.FreeBSD.org/~tmm/
PGP fingerprint: 1C97 A604 2BD0 E492 51D0  9C0F 1FE6 4F1D 419C 776C

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe cvs-all" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030226165116.GC631>