Date: Mon, 15 Mar 2004 22:20:29 -0500 From: Bosko Milekic <bmilekic@technokratis.com> To: freebsd-arch@freebsd.org Subject: mbuma: network buffers & UMA Message-ID: <20040316032029.GA3735@technokratis.com>
next in thread | raw e-mail | index | archive | help
The post below has received exactly zero attention on the
freebsd-current mailing list. I'm assuming this is because there are
other exciting threads going on there right now, such as style debates
and lovely feature requests. :-)
I would like to commit this RSN. I need feedback from the Release
Engineering people on how close we are to a RELEASE and if it's OK for
me to break netstat -m stats for a few weeks or, if not, what the
appropriate course of action is.
Fellow earthlings,
I would like to request some testing on diverse architectures of the
following patch (~130K):
http://people.freebsd.org/~bmilekic/code/mbuma-v2.diff
In short, the above accomplishes several things and is the result of
one of my now mature p4 repositories which I have been able to again
address following a generous hardware donation.
This basically gets rid of the existing mbuf allocator and replaces
it with some routines which get hooked on top of UMA, the existing
general-purpose SMP-friendly allocator in -CURRENT, after adding
some extensions to UMA which hopefully make allocations faster than
they would be without them.
The current list of implications (whether good or bad is for you to
decide):
- NMBCLUSTERS, as a compile-time option, is now gone. The
kern.ipc.nmbclusters tunable and sysctl still exist and are both
read+write. Currently, the sysctl does not effect anything but
will soon be made to allow for runtime (dynamic) modification of
the maximum number of mbuf clusters allocatable cap. If you set
the tunable to zero, though, the number of clusters allocatable is
unbounded and the system will scale according to demand. Be
careful... leaving these unbounded may not be what you really
want.
- Memory allocated to network buffers can (and now will) be
reclaimed by UMA when required and, for example, after a large
network spike.
- Current on-par performance with the existing allocator. Soon some
modifications to UMA that improve performance drastically in the
common case may change that, though - for the better.
- For developers: M_WAIT behavior now is the same as for all other
UMA allocations, and is no longer special for network buffers.
- Currently netstat(1) mbuf stats ('netstat -m') are broken and
instead the following is displayed:
fermat% netstat -m
Mbuf statistics are currently unavailable via netstat(1), see
UPDATING.
This is only a TEMPORARY measure in -CURRENT.
For current mbuf and cluster allocation stats, see sysctl vm.zone
I don't plan to fix them prior to doing some other work (which may
change the way stats work, anyway) in UMA. Hopefully this won't
be too much of a problem for most of you.
- So far has been very stable on my dual Athlon and a friend's x86
UP machine (I think it's UP, anyway). Some other architectures,
particular 64-bit ones, would be worth testing.
- Some things (e.g., dev/nsp) I just noticed might be broken by the
above changes due to slight time-delay between the vendor p4 branch
and the CVS checked out repo I used to generate the final diff.
If something you're using is broken by the diff, it is only broken
slightly, and you'll notice it immediately during kernel compile,
so you can fix it (it's generally just changing some includes) or
Email it to me and I'll send you a custom diff. This only
applies to the current diff posted above and will not be
committed.
That's all I could think of right now. I would like to commit this
soon as I have other things in the pipeline that build on top of it.
I should also mention that there are various other implications in
addition to the above but are more relevant to the developer than the
user, so they'll probably appear in a really large commit log.
Reviews and constructive criticism is both welcomed and appreciated.
Please leave me in the To: line - although I am subscribed to
-current, I'd like to see feedback in my main box.
Regards,
--
Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org
TECHNOkRATIS Consulting Services * http://www.technokratis.com/
"It is impossible for anyone to begin to learn what he believes
he already knows." -- Epictetus (c.a.d. 55-c135)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040316032029.GA3735>