Date: Sat, 20 Mar 2010 22:49:53 +0100 From: Ivan Voras <ivoras@freebsd.org> To: freebsd-arch@freebsd.org Cc: freebsd-current@freebsd.org Subject: Re: Increasing MAXPHYS Message-ID: <ho3fu3$kdk$1@dough.gmane.org> In-Reply-To: <4BA53E77.8010208@elischer.org> References: <4BA4E7A9.3070502@FreeBSD.org> <201003201753.o2KHrH5x003946@apollo.backplane.com> <891E2580-8DE3-4B82-81C4-F2C07735A854@samsco.org> <4BA52179.9030903@FreeBSD.org> <4BA532FF.6040407@elischer.org> <ho3d11$abp$1@dough.gmane.org> <4BA53E77.8010208@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Julian Elischer wrote: > Ivan Voras wrote: >> Julian Elischer wrote: >>> Alexander Motin wrote: >>>> Scott Long wrote: >>>>> On Mar 20, 2010, at 11:53 AM, Matthew Dillon wrote: >>>>>> Diminishing returns get hit pretty quickly with larger MAXPHYS >>>>>> values. >>>>>> As long as the I/O can be pipelined the reduced transaction rate >>>>>> becomes less interesting when the transaction rate is less than a >>>>>> certain level. Off the cuff I'd say 2000 tps is a good basis for >>>>>> considering whether it is an issue or not. 256K is actually quite >>>>>> a reasonable value. Even 128K is reasonable. >>>>> I agree completely. I did quite a bit of testing on this in 2008 >>>>> and 2009. >>>>> I even added some hooks into CAM to support this, and I thought >>>>> that I had >>>>> discussed this extensively with Alexander at the time. Guess it >>>>> was yet another >>>>> wasted conversation with him =-( I'll repeat it here for the record. >>> >>> In the Fusion-io driver we find that the limiting factor is not the >>> size of MAXPHYS, but the fact that we can not push more than >>> 170k tps through geom. (in my test machine. I've seen more on some >>> beefier machines), but that is only a limit on small transacrtions, >> >> Do the GEOM threads (g_up, g_down) go into saturation? Effectively all >> IO is serialized through them. > > basically.. > > You can get better throughput by using TSC for timing because the geom > and devstat code does a bit of timing.. Geom can be told to turn off > it's timing but devstat can't. The 170 ktps is with TSC as timer, > and geom timing turned off. I see. I just ran randomio on a gzero device and with 10 userland threads (this is a slow 2xquad machine) I get g_up and g_down saturated fast with ~~ 120 ktps. Randomio uses gettimeofday() for measurements. Hmm, it looks like it could be easy to spawn more g_* threads (and, barring specific class behaviour, it has a fair chance of working out of the box) but the incoming queue will need to also be broken up for greater effect.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ho3fu3$kdk$1>