Date: Mon, 22 Mar 2010 15:07:56 -0400 From: Alexander Sack <pisymbol@gmail.com> To: "M. Warner Losh" <imp@bsdimp.com> Cc: freebsd-arch@freebsd.org, freebsd-current@freebsd.org, mav@freebsd.org, ivoras@freebsd.org Subject: Re: Increasing MAXPHYS Message-ID: <3c0b01821003221207p4e4eecabqb4f448813bf5a8a8@mail.gmail.com> In-Reply-To: <20100322.124505.787670930858384500.imp@bsdimp.com> References: <1269134585.00231959.1269122405@10.7.7.3> <4BA6279E.3010201@FreeBSD.org> <D9D66012-16FD-4FB6-AB6A-9A8D17727901@samsco.org> <20100322.124505.787670930858384500.imp@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 22, 2010 at 2:45 PM, M. Warner Losh <imp@bsdimp.com> wrote: > In message: <D9D66012-16FD-4FB6-AB6A-9A8D17727901@samsco.org> > =A0 =A0 =A0 =A0 =A0 =A0Scott Long <scottl@samsco.org> writes: > : I'd like to go in the opposite direction. =A0The queue-dispatch-queue > : model of GEOM is elegant and easy to extend, but very wasteful for > : the simple case, where the simple case is one or two simple > : partition transforms (mbr, bsdlabel) and/or a simple stripe/mirror > : transform. =A0None of these need a dedicated dispatch context in order > : to operate. =A0What I'd like to explore is compiling the GEOM stack at > : creation time into a linear array of operations that happen without > : a g_down/g_up context switch. =A0As providers and consumers taste each > : other and build a stack, that stack gets compiled into a graph, and > : that graph gets executed directly from the calling context, both > : from the dev_strategy() side on the top and the bio_done() on the > : bottom. =A0GEOM classes that need a detached context can mark > : themselves as such, doing so will prevent a graph from being > : created, and the current dispatch model will be retained. > > I have a few things to say on this. > > First, I've done similar things at past companies for systems that are > similar to geom's queueing environment. =A0It is possible to convert the > queueing nodes in the graph to filtering nodes in the graph. =A0Another > way to look at this is to say you're implementing direct dispatch into > geom's stack. =A0This can be both good and bad, but should reduce > latency a lot. > > One problem that I see is that you are calling into the driver from a > different set of contexts. =A0The queueing stuff was there to protect > the driver from LoRs due to its routines being called from many > different contexts, sometimes with other locks held (fact of life > often in the kernel). > > So this certainly is something worth exploring, especially if we have > optimized paths for up/down for certain geom classes while still > allowing the current robust, but slow, paths for the more complicated > nodes in the tree. =A0It remains to be see if there's going to be issues > around locking order, but we've hit that with both geom and ifnet in > the past, so caution (eg, running with WITNESS turned on early and > often) is advised. Am I going crazy or does this sound a lot like Sun/SVR's stream based network stack? (design and problems, stream stack locking was notoriously tricky for the exact issue mentioned above, different running contexts with different locking granularity/requirements). -aps
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3c0b01821003221207p4e4eecabqb4f448813bf5a8a8>