Date: Tue, 1 Aug 2006 14:27:49 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: Andrew Gallatin <gallatin@cs.duke.edu> Cc: arch@FreeBSD.org, net@FreeBSD.org Subject: Re: Changes in the network interface queueing handoff model Message-ID: <20060801142558.M64452@fledge.watson.org> In-Reply-To: <17615.18793.700752.342809@grasshopper.cs.duke.edu> References: <20060730141642.D16341@fledge.watson.org> <17615.18793.700752.342809@grasshopper.cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 1 Aug 2006, Andrew Gallatin wrote: > > - The ifnet send queue is a separately locked object from the device driver, > > meaning that for a single enqueue/dequeue pair, we pay an extra four lock > > operations (two for insert, two for remove) per packet. > > Going forward, especially now that we support sun4v CoolThreads hardware, > we're going to want to rethink the "single lock" per transmit routine model > that most drivers have. The most expensive operation in transmit routines > is bus_dmamap_load_mbuf_sg(), especially when there is an IOMMU involved > (like on CoolThreads machines) and there is no reason why this needs to be > called with a driver's transmit lock held. I have hard data (from Solaris) > about how much fine grained locking in a 10GbE driver's transmit routine > helps. Right now, with the exception of locking for the ifnet dispatch queue, I believe our ifnet API pretty much leaves decisions about the nature and granularity of synchronization to the device driver author. The ifnet queue is high on my list to address (hence this thread) -- are there any other parts of our device driver framework that stand in the way from a device driver being modified to support greater parallelism in sending? Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060801142558.M64452>