From owner-svn-src-projects@FreeBSD.ORG Wed Mar 11 14:54:18 2015 Return-Path: Delivered-To: svn-src-projects@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 67F345C3; Wed, 11 Mar 2015 14:54:18 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3F508953; Wed, 11 Mar 2015 14:54:18 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3407CB94E; Wed, 11 Mar 2015 10:54:17 -0400 (EDT) From: John Baldwin To: Gleb Smirnoff Subject: Re: svn commit: r277122 - projects/ifnet/sys/dev/msk Date: Wed, 11 Mar 2015 10:08:30 -0400 Message-ID: <44632162.sBlScnaX0r@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <20150306024101.GT17947@FreeBSD.org> References: <201501130902.t0D927NE077024@svn.freebsd.org> <20150114144358.GD15484@FreeBSD.org> <20150306024101.GT17947@FreeBSD.org> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 11 Mar 2015 10:54:17 -0400 (EDT) Cc: svn-src-projects@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-projects@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the src " projects" tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Mar 2015 14:54:18 -0000 On Friday, March 06, 2015 05:41:01 AM Gleb Smirnoff wrote: > On Wed, Jan 14, 2015 at 05:43:58PM +0300, Gleb Smirnoff wrote: > T> On Wed, Jan 14, 2015 at 09:33:04AM -0500, John Baldwin wrote: > T> J> I posted some ideas about how to handle this in a thread several years > T> J> ago on net@ with various alternatives. In that case I was focused on > T> J> buf_ring and I settled on an approach where a draining thread marked the > T> J> queue as "busy" while it was draining it and cleared that flag before > T> J> checking the head of the queue. The enqueue process returned a > T> J> different errno value (EINPROGRESS or some such) if it queued a packet > T> J> into a "busy" queue and the transmit routines were changed to 1) always > T> J> enqueue the packet, and 2) if EINPROGRESS wasn't returned, use a > T> J> blocking mtx_lock and start transmitting. > T> J> > T> J> However, even this model has some downsides in that one thread might be > T> J> stuck transmitting packets queued by other threads and never pop back > T> J> out to userland to service its associated application (a kind of > T> J> starvation of the user side of the thread). Note that the mtx_trylock > T> J> approach has the same problem. It might be nice to have a sort of limit > T> J> on the number of packets a thread is willing to enqueue, but then you > T> J> have the problem of ensuring any packets still on the queue when it hits > T> J> its limit aren't also delayed indefinitely. > T> > T> Thanks, I will try to code that. > > John, can you please look at this patch? It is against projects/ifnet. > > The idea is that if_snd_enqueue() tells us whether we grabbed to queue and > own it or not. If we grabbed it, we go processing it to the end. However, > we keep accounting on how many packets we processed there. If other > producer notices that we processed too much, it will preempt the queue. > > Looks like a design that matches your demands. However, extra code needs > to be put into drivers foo_start() functions, since now we need to disown > the queue if we stop processing it for some reason different to queue getting > empty. I think this patch is not a bad approach. It resembles the last thing I posted to net@ except that you have added the burst length cap which is a nice addition. Of course, this uses a lock to do so which buf_ring tries to avoid. (I also eventually would love to have a way to move the enqueue out of drivers entirely still where there is a callback for "drain this queue" that only gets called in the !EBUSY case. I can't recall if that is compatible with your stacking approach, but it would make it harder for drivers to get this wrong if we can avoid duplicating it N times.) -- John Baldwin