From owner-freebsd-current@FreeBSD.ORG Mon Nov 5 16:36:50 2012 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 755814F3; Mon, 5 Nov 2012 16:36:50 +0000 (UTC) (envelope-from luigi@onelab2.iet.unipi.it) Received: from onelab2.iet.unipi.it (onelab2.iet.unipi.it [131.114.59.238]) by mx1.freebsd.org (Postfix) with ESMTP id 249FB8FC08; Mon, 5 Nov 2012 16:36:49 +0000 (UTC) Received: by onelab2.iet.unipi.it (Postfix, from userid 275) id 642CE7300A; Mon, 5 Nov 2012 17:57:48 +0100 (CET) Date: Mon, 5 Nov 2012 17:57:48 +0100 From: Luigi Rizzo To: Joe Holden Subject: Re: Dynamic Ticks/HZ Message-ID: <20121105165748.GA13098@onelab2.iet.unipi.it> References: <509758B8.1000409@rewt.org.uk> <50975F6F.6010907@rewt.org.uk> <5097898C.9080109@rewt.org.uk> <20121105163654.GA12870@onelab2.iet.unipi.it> <5097E880.8010001@rewt.org.uk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5097E880.8010001@rewt.org.uk> User-Agent: Mutt/1.4.2.3i Cc: Davide Italiano , Ryan Stone , FreeBSD Current X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 16:36:50 -0000 On Mon, Nov 05, 2012 at 04:25:36PM +0000, Joe Holden wrote: > Luigi Rizzo wrote: > >On Mon, Nov 05, 2012 at 08:11:41AM -0500, Ryan Stone wrote: > >>On Mon, Nov 5, 2012 at 4:40 AM, Joe Holden wrote: > >> > >>>doh, running kernel wasn't as GENERIC as I thought it was, looks like > >>>device polling not only breaks dynamic ticks but also reduces rx ability > >>>significantly, exactly 150,000 pps per 1000hz on igb versus 650,000 > >>>without > >>> > >>>Is this a known issue? (and if device polling isn't as useful as it once > >>>was, should it be removed?) > >>> > >>Device polling on modern multiqueue NICs isn't very useful because you're > >>limited to a single thread for handling packets. I have a patch that > >>fixes > >>this that I've let fall by the wayside. > > > >the 150,000 is result of the combination of the default value of > >sysctl kern.polling.burst_max and kern.polling.idle_poll=0 > >(i think this is the default value for the latter). > > > >The 150 was sized for the peak pps on a 100Mbit/s interface, > >back in 2001. You should at least be able to raise the number > >and see what kind of throughput you can achieve. > > > >This said, modern nics also have interrupt moderation so you > >don't really need polling. > > > >cheers > >luigi > Hi Luigi, > > This makes sense, am I likely to achieve better throughput (in the > forwarding path at this point) with netisr rather than polling, > especially as mentioned above the igb does indeed have multiple queues > for rx? at 1Gbit/s you probably don't need multiqueue (I am actually surpised you can only do 650kpps, but perhaps because you are using ipfw and not just doing plain forwarding ?) > On another note, is netmap usable in the forwarding context at all as it > is rather awesome It depends on what you need to do. If you have a v4/v6 router you won't see any advantage (at the moment; there is some work in the pipeline but probably it won't be available before spring). If you just need to implement a firewall to protect the internal network then it is another story and you can use the ipfw on netmap that I posted in august. cheers luigi