From owner-freebsd-net@FreeBSD.ORG Thu Sep 15 16:19:07 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEE3D16A421; Thu, 15 Sep 2005 16:19:07 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C31C43D5A; Thu, 15 Sep 2005 16:19:06 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id 732B146B98; Thu, 15 Sep 2005 12:19:05 -0400 (EDT) Date: Thu, 15 Sep 2005 17:19:05 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Andre Oppermann In-Reply-To: <4321BD3D.66417FA6@freebsd.org> Message-ID: <20050915171617.D75005@fledge.watson.org> References: <20050908221115.038c3abd.lists@yazzy.org> <004701c5b4df$9207d260$1200a8c0@gsicomp.on.ca> <4320EDDF.6090303@errno.com> <20050909054110.08pqjx9bi884c0sg@mail.bafirst.com> <4321BA08.9060500@errno.com> <4321BD3D.66417FA6@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: net.inet.ip.forwarding and net.inet.ip.fastforwarding X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Sep 2005 16:19:08 -0000 On Fri, 9 Sep 2005, Andre Oppermann wrote: >> 6.0 and 7.x share the same code so the settings are identical. As to >> downside you pay a penalty if the fastforwarding code has to hand the >> packet back to the "slow path". There may also be side effects from >> the run-to-completion model it uses. You should test to decide if the >> feature is worth enabling for your environment. I'm not sure it's had >> much testing (Andre?). > > When activated on a router it gives a very nice speed boost. Process > completion pays off very well here. It has got a lot of testing at > various ISP's on their production routers. For hosts it doesn't really > hurt but is totally pointless. In measurements a couple of years ago, I demonstrated to myself that on several interesting pieces of hardware, running with net.isr.enable=1 resulted in lower latency packet forwarding and processing on 5.x (at the time) than 4.x (at the time). I've not re-measured with recent 7.x/6.x or 4.x on recent hardware. Over the last couple of years, we've shaken out a number of important bugs in local network stack code that tripped up with net.isr.enable, so we're reaching the point where I might start encouraging people to work with it more actively for local (as well as routed) paths. There are still open questions about what models make the most sense, though -- run to completion has some nice latency properties, and also increases the opportunities for parallelism in the network stack. On the other hand, it increases the load born by ithreads, so if your ithread was already maxing out available CPU, you would decrease the amount of work it could do, and on UP it can result in more context switches if you have several active interfaces running out of different ithreads. Many of these questions are the same as the ones we'll be talking about for next generation polling support at the developer summit this fall, so it would make sense to talk about them at the same time. Robert N M Watson