From owner-freebsd-bugs@FreeBSD.ORG Thu Sep 28 17:40:33 2006 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7782916A595 for ; Thu, 28 Sep 2006 17:40:33 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2818443D45 for ; Thu, 28 Sep 2006 17:40:33 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k8SHeXO0068401 for ; Thu, 28 Sep 2006 17:40:33 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k8SHeWoM068400; Thu, 28 Sep 2006 17:40:33 GMT (envelope-from gnats) Date: Thu, 28 Sep 2006 17:40:33 GMT Message-Id: <200609281740.k8SHeWoM068400@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org From: Gleb Smirnoff Cc: Subject: Re: kern/100519: Suboptimal network polling X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Gleb Smirnoff List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Sep 2006 17:40:33 -0000 The following reply was made to PR kern/100519; it has been noted by GNATS. From: Gleb Smirnoff To: freebsd-gnats-submit@FreeBSD.org Cc: Subject: Re: kern/100519: Suboptimal network polling Date: Thu, 28 Sep 2006 21:37:43 +0400 attach this to PR, so that it won't be lost in my mailbox. ----- Forwarded message from Arthur Hartwig ----- >Hmm, interesting. Have you done any profiling? G'day Gleb, My work was then largely concerned with fast forwarding performance between two em interfaces on FreeBSD 6.0 and in particular with trying to get some performance improvement at high rates by using additional CPUs. I did some timing using the tsc and found that the em driver in polling mode was using roughly roughly half the time spent in ether_input() to the point of putting a frame on the output queue. So I thought if I ran the polling code on one CPU and ether_input() on another CPU I would get an observable improvement. This turned out to be the case. At some stage I noticed that vmstat showed lots of interrupts even though I was using polling mode. I noticed v_intr was incremented in swi_sched() and intr_execute_handlers(). Another system not using polling mode didn't show the same high interrupt rate. After making the described change I found the interrupt rate reported by vmstat dropped significantly. The change didn't result in a significant performance improvement but clearly reduce the overhead in the polling thread, thus making more CPU time available to the system. Arthur I found interrupt driven fast forwarding performed significantly worse on a dual CPU system than on a single CPU system. Some profiling showed a lot of contention for the routing table locks, sometimes with significant delays between a thread attempting to acquire a lock and its actual acquisition. Changing to polling mode seemed as if would help reduce lock contention and so improve performance. A number of tweaks combined to get noloss fast forwarding performance between two GigE interfaces on a dual CPU system up to about what it was on a single CPU interrupt driven system Arthur ----- End forwarded message -----