From owner-freebsd-current@FreeBSD.ORG Wed May 5 14:36:33 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E261F16A4CE for ; Wed, 5 May 2004 14:36:33 -0700 (PDT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C098843DCD for ; Wed, 5 May 2004 14:36:14 -0700 (PDT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i45LYk5F060969; Wed, 5 May 2004 17:34:46 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i45LYjBT060966; Wed, 5 May 2004 17:34:46 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Wed, 5 May 2004 17:34:45 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Gerrit Nagelhout In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org Subject: RE: 4.7 vs 5.2.1 SMP/UP bridging performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2004 21:36:34 -0000 On Tue, 4 May 2004, Gerrit Nagelhout wrote: > Thanks for all the responses so far. WITNESS is definitely disabled, as > are the other INVARIANTS. I had a look through the netperf patches, but > I don't think they will affect bridging very much. They seem be > directed more towards the socket layer and above. The reason to run with these patches is that, without them, it's not safe to run without Giant over the lower half of the network stack, so all the network interrupt threads are running with Giant otherwise. Also, in that patchset, network processing to completion runs in the ithread rather than the netisr, so you get lower latency and more parallelism even for bridging. Another known issue is the latency from interrupt generation to interrupt task execution in the interrupt thread. > I still think that one of the bigger bottlenecks is the cost of all the > mutexes in SMP mode, and some of the new bus_dma and mbuf code that was > introduced. > > With previous platforms I have worked on (vxWorks), we had similar > issues, and ended up pushing buckets of packets through the data path, > so each mutex was only taken once for every 10-100 packets. > > Also, polling is currently done by only one CPU at a time. If this were > changed to have multiple threads poll multiple devices at the same time, > the performance should become much better. Getting polling and SMP to play nicely would be a very good thing, but isn't something I currently have the bandwidth to work on. Don't suppose we could interest you in that? :-) Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research