Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 10 Oct 2002 11:18:42 +1000
From:      Christopher Smith <csmith@its.uq.edu.au>
To:        Luigi Rizzo <rizzo@icir.org>
Cc:        <hardware@freebsd.org>, <net@freebsd.org>
Subject:   Re: High interrupt load on firewalls
Message-ID:  <B9CB1292.30FD3%csmith@its.uq.edu.au>
In-Reply-To: <20021009170002.A54675@carp.icir.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On 10/10/02 10:00 AM, "Luigi Rizzo" <rizzo@icir.org> wrote:

> On Thu, Oct 10, 2002 at 09:38:40AM +1000, Christopher Smith wrote:
> ...
>> With the 2.4GHz 2650 we have currently, er, "borrowed" to do some testing
>> with, the load is down to 35% or so (highest I've seen it is 40%) and the
>> packet loss is less than 1 packet every 5 or 10 minutes.
> 
> quite reasonable then, i wouldn't spend too much time in trying to
> improve things, engineer's time is way more expensive than a new
> box -- it adds up easily.

True.  We need to redo the whole ruleset though, simply for manageability
reasons.  May as well do some performance optimising at the same time.

>> The existing firewall ruleset is not particularly optimised at all, and
>> rewriting the whole thing is an ongoing project of mine.  I might up the
>> priority of this, given your comments.  I get the impression that what
>> appears from top to be a high interrupt load may simply be a high "system"
>> load from ipfilter processing the packets.  Is this correct ?  Is there any
>> (reasonably easy) way of checking ?
> 
> you might have both problems -- the network stack operates as a soft
> interrupt, whether this is accounted as intr or system activity i
> am not sure.

I tried a quick and dirty way by just dropping the whole ruleset for ~10
seconds.  The interrupt load (in top) was about a third of "normal" for that
time period.

So, it appears the problem is definitely in the ruleset.

>> What tools do you you use to test and measure performance for things like
>> this ?
> 
> nothing special, i have a trivial c program which loops around a sendto()
> trying to push udp packets out as fast as possible, i disable icmp replies
> with
> net.inet.udp.blackhole=1

Ok, so any of the network benching products that can spit out a stream of
UDP traffic should suffice ?

> and then measure the card stats with
> 
> ns -i -w 1 -d
> 
> (ns is picobsd's version of netstat, in /usr/src/release/picobsd/tinyware/ns)
> Be warned that some cards update the stats everi 1 or 2 seconds, so you might
> have to up the refresh time from 1 second ('-w 1' above) to 2 or more seconds
> ( -w 2 ...)

Ok.  Will normal netstat do ?  I tried it on one of our machines and got
these results:

mr2fw2# netstat -w1 -i -d
            input        (Total)           output
   packets  errs      bytes    packets  errs      bytes colls drops
     39518     0   14415117      39509     0   28791210     0     0
     50231     0   18969196      50117     0   37782558     0     0
     50494     0   17912692      50212     0   35471676     0     0
     53685     0   20592345      53547     0   40987500     0     0
     44862     0   17009437      44897     0   33974566     0     0

mr2fw2# netstat -w10 -i -d
            input        (Total)           output
   packets  errs      bytes    packets  errs      bytes colls drops
    346558     0  112637886     346876     0  224797074     0     0
    392474     0  132204783     392437     0  263929848     0     0
    431339     0  139024315     431086     0  277224060     0     0
    428132     0  135837843     427895     0  271022908     0     0
    390021     0  131117003     389502     0  261278610     0     0

This only seems to indicate ca. 80kpps, which doesn't seem to agree with the
numbers I see in 'systat -ip'.  Is there a counter rolling over somewhere ?

>> Also, are there any cards/chipsets you would recommend for firewalling high
>> speed links like this ?
> 
> no idea, i have only tried the intel pro/1000 which seems to work
> reasonably well for what i have done so far (which is very little,
> basically speed testing and porting the polling support to it),
> but i have not tried any other GigE card.

Ok then.  Apparently there's a machine coming in soon with a Broadcom
5700-based 3Com card in it.  Might try and purloin that for a day or two to
do some experimenting on.

-- 
+- Christopher Smith, Systems Administrator ------------------------------+
|  Server & Security Group, Information Technology Services               |
|  The University of Queensland, Brisbane, Australia, 4072                |
+- Ph +61 7 3365 4046 | email csmith@its.uq.edu.au | Fax +61 7 3365 4065 -+



To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hardware" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B9CB1292.30FD3%csmith>