Date: Wed, 02 Aug 2006 12:27:39 +0200 From: Ian FREISLICH <if@hetzner.co.za> To: Luigi Rizzo <rizzo@icir.org> Cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw performance and random musings. Message-ID: <E1G8Dwx-000HwP-Am@hetzner.co.za> In-Reply-To: Message from Luigi Rizzo <rizzo@icir.org> of "Mon, 31 Jul 2006 07:21:45 MST." <20060731072145.A77050@xorpc.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Luigi Rizzo wrote: > On Mon, Jul 31, 2006 at 02:15:56PM +0200, Ian FREISLICH wrote: > > Hi > > > > I was wondering if anyone here had any ideas for improving the > > performance (packet rate) of ipfw. > > > > I have about 500 interfaces on my firewall and I need to match and > > filter packets on a per interface basis. > > > > I've found that while the server can move in excess of 360kpps > > bewteen arbitrary interfaces using about 5% CPU, if I turn on the > > firewall, my average packet rate falls off to about 60kpps on a UP > > system and 90kpps on a SMP system. The maximum rate I can forward > > packets with ipfw enabled is 120kpps and that is with 1 rule allowing > > this is a very strange number as it (120kpps with ipfw enabled) > is the performance i got in 2002 with a 750 MHz machine. hrmph. This is a dual 2.8GHz Xeon. SMP does not pessimise performance. It would be nice to get near wire speed filtering. > This was on 4.X - haven't checked recently on 6.x, there might > be some issue that has been introduced. > If you have a chance to try a 4.11 kernel (even a picobsd one) > on the same hardware it would be good to see some numbers. I'll see if I can get some numbers on an older version. I'm running -CURRENT on this box for various reasons. I won't be able to perform the test on this hardware. > > Perhaps these are 2 easy wins: > > > > 1. Change the order of the case statements in ipfw_chk() to move > > more frequently used items to the top. The options seem to > > that has no impact in a sane compiler - the switch() is compiled > as a jump table and i don't see how gcc could do differently > given the small range of opcodes (6-8 bits). Ok, I didn't notice that the opcodes were an enum and I guess it makes sense that the compiler to optimise that into an indirect jump. > > 2. Caching of ifp->if_index in the rule 'microinstructions' to > > remove the need for a strncmp to match interface names. Might > > be tricky if interfaces are destroyed and recreated without > > invalidating this cache. > > this is also a minor optimization - a strcmp is not that bad, > i think it is far worse to have to scan a long list of names. Maybe I'll stick an inlined version in and see if that changes things. I can also give the ifp->if_index cache a go. Since I need to virualise the firewall, I need a set of rules for each interface. I can't think of another way of sharing the firewall beween a few hundred customers than by doing this: 2 skipto 65000 ip from any to me 2 skipto 65000 ip from me to any 10 skipto 100 ip from any to any via vlan1 11 skipto 200 ip from any to any via vlan2 ... 99 deny ip from any to any #Rules for vlan1 100 ... ... 199 deny ip from any to any #Rules for vlan2 200 ... ... 299 deny ip from any to any #Rules for me 65000... So I land up doing lots of interface name comparisons, which may be an atypical work load. > > Then, state is not interface aware. I have used this effect to > > yes i agree that state is a bit limited, we only use the 5-tuple > but maybe we would need more info - the problem is, what more ? > do we always want an interface name or other metadata ? Interface would be a good start: allow tcp from any to any in recv vlan1 keep-state deny ip from any to any out via vlan1 allow tcp from any to any in recv vlan2 keep-state deny ip from any to any out xmit vlan2 check-state Will allow anything connected to vlan1 to get it's TCP packets to something on vlan2 and vice versa. > this is a design problem in the first place. Could you please have a look at the patch (kern/97951). Just storing the interface if_index with the state would make me a very happy man. Even if it's with a sysctl 'net.inet.ip.fw.interface_state'. Ian -- Ian Freislich
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1G8Dwx-000HwP-Am>