Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 20 Jun 2012 23:59:31 -0700
From:      Julian Elischer <julian@freebsd.org>
To:        Sami Halabi <sodynet1@gmail.com>
Cc:        freebsd-net@freebsd.org, "Alexander V. Chernikov" <melifaro@freebsd.org>, freebsd-ipfw@freebsd.org
Subject:   Re: ipfw rules consuming CPU
Message-ID:  <4FE2C653.40805@freebsd.org>
In-Reply-To: <CAEW+ogZhDxkydL9fMUXVdPVfe2AU=UOMg=7TaZKA0tdMxWWNOA@mail.gmail.com>
References:  <CAEW+ogZyzX6Witnx_TN0bhpygpQYb0E8xEPt8HpCFYj6yUeSRA@mail.gmail.com> <4FD3224A.3080700@FreeBSD.org> <CAEW+ogZhDxkydL9fMUXVdPVfe2AU=UOMg=7TaZKA0tdMxWWNOA@mail.gmail.com>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
On 6/9/12 4:19 AM, Sami Halabi wrote:
> Hi,
> all rules togther less than 80 rules....
>
> how tablearg helps this? each ip & pipe (up & down) are unique...
>
> any other advices?

also, make sure that all rules are only evaluate by packets that might 
actually test true..
i.e.

separate out different interfaces and directions to different rules 
using skipto...

for example

skipto 2000 ip from any to any in recv xx0
skipto 3000 ip from any to any out xmit xx0
skipto 4000 ip from any to any in yy0
skipto 5000 ip from any to any out xmit yy0

if yy0 is a 10GB ethernet and there is traffic there, that traffic 
shouldn't be evaluating  the rules that only make sense for xx0.
similarly inwards traveling packets shouldn't have to evaluate 
outwards rules.

May or may not help in your situation. you don't really give enough info.


>
> Sami
>
> On Sat, Jun 9, 2012 at 1:15 PM, Alexander V. Chernikov <melifaro@freebsd.org
>> wrote:
>> On 09.06.2012 01:56, Sami Halabi wrote:
>>
>>> Hi,
>>>
>>> I Manage a FreeBSD server as an edge router&  firewall.
>>>
>>> the setup has 10G interfaces (ixgbe-82599EB) and 1G interfaces(em-82571EB&
>>> bce-BCM5709) connected to 10G/1G switches.
>>>
>>> With the following setup i get higher cpu usage:
>>> bce1-upstream provider with little bandwidth, so i use pipes to limit
>>> users, and subnets
>>> ix0 - Internet Exchange
>>>
>>> some rules.
>>> .
>>> .
>>> .from 4000 starts pipes for specefic ips bandwidth allocations
>>> 04000    6210053001    5845967300616 pipe 1003 ip from 182.46.92.13 to any
>>> out xmit bce1
>>> 04100   41289897537    3064110648124 pipe 1004 ip from any to 182.46.92.13
>>> in recv bce1
>>>
>> You should use pipe tablearg for that. Traversing 4k rules effectively
>> kills all performance.
>>
>>
>>   .
>>> .
>>> .
>>> .7000 is the wider pipeline for the whole block
>>> 07000    9127154724    4651308720315 pipe 1000 ip from  182.46.92.0/24 to
>>> any out xmit bce1
>>> 07100    4837016828     458027989917 pipe 1002 ip from any to
>>> 182.46.92.0/24 in recv bce1
>>> last rule default to accept...
>>>
>>> specefic pipes (1003-...) have limits say between 1-10Mbps, and the wider
>>> pipe (1000 and 1002) has a global limit of 40MBps that should be reached
>>> by
>>> all other non-specefic ips, config like this:
>>> #Wide
>>> ipfw pipe 1000 config bw 40Mbit/s queue 200Kbytes
>>> ipfw pipe 1002 config bw 40Mbit/s queue 200Kbytes
>>> #specefic
>>> ipfw pipe 1003 config bw 9Mbit/s queue 200Kbytes
>>> ipfw pipe 1004 config bw 9Mbit/s queue 200Kbytes
>>> ipfw pipe 1005 config bw 3Mbit/s queue 200Kbytes
>>> ipfw pipe 1006 config bw 3Mbit/s queue 200Kbytes
>>> ipfw pipe 1007 config bw 5Mbit/s queue 200Kbytes
>>> ipfw pipe 1008 config bw 5Mbit/s queue 200Kbytes
>>> ipfw pipe 1009 config bw 10Mbit/s queue 200Kbytes
>>> ipfw pipe 1010 config bw 10Mbit/s queue 200Kbytes
>>>
>>>
>>> with this configuration when i have lots of traffic (3-6GB) going via ix0
>>> (not necessarly the ips described above, lets say to a server in my net ip
>>> 1832.46.93.4 and users behind the Internet Exchange) i see high cpu usage
>>> (70-90%).
>>>
>>> my first test was to: ipfw add 1 allow all from any to any, and cpu usage
>>> drops immediatly to 10-15%.
>>> but that not why i want (i wantto keep thelimits) so I add rule right
>>> before 4000 and the cpu usage drops down to 10-20%:
>>> 03020 1669463072808 1493341413029803 allow ip from any to any via ix0
>>>
>>>
>>> Any advice why this happens? or should it be there in the first place?
>>> I use FreeBSD 8.1-R-p10-amd64.
>>>
>>> Thanks in advance,
>>>
>>>
>> --
>> WBR, Alexander
>>
>
>





Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?4FE2C653.40805>