Date: Wed, 31 Dec 2014 16:24:51 +0600 From: info@aknet.kg To: <freebsd-net@freebsd.org> Cc: rizzo@iet.unipi.it Subject: Netmap-Ipfw: eats 90-100% of CPU, is it normal behaviour =?UTF-8?Q?=3F?= Message-ID: <bf59944795d8f4b98b7d9bfbb15ea813@aknet.kg>
next in thread | raw e-mail | index | archive | help
Hello, All !
We tried to use netmap-ipfw in production (as filtering bridge) for
traffic sanity and bandwidth limitation.
And meet a problem. Will be explaned below.
CPU: i5-4690 CPU @ 3.50GHz
RAM: 8GB x 1800Mhz
NET: Intel DA 520 (2 x 10Gbps)
kipfw starts as:
/usr/local/netmap-ipfw/kipfw netmap:ix0 netmap:ix1
ruleset:
00100 allow ip from 192.168.254.0/24 to 192.168.254.0/24
00200 allow ip from any to 192.168.0.0/16 - incoming
(for customers) traffic goes without touching
00400 pipe 665 udp from 192.168.0.0/16 to any dst-port 6881
00500 pipe 666 tcp from 192.168.0.0/16 to any tcpflags syn
00600 deny tcp from table(25) to any dst-port 25
00700 deny tcp from 192.168.0.0/16 to table(26) dst-port 25
00750 allow ip from 192.168.0.0/16 to any - this
rule we have to use (explaned below)
00800 pipe 10 ip from 192.168.0.0/16 to any - main
rule for this bridge
65535 allow ip from any to any
pipes:
# BW for packets with SYN flag and UDP-6881
${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s
${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s
# Outgoing BW for each IP
${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s
table 25 has about 100 IP's
table 26 has about 15 sub-networks
this bridge serves about 25K subscribers with IP's from network:
192.168.0.0/16
current traffic:
netstat -bdh -w1 -I ix1
input ix1 output
packets errs idrops bytes packets errs bytes colls
drops
607K 0 0 753M 452K 0 88M 0
0
603K 0 0 750M 449K 0 87M 0
0
604K 0 0 751M 448K 0 88M 0
0
604K 0 0 747M 452K 0 92M 0
0
all traffic:
netstat -bdh -w1
input (Total) output
packets errs idrops bytes packets errs bytes colls
drops
2M 0 0 1.6G 2M 0 1.6G 0
0
2M 0 0 1.6G 2M 0 1.6G 0
0
current CPU:
CPU 0: 31.1% user, 0.0% nice, 56.1% system, 5.1% interrupt, 7.7%
idle
CPU 1: 0.0% user, 0.0% nice, 0.5% system, 8.2% interrupt, 91.3%
idle
CPU 2: 0.0% user, 0.0% nice, 0.0% system, 4.6% interrupt, 95.4%
idle
CPU 3: 0.0% user, 0.0% nice, 0.5% system, 7.1% interrupt, 92.3%
idle
THE Question:
is it normal for kipfw to eat so much resoures ?
660 root 99 0 873M 325M CPU0 0 272:03 91.46% kipfw
Also, the rule #750 I have to place into ruleset, cos without it kipfw
begins to use all 100%
00750 allow ip from 192.168.0.0/16 to any
00800 pipe 10 ip from 192.168.0.0/16 to any - this rule is the main
for using of this bridge,
it assigns the same outgoing bandwidth for each of IP addresses -
5120Kbit/s (5Mbps)
# BW for packets with SYN flag and UDP-6881
${fw} pipe 665 config mask src-ip 0xffffffff bw 384Kbit/s
${fw} pipe 666 config mask src-ip 0xffffffff bw 64Kbit/s
# Outgoing BW for each IP
${fw} pipe 10 config mask src-ip 0xffffffff bw 5120Kbit/s
With working rule #800 after 30-50 mins kipfw begins to use 100% in top
-PHS and incoming (for users) traffic downs from 750Mbytes/s (about
6Gbit/s) to 330Mbytes/s (2.6Gbit/s), delay increases from 65ms to 250ms
and high percentage of drops.
Is it real limit of using netmap-ipfw ? We can give any additional info
if it will be usefull to expand limits of kipfw.
With regards and happy New Year !
Azamat B. Umurzakov
AkNet ISP
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bf59944795d8f4b98b7d9bfbb15ea813>