Date: Thu, 20 Aug 2009 01:20:20 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Michael Spratt <mike@magicislandtechnologies.com> Cc: freebsd-ipfw@freebsd.org, rizzo@icir.org Subject: Re: out xmit (demux) pipe bw accounting Message-ID: <20090819224032.B90928@sola.nimnet.asn.au> In-Reply-To: <4A8BC140.8050203@magicislandtechnologies.com> References: <4A8969FF.2070601@magicislandtechnologies.com> <F8A438EB-5A73-4E52-AC35-DE1BD5476F1A@mac.com> <4A8BC140.8050203@magicislandtechnologies.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 19 Aug 2009, Michael Spratt wrote: > Chuck Swiger wrote: > > Hi-- > > > > On Aug 17, 2009, at 7:32 AM, Michael Spratt wrote: > > > Could not find answer to following question. > > > > > > Given : > > > # pipe 1 config bw 1Mbit/sec > > > # ipfw 1000 add pipe 1 ip from any to any out xmit em0 > > > > > > Will the bandwidth limit include layer 2 Ethernet headers or will only > > > the IP datagram itself be included in the accounting mechanism? > > > > IPFW normally processes traffic at layer-3, but you can toggle > > net.link.ether.ipfw sysctl to on, in which case the above rule would also > > be run against layer-2 packets including the 802.3 ethernet header. See > > the diagram in "PACKET FLOW" of "man ipfw".... > > > > Regards, > > Dear Chuck thanks for your response. I am not sure though that my question is > answered. Yes switching net.link.ether.ipfw->1 does cause processing against > layer 2 as indicated in the man ipfw diagram and the command > # sysctl -d net.link.ether.ipfw Yields the description net.link.ether.ipfw: > Pass ether pkts through firewall > > Setting this sysctl variable to 0 does disengage the processing of Ethernet > frames and corresponding IP payloads according to the rule set specified and > visa-versa. > > I do not see anything indicating this sysctl variable affects the way > dummynet does bit counting as it enforces bw limits on a pipe or queue. That's right, dummynet pipes and queues work at the IP packet level. > Will the ruleset I provided above impliment the 1Mbit/s limit at layer 2 > including the Ethernet frames in the 1Mbit/s budget or (VS.) Only layer 3 IP > datagrams (and their coresponding layer 4 payloads) are incrementing the > counters responsible for counting against the 1Mbit/s limit I think only IP packets are queued in dummynet pipes, and it's IP packet throughput rates being managed. Check out Luigi's site at http://info.iet.unipi.it/~luigi/dummynet/ There's a couple of really good .pdfs listed at the bottom describing the porting of ipfw/dummynet to Linux, with lots of detail background. It'd seem unusual if the 1Mbit/s limit you've presumably been allocated from 'higher up' would include ethernet headers as well as IP traffic. And even if it did, it's not so hard to calculate what factor to use to keep the rate below 1Mbit/s at ethernet layer, if they count that way. > Simplification of question: > Given: > # pipe 1 config bw 1Mbit/sec > # ipfw 1000 add pipe 1 ip from any to any out xmit em0 > Dichotomy: > layer 2 is limit to 1Mbit/s out em0 (Vs.) layer 3 is limit to 1Mbit/s out > em0 > > Only one of these can be correct and I do not see documentation anywhere > indicating to me which one is indeed correct. I am not educated enough to > look in the source code to determine this for myself. man ipfw read a few times forwards and backwards pretty well covers it, though the code's not bad bedtime reading :) > I did not state before the dummynet is on a freebsd transparent bridge. > Here are my sysctl variables > > net.link.ether.ipfw: 0 > net.link.bridge.ipfw: 1 > net.link.bridge.ipfw_arp: 0 > # sysctl -a | grep dummynet > net.inet.ip.dummynet.debug: 0 > net.inet.ip.dummynet.pipe_byte_limit: 7000000 > net.inet.ip.dummynet.pipe_slot_limit: 100 > net.inet.ip.dummynet.io_pkt_drop: 33250421 > net.inet.ip.dummynet.io_pkt_fast: 0 > net.inet.ip.dummynet.io_pkt: 3512830485 > net.inet.ip.dummynet.io_fast: 0 > net.inet.ip.dummynet.tick_lost: 0 > net.inet.ip.dummynet.tick_diff: 1662798 > net.inet.ip.dummynet.tick_adjustment: 1679911 > net.inet.ip.dummynet.tick_delta_sum: 721 > net.inet.ip.dummynet.tick_delta: 2 > net.inet.ip.dummynet.red_max_pkt_size: 1500 > net.inet.ip.dummynet.red_avg_pkt_size: 512 > net.inet.ip.dummynet.red_lookup_depth: 256 > net.inet.ip.dummynet.max_chain_len: 16 > net.inet.ip.dummynet.expire: 1 > net.inet.ip.dummynet.search_steps: 0 > net.inet.ip.dummynet.searches: 0 > net.inet.ip.dummynet.extract_heap: 0 > net.inet.ip.dummynet.ready_heap: 16 > net.inet.ip.dummynet.curr_time: 1804818806 > net.inet.ip.dummynet.hash_size: 64 > # sysctl -a | grep fw > net.inet.ip.fw.dyn_keepalive: 1 > net.inet.ip.fw.dyn_short_lifetime: 5 > net.inet.ip.fw.dyn_udp_lifetime: 10 > net.inet.ip.fw.dyn_rst_lifetime: 1 > net.inet.ip.fw.dyn_fin_lifetime: 1 > net.inet.ip.fw.dyn_syn_lifetime: 20 > net.inet.ip.fw.dyn_ack_lifetime: 300 > net.inet.ip.fw.static_count: 26 > net.inet.ip.fw.dyn_max: 4096 > net.inet.ip.fw.dyn_count: 0 > net.inet.ip.fw.curr_dyn_buckets: 256 > net.inet.ip.fw.dyn_buckets: 256 > net.inet.ip.fw.default_rule: 65535 > net.inet.ip.fw.verbose_limit: 0 > net.inet.ip.fw.verbose: 0 > net.inet.ip.fw.debug: 1 > net.inet.ip.fw.one_pass: 1 > net.inet.ip.fw.autoinc_step: 100 > net.inet.ip.fw.enable: 1 > > rule set > > 00900 15945359 1264958184 allow ip from 10.18.0.6 to any out xmit bce0 > 00901 44378882 8041056830 pipe 1 ip from 10.18.0.0/24 to any out xmit bce0 > 00910 225165 18913860 pipe 1 ip from 10.18.2.1 to any out xmit bce0 > 00915 0 0 pipe 2 ip from 10.18.2.2 to any out xmit bce0 > 00920 0 0 pipe 3 ip from 10.18.2.3 to any out xmit bce0 > 01000 265959 40261978 pipe 1 ip from 10.21.192.8/29 to any out xmit bce0 > 01001 19627389 3444748305 pipe 1 ip from 10.20.0.0/20 to any out xmit bce0 > 01002 180922838 31277916333 pipe 1 ip from 10.21.32.0/20 to any out xmit bce0 > 01003 376697028 75171259985 pipe 1 ip from 10.20.64.0/20 to any out xmit bce0 > 01004 68629348 12603497407 pipe 1 ip from 10.20.192.0/20 to any out xmit bce0 > 01005 185005607 37413644435 pipe 1 ip from 10.20.176.0/20 to any out xmit bce0 > 01006 59179395 10051795154 pipe 1 ip from 10.20.144.0/20 to any out xmit bce0 > 01007 235863676 41845371053 pipe 1 ip from 10.20.128.0/20 to any out xmit bce0 > 01008 51518999 10847386956 pipe 1 ip from 10.20.96.0/20 to any out xmit bce0 > 01009 79734709 13723937026 pipe 1 ip from 10.20.112.0/20 to any out xmit bce0 > 01010 30062283 4078276506 pipe 1 ip from 10.20.80.0/20 to any out xmit bce0 > 01011 36318511 6974450810 pipe 1 ip from 10.20.208.0/20 to any out xmit bce0 > 01012 19642496 3098922560 pipe 1 ip from 10.20.160.0/20 to any out xmit bce0 > 01013 18209570 2964820411 pipe 1 ip from 10.20.224.0/20 to any out xmit bce0 > 01014 13707931 3915527156 pipe 1 ip from 10.22.160.0/20 to any out xmit bce0 > 01015 9893051 1175739042 pipe 1 ip from 10.21.112.0/20 to any out xmit bce0 > 01016 39876049 7905051976 pipe 1 ip from 10.22.144.0/20 to any out xmit bce0 > 01017 0 0 pipe 1 ip from 10.22.147.0/24 to any out xmit bce0 > 01018 128251743 24972920481 pipe 1 ip from 10.21.0.0/20 to any out xmit bce0 > 01019 102099411 27172050432 pipe 1 ip from 10.21.16.0/20 to any out xmit bce0 > 65535 53581341116 24295389614128 allow ip from any to any > > and pipe spec > > # ipfw pipe list > 00001: 11.500 Mbit/s 0 ms 75 KB 1 queues (1 buckets) droptail > mask: 0x00 0x00000000/0x0000 -> 0x00000000/0x0000 > BKT Prot ___Source IP/port____ ____Dest. IP/port____ Tot_pkt/bytes Pkt/Byte > Drp > 0 tcp 10.21.44.163/47934 65.55.131.121/80 1735939204 358391598184 > 1 191 10152696 > > > You can see that the bw has been configured to 11.5Mbit/s which should be the > base 10 expression for bit/s meaning exactly 11,500,000 bit/s Yes, but I don't see how you get 11.5Mbit/s from your specification above of 1Mbit/s? Or is this a different example? > And here lies the question again does this limit include layer 2 Ethernet > frame headers in the bit/s counting mechanism I'd say not, but check with your provider that 1Mbit/s of IP is allowed; that would be my expectation of such a spec, hereabouts anyway. cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090819224032.B90928>