Date: Tue, 9 Mar 2010 12:49:27 -0800 From: Pyun YongHyeon <pyunyh@gmail.com> To: Ian FREISLICH <ianf@clue.co.za> Cc: current@freebsd.org Subject: Re: dev.bce.X.com_no_buffers increasing and packet loss Message-ID: <20100309204927.GN1311@michelle.cdnetworks.com> In-Reply-To: <E1NozXP-000GGH-T9@clue.co.za> References: <20100305215539.GG14818@michelle.cdnetworks.com> <20100305210435.GF14818@michelle.cdnetworks.com> <20100305184046.GD14818@michelle.cdnetworks.com> <20100305175639.GB14818@michelle.cdnetworks.com> <E1NnVaT-0003Ft-3p@clue.co.za> <E1Nnc4d-0003mB-6e@clue.co.za> <E1Nne0Q-0003uZ-OR@clue.co.za> <E1Nnesz-00040L-AQ@clue.co.za> <E1NoeCu-0006P9-12@clue.co.za> <E1NozXP-000GGH-T9@clue.co.za>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 09, 2010 at 03:31:55PM +0200, Ian FREISLICH wrote: > Pyun YongHyeon wrote: > > On Mon, Mar 08, 2010 at 04:45:20PM +0200, Ian FREISLICH wrote: > > > Pyun YongHyeon wrote: > > > > On Fri, Mar 05, 2010 at 11:16:41PM +0200, Ian FREISLICH wrote: > > > > > Pyun YongHyeon wrote: > > > > > > Thanks for the info. Frankly, I have no idea how to explain the > > > > > > issue given that you have no heavy load. > > > > > > > > > > How many cores would be involved in handling the traffic and runnig > > > > > PF rules on this machine? There are 4x > > > > > CPU: Quad-Core AMD Opteron(tm) Processor 8354 (2194.51-MHz K8-class CPU > ) > > > > > In this server. I'm also using carp extensively. > > > > > > > > > > > > > pf(4) uses a single lock for processing, number of core would have > > > > no much benefit. > > > > > > What's interesting is the effect on CPU utilisation and interrupt > > > generation that net.inet.ip.fastforwarding has: > > > > > > net.inet.ip.fastforwarding=1 > > > interrupt rate is around 10000/s per bce interface > > > cpu 8.0% interrupt > > > > > > > Yes, this is one of intentional change of the patch. Stock bce(4) > > seems to generate too much interrupts on BCM5709 so I rewrote > > interrupt handling with the help of David. sysctl nodes are also > > exported to control interrupt moderation so you can change them if > > you want. Default value was tuned to generate interrupts less than > > 10k per second and try to minimize latencies. > > Can you explain the tunables please - I'm guessing it's these: > > dev.bce.$i.tx_quick_cons_trip_int This value controls the number of TX Quick BD Chain entries that must be completed before a status block is generated during an interrupt. > dev.bce.$i.tx_quick_cons_trip This value controls the number of TX Quick BD Chain entries that must be completed before a status block is generated. Setting this to 0 disables TX Quick BD Chain consumption from generating status blocks. > dev.bce.$i.tx_ticks_int This value controls the number of 1us ticks that will be counted for status block updates generated due to TX activity during interrupt processing. Setting this value to 0 disables the TX timer feature during interrupts. > dev.bce.$i.tx_ticks This value controls the number of 1us ticks that will be counted before a status block update is generated due to TX activity. Setting this value to 0 disables the TX timer feature. > dev.bce.$i.rx_quick_cons_trip_int This value controls the number of RX Quick BD entries that must be completed before a status block is generated during interrupt processing. > dev.bce.$i.rx_quick_cons_trip This value controls the number of RX Quick BD Chain entries that must be completed before a status block is generated. Setting this to 0 disables RX Event consumption from generating status blocks. > dev.bce.$i.rx_ticks_int This value controls the number of 1us ticks that will be counted for status block updates generated due to RX activity during interrupt processing. Setting this value to 0 disables the RX timer feature during interrupts. > dev.bce.$i.rx_ticks > This value controls the number of 1us ticks that will be counted before a status block update is generated due to RX activity. Setting this value to 0 disables the RX timer feature.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100309204927.GN1311>