Date: Fri, 5 Nov 2010 16:55:43 -0700 From: "Ricky Charlet" <RCharlet@adaranet.com> To: "Ricky Charlet" <RCharlet@adaranet.com>, =?iso-8859-1?Q?'Ermal_Lu=E7i'?= <eri@freebsd.org> Cc: "altq@csl.sony.co.jp" <altq@csl.sony.co.jp>, "freebsd-pf@freebsd.org" <freebsd-pf@freebsd.org> Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC08F@SJ-EXCH-1.adaranet.com> In-Reply-To: <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> <AANLkTi=pga3UpDJ2dzHseb%2BQpX98vtVmjxi430vQvqpP@mail.gmail.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
OK, so what is the theory of operation behind this (empirically effective) =
solution? I'll post my guess, please correct me.
<guess>
The slowness in the system is waiting on the timer to pop. The timer did no=
t need to be set any quicker, just the granularity of how quickly the timer=
s are run needed to be updated.
</guess>
> -----Original Message-----
> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd-
> pf@freebsd.org] On Behalf Of Ricky Charlet
> Sent: Friday, November 05, 2010 1:43 PM
> To: 'Ermal Lu=E7i'
> Cc: altq@csl.sony.co.jp; freebsd-pf@freebsd.org
> Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or
> above 40 Mb on e1000
>
> Yes.. this works very well.
>
> the first thing I tried:
> I had a kern.hz of 1000 and changed to 10000 (through loader.conf +
> reboot)
> I had a queue depth of 50 and changed to 900 (through `qlimit`
> parameter in pf.conf)
>
>
> And now I am hitting my configured rate limits beyond my goal of 45Mb
> (T3 speed) even into 100 Mb range. I'm going to continue to experiment
> around with hz and q-depth and pick some eventual values for my
> implementation.
>
> But thanks, Emeral. This seems to be exactly the parameters to play
> around with to increase altq performance into the 50~100 Mb range.
>
> ---
> Ricky Charlet
> Adara Networks
> USA 408-433-4942
>
>
>
>
>
>
>
> > -----Original Message-----
> > From: ermal.luci@gmail.com [mailto:ermal.luci@gmail.com] On Behalf Of
> > Ermal Lu=E7i
> > Sent: Friday, November 05, 2010 10:18 AM
> > To: Ricky Charlet
> > Cc: freebsd-pf@freebsd.org; altq@csl.sony.co.jp
> > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or
> > above 40 Mb on e1000
> >
> > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet
> <RCharlet@adaranet.com>
> > wrote:
> > > Has anyone out there run altq with cbq with bandwidth limits set
> > around 40 ~ 50 Mb and seen it work well (actual through put allowed
> to
> > come near that speed)?
> > >
> > > Thanks
> > > ---
> > > Ricky Charlet
> > > Adara Networks
> > > USA 408-433-4942
> > >
> >
> > I can suggest to try with higher hz and increase queue lengths in
> altq.
> > By default they are 50 which is quite low in that regard.
> >
> > >
> > >> -----Original Message-----
> > >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd-
> > >> pf@freebsd.org] On Behalf Of Ricky Charlet
> > >> Sent: Wednesday, November 03, 2010 9:47 PM
> > >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp'
> > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or
> > above
> > >> 40 Mb on e1000
> > >>
> > >> Howdy,
> > >>
> > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1
> altq
> > >> capabilities, amd64, 8 core)
> > >>
> > >> I have been measuring altq performance on my system. I set
> a
> > >> cbq bandwith limit for a queue, match traffic to the queue with an
> > >> appropriate rule and pump data.
> > >>
> > >> Without altq enabled, my traffic gen tool (nuttcp) can
> pump
> > >> 800Mb across my gig links.
> > >>
> > >> With altq(cbq) setting bandwith limits (no borrow), up to
> > about
> > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up,
> altq
> > >> seems to slow down my traffic much slower than the bandwidth
> limits.
> > >>
> > >> Same effect is observed on gif interfaces (where my real
> > >> interest lies) but much worse. With altq(cbq) setting bandwith
> > limits
> > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb
> > >> bandwidth limit and up, altq seems to slow down my traffic much
> > lower
> > >> than the bandwidth limits.
> > >>
> > >> I have a large file of sample data in excel formant which
> > I'd
> > >> like to share. Is there an appropriate way to share it with this
> > list?
> > >>
> > >> I know I'm not cpu bound (97% idle while nuttcp pushes
> 800Mb
> > >> traffic and altq configured with a bandwidth limit of 50Mb
> actually
> > >> passes 30Mb.)
> > >>
> > >> I sort of suspect I'm mutex bound but have no idea how to
> > test
> > >> that.
> > >>
> > >>
> > >>
> > >> (1) this config allows about 4Mb tcp traffic to pass:
> > >> ----------cut------------------
> > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> > >> queue queue1 bandwidth 1Mb priority 7 cbq (default)
> > >> queue queue2 bandwidth 50Mb priority 1 cbq
> > >> pass out quick on gif10 inet proto tcp no state queue queue2
> > >> pass out quick on gif10 proto icmp queue queue2
> > >> ----------paste----------------
> > >>
> > >> (2) this config allows about 40 Mb tcp traffic to pass
> > >> ----------cut------------------
> > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 }
> > >> queue queue1 bandwidth 1Mb priority 7 cbq (default)
> > >> queue queue2 bandwidth 50Mb priority 1 cbq
> > >> pass out quick on em0 inet proto tcp no state queue queue2
> > >> pass out quick on em0 proto icmp queue queue2
> > >> ----------paste----------------
> > >>
> > >> Yes, I send appropriate traffic in each case to be
> 'caught'
> > by
> > >> the tcp pass out rule wether it be found on a gif or an em.
> > >>
> > >>
> > >> My goal is to get 'expected' behavior (a bandwidth limit
> of
> > 40
> > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a
> > reasonable
> > >> expectation? Any ideas about config jigs or tests to run?
> > >>
> > >>
> > >>
> > >> ---
> > >> Ricky Charlet
> > >> Adara Networks
> > >> USA 408-433-4942
> > >>
> > >>
> > >> _______________________________________________
> > >> freebsd-pf@freebsd.org mailing list
> > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> > >> To unsubscribe, send any mail to "freebsd-pf-
> > unsubscribe@freebsd.org"
> > > _______________________________________________
> > > freebsd-pf@freebsd.org mailing list
> > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> > > To unsubscribe, send any mail to "freebsd-pf-
> unsubscribe@freebsd.org"
> > >
> >
> >
> >
> > --
> > Ermal
> _______________________________________________
> freebsd-pf@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-pf
> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?32AB5C9615CC494997D9ABB1DB12783C024C6FC08F>