Date: Thu, 6 Nov 2014 09:21:11 -0800 From: Adrian Chadd <adrian@freebsd.org> To: "Alexander V. Chernikov" <melifaro@freebsd.org> Cc: "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org> Subject: Re: svn commit: r273112 - head/sys/dev/ixgbe Message-ID: <CAJ-VmokhhBiJ%2Buj8yEsfGj4i6FsaBFiZDdYhtFnHgJC3GAokZQ@mail.gmail.com> In-Reply-To: <545B55FA.6070408@FreeBSD.org> References: <201410150122.s9F1Mudu083306@svn.freebsd.org> <545B55FA.6070408@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hm, are there any other hardware counters that we can use to see what was dropped? I didn't see performance drops doing this; in fact I saw performance increases with FC disabled and DROP_EN set. Thanks, -adrian On 6 November 2014 03:05, Alexander V. Chernikov <melifaro@freebsd.org> wrote: > On 15.10.2014 05:22, Adrian Chadd wrote: >> >> Author: adrian >> Date: Wed Oct 15 01:22:56 2014 >> New Revision: 273112 >> URL: https://svnweb.freebsd.org/changeset/base/273112 >> >> Log: >> Set the DROP_EN bit before the RX queue is brought up and active. >> He noticed issues setting this bit in SRRCTL after the queue was up, >> so doing it from the sysctl handler isn't enough and may not actually >> work correctly. > > While investigating/measuring ixgbe performance w/ and without flow control > I noticed that disabling fc entirely (e.g. disabling 512K skid buffer and > turning drop_en on) > not only makes things a bit worse, but also makes NIC stop accounting > tail-drops as errors > at all. > It is a bit unhandy with HW counters, since you, for example, see 2M packets > being received > in netstat, but in reality system receives only 1M (And there is no easy way > to determine exact > count). > >> This commit doesn't remove the sysctl path or try to change its >> behaviour. I'll talk with others about how to finish fixing that >> before I tackle that. >> PR: kern/194311 >> Submitted by: luigi >> MFC after: 3 days >> Sponsored by: Norse Corp, Inc >> >> Modified: >> head/sys/dev/ixgbe/ixgbe.c >> >> Modified: head/sys/dev/ixgbe/ixgbe.c >> >> ============================================================================== >> --- head/sys/dev/ixgbe/ixgbe.c Wed Oct 15 01:16:11 2014 (r273111) >> +++ head/sys/dev/ixgbe/ixgbe.c Wed Oct 15 01:22:56 2014 (r273112) >> @@ -4377,6 +4377,20 @@ ixgbe_initialize_receive_units(struct ad >> srrctl &= ~IXGBE_SRRCTL_BSIZEPKT_MASK; >> srrctl |= bufsz; >> srrctl |= IXGBE_SRRCTL_DESCTYPE_ADV_ONEBUF; >> + >> + /* >> + * Set DROP_EN iff we have no flow control and >1 queue. >> + * Note that srrctl was cleared shortly before during >> reset, >> + * so we do not need to clear the bit, but do it just in >> case >> + * this code is moved elsewhere. >> + */ >> + if (adapter->num_queues > 1 && >> + adapter->hw.fc.requested_mode == ixgbe_fc_none) { > > This code seems to turn drop_en always on HEAD due to > hw->fc.requested_mode is inherited from adapter->fc at the end of > ixgbe_init_locked(), _after_ calling ixgbe_initialize_receive_units() > >> + srrctl |= IXGBE_SRRCTL_DROP_EN; >> + } else { >> + srrctl &= ~IXGBE_SRRCTL_DROP_EN; >> + } >> + >> IXGBE_WRITE_REG(hw, IXGBE_SRRCTL(i), srrctl); >> /* Setup the HW Rx Head and Tail Descriptor Pointers */ >> >> >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmokhhBiJ%2Buj8yEsfGj4i6FsaBFiZDdYhtFnHgJC3GAokZQ>