Date: Wed, 21 Jan 2015 08:09:16 -0800 From: Adrian Chadd <adrian@freebsd.org> To: Lawrence Stewart <lstewart@freebsd.org> Cc: "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, Hans Petter Selasky <hselasky@freebsd.org> Subject: Re: svn commit: r277213 - in head: share/man/man9 sys/kern sys/ofed/include/linux sys/sys Message-ID: <CAJ-Vmo=GKfnD_q7=LB65WpyCaAanC1PE1dO2-o8jZA8OZ9317A@mail.gmail.com> In-Reply-To: <54BFB10C.9080004@freebsd.org> References: <201501151532.t0FFWV2Y037455@svn.freebsd.org> <CAJ-Vmok0GXZoojyi=jE=b5D-d338APztaf3Pw0_AAQ-173XSWw@mail.gmail.com> <CAJ-VmomOE6AzbEFw45VOtVbLTtDMbnixkYDVK951pSCkcMrQjQ@mail.gmail.com> <54BFB10C.9080004@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 21 January 2015 at 06:00, Lawrence Stewart <lstewart@freebsd.org> wrote: > On 01/20/15 09:22, Adrian Chadd wrote: >> Yeah, it looks like you set c_cpu to timeout_cpu in >> _callout_init_locked(), but then you only handle the case of the CPU >> being changed in certain circumstances. You aren't handling the CPU >> being initialised when the callout is first added. >> >> And, callout_restart_async() calls callout_lock(), which calls >> CC_LOCK() on the callout CPU, which initially is zero. Then, it never >> seems to be 'moved' into the correct CPU, even though it's being >> called with a CPU id. >> >> So, if I am reading this all correctly, you aren't really handling >> multi CPU callwheels at all. ;) >> >> In my instance, I'm seeing quite a lot of lock contention between the >> userland threads, the network RX threads and the clock thread, all >> contending on a single callout wheel being used for TCP timers. One of >> the early goals for fixing up the RSS stuff in -HEAD was to make >> per-CPU (Well, per-RSS-bucket really) TCP timers not contend with the >> NIC RX processing path, and now it does. :( > > To clarify, you're seeing this with net.inet.tcp.per_cpu_timers=1? Yup, RSS enables that by default. -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=GKfnD_q7=LB65WpyCaAanC1PE1dO2-o8jZA8OZ9317A>