Date: Tue, 23 Jul 2013 10:54:41 -0700 From: trafdev <trafdev@mail.ru> To: Adrian Chadd <adrian@freebsd.org> Cc: Sepherosa Ziehau <sepherosa@gmail.com>, freebsd-net@freebsd.org, John-Mark Gurney <jmg@funkthat.com> Subject: Re: SO_REUSEPORT: strange kernel balancer behaviour Message-ID: <51EEC361.9050806@mail.ru> In-Reply-To: <CAJ-Vmomfb29QqOBi%2B8h6O-NV_jKoti9EhQW9tWyXJowYqCftNQ@mail.gmail.com> References: <51E0E2AF.7090404@mail.ru> <CAMOc5cz6gP2N62T4QhbTdVar94O4FSdPDsqktD_9vJ0mYVqt_Q@mail.gmail.com> <51E44E2F.8060700@mail.ru> <CAJ-VmomHHfhExa4g63tT_sf0hTPa2T7jPKQGHrD0fchq=-k%2B=g@mail.gmail.com> <51E455D5.2090403@mail.ru> <20130722200205.GO26412@funkthat.com> <51EDA37A.9040200@mail.ru> <CAJ-Vmon-WdsUnH33smkf%2B33yrUHP0p-Fp3ng1ZyTsvLb9tQfVA@mail.gmail.com> <51EE198B.7040509@mail.ru> <CAJ-Vmokxo1LniNvKYerap8bRe9hpUWLvdzSYquNVyZX4Ata_sQ@mail.gmail.com> <CAJ-Vmons_6XfywPm2SPnE1-aac8Lc-wBEN06kiezD2m49z=PYw@mail.gmail.com> <51EE2C2B.4020800@mail.ru> <CAJ-VmomoGzZUUKSz3mjssx4Yd93xRx9sBDwhyRLXFesNuiAayA@mail.gmail.com> <CAJ-Vmomfb29QqOBi%2B8h6O-NV_jKoti9EhQW9tWyXJowYqCftNQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Yes, and if you kill this first thread - second thread will start to receive connections and so on. That's why I've used one processes-shared acceptor socket which behaves better (load is balancing between processes but equality of distribution is far from ideal). Btw as per https://lwn.net/Articles/542629/ Linux 3.1 SOLVES all these problems via SO_REUSEPORT. On Tue Jul 23 10:18:07 2013, Adrian Chadd wrote: > Answering my own email: > > SO_REUSEPORT on FreeBSD doesn't load balance incoming connections. > > Test case: > > * 8 threads > * each creates a TCP socket, listening on port 1667, with SO_REUSEPORT > * only the first thread ever sees incoming requests. > > I think this load distribution feature is useful to implement, but it > shouldn't be called SO_REUSEPORT. > > (Silly Linux, why would you do that too..) > > > > -adrian > > On 23 July 2013 07:39, Adrian Chadd <adrian@freebsd.org> wrote: >> On 23 July 2013 00:09, trafdev <trafdev@mail.ru> wrote: >>> It's like shared acceptor FD and N processes: >> >> [snip] looks like mine, but I use threads. >> >>> Accept conn callback is called in N processes on each connection, only one >>> wins, >>> others exit by errno == EAGAIN case. Overhead is almost zero. >>> Problem is that "wins" distribution is far from equal. >> >> Right. I'm not at that stage yet, but I can totally see that happening. >> >> Ok. Time to hit up the TCP stack people to weigh in on the recent lkml >> posts about this: >> >> http://lwn.net/Articles/542629/ >> >> With SO_REUSEPORT, we should create one listen FD per thread, and let >> the OS balance how that gets distributed. Rather than one listen >> socket that is shared between all processes/threads. I'll try that >> locally and see if that works right. Would you mind trying it locally >> and see if it improves the distribution of work? >> >> Thanks, >> >> >> -adrian > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51EEC361.9050806>