Date: Tue, 23 Jul 2013 10:18:07 -0700 From: Adrian Chadd <adrian@freebsd.org> To: trafdev <trafdev@mail.ru> Cc: Sepherosa Ziehau <sepherosa@gmail.com>, freebsd-net@freebsd.org, John-Mark Gurney <jmg@funkthat.com> Subject: Re: SO_REUSEPORT: strange kernel balancer behaviour Message-ID: <CAJ-Vmomfb29QqOBi%2B8h6O-NV_jKoti9EhQW9tWyXJowYqCftNQ@mail.gmail.com> In-Reply-To: <CAJ-VmomoGzZUUKSz3mjssx4Yd93xRx9sBDwhyRLXFesNuiAayA@mail.gmail.com> References: <51E0E2AF.7090404@mail.ru> <CAMOc5cz6gP2N62T4QhbTdVar94O4FSdPDsqktD_9vJ0mYVqt_Q@mail.gmail.com> <51E44E2F.8060700@mail.ru> <CAJ-VmomHHfhExa4g63tT_sf0hTPa2T7jPKQGHrD0fchq=-k%2B=g@mail.gmail.com> <51E455D5.2090403@mail.ru> <20130722200205.GO26412@funkthat.com> <51EDA37A.9040200@mail.ru> <CAJ-Vmon-WdsUnH33smkf%2B33yrUHP0p-Fp3ng1ZyTsvLb9tQfVA@mail.gmail.com> <51EE198B.7040509@mail.ru> <CAJ-Vmokxo1LniNvKYerap8bRe9hpUWLvdzSYquNVyZX4Ata_sQ@mail.gmail.com> <CAJ-Vmons_6XfywPm2SPnE1-aac8Lc-wBEN06kiezD2m49z=PYw@mail.gmail.com> <51EE2C2B.4020800@mail.ru> <CAJ-VmomoGzZUUKSz3mjssx4Yd93xRx9sBDwhyRLXFesNuiAayA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Answering my own email: SO_REUSEPORT on FreeBSD doesn't load balance incoming connections. Test case: * 8 threads * each creates a TCP socket, listening on port 1667, with SO_REUSEPORT * only the first thread ever sees incoming requests. I think this load distribution feature is useful to implement, but it shouldn't be called SO_REUSEPORT. (Silly Linux, why would you do that too..) -adrian On 23 July 2013 07:39, Adrian Chadd <adrian@freebsd.org> wrote: > On 23 July 2013 00:09, trafdev <trafdev@mail.ru> wrote: >> It's like shared acceptor FD and N processes: > > [snip] looks like mine, but I use threads. > >> Accept conn callback is called in N processes on each connection, only one >> wins, >> others exit by errno == EAGAIN case. Overhead is almost zero. >> Problem is that "wins" distribution is far from equal. > > Right. I'm not at that stage yet, but I can totally see that happening. > > Ok. Time to hit up the TCP stack people to weigh in on the recent lkml > posts about this: > > http://lwn.net/Articles/542629/ > > With SO_REUSEPORT, we should create one listen FD per thread, and let > the OS balance how that gets distributed. Rather than one listen > socket that is shared between all processes/threads. I'll try that > locally and see if that works right. Would you mind trying it locally > and see if it improves the distribution of work? > > Thanks, > > > -adrian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmomfb29QqOBi%2B8h6O-NV_jKoti9EhQW9tWyXJowYqCftNQ>