Date: Fri, 27 Dec 2013 22:02:50 -0800 From: Adrian Chadd <adrian@freebsd.org> To: Adrian Chadd <adrian@freebsd.org>, "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: Default knote hash table size is too .. small Message-ID: <CAJ-Vmo=%2Bg_c-YBV-a8zdqgkf%2B7sMn3vZv77x9RDCV%2B0y-mZCEQ@mail.gmail.com> In-Reply-To: <20131228025948.GP99167@funkthat.com> References: <CAJ-VmomteHhbNv2pLX1cma74hExbtQKhxiRd3D653abneuJwhg@mail.gmail.com> <20131228025948.GP99167@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 27 December 2013 18:59, John-Mark Gurney <jmg@funkthat.com> wrote: > Adrian Chadd wrote this message on Fri, Dec 27, 2013 at 14:41 -0800: >> The default knote hash table size (KN_HASHSIZE) is 64. When doing >> dirty things with lots of sockets (say, 64k and more) that involve >> plenty of knote insert/remove (ie, oneshot events - think posix AIO >> and my upcoming kqueue sendfile notification stuff) it consumes stupid >> amounts of CPU. > > Just for the record, KN_HASHSIZE is not used by sockets, since they > have an fd to do the look up... It's AIO that's probably the one > using the CPU time, but definately not sockets.. > > The notes put on the hash are _AIO/_LIO, _PROC, _TIMER, _USER, _FS, > and _SIGNAL... > > So, should we look at another data structure like RB trees for this > then? At least RB trees would scale better, or we should look at > making the hash table growable? It looks like there is support to > grow it, since kq_knhas and kq_knhashmask are both local to kq... > > From a quick look, we can resize the hash only by holding the KQ > lock... > >> I'd like to make this a tunable so people like Adrian at Netflix can >> bump this to higher values (say 32k) but people like Adrian at >> Embedded can bump this to lower values (say 64.) > > Yeh, a smaller default, say of 8 or 16 would make sense for embedded > since there aren't many non-fd consumers of kq out there... This would > mean each kq only consumes 32 or 64 bytes instead of 2k... Heck, > even for normal systems, a default of 8 could/would make sense... > > Hmm... maybe we should get some stats on how many notes use the hash? It again is highly application dependent. If you're just doing socket IO then you'r elikely fine. If you're also doing other things in high volumes and lots of pending operations (eg posix aio, or as you said timers, etc) then it could get highly crappy very quickly. -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=%2Bg_c-YBV-a8zdqgkf%2B7sMn3vZv77x9RDCV%2B0y-mZCEQ>