Date: Fri, 27 Dec 2013 18:59:48 -0800 From: John-Mark Gurney <jmg@funkthat.com> To: Adrian Chadd <adrian@freebsd.org> Cc: "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: Default knote hash table size is too .. small Message-ID: <20131228025948.GP99167@funkthat.com> In-Reply-To: <CAJ-VmomteHhbNv2pLX1cma74hExbtQKhxiRd3D653abneuJwhg@mail.gmail.com> References: <CAJ-VmomteHhbNv2pLX1cma74hExbtQKhxiRd3D653abneuJwhg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Adrian Chadd wrote this message on Fri, Dec 27, 2013 at 14:41 -0800:
> The default knote hash table size (KN_HASHSIZE) is 64. When doing
> dirty things with lots of sockets (say, 64k and more) that involve
> plenty of knote insert/remove (ie, oneshot events - think posix AIO
> and my upcoming kqueue sendfile notification stuff) it consumes stupid
> amounts of CPU.
Just for the record, KN_HASHSIZE is not used by sockets, since they
have an fd to do the look up... It's AIO that's probably the one
using the CPU time, but definately not sockets..
The notes put on the hash are _AIO/_LIO, _PROC, _TIMER, _USER, _FS,
and _SIGNAL...
So, should we look at another data structure like RB trees for this
then? At least RB trees would scale better, or we should look at
making the hash table growable? It looks like there is support to
grow it, since kq_knhas and kq_knhashmask are both local to kq...
>From a quick look, we can resize the hash only by holding the KQ
lock...
> I'd like to make this a tunable so people like Adrian at Netflix can
> bump this to higher values (say 32k) but people like Adrian at
> Embedded can bump this to lower values (say 64.)
Yeh, a smaller default, say of 8 or 16 would make sense for embedded
since there aren't many non-fd consumers of kq out there... This would
mean each kq only consumes 32 or 64 bytes instead of 2k... Heck,
even for normal systems, a default of 8 could/would make sense...
Hmm... maybe we should get some stats on how many notes use the hash?
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20131228025948.GP99167>