Date: Fri, 14 May 2004 01:30:27 -0700 (PDT) From: Mark Terribile <materribile@yahoo.com> To: arch@freebsd.org Cc: Garrett Wollman <wollman@khavrinen.lcs.mit.edu> Subject: Re: kqueue giant-locking (&kq_Giant, locking) Message-ID: <20040514083027.38172.qmail@web21103.mail.yahoo.com>
next in thread | raw e-mail | index | archive | help
Sirs,
Please excuse my coming in late on this; I don't ordinarily follow this list.
Brian F. Feldman writes
> I can't imagine a well-designed applications has kqueues of kqueues. I
> didn't remove the file descriptor polling interface, I removed the file
> descriptor kqueue interface.
I took over a server to which I added a kqueue of kqueues. It used separate
kqueues as priority queues, with controls on starvation. I found this
essential for getting full performance out of the box. It was a caching web
proxy running on a 1 GHz PIII, serving about 1,500 requests per second. That
actually meant about 7,200 stimuli/second, including disk completion, TCP
connects, TCP disconnects, TCP read data available, and TCP write buffer
space available. It did its own async DNS, with caching. We often ran
with 40,000 file descriptors open. After tuning of IP buffer space and
connect queues, it withstood a 20 percent overload with graceful degradation,
load shedding, and no lockup.
Without multiple kqueues, this would have been far more difficult--or
impossible.
Roughly 70% of the processor was spent in the kernel, and measurements
with different load mixes indicated the bulk of that was in the TCP stack.
This means that the average service time at user level was about 40 usec.
FreeBSD and recursive kqueue deserve a lot of the performance credit.
I consider the server well-designed. When I was doing this, other developers
at other companies were getting similar results from similar hardware, some
of them using FreeBSD and kqueue. I have reason to believe that they were
using similar priority schemes.
I recommend allowing kqueues of kqueues.
Mark Terribile
__________________________________
Do you Yahoo!?
SBC Yahoo! - Internet access at a great low price.
http://promo.yahoo.com/sbc/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040514083027.38172.qmail>