Date: 30 Oct 2000 22:06:42 -0500 From: Nat Lanza <magus@cs.cmu.edu> To: Andre Oppermann <oppermann@telehouse.ch> Cc: Alfred Perlstein <bright@wintelcom.net>, Bosko Milekic <bmilekic@dsuper.net>, freebsd-net@FreeBSD.ORG Subject: Re: MP: per-CPU mbuf allocation lists Message-ID: <uoc7l6pyail.fsf@hurlame.pdl.cs.cmu.edu> In-Reply-To: Andre Oppermann's message of "Mon, 30 Oct 2000 20:47:05 %2B0100" References: <Pine.BSF.4.21.0010301256580.30271-100000@jehovah.technokratis.com> <20001030104457.E22110@fw.wintelcom.net> <39FDD039.594CED43@telehouse.ch>
next in thread | previous in thread | raw e-mail | index | archive | help
Andre Oppermann <oppermann@telehouse.ch> writes:
> Lets assume there is an apache webserver running on the MP machine.
> Now this apache process tends to stick to one CPU (does it? would
> make sense for cache locality). This process generates a ton of
> traffic and needs thousands of mbufs. If I keep the watermark
> defaults then this is sub-optimal because on CPU1 I need thousands
> of mbuf whereas CPU2 is not using much. If I tune the watermarks to
> the required level of CPU1 then I'm wasting an awful lot of mbufs on
> the other CPU('s)...
Actually, Apache uses a pool of processes. So ideally the active httpd
processes would be distributed fairly evenly across the machine's CPUs.
Also, if there is only one process heavily consuming mbufs, then
presumably acquiring the locks necessary to grab mbufs from the global
pool is fast, since there's not going to be any contention for them.
> The proposal; Lets have the minimum watermark set on a system-wide
> level (sysctl) but the high-watermark would be based per-cpu with
> an sliding window calculating the average use of mbufs over a certain
> period of time (eg. 10sec). The initial high watermark would be
> either min*2 or set upon system initialization to some initial (base)
> value. This also takes automagically care of process migration between
> the CPU's. Also this system would be self adjusting and self tuning
> without admin intervention in common edge cases.
I'm not convinced this is really necessary; if there isn't any
contention for the mbuf lists, then it doesn't matter too much where
they are -- as I understand it, the benefit in per-cpu mbuf lists is
that you avoid contention when there are multiple consumers of
mbufs. If you don't have multiple consumers, you're not going to have
contention, so I don't see the point in adding fancy logic for that
case.
--nat
--
nat lanza --------------------- research programmer, parallel data lab, cmu scs
magus@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/
there are no whole truths; all truths are half-truths -- alfred north whitehead
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?uoc7l6pyail.fsf>