From owner-freebsd-net Mon Oct 30 19: 7:14 2000 Delivered-To: freebsd-net@freebsd.org Received: from hurlame.pdl.cs.cmu.edu (HURLAME.PDL.CS.CMU.EDU [128.2.189.78]) by hub.freebsd.org (Postfix) with ESMTP id 64D2137B479 for ; Mon, 30 Oct 2000 19:07:11 -0800 (PST) Received: (from magus@localhost) by hurlame.pdl.cs.cmu.edu (8.11.0/8.11.0) id e9V36h225279; Mon, 30 Oct 2000 22:06:43 -0500 (EST) (envelope-from magus) To: Andre Oppermann Cc: Alfred Perlstein , Bosko Milekic , freebsd-net@FreeBSD.ORG Subject: Re: MP: per-CPU mbuf allocation lists References: <20001030104457.E22110@fw.wintelcom.net> <39FDD039.594CED43@telehouse.ch> From: Nat Lanza Date: 30 Oct 2000 22:06:42 -0500 In-Reply-To: Andre Oppermann's message of "Mon, 30 Oct 2000 20:47:05 +0100" Message-ID: Lines: 42 User-Agent: Gnus/5.0807 (Gnus v5.8.7) XEmacs/21.1 (Channel Islands) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Andre Oppermann writes: > Lets assume there is an apache webserver running on the MP machine. > Now this apache process tends to stick to one CPU (does it? would > make sense for cache locality). This process generates a ton of > traffic and needs thousands of mbufs. If I keep the watermark > defaults then this is sub-optimal because on CPU1 I need thousands > of mbuf whereas CPU2 is not using much. If I tune the watermarks to > the required level of CPU1 then I'm wasting an awful lot of mbufs on > the other CPU('s)... Actually, Apache uses a pool of processes. So ideally the active httpd processes would be distributed fairly evenly across the machine's CPUs. Also, if there is only one process heavily consuming mbufs, then presumably acquiring the locks necessary to grab mbufs from the global pool is fast, since there's not going to be any contention for them. > The proposal; Lets have the minimum watermark set on a system-wide > level (sysctl) but the high-watermark would be based per-cpu with > an sliding window calculating the average use of mbufs over a certain > period of time (eg. 10sec). The initial high watermark would be > either min*2 or set upon system initialization to some initial (base) > value. This also takes automagically care of process migration between > the CPU's. Also this system would be self adjusting and self tuning > without admin intervention in common edge cases. I'm not convinced this is really necessary; if there isn't any contention for the mbuf lists, then it doesn't matter too much where they are -- as I understand it, the benefit in per-cpu mbuf lists is that you avoid contention when there are multiple consumers of mbufs. If you don't have multiple consumers, you're not going to have contention, so I don't see the point in adding fancy logic for that case. --nat -- nat lanza --------------------- research programmer, parallel data lab, cmu scs magus@cs.cmu.edu -------------------------------- http://www.cs.cmu.edu/~magus/ there are no whole truths; all truths are half-truths -- alfred north whitehead To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message