From owner-freebsd-arch Mon Feb 17 19:14:49 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BAE6137B401 for ; Mon, 17 Feb 2003 19:14:47 -0800 (PST) Received: from tesla.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0746743F75 for ; Mon, 17 Feb 2003 19:14:47 -0800 (PST) (envelope-from bmilekic@unixdaemons.com) Received: (from bmilekic@localhost) by tesla.distributel.net (8.11.6/8.11.6) id h1I3Dnc67975; Mon, 17 Feb 2003 22:13:49 -0500 (EST) (envelope-from bmilekic@unixdaemons.com) Date: Mon, 17 Feb 2003 22:13:49 -0500 From: Bosko Milekic To: Julian Elischer Cc: freebsd-arch@FreeBSD.ORG Subject: Re: mb_alloc cache balancer / garbage collector Message-ID: <20030217221349.A67942@unixdaemons.com> References: <20030217192418.A67144@unixdaemons.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: ; from julian@elischer.org on Mon, Feb 17, 2003 at 05:38:27PM -0800 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Feb 17, 2003 at 05:38:27PM -0800, Julian Elischer wrote: > On Mon, 17 Feb 2003, Bosko Milekic wrote: > [...] > Bosko, If I have one NIC bound to one CPU (a future capability say,) > and another bound to a second, and there is a stream of packets fron > NIC1 to NIC2 (we are routing) at (say) 30,000 packets per second, > what is the path by which those 30,000 packets make their way from > CPU2's cache of mbufs, back to CPU1 to be used again? (each second). > We'll imagine there are no packets going the other way. (maybe they take > a different route). If the mbufs are allocated from CPU1's cache then they'll most likely be freed back to CPU1's cache. The way it works is that the mbuf is freed back to its bucket and so to whichever cache the bucket is sitting in. Most likely the bucket will not have migrated caches so you're going to be using CPU1's cache in this scenario, since that's the cache you're allocating from. This is probably not ideal when you do something like bind the NIC to a CPU but it is better than having the freing thread free to its own cache, in which case you'd have a serious debalancing of caches going on. I'm not sure how performance would be impacted either way, but I guess I can't say until we actually bind the NICs each to their own CPUs and measure. -- Bosko Milekic * bmilekic@unixdaemons.com * bmilekic@FreeBSD.org "If we open a quarrel between the past and the present, we shall find that we have lost the future." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message