From owner-freebsd-net Tue Oct 31 14:18:26 2000 Delivered-To: freebsd-net@freebsd.org Received: from field.videotron.net (field.videotron.net [205.151.222.108]) by hub.freebsd.org (Postfix) with ESMTP id 634A437B4CF for ; Tue, 31 Oct 2000 14:18:24 -0800 (PST) Received: from modemcable213.3-201-24.mtl.mc.videotron.ca ([24.201.3.213]) by field.videotron.net (Sun Internet Mail Server sims.3.5.1999.12.14.10.29.p8) with ESMTP id <0G3B004LQFAM3C@field.videotron.net> for freebsd-net@FreeBSD.ORG; Tue, 31 Oct 2000 17:18:23 -0500 (EST) Date: Tue, 31 Oct 2000 17:22:58 -0500 (EST) From: Bosko Milekic Subject: Re: MP: per-CPU mbuf allocation lists In-reply-to: <20001030115209.G22110@fw.wintelcom.net> X-Sender: bmilekic@jehovah.technokratis.com To: Alfred Perlstein Cc: freebsd-net@FreeBSD.ORG Message-id: MIME-version: 1.0 Content-type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 30 Oct 2000, Alfred Perlstein wrote: > Basically when you have a procducer/consumer, the producer is writing > to the memory while the consumer is only reading. Now if the consumer > free's to his own list rather than the producers list, what happens > is that when the consumer does his next allocation he most likely > is going to write to a page that has it's writablility "owned" by > the producer, this generates bus traffic to inform the producer that > he can no longer write/read cache the memory. By freeing to the > producer's pool you avoid invalidating the producer's cache. Yeah, as you mention below, you don't exactly have a producer/consumer relationship for what concerns mbufs, so improving cache efficiency by grouping together mbufs on per-CPU lists likely won't get you too far. > There is a gotcha here though that would need some thourough > investigation, thinking about how mbufs work, a lot of the time > the mbuf headers are accessed right before they are free'd to > either trim or remove them from linked lists, so perhaps the > free'ing to the CPU that the mbuf was allocated from should only > happen for mbuf clusters, not mbuf headers. Regarding clusters; although I could be wrong, think about what could happen with a socket buffer that can be touched by several CPUs; you would have an invalidation of the cache anyway, AFAIK, and the performance benefit of doing this would no longer be apparent. I suppose that when writing a general purpose allocator (such as hoard), that one should take note of this mainly because in that case, the possibility of having a producer/consumer relationship is likely greater, or more reasonable? > -- > -Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] > "I have the heart of a child; I keep it in a jar on my desk." Regards, Bosko Milekic bmilekic@technokratis.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message