Date: Thu, 21 Jun 2001 14:56:44 -0400 From: Bosko Milekic <bmilekic@technokratis.com> To: Terry Lambert <tlambert2@mindspring.com> Cc: Terry Lambert <tlambert@primenet.com>, freebsd-alpha@FreeBSD.ORG Subject: Re: vx, lge, nge, dc, rl, sf, sis, sk, vr, wb users please TEST Message-ID: <20010621145644.A923@technokratis.com> In-Reply-To: <3B31B4F9.FA2D43F1@mindspring.com>; from tlambert2@mindspring.com on Thu, Jun 21, 2001 at 01:48:57AM -0700 References: <20010619191602.A28591@technokratis.com> <200106200224.TAA24251@usr05.primenet.com> <20010619232624.A29829@technokratis.com> <3B304ADF.C5131399@mindspring.com> <20010620123029.A34452@technokratis.com> <3B30D941.6AE93443@mindspring.com> <20010620135939.A34888@technokratis.com> <3B31B4F9.FA2D43F1@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jun 21, 2001 at 01:48:57AM -0700, Terry Lambert wrote: > I think you now see why I'm a bit more interested in a > generalized version of the allocator, and thing this > change is benign enough that, if Bill Paul approves, it > is OK with me. The change regarding m_devget() is not that closely tied with mb_alloc. I merely happened to discover the problem while testing mb_alloc, that's all. [...] > You could put this in INVARIANTS or DEBUG_MB_ALLOC, or whatever. Will do, (already done locally). > > > Do you have any interest in generalizing your allocator? > > > > Well, not really. See, the allocator is a specialization > > of a general allocator. One of the main design goals of the > > allocator is performance. I tried very hard to give the new > > allocator the advantages required for scalability and the > > "infrastructure" required to reclaim memory while keeping > > approximately the same allocation/deallocation standards of > > the present allocator. One important performance advantage > > of the present allocator relative to say the NetBSD or OpenBSD > > mbuf allocations/deallocations is that > > we *never* have to free pages back to the map from m_free() > > (an expensive procedure). This is precisely why I'd like to > > have freeing eventually implemented from a kproc, when it can > > be handled only when really needed without affecting network > > performance. General purpose allocations should probably be > > handled differently. > > Turning the mbufs type-stable didn't seem to affect the > performance postively to any significant extent; this Not having to perform calls to the VM code made things quite a bit faster. I speak from experience: some time ago, I implemented full-blown freeing to the map from the (old) "free an mbuf" function. Freeing pages directly from m_free() is significantly expensive. I struggled quite a bit and at one point had a discussion with Poul-Henning via Email. He offered some interesting suggestions that involved the actions of a kproc. I saved all the Emails and some of those ideas are found in mb_alloc. > might have just been balancing of the lock overhead, but > I have a hard time believing that it could be that balanced; > the spiking seen in the without/with comparison occurred on > both, so that probably wasn't it. > > I also have a minor concern with the deallocation back > to the bucket locking things up; maybe a kproc per CPU > now would let you queue deallocations that were non-local, > and get rid of many of those locks (substituting locks > that only IPI two CPUs instead of all of them to do a queue > content exchange). Yes and no. I just think that optimizing a case that is not supposed to occur often is a waste of resources (not to mention energy). One looks to optimize the most common case, as for the others, well, we hope to avoid them. I plan to commit the mb_alloc code today. When things in the net code play out eventually, we may want to test cases like the one you mention and see if they really offer an overall performance boost. > If you have a general version of the allocator, it would be > better to show it now, rather than a cut-down version. My > guess is that it would not have an effect on John Lemon's > or your benchmarks, one way or the other. > > Obviously, I'm a bit biased, given that I don't really > allocate mbufs any more, I only pull them off or put them > on a freelist. Well, that's nice. But, unfortunately, not everyone has the physical memory to give away [permanently] to the expense of the mbuf subsystem. The problem is obvious: you have unused wired-down pages dispersed across a free list. > > The whole reason I wrote mb_alloc separately from > > memcache altogether is to allow for the different type of > > freeing to occur, > > I think it should be a flag in the object description, it > would add a single compare, but the code would be much more > general. Perhaps. Ideally, we may want to look at allocating network related things from mb_alloc and have them all optionally freed from a kproc. I'd like to stay away from the possibility of dealing with freeing back to the map from the mbuf free code. > -- Terry Later, -- Bosko Milekic bmilekic@technokratis.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-alpha" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010621145644.A923>