Date: Wed, 27 Feb 2002 12:39:29 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Bosko Milekic <bmilekic@unixdaemons.com> Cc: Jeff Roberson <jroberson@chesapeake.net>, arch@FreeBSD.ORG Subject: Re: Slab allocator Message-ID: <3C7D4401.9E57D6AD@mindspring.com> References: <20020227005915.C17591-100000@mail.chesapeake.net> <3C7D1E31.B13915E7@mindspring.com> <20020227143330.A34054@unixdaemons.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Bosko Milekic wrote: > On Wed, Feb 27, 2002 at 09:58:09AM -0800, Terry Lambert wrote: > > First, let me say OUTSTANDING WORK! > > > > Jeff Roberson wrote: > > > There are also per cpu queues of items, with a per cpu lock. This allows > > > for very effecient allocation, and also it provides near linear > > > performance as the number of cpus increase. I do still depend on giant to > > > talk to the back end page supplier (kmem_alloc, etc.). Once the VM is > > > locked the allocator will not require giant at all. > > > > What is the per-CPU lock required for? I think it can be > > gotten rid of, or at least taken out of the critical path, > > with more information. > > Per-CPU caches. Reduces lock contention and trashes caches less often. I thinl you are misunderstanding. If the caches are per-CPU, then by definition, they will only ever be accessed by a single CPU, and so contention can be eliminated by ordered atomicity of operations, unlike where there is inter-CPU contention. Per CPU resources are really not something you would expect to be contended between CPUs, and within the context of a single CPU, contention is controllable. > > > I would eventually like to pull other allocators into uma (The slab > > > allocator). We could get rid of some of the kernel submaps and provide a > > > much more dynamic amount of various resources. Something I had in mind > > > were pbufs and mbufs, which could easily come from uma. This gives us the > > > ability to redistribute memory to wherever it is needed, and not lock it > > > in a particular place once it's there. > > > > How do you handle interrupt-time allocation of mbufs, in > > this case? The zalloci() handles this by pre-creation of > > the PTE's for the page mapping in the KVA, and then only > > has to deal with grabbing free physical pages to back them, > > which is a non-blocking operation that can occur at interrupt, > > and which, if it fails, is not fatal (i.e. it's handled; I've > > considered doing the same for the page mapping and PTE's, but > > that would make the time-to-run far less deterministic). > > Terry, how long will you keep thinking that mbufs come through the > zone allocator? :-) For G*d's sake man, we've been over this before! Then take that part out, and answer the question about interrupt time allocations. Whether I'm still substituting mbufs in there when I shouldn't be or not is irrelevent to the question. > > > The other problem is with the per cpu buckets. They are a > > > fixed size right now. I need to define several zones for > > > the buckets to come from and a way to manage growing/shrinking > > > the buckets. > > > > I built a "chain" allocator that dealt with this issue, and > > also the object granularit issue. Basically, it calculated > > the LCM of the object size rounded to a MAX(sizeof(long),8) > > boundary for processor alignment sensitivity reasons, and > > the page size (also for processor sensitivity reasons), and > > then allocated a contiguous region from which it obtained > > objects of that type. All in all, it meant zero unnecessary > > space wastage (for 1,000,000 TCP connections, the savings > > were 1/4 of a Gigabyte for one zone alone). > > That's great, until you run out of pre-allocated contiguous space. At which point you've reached the load bearing capacity of the system, and will have to stop, no matter what. It's not like you can swap mbufs. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C7D4401.9E57D6AD>