Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 11 Mar 2015 10:24:01 -0700
From:      Adrian Chadd <adrian@freebsd.org>
To:        "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org>
Subject:   re-visiting the NUMA physical allocator work in -HEAD
Message-ID:  <CAJ-VmokVsB0Pbd3bY6u14QoAiOOsE9=dbqg-dvK8R3x0MGrdMw@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
hi,

I've started fleshing out the vm_phys allocator work in -HEAD. This is
based on what's in -9 with some discussions from others - namely, we'd
like policies and iterators, rather than just hard-coded stuff.

So, here's where i'm doing it:

https://github.com/freebsd/freebsd/compare/master...erikarn:local/adrian_numa_policy

My aim is to get the following done:

* add vm_phys default allocator policies - round robin, first touch,
first touch + round robin (done);
* convert the vm_phys domain iteration to use an iterator instead of
hard-coded stuff (done);
* add some basic policy description objects and API - likely just
something that's copied around to begin with;
* add counters for allocations - by domain; looking at allocation
types and whether they were met (ie, things like "domain-local
allocation to domain 0 success, domain-local allocation to domain 0
failure; domain 0 round-robin allocation, etc."
* add some very basic domain and domain policy awareness to busdma.

At this point I'll stop and get buy-in for pushing it into -HEAD.

Trying to plumb things through contigmalloc / kmem_alloc / malloc all
the way down to vm_phys for domain aware kernel allocations won't
happen this pass - the vm_reserv code needs to know about domains and
some basic domain policy. I want to get the above done and pushed into
-HEAD so we can then build upon it, and that includes teaching
vm_reserv about domains and domain policies.

I'm treating all of this NUMA stuff as "if it's in -HEAD then great,
but the API will be subject to change."

With the basic iterator stuff and policy stuff done, I can already
measure the effectiveness of first-touch on my dual and quad socket
test boxes. It's indeed doing the right thing for some userland use
cases. It seems completely silly to not have this bare minimum
functionality in -HEAD. :(



-adrian



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-VmokVsB0Pbd3bY6u14QoAiOOsE9=dbqg-dvK8R3x0MGrdMw>