Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 10 Oct 2014 01:23:50 -0700
From:      Adrian Chadd <adrian@freebsd.org>
To:        Warner Losh <imp@bsdimp.com>
Cc:        "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org>
Subject:   Re: [rfc] enumerating device / bus domain information
Message-ID:  <CAJ-Vmo=GJnk3U-C6cbKNGv9zJhf2MeVwMWAVDaAyFMwoWk3-Ow@mail.gmail.com>
In-Reply-To: <838B58B2-22D6-4AA4-97D5-62E87101F234@bsdimp.com>
References:  <CAJ-VmokF7Ey0fxaQ7EMBJpCbgFnyOteiL2497Z4AFovc%2BQRkTA@mail.gmail.com> <2975E3D3-0335-4739-9242-5733CCEE726C@bsdimp.com> <CAJ-VmonbGW1JbEiKXJ0sQCFr0%2BCRphVrSuBhFnh1gq6-X1CFdQ@mail.gmail.com> <838B58B2-22D6-4AA4-97D5-62E87101F234@bsdimp.com>

next in thread | previous in thread | raw e-mail | index | archive | help
... because for some topologies, as you've said, the devices are equal
cost to all CPUs/memories. In the current parlance they don't have a
VM domain and they're not assigned a proximity domain at all. We
currently don't have a concept of "all domains" when compiling a
kernel with a VM domain count of > 1.

Now, should there be one? Yes. Should it be -1? Who knows. Let's have
that discussion. Maybe we can just label -1 as the "default whole
machine VM domain" and the behaviour won't change.

And yes, I know about device costs. I was thinking about implementing
this using the device/CPU/memory _PXM and for platforms with SLIT
tables mapping out the actual costs to various other proximity
domains. I'll eventually write a SLIT parser anyway as we may want the
memory allocation to be "local" vs varying levels of "less local"
rather than "local" vs "non-local".

Now, whether the bus code can completely enumerate all of the
requirements for allocating device local memory is another discussion.
But drivers right now don't consistently at all say "please route
stuff locally"; nor do they consistently say "please give me the
cpuset of things that are local so I can allocate how many default
queues and where to constrain them to run" - they just arbitrarily
pick how much of what to run where and how.

One simple version of the device/bus locality for this stuff is now in
-HEAD. There's a little missing glue to do. Let's have the
conversation of "how should drivers do this stuff and have the
defaults behave consistently but be easily changed" so we can start
experimenting with different ways to do all of this and move towards
something that'll appear in -11.



-a



-a



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=GJnk3U-C6cbKNGv9zJhf2MeVwMWAVDaAyFMwoWk3-Ow>