Date: Fri, 10 Oct 2014 01:23:50 -0700 From: Adrian Chadd <adrian@freebsd.org> To: Warner Losh <imp@bsdimp.com> Cc: "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: [rfc] enumerating device / bus domain information Message-ID: <CAJ-Vmo=GJnk3U-C6cbKNGv9zJhf2MeVwMWAVDaAyFMwoWk3-Ow@mail.gmail.com> In-Reply-To: <838B58B2-22D6-4AA4-97D5-62E87101F234@bsdimp.com> References: <CAJ-VmokF7Ey0fxaQ7EMBJpCbgFnyOteiL2497Z4AFovc%2BQRkTA@mail.gmail.com> <2975E3D3-0335-4739-9242-5733CCEE726C@bsdimp.com> <CAJ-VmonbGW1JbEiKXJ0sQCFr0%2BCRphVrSuBhFnh1gq6-X1CFdQ@mail.gmail.com> <838B58B2-22D6-4AA4-97D5-62E87101F234@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
... because for some topologies, as you've said, the devices are equal cost to all CPUs/memories. In the current parlance they don't have a VM domain and they're not assigned a proximity domain at all. We currently don't have a concept of "all domains" when compiling a kernel with a VM domain count of > 1. Now, should there be one? Yes. Should it be -1? Who knows. Let's have that discussion. Maybe we can just label -1 as the "default whole machine VM domain" and the behaviour won't change. And yes, I know about device costs. I was thinking about implementing this using the device/CPU/memory _PXM and for platforms with SLIT tables mapping out the actual costs to various other proximity domains. I'll eventually write a SLIT parser anyway as we may want the memory allocation to be "local" vs varying levels of "less local" rather than "local" vs "non-local". Now, whether the bus code can completely enumerate all of the requirements for allocating device local memory is another discussion. But drivers right now don't consistently at all say "please route stuff locally"; nor do they consistently say "please give me the cpuset of things that are local so I can allocate how many default queues and where to constrain them to run" - they just arbitrarily pick how much of what to run where and how. One simple version of the device/bus locality for this stuff is now in -HEAD. There's a little missing glue to do. Let's have the conversation of "how should drivers do this stuff and have the defaults behave consistently but be easily changed" so we can start experimenting with different ways to do all of this and move towards something that'll appear in -11. -a -a
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ-Vmo=GJnk3U-C6cbKNGv9zJhf2MeVwMWAVDaAyFMwoWk3-Ow>