Date: Sun, 8 Apr 2018 09:29:25 -0700 From: Mark Millard <marklmi26-fbsd@yahoo.com> To: freebsd-hackers@freebsd.org, freebsd-amd64@freebsd.org, FreeBSD Current <freebsd-current@freebsd.org> Subject: Ryzen Threadripper, Hyper-V, and NUMA vs. DIMMs: 3 DIMMs on each side seems to always be in "Local" mode, not "Distributed" Message-ID: <BE68A5DE-5192-4FBF-BD6F-C870A1A423BF@yahoo.com>
next in thread | raw e-mail | index | archive | help
Context: Ryzen Threadripper 1950X under Windows 10 Pro
with Hyper-V (used to run FreeBSD).
In experimenting with switching a Threadripper 1950X to
have ECC RAM I discovered:
A) The maximum ECC memory it would put to use was 96 GiBytes
(3 DIMMs on each side, a 4th on each side was recognized
but was ignored/disabled if present).
B) AMD Ryzen Master classified the 96 GiByte configurations
(with or without the ignored DIMMs) as "Local" without an
ability switch to "Distributed".
C) The downloaded Windows CoreInfo.exe utility agreed on there
being 2 NUMA nodes.
D) As did the result of the User Hardware Topology button
in the Hyper-V Processor > NUMA settings:
On a single virtual non-uniform memory architecture node:
Maximum number of processors : 16
Maximum amount of memory (MB) : 48070
Maximum NUMA nodes allowed on a socket: 2
Only 1 socket.
E) The CoreInfo.exe quick "Approximate Cross-NUMA Node Access
Cost (relative to fastest)" tends to show the 4 numbers
varying from 1.0 to 1.7 when retried repeatedly. An oddity
is that sometimes the 1.0's are between 00 and 01, in fact
this seems usual, and normally at most one 1.0 exists. The
00 row seems to always have the smaller numbers. An example:
00 01
00: 1.2 1.0
01: 1.3 1.5
I had no original intent of playing with NUMA but I figured
that the Threadripper could be configured for such, and even
has configurations that apparently require such as far as
AMD Ryzen Master is concerned, could be of interest and
possible use for folks testing FreeBSD NUMA support.
Since I'd done nothing to build a kernel with NUMA enabled,
FreeBSD 12.0 under Hyper-V did not see the NUMA structure
from (D). One thing that did show during booting was
getting 4 lines of: "SRAT: Ignoring memory at addr" instead
of 2.
===
Mark Millard
marklmi26-fbsd at yahoo.com
( dsl-only.net went
away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE68A5DE-5192-4FBF-BD6F-C870A1A423BF>