Date: Tue, 4 Oct 2011 01:39:50 +0400 From: Lev Serebryakov <lev@FreeBSD.org> To: mdf@FreeBSD.org Cc: freebsd-hackers@freebsd.org, satish kondapalli <nitw.satish@gmail.com>, Arnaud Lacombe <lacombar@gmail.com> Subject: Re: NUMA Support is there in FreeBSD. Message-ID: <176698417.20111004013950@serebryakov.spb.ru> In-Reply-To: <CAMBSHm-DtXheSETNQ3UFfB9gsehFWOomiDCsUC%2B9MVf7QV_r2w@mail.gmail.com> References: <1317653757588-4865200.post@n5.nabble.com> <CAMBSHm_Gh26HaLfzmmkfNoQaCaqajxwVtHKm8s32aS1AneA80A@mail.gmail.com> <CACqU3MXrN-y6cpJ1_a=revBVGiCYHaUS1xmoqbzC_cfOa=pAwQ@mail.gmail.com> <CAMBSHm-DtXheSETNQ3UFfB9gsehFWOomiDCsUC%2B9MVf7QV_r2w@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello, Mdf.
You wrote 3 =EE=EA=F2=FF=E1=F0=FF 2011 =E3., 21:34:29:
> Your statement isn't incorrect. What I'm saying is that there's no
> KPI for requesting bound memory because, while the netstat example is
> a fine one for where local memory is desired, the majority [1] of
> processing is not bound to a CPU and so round-robin allocations will
> produce uniform performance results -- that is, not the best possible,
> but not wildly fluctuating as scheduling decisions over different runs
> give different remote memory penalties.
We have exactly the same config at ${WORK}, as Arnaud describes. And
we need to process huge (4Gbit+ wire speed in small -- 100-1000
bytes -- packets) UDP traffic. Without fixed affinity of "netisr"
threads our system drops some packets on the way between DMA-mapped
network card buffers and kernel structures. One big difference: we
use Solaris and it have all needed API, KPI and userland control
utilities to tune system, both kernel-side and userland-side. Even
Solaris, though, could no process such traffic "automagically". We
didn't try FreeBSD, as our ops knows nothing about it (I'm only
FreeBSD fan in team and I'm developer, not operations)...
I wrote this as example, that for some tasks system NEEDS all these
NUMA-specific knobs.
BTW, NUMA-aware allocator in HotSpot (Sun's, errr, sorry, Oracle's
Java VM), added between Java6 and Java7, increased performance for some
workloads up to 300% on 72-way system (SunFire 15K), and gives about
3% performance drop on worst situations :) And it was allocator in
virtual machine! But it would have been impossible without kernel
API, so this changes works well only on HotSpot/Solaris ;-)
Again, I wrote this all to show, that NUMA-awareness could be very
useful on big iron.
--=20
// Black Lion AKA Lev Serebryakov <lev@FreeBSD.org>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?176698417.20111004013950>