From owner-freebsd-hackers@FreeBSD.ORG Wed Jan 8 08:03:47 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D0FAA6D for ; Wed, 8 Jan 2014 08:03:47 +0000 (UTC) Received: from mail-qe0-x231.google.com (mail-qe0-x231.google.com [IPv6:2607:f8b0:400d:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CF95E1B2A for ; Wed, 8 Jan 2014 08:03:46 +0000 (UTC) Received: by mail-qe0-f49.google.com with SMTP id w7so1447115qeb.36 for ; Wed, 08 Jan 2014 00:03:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=Oe4UIG0vyoFNisuPyKdQxkBUP/Wf934AmwYb2Nu8wS0=; b=NQj60L8j5lXLpa3ckftNVOCmCRUfH49P9afGEgZt64AYVMwgYgbyVKTRF8AXrtZtDz Y5x7y5/7tXoPon1ChGpLh2bKu+8KGdRIyecASMxdVe1efon03rODfOj32nf4LdiSSrnK ewc8TPv6WOHRIZomScCq1BT5tev2iO2F2EYiXdHScB/rXDy4dSztOlimDqK2y4PHLgB8 4D6HBWfOu5NZOoNQp1MEqRveIz1vx9hVOIFpxnEmsslu7DX8tB746MOcjUif2sbck7gv 2iz2JKAWvjisd3XWYSSpyzaHIFHcA4hd/oDcVkwbf1uRblePQinxgvVtm+yZvYPO5uP8 Z+Yw== MIME-Version: 1.0 X-Received: by 10.49.30.197 with SMTP id u5mr11355603qeh.33.1389168226110; Wed, 08 Jan 2014 00:03:46 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.224.52.8 with HTTP; Wed, 8 Jan 2014 00:03:46 -0800 (PST) In-Reply-To: References: Date: Wed, 8 Jan 2014 00:03:46 -0800 X-Google-Sender-Auth: sr1kLfpXW3KD12WaCki0OUHG2XQ Message-ID: Subject: Re: Working on NUMA support From: Adrian Chadd To: Andrew Bates Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jan 2014 08:03:47 -0000 Cool! Do you have any working code to implement the API, or is this just in the design phase right now? -a On 6 January 2014 12:11, Andrew Bates wrote: > Hey all, > > My name is Andrew Bates, and I would like to take a bit of your time to > talk about NUMA support. > > Supporting Non-Uniform Memory Access in FreeBSD is something that has bee= n > brought up in the past < > http://freebsd.1045724.n5.nabble.com/NUMA-Support-is-there-in-FreeBSD-td4= 865200.html>. > This is becoming increasingly important now that multiprocessor > systems > are an expanding technology, thus performance is scaling in terms of cpu > count, rather than just clock rate. > > There is a great opportunity here to optimize performance. After being > asked to look into this by the EMC Isilon Storage Division, myself and a > few colleagues advised by Andrew Pilloud and Jeff Roberson would like to > propose APIs to handle basic memory allocation/management to specific NUM= A > domains. > > What we have devised so far consists of two levels. First there are the > KPIs, to expose NUMA functionality at a thread level of domain affinity. > Secondly, there would be a userspace/interface to take advantage of the > proposed APIs, thus giving users the capability to make their application= s > NUMA-aware. > > We took the time to look into how many other systems (Linux, Macintosh, > Solaris, Windows) already approach this problem, so there are some aspect= s > of our solution that are similar to how Linux and Solaris handle NUMA. > Unlike Linux libnuma, we are only proposing a few additions and a minimal > library that can easily be expanded later to suit users=92 needs. > > > KISS in mind, we came up with the following KPI prototypes (freebsdnuma.h= ) > to uncover NUMA in a usable fashion: > > > - > > cpuset_get_memory_affinity() > - > > cpuset_set_memory_affinity() > - > > move_pages() > - > > migrate_pages() > - > > get_numa_cpus() > - > > get_numa_weights() > > > Then to the second part, we have the following userspace API prototypes > (numanor.h) for our interface and testing purposes: > > > - > > is_numa_available() > - > > set_thread_on_domain() > - > > set_memory_policy() > - > > move_thread() > > > In much much more detail, you can learn more about these prototypes, this > project, view our progress, track along, and give input on our github rep= o > < https://github.com/andrewbates09/freebsd-numa > or simply via email. Th= is > repo currently includes fully commented prototypes (like a mini man page) > and will later include additions to the project. > > If anyone has any comments, suggestions, concerns, quandaries, or just > general thoughts please feel free to contact us, as we would love to hear > your input! > > The Leaders: Sakire Arslan Ay, Andrew Pilloud, Jeff Roberson > The Team: Andrew Bates, Joshua Clark, Alex Schuldberg, Dustin Walker > > -- > V/Respectfully, > Andrew M Bates > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= "