From owner-freebsd-arch@FreeBSD.ORG Fri May 20 08:36:34 2005 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9A02016A4CE; Fri, 20 May 2005 08:36:34 +0000 (GMT) Received: from mail.chesapeake.net (chesapeake.net [208.142.252.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0EC6B43D7F; Fri, 20 May 2005 08:36:34 +0000 (GMT) (envelope-from jroberson@chesapeake.net) Received: from mail.chesapeake.net (localhost [127.0.0.1]) by mail.chesapeake.net (8.12.10/8.12.10) with ESMTP id j4K8aXS3089157; Fri, 20 May 2005 04:36:33 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost)j4K8aXbH089153; Fri, 20 May 2005 04:36:33 -0400 (EDT) (envelope-from jroberson@chesapeake.net) X-Authentication-Warning: mail.chesapeake.net: jroberson owned process doing -bs Date: Fri, 20 May 2005 04:36:32 -0400 (EDT) From: Jeff Roberson To: Scott Long In-Reply-To: <428CDD17.7020207@samsco.org> Message-ID: <20050520043319.N82493@mail.chesapeake.net> References: <200505191431.08189.jkim@niksun.com> <428CDD17.7020207@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-arch@freebsd.org cc: rwatson@freebsd.org cc: freebsd-amd64@freebsd.org cc: Jung-uk Kim Subject: Re: AMD64 NUMA-awareness? X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2005 08:36:34 -0000 On Thu, 19 May 2005, Scott Long wrote: > Jung-uk Kim wrote: > > > ULE scheduler paper > > (http://www.usenix.org/publications/library/proceedings/bsdcon03/tech/roberson.html) > > says: > > > > 'SMT introduces a concept of non-uniform processors into ULE which > > could be extended to support NUMA. The concept of expressing the > > penalty for migration through the use of separate queues could be > > further developed to include a local and global load-balancing > > policy. At the time of this writing, however, FreeBSD does not > > support any true NUMA capable machines and so this is left until such > > time that it does.' > > > > I am not sure about the meaning of 'true NUMA capable machines' but > > AMD64 is ccNUMA unless I am completely mistaken, and FreeBSD/amd64 is > > well-supported. Even multicore processors are available now and > > Intel is going to release dual-core processors with HTT to make > > matters worse. > > > > Even HTT by itself presents some interesting scheduling challenges that > need to be accounted for. > > > Is there anybody working on this? > > Jeff stated recently that both ULE and 4BSD have some very primitive > awareness of topology, but that much more work is needed. NUMA would > also benefit from the UMA memory allocator understanding memory topology > and working with the scheduler to keep processes on CPUs that are > closest to their memory. Unfortunately, there doesn't appear to be > much work going on, so if anyone is interested, it's a great project and > I encourage all to help. I have put a great deal of thought into the issues in both UMA and schedulers for NUMA. I have not put any code in as of yet, however. I think that linux has had limited success with numa support on AMD64. The remote penalty doesn't seem to be great enough to really warrant much more complicated algorithms. Interested parties are certainly welcome to explore this more. I believe remote accesses are in the range of 33-50% more expensive than local. We might benefit by making a copy of the kernel text for each processor. That would be a good starting point to see if any gains are observed in the best case. > > Scott > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" >