From owner-freebsd-amd64@FreeBSD.ORG Mon Feb 2 03:45:33 2004 Return-Path: Delivered-To: freebsd-amd64@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D153516A4CE; Mon, 2 Feb 2004 03:45:33 -0800 (PST) Received: from smtp.helinet.de (smtp.helinet.de [212.37.38.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B86D43D2F; Mon, 2 Feb 2004 03:45:29 -0800 (PST) (envelope-from braukmann@tse-online.de) Date: Mon, 02 Feb 2004 12:45:18 +0100 From: Andreas Braukmann To: obrien@freebsd.org Message-ID: <2147483647.1075725918@[192.168.111.140]> In-Reply-To: <20040202005326.GB60117@dragon.nuxi.com> References: <20040202005326.GB60117@dragon.nuxi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Virus-Scanned: by amavisd-new-20030314-p2 (Debian) at helinet.de cc: amd64@freebsd.org Subject: Re: Dual processor, AMD 64 machine freezing. X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Feb 2004 11:45:33 -0000 --On Sonntag, 1. Februar 2004 16:53 Uhr -0800 David O'Brien wrote: > On Mon, Feb 02, 2004 at 12:51:34AM +0100, Andreas Braukmann wrote: >> > P.S. For best performance, I think you really want to run 4x 512 MB. >> > Running with two DIMMs means either you only get 64-bit memory access (not >> > 128-bit) or else you need to put both DIMMs into the CPU1 memory slots >> > (which means CPU2 will have to access those through hypertransport). >> >> That's theory. ;-) >> Since the allocators don't know about the numa-like architecture >> memory would be accessed through hyptertransport (statistically) >> more or less "half of the time". (CPU0 ---> HT ---> MEM1 ; >> CPU1 ---> HT ---> MEM0) > > Its not theory, its fact -- even w/o a NUMA aware OS. Statistically, 1/2 > the accesses by a CPU are to local memory, 1/2 to distant memory. If you > put all them memory on a single CPU then you've got two processors trying > to access memory, saturating the memory controller on the single CPU with > memory -- thus giving you less BW. You are right, of course. I have two nearly identical systems. One is equipped with two 1 GByte Dimms both connected to CPU0, the second is equipped with four 512 MByte Dimms (2 at CPU0 and 2 at CPU1). My "real world" benchmarks (no micro- benchmarking involved) showed, that the system with memory on both CPUs performs minimal better (<< 5%) than the system having its complete RAM on one CPU. Choosing a configuration that allows to make use of both memory controller channels (even using only one CPU's controller) pays of significantly (20% - 30%). Especially for the K8S (having only 4 + 2 DIMM-slots) I would rather buy 2 * 1 GByte Dimms (to be able to simply add two more). > Your diagram above leaves out the memory controller (and its request buffer). Point taken. My remark was over simplified and not to the point. -Andreas