From owner-freebsd-fs@FreeBSD.ORG  Fri Sep 23 11:06:46 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id A1AB61065672
	for <freebsd-fs@freebsd.org>; Fri, 23 Sep 2011 11:06:46 +0000 (UTC)
	(envelope-from daniel@digsys.bg)
Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230])
	by mx1.freebsd.org (Postfix) with ESMTP id 2AEA18FC19
	for <freebsd-fs@freebsd.org>; Fri, 23 Sep 2011 11:06:45 +0000 (UTC)
Received: from digsys236-136.pip.digsys.bg (digsys236-136.pip.digsys.bg
	[193.68.136.236]) (authenticated bits=0)
	by smtp-sofia.digsys.bg (8.14.4/8.14.4) with ESMTP id p8NB6Zrd055136
	(version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO);
	Fri, 23 Sep 2011 14:06:41 +0300 (EEST)
	(envelope-from daniel@digsys.bg)
Mime-Version: 1.0 (Apple Message framework v1244.3)
Content-Type: text/plain; charset=windows-1252
From: Daniel Kalchev <daniel@digsys.bg>
In-Reply-To: <1316729475.54812.YahooMailClassic@web121211.mail.ne1.yahoo.com>
Date: Fri, 23 Sep 2011 14:06:35 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <81C28D6A-47E6-4680-BFB2-BE36CAE4DD8C@digsys.bg>
References: <1316729475.54812.YahooMailClassic@web121211.mail.ne1.yahoo.com>
To: Jason Usher <jusher71@yahoo.com>
X-Mailer: Apple Mail (2.1244.3)
Cc: freebsd-fs@freebsd.org, Rich <rincebrain@gmail.com>
Subject: Re: redux: 48 or 96 sata3 paths ... specific ZFS hardware proposal
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 23 Sep 2011 11:06:46 -0000


On Sep 23, 2011, at 01:11 , Jason Usher wrote:

> You are saying I want the IR, yes ?
>=20

You want IT firmware. You also need to follow exactly the procedure to =
update the firmware and you also should use the Supermicro provided =
software. In particular, pay attention to write down the original SAS =
addresses.

>=20
>> - If memory serves, the way the PCIe bandwidth works on
>> that board is
>> that slots 1-3 go to CPU1's interface and slots 4-7 go to
>> CPU2.
>=20
>=20
> So, if I add four LSI cards, I should probably put two of them on =
slots 1,2 and the other two on 4,5 ... yes ?

You also need to make sure you have two processors. In you only have =
one, then ports 4-7 will get routed to the first CPU, via the same QPI =
link to the hub chipset.

You also need to make sure RAM is evenly populated for both CPUs, but =
this should already be in the motherboard's documentation.

It is somewhat unclear how good current FreeBSD handles NUMA and similar =
architecture=85 At worst, you will end up with just the capacity of one =
QPI link to the peripherals.

Daniel=