From owner-freebsd-questions@FreeBSD.ORG Tue Nov 18 18:05:18 2014 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 532D1C8A for ; Tue, 18 Nov 2014 18:05:18 +0000 (UTC) Received: from newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) by mx1.freebsd.org (Postfix) with ESMTP id 154CBD69 for ; Tue, 18 Nov 2014 18:05:18 +0000 (UTC) Received: from newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) by newmail.rmm.fr (Postfix) with ESMTP id C2B8B461C8 for ; Tue, 18 Nov 2014 19:05:13 +0100 (CET) X-Virus-Scanned: amavisd-new 2.8.0 (20120630) at rmm.fr Received: from newmail.rmm.fr ([213.251.152.9]) by newmail.rmm.fr (newmail.rmm.fr [213.251.152.9]) (amavisd-new, port 10026) with ESMTP id d8PnuUnrMM6b for ; Tue, 18 Nov 2014 19:05:13 +0100 (CET) Received: from newmail.rmm.fr (newmail.rmm.fr [87.98.206.99]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: hidden) by newmail.rmm.fr (Postfix) with ESMTPSA id 8BE01461BD for ; Tue, 18 Nov 2014 19:05:13 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.1 \(1993\)) Subject: Re: Hardware RAID or software for ZFS From: "bsd@todoo.biz" In-Reply-To: <546B820A.7040806@freebsd.org> Date: Tue, 18 Nov 2014 19:05:13 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <2A45AB64-6EB4-4A22-9625-4931605F62F0@todoo.biz> References: <7F1B1DC4-46CA-4B96-836B-2D2D494F60E8@todoo.biz> <546B820A.7040806@freebsd.org> To: Liste FreeBSD X-Mailer: Apple Mail (2.1993) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Nov 2014 18:05:18 -0000 > Le 18 nov. 2014 =C3=A0 18:29, Matthew Seaman a = =C3=A9crit : >=20 > On 11/18/14 15:28, bsd@todoo.biz wrote: >> I would need your help in order to figure out which will be the best = in order to setup a server which will be used as a Poudriere server.=20 >>=20 >> My question is related to the usage of hardware RAID controller = (JBOD) or software based controller (graid)=20 >>=20 >> Server we are targeting is a SuperMicro 1018R-WC0R - 1U >>=20 >> I used to build my server using an HBA such as LSI SAS 9207-8i or LSI = SAS 9201-16i >> Then I=E2=80=99ll configure the HBA as JBOD so that disks can be = presented correctly to the system so that we can build our ZFS FS on top = of that.=20 >>=20 >>=20 >> My sysadmin suggested me to get rid of the HBA and use direct = connectivity and Graid.=20 >>=20 >>=20 >> I would like to have some feedback on various aspects :=20 >>=20 >> 1. performance=20 >> 2. usability=20 >> 3. problem one might have encountered >>=20 >=20 > OK -- Your sysadmin is correct about the direct connectivity thing, = but > graid? Seriously? That's an option for really cheap and some might = say > quite nasty pseudo-raid things you get on some desktop hardware. If > you're going to be building a significant number of packages with any > regularity, then you're going to need something better than that. >=20 > The LSI HBAs you suggest are known to work well with FreeBSD. If = you're > going to be using ZFS -- which is really a no-brainer for a poudriere > box, as poudriere has a lot of functionality built around cloning and > snapshotting and other things that come naturally to ZFS[*] -- well, = in > that case, ideally you want ZFS to have direct access to the disk = devices. >=20 > With LSI, when they say 'JBOD' this direct access is generally what = they > mean -- although depending on the precise model you get, you may need = to > flash the device with different firmware to get the characteristics = you > want. Some other RAID controller manufacturers essentially give you a > load of single-disk RAID-0 devices, so there's still a layer of RAID > controller stuff between ZFS and the disk. That isn't the kiss of = death > to the idea of building a zpool from it, but it's less optimal. >=20 > On the three aspecs you mention: >=20 > performance depends very much on the precise type of disk hardware > you specify, how you configure the zpool (for instance, will you be > using ZIL or ARC devices?) and on having enough RAM in the system to > provide an effective buffer-cache for all those ZFS filesystems. >=20 > usability. I'd say ZFS is pretty usable. Once it's set up, you > don't really need to interact with things at the level of the HBA. It > just works. >=20 > problems: this depends very much on the details of what you're = trying > to do. But IMHO you're on the right track with what you've described = so > far. >=20 > Cheers, >=20 > Matthew >=20 > [*] Although poudriere does now work on UFS as well and has done for a > year or so. >=20 Thank you very much for this documented and precise answer.=20 I think that the hardware we were targeting is a bit too new and is not = yet marked as =C2=AB ok =C2=BB for FreeBSD by SuperMicro.=20 We will probably go for a little bit older hardware that=E2=80=99s 100% = known to work ok with FreeBSD.=20 I have already been building quite large NAS using this setup (JBOD = Controller + ZFS on top) and I had very interesting perfs.=20 I didn=E2=80=99t know about the problem with native JBOD and controller = SW update/upgrade=E2=80=A6 =20 > SuperMicro 1018R-WC0R - 1U This is the one we wanted to target but It doesn=E2=80=99t seem to be on = the hardware list marked as FreeBSD compatible.=20 Thanks all - long leave FBSD.=20 ;-)=20 >=20 >=20 >=20 >=20 ________________________________________________ = =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= = =C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2= =AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF= =C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2= =AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF BSD - BSD - BSD - BSD - BSD - BSD - BSD - BSD - ________________________________________________ = =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= =C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7=C2=AB?=C2=BB=C2=A5=C2=AB?=C2=BB=C2=A7= = =C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2= =AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF= =C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2= =AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF=C2=AF PGP ID --> 0x1BA3C2FD