Skip site navigation (1)Skip section navigation (2) 
Date:      Fri, 21 Dec 2018 11:16:40 -0600
From:      Karl Denninger <karl@denninger.net>
To:        freebsd-fs@freebsd.org
Subject:   Re: Suggestion for hardware for ZFS fileserver
Message-ID:  <4f816be7-79e0-cacb-9502-5fbbe343cfc9@denninger.net>
In-Reply-To: <CAEW%2BogYWKPL5jLW2H_UWEsCOiz=8fzFcSJ9S5k8k7FXMQjywsw@mail.gmail.com>
References:  <CAEW%2BogZnWC07OCSuzO7E4TeYGr1E9BARKSKEh9ELCL9Zc4YY3w@mail.gmail.com> <C839431D-628C-4C73-8285-2360FE6FFE88@gmail.com> <CAEW%2BogYWKPL5jLW2H_UWEsCOiz=8fzFcSJ9S5k8k7FXMQjywsw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help 

[-- Attachment #1 --]
On 12/21/2018 10:51, Sami Halabi wrote:
> Hi,
> First thanks for everyone who responded, very helpful.
>
> One 2 more questions/clarifications:
> 1. As many mentioned 2008/3008 LSI HBA I searched (for lsi sas hba 24
> ports) and found many but with lsi i/o controller other than 2008/3008, for
> example 3224)..it seems 2008/3008 are more popular in 8port hba cards.
> 2. I got That as much ram is better however I saw almost no docs /
> recommendations and how to estimate CPU.. So I need more clarification on
> how to size CPU with the following configurations ZFS for raidz2, all
> without dedup:
> A) lz4 compression.
> B) encryption (I must say I still not quiet know what are my options)
> C) A+B
>
> Thanks,
> Sami
> בתאריך יום ו׳, 21 בדצמ׳ 2018, 12:33, מאת
IMHO --

1. RAM *must* be ECC.  No wiggle room here.  Undetected RAM corruption
on a ZFS-heavy system is EXTREMELY likely to get onto the disk with a
correct checksum, which means permanent, undetectable corruption of the
data and possibly a pool that, when certain operations are performed on
it, immediately panics the system.  In other words it's entirely
possible to get into a situation where the only "remedy" is to destroy
the pool impacted and re-create it.  Incidentally this means that
/backups are not optional; you must have them and develop something that
results in a PROVABLE good copy that can be restored in the event of
disaster because of not only this risk but also human error that the
operating system and hardware executes exactly as you requested, either
of which results in unrecoverable data loss./

2. More RAM is better, up to a point, in that cache is faster than disk
I/O in all cases as operations are avoided.  HOWEVER, there are
pathologies in both the FreeBSD VFS and the ARC when considered as a
group.  I and others have tried to eliminate the pathological behavior
under certain workloads (and some of us have had long-running debates on
same.)  Therefore, your workload must be considered -- simply saying
"more is better" may not be correct for your particular circumstances.

3. LZ4 is good for compressible data.  It is worthless for
non-compressible (or already-compressed) data, such as for example MP3s,
MP4s (video), etc. and in fact makes performance worse since the ZFS
code has to detect that the compression is futile and that takes cycles.

4. On FreeBSD I prefer GELI on the base partition to which ZFS is then
pointed as a pool member for encryption at the present time.  It's
proven, uses AES hardware acceleration on modern processors and works. 
Read the documentation carefully and understand your options for keying
(e.g. password only, password + key file, etc) and how you will manage
the security of the key component(s).


-- 
Karl Denninger
karl@denninger.net <mailto:karl@denninger.net>
/The Market Ticker/
/[S/MIME encrypted email preferred]/

[-- Attachment #2 --]
0	*H
010
	`He0	*H

00H^Ōc!5
H0
	*H
010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CA0
170817164217Z
270815164217Z0{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CA0"0
	*H
0
h-5B>[;olӴ0~͎O9}9Ye*$g!ukvʶLzN`jL>MD'7U45CB+kY`bd~b*c3Ny-78ju]9HeuέsӬDؽmgwER?&UURj'}9nWD i`XcbGz\gG=u%\Oi13ߝ4
K44pYQr]Ie/r0+eEޝݖ0C15Mݚ@JSZ(zȏNTa(25DD5.l<g[[ZarQQ%Buȴ~~`IohRbʳڟu2MS8EdFUClCMaѳ!}ș+2k/bųE,n当ꖛ\(8WV8	d]b	yXw	܊:I39
00U]^§Q\ӎ0U#0T039N0b010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CA	@Ui0U00U0
	*H
:P U!>vJnio-#ן]WyujǑR̀Q
nƇ!GѦFg\yLxgw=OPycehf[}ܷ['4ڝ\[p6\o.B&JF"ZC{;*o*mcCcLY߾`
t*S!񫶭(`]DHP5A~/NPp6=mhk밣'doA$86hm5ӚS@jެEgl
)0JG`%k35PaC?σ
׳HEt}!P㏏%*BxbQwaKG$6h¦Mve;[o-Iی&
I,Tcߎ#t wPA@l0P+KXBպT	zGv;NcI3&JĬUPNa?/%W6G۟N000k#Xd\=0
	*H
0{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CA0
170817212120Z
220816212120Z0W10	UUS10UFlorida10U
Cuda Systems LLC10Ukarl@denninger.net0"0
	*H
0
T[I-ΆϏdn;Å@שy.us~_ZG%<MYd\gvfnsa1'6Egyjs"C [{~_KPn+<*pv#Q+H/7[-vqDV^U>f%GX)H.|l`M(Cr>е͇6#odc"YljҦln8@5SA0&ۖ"OGj?UDWZ5	dDB7k-)9Izs-JAv
J6L$Ն1SmY.Lqw*SH;EF'DĦH]MOgQQ|Mٙג2Z9y@y]}6ٽeY9Y2xˆ$T=eCǺǵbn֛{j|@LLt1[Dk5:$=	`	M00<+00.0,+0 http://ocsp.cudasystems.net:88880	U00	`HB0U0U%0++03	`HB
&$OpenSSL Generated Client Certificate0U%՞V=؁;bzQ0U#0]^§Q\ӎϡ010	UUS10UFlorida10U	Niceville10U
Cuda Systems LLC10UCuda Systems CA1!0UCuda Systems LLC 2017 CAH^Ōc!5
H0U0karl@denninger.net0
	*H
۠A0-j%--$%g2#ޡ1^>{K+uGEv1ş7Af&b&O;.;A5*U)ND2bF|\=]<sˋL!wrw٧>YMÄ3\mWR hSv!_zvl? 3_ xU%\^#O*Gk̍YI_&Fꊛ@&1n”} ͬ:{hTP3B.;bU8:Z=^Gw8!k-@xE@i,+'Iᐚ:fhztX7/(hY` O.1}a`%RW^akǂpCAufgDixUTЩ/7}%=jnVZvcF<M=
2^GKH5魉
_O4ެByʈySkw=5@h.0z>
W1000{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0
	`HeE0	*H
	1	*H
0	*H
	1
181221171640Z0O	*H
	1B@KVRv.PuY\?-dÍk}لZȽ-mXE./*OB20l	*H
	1_0]0	`He*0	`He0
*H
0*H
0
*H
@0+0
*H
(0	+7100{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0*H
	10{10	UUS10UFlorida10U
Cuda Systems LLC10UCuda Systems CA1%0#UCuda Systems LLC 2017 Int CAk#Xd\=0
	*H
q܉:ǗX;X](γ޽k?:HemJdz:.9M@\?E>#E#	HD`"l2	DKyo}W(TOtW>,mBCͳ#@/ьl_[%BWuq;0a%+GK.R(m-;"^ocPm&lm ?CEGKaS|y:0
Pd=X,\2XR)NxfD?ojȲ)YRB1yOE؛'J0NY|AKKujOF,fѮ^q709+|5+j|Bp:50/Xײ/w#
Qdg*)=	/Y
|tfRs0Ehwz{D(N}
Ƞ

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4f816be7-79e0-cacb-9502-5fbbe343cfc9>