Date: Fri, 16 Sep 2011 18:22:06 -0700 (PDT) From: Jason Usher <jusher71@yahoo.com> To: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org> Subject: ZFS obn FreeBSD hardware model for 48 or 96 sata3 paths... Message-ID: <1316222526.31565.YahooMailNeo@web121205.mail.ne1.yahoo.com>
next in thread | raw e-mail | index | archive | help
=0AHello,=0A=0AI am building my first FreeBSD based ZFS system and am decid= ing on a hardware model.=A0 The overriding requirement is:=0A=0A1) immediat= ely support 48 internal sata3 drives at full bandwidth - every drive has in= dependent path to CPU=0A=0A2) future expansion to support another 48 drives= on an attached JBOD, all of which ALSO have their own independent path to = CPU=0A=0AThe first question is:=A0 how many pcie 2.0 lanes does a motherboa= rd need to run 96 independent sata3 connections ?=A0 Am I correct that this= is extremely important ?=0A=0ANext, I see a lot of implementations done wi= th LSI adaptors - is this as simple as choosing (3) LSI SAS 9201-16i for th= e 48 internal drives and (3) LSI SAS 9201-16e for the external drives ?=A0 = I remember that these cards are supported with mps(4) in FreeBSD, but only = in 9.x (?) - is that still the case, or is that support in 8.2 or later in = 8.3 ?=0A=0ASo I will boot of a pair of mirrored SSDs formatted UFS2 - easy.= =A0 But I would also like to spec and use a ZIL+L2ARC and am not sure where= to go ... the system will be VERY write-biased and use a LOT of inodes - s= o lots of scanning of large dirs with lots of inodes and writing data.=A0 S= omething like 400 million inodes on a filesystem with an average file size = of 150 KB.=0A=0A- can I just skip the l2arc and just add more RAM ?=A0 Woul= dn't the RAM always be faster/better ?=A0 Or do folks build such large L2ar= cs (4x200 GB SSD ?) that it outweighs an extra 32 GB of RAM ?=0A=0A- provid= ed I maintain the free pcie slot(s) and/or free 2.5" drive slots, can I alw= ays just add a ZIL after the fact ?=A0 I'd prefer to skip it for now and sa= ve that complexity for later...=0A=0AThanks very much for any comments/sugg= estions.=0A
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1316222526.31565.YahooMailNeo>