Date: Fri, 3 Sep 2010 00:39:01 -0400 From: Zaphod Beeblebrox <zbeeble@gmail.com> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: Michal <michal@sharescope.co.uk>, Joshua Boyd <boydjd@jbip.net>, freebsd-stable@freebsd.org Subject: Re: Extending your zfs pool with multiple devices Message-ID: <AANLkTikYNC9nM21EsRqqGPJYWTPeVRCVHi38GV4j%2BCnH@mail.gmail.com> In-Reply-To: <20100903040841.GA59175@icarus.home.lan> References: <4C7FA50D.4000409@sharescope.co.uk> <AANLkTimaqp6Zu23ADA%2BhcU8Gr8nZtzecRwPLGJaRaVpj@mail.gmail.com> <AANLkTi=38yK0eb6L27X_J6fO4qHHc2qxZz-hD-6wwNqx@mail.gmail.com> <20100903040841.GA59175@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Sep 3, 2010 at 12:08 AM, Jeremy Chadwick <freebsd@jdc.parodius.com> wrote: > On Thu, Sep 02, 2010 at 04:56:04PM -0400, Zaphod Beeblebrox wrote: >> With 1.5T disks, I find that the 4 to 1 multipliers have a small >> effect on speed. =A0The 4 drives I have on the multipler are saturated >> at 100% a little bit more than the drives directly connected. >> Essentially you have 3 gigabit for 4 drives instead of 3 gigabit for 1 >> drive. > > 1:4 SATA replicators impose a bottleneck on the overall bandwidth > available between the replicator and the disks attached, as you stated. > Diagram: > > ICH10 > =A0|||___ (SATA300) Port 0, Disk 0 > =A0||____ (SATA300) Port 1, Disk 1 > =A0|_____ (SATA300) Port 2, eSATA Replicator > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ||||________ (SATA300= ) Port 0, Disk 2 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 |||_________ (SATA300= ) Port 1, Disk 3 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 ||__________ (SATA300= ) Port 2, Disk 4 > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 |___________ (SATA300= ) Port 3, Disk 5 > > If Disks 2 through 5 are decent disks (pushing 100MB/sec), essentially > you have 100*4 =3D 400MB/sec worth of bandwidth being shoved across a > 300MB/sec link. =A0That's making the assumption the disks attached are > magnetic and not SSD, and not taking into consideration protocol > overhead. > A better choice is a SATA multilane HBA, which are usually PCIe-based > with a single connector on the back of the HBA which splits out to > multiple disks (usually 4, but sometimes more). That's just connector-foo. The cards are still very expensive. Many ZFS loads don't saturate disks ... or don't saturate them consistently. I just built several systems with one port per disk --- and those cards tended towards $100/port. 1:4 replicators are less than $10/port and the six port motherboards don't seem to add any cost (4 or 6 SATA ports seem standard now). My point is: if you're building a database server and speed is all you care about, then one port per disk makes sense. If you are building a pile of disk and you're on a budget, port replicators are a good solution.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikYNC9nM21EsRqqGPJYWTPeVRCVHi38GV4j%2BCnH>