From owner-freebsd-fs@FreeBSD.ORG Fri Nov 16 19:24:53 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4DC53552; Fri, 16 Nov 2012 19:24:53 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id 84F8A8FC14; Fri, 16 Nov 2012 19:24:52 +0000 (UTC) Received: by mail-lb0-f182.google.com with SMTP id go10so357342lbb.13 for ; Fri, 16 Nov 2012 11:24:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=RhMg7OEVp+qIJZtGnKcA77Ch8EgP6o10JmvSvJmE3aI=; b=eab37bxGgnRVwKQkxvWAFlNRAjgzn3R+QY8CDNC1fc2a4tTNTmLjjmWxYr8s6uJ9xZ Klo51y4GpqxuWSFPbI+gA+Cozek8yIGY4SQ15qaD7u/jbK/UJoxShNBUcMD2QwGD8f/f V3BbiiJGknXgiAO/E3uBsc7ylTkozIFVus5xnG3wLsLe7YlDDzB38fik6KE0x93IXi/P FW5w7DMuYvXBVSLkHRjBmKRkmAWQosxcClOaZiDOdRVPTOzWmTc4xsuIOQivxuyoRSN0 zaoB9hMzGSHE89EUZKiaxG2DrIleZP/sWidzGxU2EAwIoiBmxLJ+lHDuOKDl9a5lOU19 s+Kg== MIME-Version: 1.0 Received: by 10.152.104.115 with SMTP id gd19mr5250179lab.13.1353093891217; Fri, 16 Nov 2012 11:24:51 -0800 (PST) Received: by 10.112.49.138 with HTTP; Fri, 16 Nov 2012 11:24:51 -0800 (PST) In-Reply-To: <20121116091747.2c1bfc55@suse3> References: <50A130B7.4080604@cse.yorku.ca> <20121113043409.GA70601@neutralgood.org> <50A2B95D.4000400@cse.yorku.ca> <50A2F804.3010009@freebsd.org> <20121115001840.GA27399@FreeBSD.org> <20121115102704.6657ee52@suse3> <20121116091747.2c1bfc55@suse3> Date: Fri, 16 Nov 2012 14:24:51 -0500 Message-ID: Subject: Re: RHEL to FreeBSD file server From: Zaphod Beeblebrox To: Rainer Duffner Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org, John X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Nov 2012 19:24:53 -0000 On Fri, Nov 16, 2012 at 3:17 AM, Rainer Duffner wr= ote: > Am Fri, 16 Nov 2012 01:03:19 -0500 > schrieb Zaphod Beeblebrox : >> The only price I see is $2449 on Dell's website. Ouch. For 8G. > I saw it for 1800=80 somewhere. > FusionIO is 10k for 320G. > > But I've never seen FusionIO used in a Fileserver - probably doesn't > make sense, price-wise. > > IIRC, the OP wanted to build a 16T fileserver. So he will have to spend > some cash anyway. I think it's a safe assumption that Nexenta and its > partners have spent quite some time evaluating the various options. That seems about the same price. I've stamped out a few of the following builds which are surprisingly cheap and fast. You start with an ICH10 motherboard where the 6 ICH10 SATA ports support port multipliers (this is random and you may have to test --- some do and some don't). Bonus marks if your motherboard also has a few other SATA ports. Check out "gamer" boards from the ASUS "RoG" series. Now... port multipliers are built into quite a few things ... 4 drive external boxes are fairly easy to find, but if you look, the port multiplier itself (1-to-4 or 1-to-5) is available separately for about $40. attach port multipliers to each of the 6 ICH10 ports and go on to connect 24 (or 30) drives. If you're using "green" 2T's, the speed of 4 disks on one channel is about half of the speed of 1-disk-per-channel. If your motherboard has 2 extra SATA (sometimes 6G, for instance), they likely _don't_ support multipliers (this seems deliberate). But you can have a 2 disk raid-1 boot of either FFS or ZFS. Another choice is USB for either flash or HD boot. Put as much memory as it supports. Desktop board will often take 8G or 16G now... or more. You'll want a big power supply and an nice i5 :). You'll get close to wire speed GigE out of this setup. 3x8 vdevs in RAID-Z1 or Z2 work nicely. The reason I say all this... is that this config runs about $3500-ish here in Canada (where green 2T's are ~$109). The ZeusRAM drive up there is 2/3 of that.