From owner-freebsd-fs@FreeBSD.ORG Fri Nov 16 22:50:17 2012 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id B74E74EC; Fri, 16 Nov 2012 22:50:17 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id F02498FC14; Fri, 16 Nov 2012 22:50:16 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id j13so3105310lah.13 for ; Fri, 16 Nov 2012 14:50:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=T0/dvfTMr01kye7SE/Ivn0AdAbmc3KkuLsZ/9zqHzdo=; b=ns9RxgWLYyPT9LBG7m+HNe6sdqHPFg6LJrX/q3lVIZGYyyblMqURRkyXLAFeXcy4Fy TLiZZhOSGnBQlU38i099dhuO4PmBHRmJYzXP7auy1iYgP5x14hIGDw27W7pPiQ+IYMfV uhsE0iFesJUsxh+fx6VVgxfULqd3vq0LQWrrkHVxqfNEG2jhe47ZKuehvcMG0W9dStdI sdjIm5naThqnBPO2c4ligsRYDMBma7210OJJBTC0nHJbKeNq93Nydz9ijkMu3YLglVl6 sVKvQ3DtjiDhYi77shBwzs8ce2DtIK34wpWkQLwqKpp+vB86C6io5ZewhwGVHy5h1afa PkNw== MIME-Version: 1.0 Received: by 10.152.106.110 with SMTP id gt14mr5573158lab.1.1353106214812; Fri, 16 Nov 2012 14:50:14 -0800 (PST) Received: by 10.112.49.138 with HTTP; Fri, 16 Nov 2012 14:50:14 -0800 (PST) In-Reply-To: <16B803FB-0964-4237-8F25-291470E7EFB5@gmail.com> References: <50A130B7.4080604@cse.yorku.ca> <20121113043409.GA70601@neutralgood.org> <50A2B95D.4000400@cse.yorku.ca> <50A2F804.3010009@freebsd.org> <20121115001840.GA27399@FreeBSD.org> <20121115102704.6657ee52@suse3> <20121116091747.2c1bfc55@suse3> <16B803FB-0964-4237-8F25-291470E7EFB5@gmail.com> Date: Fri, 16 Nov 2012 17:50:14 -0500 Message-ID: Subject: Re: RHEL to FreeBSD file server From: Zaphod Beeblebrox To: Steven Schlansker Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, John X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Nov 2012 22:50:17 -0000 On Fri, Nov 16, 2012 at 2:38 PM, Steven Schlansker wrote: > > On Nov 16, 2012, at 11:24 AM, Zaphod Beeblebrox wrote: > >> to connect 24 (or 30) drives. If you're using "green" 2T's, the speed >> of 4 disks on one channel is about half of the speed of >> 1-disk-per-channel. >> >> >> The reason I say all this... is that this config runs about $3500-ish >> here in Canada (where green 2T's are ~$109). The ZeusRAM drive up >> there is 2/3 of that. > > Curious -- have you been running this setup for any length of time? There's > a fair number of horror stories about the "green" drives in particular. > > The power management is very aggressive about spin down, causing many unneeded > power on/off cycles, dramatically reducing lifespan in a RAID configuration. > > Additionally, supposedly the error recovery is inappropriate leading to drive > failure events. (I believe the feature is known as TLER, time-limited error recovery) > > Have you run into this? I do see the power down events. If they array is quiet for some time, it takes 10-ish seconds to respond... but then it's supposedly saving power. I've been running about a dozen of these setups for myself and clients... the longest has been running since the early days of ZFS on FreeBSD (my home array). I find each array looses about 1 drive a year. If occasional errors pop up, they almost always indicate that a drive _will_ fail. Smart seems rather universally dumb on this issue... at least for the green drives. Due to the limitation of the port multipliers, increasing the speed of this setup is often quite expensive (requiring RAID cards and 8x slots and whatnot). Without changing the drives, 24 ports of SATA cost roughly $1200, last I looked at it. While the majority of these are "backup" or "archive" file servers, some do run production loads.