From owner-freebsd-fs@FreeBSD.ORG Wed Dec 23 20:26:38 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3B03F106566C for ; Wed, 23 Dec 2009 20:26:38 +0000 (UTC) (envelope-from solon@pyro.de) Received: from srv23.fsb.echelon.bnd.org (mail.pyro.de [83.137.99.96]) by mx1.freebsd.org (Postfix) with ESMTP id BA47C8FC1A for ; Wed, 23 Dec 2009 20:26:37 +0000 (UTC) Received: from port-87-193-183-44.static.qsc.de ([87.193.183.44] helo=MORDOR) by srv23.fsb.echelon.bnd.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NNXmy-0003h3-Jq for freebsd-fs@freebsd.org; Wed, 23 Dec 2009 21:26:36 +0100 Date: Wed, 23 Dec 2009 21:26:12 +0100 From: Solon Lutz X-Mailer: The Bat! (v3.99.25) Professional Organization: pyro.labs berlin X-Priority: 3 (Normal) Message-ID: <1696529130.20091223212612@pyro.de> CC: freebsd-fs@freebsd.org In-Reply-To: <26F8D203-A923-47D3-9935-BE4BC6DA09B7@corp.spry.com> References: <568624531.20091215163420@pyro.de> <42952D86-6B4D-49A3-8E4F-7A1A53A954C2@spry.com> <957649379.20091216005253@pyro.de> <26F8D203-A923-47D3-9935-BE4BC6DA09B7@corp.spry.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.1 (/) X-Spam-Report: Spam detection software, running on the system "srv23.fsb.echelon.bnd.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: Hi, I opted for two 12-disc raidz2. Reasons were: Space is more important than performance. But performance is very poor - have a look at the iostats - sometimes nothing really seems to happen for up to ten seconds, or very little data gets written. Might this be a problem of the amd64 system having only 4GB of RAM? Any tuneable sysctls? Enabling prefetch didn't help...: [...] Content analysis details: (0.1 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.4 ALL_TRUSTED Passed through trusted hosts only via SMTP 1.6 MISSING_HEADERS Missing To: header X-Spam-Flag: NO Subject: Re: ZFS RaidZ2 with 24 drives? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Dec 2009 20:26:38 -0000 Hi, I opted for two 12-disc raidz2.=20 Reasons were: Space is more important than performance. But performance is very poor - have a look at the iostats - sometimes nothi= ng really seems to happen for up to ten seconds, or very little data gets written. Might t= his be a problem of the amd64 system having only 4GB of RAM? Any tuneable sysctls? Enabling = prefetch didn't help...: capacity operations bandwidth capac= ity operations bandwidth pool used avail read write read write pool used a= vail read write read write ---------- ----- ----- ----- ----- ----- ----- ---------- ----- -= ---- ----- ----- ----- ----- backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 1.23K 0 153M backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 510 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 510 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 835 0 104M backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 645 0 68.7M backup-7 1.23T 130G 0 0 0 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 474 0 59.3M 0=09store-1 616G 1= 5.6T 0 4 0 5.49K backup-7 1.23T 130G 494 0 61.8M 0=09store-1 616G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 53 0 6.73M 0=09store-1 617G 1= 5.6T 0 19 0 39.4K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 30 0 140K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 33 0 72.8K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 65 0 162K backup-7 1.23T 130G 71 0 8.92M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 636 0 79.4M 0=09store-1 617G 1= 5.6T 0 4 0 10.5K backup-7 1.23T 130G 485 0 60.6M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 85 0 10.7M 0=09store-1 617G 1= 5.6T 0 15 0 39.9K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 17 0 40.4K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 31 0 73.3K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 62 0 156K backup-7 1.23T 130G 17 0 2.24M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 189 0 23.6M 0=09store-1 617G 1= 5.6T 0 9 0 14.0K backup-7 1.23T 130G 2 74 133K 247K=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 12 0 17.5K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 1 0 255K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 100 0 12.6M backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 1012 0 126M backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 33 0 137K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 19 0 293K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 153 0 19.2M backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 109 0 13.7M backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 4 0 15.5K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 1.54K 0 188M backup-7 1.23T 130G 6 0 888K 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 556 0 69.5M 0=09store-1 617G 1= 5.6T 0 8 0 11.0K backup-7 1.23T 130G 500 0 62.5M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 5 0 766K 0=09store-1 617G 1= 5.6T 0 16 0 30.4K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 29 0 136K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 38 0 84.7K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 57 0 145K backup-7 1.23T 130G 92 0 11.6M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 497 0 62.1M 0=09store-1 617G 1= 5.6T 0 8 0 9.48K backup-7 1.23T 130G 534 0 66.7M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 153 0 19.2M 0=09store-1 617G 1= 5.6T 0 12 0 12.0K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 26 0 101K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 34 0 76.3K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 62 0 155K backup-7 1.23T 130G 75 0 9.48M 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 690 0 86.3M 0=09store-1 617G 1= 5.6T 0 6 0 12.0K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0 backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 10 0 14.0K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 1 0 255K backup-7 1.23T 130G 0 0 0 0=09store-1 617G 1= 5.6T 0 0 0 0