From owner-freebsd-current@FreeBSD.ORG Mon Dec 19 20:42:57 2011 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC33B106566B for ; Mon, 19 Dec 2011 20:42:57 +0000 (UTC) (envelope-from se@freebsd.org) Received: from nm34-vm3.bullet.mail.bf1.yahoo.com (nm34-vm3.bullet.mail.bf1.yahoo.com [72.30.239.75]) by mx1.freebsd.org (Postfix) with SMTP id 89D9A8FC19 for ; Mon, 19 Dec 2011 20:42:56 +0000 (UTC) Received: from [98.139.212.145] by nm34.bullet.mail.bf1.yahoo.com with NNFMP; 19 Dec 2011 20:42:55 -0000 Received: from [98.139.213.7] by tm2.bullet.mail.bf1.yahoo.com with NNFMP; 19 Dec 2011 20:42:55 -0000 Received: from [127.0.0.1] by smtp107.mail.bf1.yahoo.com with NNFMP; 19 Dec 2011 20:42:55 -0000 X-Yahoo-Newman-Id: 842931.99377.bm@smtp107.mail.bf1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: vcKI.vYVM1mt9XADgJzJryq6KBAePBJWfQKblOV3vd36GaC Xbzc.cy0oVfj6PrvWhroKaldRN6vIg5nRx_tzAomsC91YGlpEJhtD.7HUq0U 3sR42tt3VBNVdMvDeUt1KavC1f87QT_D8FpgUrRCrlms6TphgVfa27zvfQWQ u0FICzuLk_kl17MPO1Qra1TN8hB2LtH9dO.hpLX.sVYaJx3aa3TW6Hk4Sgoy OdHUI2qNYLkzJ89sqcb8IvqyPx8hgXZgl5iFST8MOu30Inyl0woXUP2hiMor x8lqxWuwBwSOleVGfBp3FlHbvSrSQ0yIevOMRoMUYtTOC104dcOp75iAhkkY wwhH1ibG9Z034uIr12j4MNU9ms.nKobCouvxkbuKp2zTVsy58_ZmEdts8f.L QunIFEU5VzFqsCWE- X-Yahoo-SMTP: iDf2N9.swBDAhYEh7VHfpgq0lnq. Received: from [192.168.119.20] (se@81.173.146.234 with plain) by smtp107.mail.bf1.yahoo.com with SMTP; 19 Dec 2011 12:42:55 -0800 PST Message-ID: <4EEFA1CD.4040001@freebsd.org> Date: Mon, 19 Dec 2011 21:42:53 +0100 From: Stefan Esser User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: Michael Reifenberger References: <4EEF488E.1030904@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Current Subject: Re: Uneven load on drives in ZFS RAIDZ1 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Dec 2011 20:42:58 -0000 Am 19.12.2011 17:36, schrieb Michael Reifenberger: > Hi, > a quick test using `dd if=/dev/zero of=/test ...` shows: > > dT: 10.004s w: 10.000s filter: ^a?da?.$ > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 0 378 0 0 12.5 376 36414 11.9 60.6| ada0 > 0 380 0 0 12.2 378 36501 11.8 60.0| ada1 > 0 382 0 0 7.7 380 36847 11.6 59.2| ada2 > 0 375 0 0 7.4 374 36164 9.6 51.3| ada3 > 0 377 0 1 10.2 375 36325 10.1 53.3| ada4 > 10 391 0 0 39.3 389 38064 15.7 80.2| ada5 Thanks! There are surprising differences (ada5 has a queue length of 10 and much higher latency than the other drives). > Seems to be sufficiently equally distributed for a life system... Hmmm, 50%-55% busy on ada3 and ada4 contrasts with 80% busy on ada5. > zpool status shows: > ... > NAME STATE READ WRITE CKSUM > boot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > ada0p3 ONLINE 0 0 0 > ada1p3 ONLINE 0 0 0 > ada2p3 ONLINE 0 0 0 > ada3p3 ONLINE 0 0 0 > ada4p3 ONLINE 0 0 0 > ada5p3 ONLINE 0 0 0 > ... > > The only cases I've seen (and expected to see) unequal load > distributions on ZFS was after extending a nearly full four disk mirror > pool by additional two disks. In my case the pool was created from disk drives with nearly identical serial numbers in its current configuration. Some of the drives have a few more power-on hours, since I performed some tests with them, before moving all data from the old pool the new one, but else everything should be symmetric. Best regards, STefan