From owner-freebsd-fs@FreeBSD.ORG Wed May 29 15:19:44 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 8B386E22 for ; Wed, 29 May 2013 15:19:44 +0000 (UTC) (envelope-from nowakpl@platinum.linux.pl) Received: from platinum.linux.pl (platinum.edu.pl [81.161.192.4]) by mx1.freebsd.org (Postfix) with ESMTP id 516F6EE4 for ; Wed, 29 May 2013 15:19:43 +0000 (UTC) Received: by platinum.linux.pl (Postfix, from userid 87) id 90C3947E1A; Wed, 29 May 2013 17:19:35 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on platinum.linux.pl X-Spam-Level: X-Spam-Status: No, score=-1.3 required=3.0 tests=ALL_TRUSTED,AWL autolearn=disabled version=3.3.2 Received: from [10.255.0.2] (c38-073.client.duna.pl [83.151.38.73]) by platinum.linux.pl (Postfix) with ESMTPA id 3129547E15 for ; Wed, 29 May 2013 17:19:35 +0200 (CEST) Message-ID: <51A61C7F.5060007@platinum.linux.pl> Date: Wed, 29 May 2013 17:19:27 +0200 From: Adam Nowacki User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS 4k aligned space overhead References: <51A6002C.2040803@nbi.dk> In-Reply-To: <51A6002C.2040803@nbi.dk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 May 2013 15:19:44 -0000 Free space calculation is done with the assumption of 128k block size. Each block is completely independent so sector aligned and no parity shared between blocks. This creates overhead unless the number of disks minus raidz level is a power of two. Above that is allocation overhead where each block (together with parity) is padded to occupy the multiple of raidz level plus 1 (sectors). Zero overhead from both happens at raidz1 with 2, 3, 5, 9 and 17 disks and raidz2 with 3, 6 or 18 disks. On 2013-05-29 15:18, Hans Henrik Happe wrote: > Hi, > > I've a system with 3TB WD NAS disks for ZFS. I created a 4k aligned 10 > disk RAIDZ2. I noticed the overhead was ~1.4TB for the file system > (3*8*10^12/2^40 - ). Then I tried with different number of > disks: > > 6: 0.2602885812520981 > 7: 1.1858475767076015 > 8: 1.149622268974781 > 9: 0.7288754601031542 > 10: 1.3953630803152919 > 11: 2.061850775964558 > 12: 2.915792594663799 > 13: 1.5491229379549623 > 14: 2.056995471008122 > 15: 2.5648680040612817 > 16: 3.0727405650541186 > 17: 3.5806130981072783 > 18: 0.7912140190601349 > > the other good configs (6 and 18) is okay, but it seem strange that 10 > has higher spaceoverhead than 18. High overhead with 10 disks is because of allocation overhead. 128k / 4k = 32 sectors, 32 sectors / 8 data disks = 4 sectors per disk, 4 sectors per disk * (8 data disks + 2 parity disks) = 40 sectors. 40 is not a multiple of 3 so 2 sector padding is added. (5% overhead) > > I then tried with RAIDZ: > > 5: 0.19996666628867388 > 9: 0.39560710452497005 > 17: 0.8849408514797688 > > This seems correct. Then RAIDZ2 again but with ashift=9: > > 6: 0.2602881230413914 > 10: 0.4523236369714141 > 18: 0.7912133224308491 > > This also seems correct. The 6 and 18 results are basically the same for > ashift 9 and 12. > > Is there an explanation to this? > > I'm running FreeBSD 9.1-RELEASE-p3. > > Cheers, > Hans Henrik Happe > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"