From owner-freebsd-fs@FreeBSD.ORG Thu Dec 18 00:10:57 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D61681065675 for ; Thu, 18 Dec 2008 00:10:57 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (email.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 8EBF38FC12 for ; Thu, 18 Dec 2008 00:10:57 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id AE8EB17E55; Thu, 18 Dec 2008 11:10:54 +1100 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=0.1 required=10.0 tests=ALL_TRUSTED,MISSING_HEADERS autolearn=no version=3.2.3 Received: from [10.1.50.60] (ppp121-44-3-41.lns10.syd7.internode.on.net [121.44.3.41]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id B427017E49; Thu, 18 Dec 2008 11:10:49 +1100 (EST) Message-ID: <494994F9.4010105@modulus.org> Date: Thu, 18 Dec 2008 11:10:33 +1100 From: Andrew Snow User-Agent: Thunderbird 2.0.0.14 (X11/20080523) MIME-Version: 1.0 References: <5f67a8c40812171351j66dc5484pee631198030a5739@mail.gmail.com> <20081217231757.GE27041@lor.one-eyed-alien.net> In-Reply-To: <20081217231757.GE27041@lor.one-eyed-alien.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: More on ZFS filesystem sizes. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Dec 2008 00:10:57 -0000 > I now have another quandry. I have ZFS on my laptop (two drives, mirrored) > and I "zfs send" backups to my big array (6 drives, raid-Z1). The problem > is that they don't match up As you know, ZFS has variable block sizes from 512 bytes to 128kb with every power of 2 in between. Each block has a fair chunk of meta-data to go with it (those 128 bit pointers aren't very space efficient!) I suppose what you're seeing is due to fragmentation, since with copy-on-write for snapshots, big blocks can be replaced with smaller ones when a file is partially updated, but these can be written more efficiently during the send/receive process, as only the actually referenced data needs to be stored. Given all of that, your numbers are only out by 1 to 1.5%, so is it really that surprising? Regarding du on ZFS, it calculates the result based on the number of blocks consumed by the file, excluding metadata and parity and checksums, and after compression. /usr/ports will be full of tiny, compressable files resulting in a large ratio of metadata to actual file data. "zfs list" returns the space consumed including metadata, parity, and checksums. (Also, filesystem metadata is stored twice by default, or three times optionally, in addition to whatever RAID you are using.) So it is weird, but I believe what you're seeing is normal. Maybe you need special ZFS sunglasses which black out whenever you start trying to look at what ZFS is doing to your files :-) - Andrew