From owner-freebsd-stable@FreeBSD.ORG Tue Aug 2 12:55:40 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D62BE1065670 for ; Tue, 2 Aug 2011 12:55:40 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3fd3:cd67:fafa:3d78]) by mx1.freebsd.org (Postfix) with ESMTP id 414FE8FC12 for ; Tue, 2 Aug 2011 12:55:40 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.187.76.163]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.5/8.14.5) with ESMTP id p72CtZsc066415 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Tue, 2 Aug 2011 13:55:36 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) X-DKIM: Sendmail DKIM Filter v2.8.3 smtp.infracaninophile.co.uk p72CtZsc066415 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=infracaninophile.co.uk; s=201001-infracaninophile; t=1312289736; bh=eBw6j4b2hdx+W457qKqWNSNlJgL9opOh0qa099FRY0A=; h=Message-ID:Date:From:MIME-Version:To:Subject:References: In-Reply-To:Content-Type:Cc:Content-Type:Date:From:In-Reply-To: Message-ID:Mime-Version:References:To; z=Message-ID:=20<4E37F3BE.5040207@infracaninophile.co.uk>|Date:=20T ue,=2002=20Aug=202011=2013:55:26=20+0100|From:=20Matthew=20Seaman= 20|User-Agent:=20Mozilla/5.0=20(M acintosh=3B=20Intel=20Mac=20OS=20X=2010.6=3B=20rv:5.0)=20Gecko/201 10624=20Thunderbird/5.0|MIME-Version:=201.0|To:=20freebsd-stable@f reebsd.org|Subject:=20Re:=20ZFS=20directory=20with=20a=20large=20n umber=20of=20files|References:=20=20<20110802090830.GA92646@icar us.home.lan>=20=20<20110802094226.GA93114@icarus.home.lan>=20<42 039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au>=20<4E37CD13.10704 02@digsys.bg>|In-Reply-To:=20<4E37CD13.1070402@digsys.bg>|X-Enigma il-Version:=201.2|OpenPGP:=20id=3D60AE908C|Content-Type:=20multipa rt/signed=3B=20micalg=3Dpgp-sha1=3B=0D=0A=20protocol=3D"applicatio n/pgp-signature"=3B=0D=0A=20boundary=3D"------------enigB5D6A62DEB 22158D1A0D9F01"; b=sNCB/tK+M1647ft7h6WROA3MWdSOUM7XTPBxmCdRSbTtow96ro1gmwecNEBde8Tpn 9SPFksJmn279yXxsXAzIK7OYuLaF+j4G5IFtwXMuPtJGviBnKtazq+PxpP5KFxla3v opyDGpEMI+wCPwVxaUFu1Y5A9i4XRUGa/a8+uR1s= Message-ID: <4E37F3BE.5040207@infracaninophile.co.uk> Date: Tue, 02 Aug 2011 13:55:26 +0100 From: Matthew Seaman User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20110624 Thunderbird/5.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <20110802090830.GA92646@icarus.home.lan> <20110802094226.GA93114@icarus.home.lan> <42039B84-D6CE-4780-AA70-8500B1B32036@gsoft.com.au> <4E37CD13.1070402@digsys.bg> In-Reply-To: <4E37CD13.1070402@digsys.bg> X-Enigmail-Version: 1.2 OpenPGP: id=60AE908C Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB5D6A62DEB22158D1A0D9F01" X-Virus-Scanned: clamav-milter 0.97.2 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-0.1 required=5.0 tests=BAYES_40,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,SPF_FAIL autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk Subject: Re: ZFS directory with a large number of files X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 12:55:40 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB5D6A62DEB22158D1A0D9F01 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 02/08/2011 11:10, Daniel Kalchev wrote: > Other than that, perhaps in ZFS it would be easier to prune the unused > directory entries, than it is in UFS. It looks like this is not > implemented. Remember that ZFS uses copy-on-write for all filesystem updates. Any change to a directory contents means the whole directory data is rewritten. In which case, there's no benefit to retaining a large data structure with lots of empty slots (as happened on Unix FSes in the past.) I'd expect, and I see in my (admittedly fairly cursory) testing that ZFS directory data sizes update immediately whenever files are added or removed from the directory. Where this gets interesting is when the directory gets sufficiently large that the directory data is larger than the 128kB block size used by ZFS. As that takes many more files than any sensible person would put into one directory it's possible there's a bug in handling such large structures which is only rarely tickled. But this is all speculation on my behalf, and I have no evidence to back it up. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate JID: matthew@infracaninophile.co.uk Kent, CT11 9PW --------------enigB5D6A62DEB22158D1A0D9F01 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.16 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk4388YACgkQ8Mjk52CukIxSbwCfRFz96Y2hycFGs2NPQPJepM87 iWgAnilMjLGPTepRGr37CBMd2M0DoY3f =V6y7 -----END PGP SIGNATURE----- --------------enigB5D6A62DEB22158D1A0D9F01--