Date: Tue, 29 May 2012 16:38:03 +1000 From: Peter Jeremy <peter@rulingia.com> To: Alessio Focardi <alessiof@gmail.com> Cc: freebsd-fs@freebsd.org Subject: Re: Millions of small files: best filesystem / best options Message-ID: <20120529063803.GG2675@aspire.rulingia.com> In-Reply-To: <2134924725.5040.1338211317460.JavaMail.root@zimbra.interconnessioni.it> References: <588211375.4794.1338210497900.JavaMail.root@zimbra.interconnessioni.it> <2134924725.5040.1338211317460.JavaMail.root@zimbra.interconnessioni.it>
next in thread | previous in thread | raw e-mail | index | archive | help
--iAL9S67WQOXgEPD9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-May-28 15:21:57 +0200, Alessio Focardi <alessiof@gmail.com> wrote: >I'm looking for some advice to efficiently pack millions of small files (2= 00 bytes or less) over a freebsd fs. "Millions of files" isn't an issue - any self-respecting filesystem will manage that. The "200 bytes or less" is a real problem. The best you will be able to do is a 4K/512 UFS1 filesystem (UFS1 because the inodes are half the size) - that gives you a minimum of 640 bytes (plus directory entry) per file (it would be 768 bytes for UFS2 and 1536 or 2048 bytes for ZFS). I would suggest that no normal filesystem will be a good match for your requirements. IMHO, you would be better off storing your data as an array of "file+metadata" objects within a single physical file, together with some sort of index structure that suits your "file" names. --=20 Peter Jeremy --iAL9S67WQOXgEPD9 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAk/EbsoACgkQ/opHv/APuIdrtACgs5rseqWRzW/8nsc+XxE3SCM6 fOEAoK3c77JusLQomJ+YygpKhYCh9XO8 =gO6N -----END PGP SIGNATURE----- --iAL9S67WQOXgEPD9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120529063803.GG2675>