Date: Sun, 01 Jul 2007 12:09:06 +0200 From: Thomas Herrlin <junics-fbsdstable@atlantis.maniacs.se> To: Nguyen Tam Chinh <unixvn@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: UFS2 optimization for many small files Message-ID: <46877D42.8010606@atlantis.maniacs.se> In-Reply-To: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com> References: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Nguyen Tam Chinh wrote: > Greetings, > > We're going to build a server with some 1Tb of over 500 million small > files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle > this kind of system well. From newfs(8) the min block size is 4k. This > is not optimal in our case, a 1k or 0,5k block is more effective IMHO. > I'd be happy if anyone can suggest what does fragment (block/8) in the > ufs2 mean and how this parameter works. I know It's better to read the > full ufs2 specification, but hope that someone here can give a hint. > Please advice with optimizations or tricks. > Thank you very much. > If all else fails; try "divide and conquer" by having one filesystem per subdirectory. Unless you plan on having all files in a single dir?! Also look at how the squid proxy stores its files using a hashed dir structure. Another alternative is storing the data in a database if you don't need direct random RW file access..
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46877D42.8010606>