Date: Wed, 27 Jun 2007 08:33:42 -0400 From: Bill Moran <wmoran@potentialtech.com> To: "Nguyen Tam Chinh" <unixvn@gmail.com> Cc: FreeBSD-Questions <freebsd-questions@freebsd.org> Subject: Re: UFS2 optimization for many small files Message-ID: <20070627083342.f965009b.wmoran@potentialtech.com> In-Reply-To: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com> References: <64b284310706270311j2a6af2f6i6766b483a4b66a5c@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In response to "Nguyen Tam Chinh" <unixvn@gmail.com>: > > We're going to build a server with some 1Tb of over 500 million small > files with size from 0,5k to 4k. I'm wonder if the ufs2 can handle > this kind of system well. From newfs(8) the min block size is 4k. This > is not optimal in our case, a 1k or 0,5k block is more effective IMHO. > I'd be happy if anyone can suggest what does fragment (block/8) in the > ufs2 mean and how this parameter works. I know It's better to read the > full ufs2 specification, but hope that someone here can give a hint. > Please advice with optimizations or tricks. > Thank you very much. Read the newfs man page. Based on your assessment of your files, I'd go with a block size of 4K and a frag size of 500 bytes. Blocks are broken in to frags when a file doesn't fill an entire block. Make sure to set -i to about 250 or so. An inode is needed for each file or directory on the filesystem, so you're liable to run out of inodes with the default values. Make sure your files are organized in a directory hierarchy. No filesytem that I know of performs well with huge numbers of files in a single directory. Please don't cross-post. I see no reason to copy stable@ with this message. -- Bill Moran http://www.potentialtech.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070627083342.f965009b.wmoran>