From owner-freebsd-stable Thu Feb 6 3:22:23 2003 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 20BD237B401 for ; Thu, 6 Feb 2003 03:22:22 -0800 (PST) Received: from 12-234-22-23.client.attbi.com (12-234-22-23.client.attbi.com [12.234.22.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id A2C1843F93 for ; Thu, 6 Feb 2003 03:22:21 -0800 (PST) (envelope-from DougB@FreeBSD.org) Received: from slave.gorean.org (1s8hg9s9xwx7xizk@slave.gorean.org [10.0.0.1]) by 12-234-22-23.client.attbi.com (8.12.6/8.12.6) with ESMTP id h16BMIh9053755; Thu, 6 Feb 2003 03:22:18 -0800 (PST) (envelope-from DougB@FreeBSD.org) Date: Thu, 6 Feb 2003 03:22:18 -0800 (PST) From: Doug Barton To: Ryan Dooley Cc: stable@FreeBSD.org Subject: Re: recommendations on the newfs of a 1.0TB fs... In-Reply-To: <20030204050631.GB81935@elvis.mu.org> Message-ID: <20030206031326.K40993@12-234-22-23.pyvrag.nggov.pbz> References: <20030203194828.GA55143@elvis.mu.org> <20030204043726.GA4323@HAL9000.homeunix.com> <20030204050631.GB81935@elvis.mu.org> Organization: http://www.FreeBSD.org/ X-message-flag: Outlook -- Not just for spreading viruses anymore! MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, 3 Feb 2003, Ryan Dooley wrote: > > IIRC, block sizes greater than 16384 can cause significant buffer > > cache fragmentation, which can reduce I/O performance. Moreover, > > blocks that large will waste space and I/O bandwidth unless most > > of the files on the disk are very large. A smaller setting, > > e.g. the default, is probably more appropriate. > > The default wasted too much disk space (we didn't get that 1.0TB of > usable space out of it (it was more like 893GB of total usable space.) FWIW, when I was approaching a similar problem a few years ago (big fs with lots of little files), I was told that a good way to get maximum disk utilization and performance was to figure the size that roughly 75-80% of your files would fit into, and make this the fragment size. Multiply this by 8 to get the block size, and newfs to your heart's content. This formula has worked really well for me, and drastically improved the performance of the systems in question. My rough understanding of why this works is that when reading a block, you want to pick up as many files as you can in one pass. For files larger than your fragment size, the amount of space "wasted" by non-full fragments will have a minimal impact on your disk utilization because most of your files will be within a fragment already. Make sure to heed the warnings in the man page for the -b and -f options. HTH, Doug -- If it's moving, encrypt it. If it's not moving, encrypt it till it moves, then encrypt it some more. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message