From owner-freebsd-hackers@FreeBSD.ORG Sat Jan 10 16:53:26 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1444816A4CE for ; Sat, 10 Jan 2004 16:53:26 -0800 (PST) Received: from server.vk2pj.dyndns.org (c211-30-75-229.belrs2.nsw.optusnet.com.au [211.30.75.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3D33643D5E for ; Sat, 10 Jan 2004 16:53:24 -0800 (PST) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1])i0B0rE7B062133; Sun, 11 Jan 2004 11:53:14 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.12.10/8.12.10/Submit) id i0B0r8Kq062132; Sun, 11 Jan 2004 11:53:08 +1100 (EST) (envelope-from peter) Date: Sun, 11 Jan 2004 11:53:08 +1100 From: Peter Jeremy To: Dag-Erling =?iso-8859-1?Q?Sm=F8rgrav?= Message-ID: <20040111005308.GD60996@server.vk2pj.dyndns.org> References: <20040109193551.GD39751@moo.sysabend.org> <20040110225509.GA60996@server.vk2pj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.4.1i cc: freebsd-hackers@freebsd.org Subject: Re: Large Filesystem Woes X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Jan 2004 00:53:26 -0000 On Sun, Jan 11, 2004 at 12:25:46AM +0100, Dag-Erling Smørgrav wrote: >Peter Jeremy writes: >> Our main fileserver has a filesystem with 2.7e6 files and we >> are continually running into undocumented "features" (aka bugs) as a >> result of the large number of files. > >Is 2.7e6 a typo for 2.7e9? I can't imagine *any* modern file system >having trouble storing barely three million files. My ~ alone has >almost a million. No, I am serious about the the 2.7e6. It's not so much a hard limit as things like: - A statement "these options are no longer necessary and will be be removed in a future release" in the newfs(8)-equivalent man page should read more like "these options are essential" (this relates to dimensioning metadata allocation based on the expected total number of files). The default values hit undocumented metadata extent count limits at about 500,000 files. The table showing suggested dimensioning guidelines only goes to 800,000 files. - Failure to correctly dimension metadata during filesystem creation will lead to "out of disk space" errors but there are no tools to determine how close you are to this situation. - Recommendations that a minimum of 5% free space should exist for performing snapshots neglect to mention that the underlying code apparently changes behaviour at about 12% free space. - Metadata fragmentation can result in directories turning into mode 0 files within a snapshot. (This is apparently more likely to occur with less than 12% free space) Admittedly, there has been a major upgrade of the FS code in a newer OS release but my application requires the older release. Peter