From owner-freebsd-fs@FreeBSD.ORG Sat Feb 10 01:54:14 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D8F4C16A4DC for ; Sat, 10 Feb 2007 01:54:14 +0000 (UTC) (envelope-from fbsd-fs@mawer.org) Received: from fallbackmx03.syd.optusnet.com.au (fallbackmx03.syd.optusnet.com.au [211.29.133.136]) by mx1.freebsd.org (Postfix) with ESMTP id CD7A013C471 for ; Sat, 10 Feb 2007 01:54:13 +0000 (UTC) (envelope-from fbsd-fs@mawer.org) Received: from mail05.syd.optusnet.com.au (mail05.syd.optusnet.com.au [211.29.132.186]) by fallbackmx03.syd.optusnet.com.au (8.12.11.20060308/8.12.11) with ESMTP id l1A07dPl023469; Sat, 10 Feb 2007 11:07:40 +1100 Received: from [127.0.0.1] (c211-30-198-155.thorn1.nsw.optusnet.com.au [211.30.198.155]) by mail05.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l1A05gib022279; Sat, 10 Feb 2007 11:05:45 +1100 Message-ID: <45CD0C5A.3070804@mawer.org> Date: Fri, 09 Feb 2007 14:05:46 -1000 From: Antony Mawer User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: Eric Anderson References: <646424.65334.qm@web58613.mail.re3.yahoo.com> <45CC793F.7090003@freebsd.org> In-Reply-To: <45CC793F.7090003@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Ivan Voras Subject: Re: comments on newfs raw disk ? Safe ? (7 terabyte array) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Feb 2007 01:54:14 -0000 On 9/02/2007 3:38 AM, Eric Anderson wrote: >> Only "collateral" problems because of the partition size: a regular >> (non-softupdates) fsck will take a LONG time to finish and eat a LOT of >> memory while it's doing its stuff. You'll need a lot of swap space (1GB >> per TB? someone had empirical numbers on this, I'm sure) if you think >> you'll need to fsck it entirely. Creating snapshots will also take a >> long time on it, and you probably want to search the lists for >> recommendations about creating snapshots in a second level directory in >> order not to block the root directory. Related to this is >> background-fsck which works by creating snapshots, so you'll probably >> want to disable it. > > I have 5 10Tb file systems (and some 2Tb ones, but who cares about those > tiny things? :)), and I can tell you that an empty huge file system is > pretty easily fsck-able, but a full one will kill you. It greatly > depends on how many files (inodes) you have used on the file system. If > you have a massive amount of small files, you'll be eating up a ton of > memory. My 'rule of thumb' for my data (which averages to about > 16k/file) is 1G of memory for each 1Tb of disk space used. So, on a > 10Tb file system, if I ever want the fsck to complete, I need an AMD64 > box with *at least* 10G of memory, plus a lot of time. A *lot* of time. > By 'a lot', I mean anywhere from a day, to several days. Has anyone looked at the changes in DragonFly that were made in the 1.8 release? I noticed the other day, reading the release notes (http://www.dragonflybsd.org/community/release1_8.shtml) the point: "Greatly reduce the memory allocated by fsck when fscking filesytems with a huge number of directories (primarily mirors with lots of hardlinked files). Otherwise fsck can run out of memory on such filesystems." Whether or not this helps in the general case, or only the scenario described, I do not know... but it would be interesting for someone with enough filesystem-foo to have a look at! --Antony