From owner-freebsd-fs@FreeBSD.ORG Mon Jun 4 19:02:42 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E020616A46D for ; Mon, 4 Jun 2007 19:02:42 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.237]) by mx1.freebsd.org (Postfix) with ESMTP id 62D2213C465 for ; Mon, 4 Jun 2007 19:02:42 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: by wr-out-0506.google.com with SMTP id 69so803899wra for ; Mon, 04 Jun 2007 12:02:41 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; b=RfjaKjCRVyWuAYwpLDTpXW90I4TSPvgAEmoQ/7Hva3NR25miQ3c5kFo6l5XgdnZKxDHBunuOxFoC2p5T9PBCtV0A0fvTKvDn/CPfPxEPtDbf88tfFgJbaczim+RgE04SRB2kfL/apf9pOzkyzIR3aTFlOhp7bicbYcE54GSpmSY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=qohDV1nONpiJCYFI4nRocd42MSUt/lDJvwu7r3wPxauQgzhuskv2qPbzqlOg+KqHydnLj/Uev0WUG6PAnRHos0vKYjaIRR9HEx+Q7uxQXynSo9ylo/R4UY4xi4xFi96691dpYbsZONMCRb27RB2ck/S3VudzcC9QdArmPj8SfkE= Received: by 10.142.86.7 with SMTP id j7mr234697wfb.1180983761130; Mon, 04 Jun 2007 12:02:41 -0700 (PDT) Received: by 10.142.251.8 with HTTP; Mon, 4 Jun 2007 12:02:41 -0700 (PDT) Message-ID: <440b3e930706041202m7cbc7cap52947ae1d659d115@mail.gmail.com> Date: Mon, 4 Jun 2007 15:02:41 -0400 From: "Ali Mashtizadeh" To: freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: ZFS Panic on USB "SCSI" Device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jun 2007 19:02:43 -0000 SSBoYXZlIGJlZW4gdXNpbmcgWkZTIG9uIG15IGxhcHRvcCB3aXRoIGEgWkZTIHJvb3Qgd29ya3Mg Zmxhd2xlc3NseS4gVGhlCnByb2JsZW0gaXMgdGhhdCBJIHB1dCBhIFpGUyBmaWxlc3lzdGVtIG9u IG15IGV4dGVybmFsIHVzYiBoYXJkZHJpdmUgd2hpY2gKc2hvd3MgdXAgYXMgImRhMCIgc28gaXQg bG9va3MgbGlrZSBhIHNjc2kgZGV2aWNlLiBBbmQgSSBrZWVwIGdldHRpbmcgQ0FNCmVycm9ycyBz YXlpbmcgbG9naWNhbCBibG9jayBhZGRyZXNzIG91dCBvZiByYW5nZS4gQW0gSSBkb2luZyBzb21l dGhpbmcgd3JvbmcKaXMgdGhlcmUgc29tZSBzcGVjaWFsIHRoaW5nIEkgbmVlZCB0byBkbz8gVGhl IGVycm9ycyBhcmUgb2theSBhdCBmaXJzdCBidXQKYWZ0ZXIgYSBjb3VwbGUgb2YgdGhlbSBpdCBQ QU5JQ3MuIEluIGNhc2UgeW91ciB3b25kZXJpbmcgaXRzIE9OTFkgWkZTLCBVRlMKYW5kIHRoZSBN U0RPUyBwYXJ0aXRpb25zIHdvcmsgZmluZSBvbiB0aGUgc2FtZSBkcml2ZS4KCk15IGJ1aWxkIG9m IGN1cnJlbnQgaXMgYWJvdXQgYSB3ZWVrIG9sZC4KCi0tIApBbGkgTWFzaHRpemFkZWgK2LnZhNuM INmF2LTYqtuMINiy2KfYr9mHCg== From owner-freebsd-fs@FreeBSD.ORG Tue Jun 5 17:13:20 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 440D816A41F for ; Tue, 5 Jun 2007 17:13:20 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: from nz-out-0506.google.com (nz-out-0506.google.com [64.233.162.229]) by mx1.freebsd.org (Postfix) with ESMTP id E2EC913C455 for ; Tue, 5 Jun 2007 17:13:19 +0000 (UTC) (envelope-from mashtizadeh@gmail.com) Received: by nz-out-0506.google.com with SMTP id 14so1267545nzn for ; Tue, 05 Jun 2007 10:13:19 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=AKauar/rytGsdkmkAEOvIJjyCvU1lc05RS0hxXOQwlWlfWfYWghd7kpuJjM6dENtQXTxXh1V1J5wCQA7HjoRR57MLx1UauXOWVGWJasj+cF2Rxsus2gh6fHu4C+GV+eThRZ5XdbRZN0Ce36NWosK3bW/oAOSVuGaeOX1WqBO1BQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=fUEHZLrCTaFv0PCG8UcfYuzS+zCn2/JGqf4hkUsmZy+39bRQrYb6YBYxtzrGlFjeMRUcNUibE0reJfzxdyMOJ8yg9Q6ZIG8kf25OVRQIJUQ9mdAnc8mL78N/OSeLqjSZhc0LGD/Bu47WwLntm/T7yU15b1vPgKGwOsD8r4rrulM= Received: by 10.142.101.17 with SMTP id y17mr291967wfb.1181063599007; Tue, 05 Jun 2007 10:13:19 -0700 (PDT) Received: by 10.142.251.8 with HTTP; Tue, 5 Jun 2007 10:13:18 -0700 (PDT) Message-ID: <440b3e930706051013n65c171feta10a5872026c8f00@mail.gmail.com> Date: Tue, 5 Jun 2007 13:13:18 -0400 From: "Ali Mashtizadeh" To: freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org In-Reply-To: <440b3e930706041202m7cbc7cap52947ae1d659d115@mail.gmail.com> MIME-Version: 1.0 References: <440b3e930706041202m7cbc7cap52947ae1d659d115@mail.gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: ZFS Panic on USB "SCSI" Device X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jun 2007 17:13:20 -0000 SWYgYW55b25lIGlzIGN1cmlvdXMgaXQgc2VlbXMgdGhpcyBpcyBhIHNvbWV3aGF0IHJhbmRvbSBv Y2N1cnJlbmNlLiBJIHRoaW5rCml0cyB1bnJlbGF0ZWQgdG8gWkZTIHByb2JhYmx5IHNvIG5ldmVy bWluZC4gSXQgd29ya3MgZmxhd2xlc3NseSAgbm93IDotKQoKT24gNi80LzA3LCBBbGkgTWFzaHRp emFkZWggPG1hc2h0aXphZGVoQGdtYWlsLmNvbT4gd3JvdGU6Cj4KPiBJIGhhdmUgYmVlbiB1c2lu ZyBaRlMgb24gbXkgbGFwdG9wIHdpdGggYSBaRlMgcm9vdCB3b3JrcyBmbGF3bGVzc2x5LiBUaGUK PiBwcm9ibGVtIGlzIHRoYXQgSSBwdXQgYSBaRlMgZmlsZXN5c3RlbSBvbiBteSBleHRlcm5hbCB1 c2IgaGFyZGRyaXZlIHdoaWNoCj4gc2hvd3MgdXAgYXMgImRhMCIgc28gaXQgbG9va3MgbGlrZSBh IHNjc2kgZGV2aWNlLiBBbmQgSSBrZWVwIGdldHRpbmcgQ0FNCj4gZXJyb3JzIHNheWluZyBsb2dp Y2FsIGJsb2NrIGFkZHJlc3Mgb3V0IG9mIHJhbmdlLiBBbSBJIGRvaW5nIHNvbWV0aGluZyB3cm9u Zwo+IGlzIHRoZXJlIHNvbWUgc3BlY2lhbCB0aGluZyBJIG5lZWQgdG8gZG8/IFRoZSBlcnJvcnMg YXJlIG9rYXkgYXQgZmlyc3QgYnV0Cj4gYWZ0ZXIgYSBjb3VwbGUgb2YgdGhlbSBpdCBQQU5JQ3Mu IEluIGNhc2UgeW91ciB3b25kZXJpbmcgaXRzIE9OTFkgWkZTLCBVRlMKPiBhbmQgdGhlIE1TRE9T IHBhcnRpdGlvbnMgd29yayBmaW5lIG9uIHRoZSBzYW1lIGRyaXZlLgo+Cj4gTXkgYnVpbGQgb2Yg Y3VycmVudCBpcyBhYm91dCBhIHdlZWsgb2xkLgo+Cj4gLS0KPiBBbGkgTWFzaHRpemFkZWgKPiDY udmE24wg2YXYtNiq24wg2LLYp9iv2YcKCgoKCi0tIApBbGkgTWFzaHRpemFkZWgK2LnZhNuMINmF 2LTYqtuMINiy2KfYr9mHCg== From owner-freebsd-fs@FreeBSD.ORG Thu Jun 7 15:04:12 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7221916A400 for ; Thu, 7 Jun 2007 15:04:12 +0000 (UTC) (envelope-from lists.freebsd@gmail.com) Received: from hu-out-0506.google.com (hu-out-0506.google.com [72.14.214.230]) by mx1.freebsd.org (Postfix) with ESMTP id 0079513C483 for ; Thu, 7 Jun 2007 15:04:11 +0000 (UTC) (envelope-from lists.freebsd@gmail.com) Received: by hu-out-0506.google.com with SMTP id 28so351378hug for ; Thu, 07 Jun 2007 08:04:10 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type; b=UQpwvoYTzPF3RhPRNlp5Z/5GcfSKRhs3SpCh1DhPrrD1Vm0NUMTIu6634Y6m6CY3BH3ngTaXMDdUfzo7K/c4IeINvn3aL6SGmIKR17x7brm2iaRpr5kGmQ9sGx64cRQxuLT9YMHQqzMBpud4P71woVn0OvYo7uYBFuF+msTkXEg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type; b=rzQ7+DHRfejCtWkb9BagZN3LCs9+FIJSBoZuTgVdSxHsdpqUYokoPus/zbvcnWulwrk0F16Io6z6KSP/qIXt6XHjvOULi4ypMLkqku4fyT/6ywB4BBeZ9H8sZ0i+UvYR9fOphYPzShNV3jFNyZn2Y3rMBV6F74v5SmiwZCO8zgA= Received: by 10.143.164.19 with SMTP id r19mr87515wfo.1181228649687; Thu, 07 Jun 2007 08:04:09 -0700 (PDT) Received: by 10.143.155.17 with HTTP; Thu, 7 Jun 2007 08:04:09 -0700 (PDT) Message-ID: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> Date: Thu, 7 Jun 2007 17:04:09 +0200 From: "Richard Noorlandt" To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2007 15:04:12 -0000 Hi everybody, While reading a bit about tunefs I noticed that UFS reserves 8% of the drive space for the root user and the system. However, I don't really understand what this space is actually used for. From the tunefs man page I understand that it is primarily used to guard against fragmentation, and that's about it. Is this the only thing that the reserved space is used for? I'm building a large array for my fileserver, and it actually hurts a bit to see that so much space is "wasted" without a very clear reason to do so. Especially because the data on the array won't be modified very often, it appears to be quite a lot of disk space just to prevent fragmentation. Does anybody have some more information on this? And while I'm at it: what is the effect of the expected average file size option? What are the benefits and dangers of tweaking it? From the FreeBSD handbook I understand that the FS actually optimizes itself as time passes, but that's about all that's said about it. Regards, Richard From owner-freebsd-fs@FreeBSD.ORG Thu Jun 7 15:17:37 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 804D616A400 for ; Thu, 7 Jun 2007 15:17:37 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (grnl-static-02-0046.dsl.iowatelecom.net [69.66.56.110]) by mx1.freebsd.org (Postfix) with ESMTP id 23F9D13C45D for ; Thu, 7 Jun 2007 15:17:36 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.13.8/8.13.8) with ESMTP id l57FHaRc098873; Thu, 7 Jun 2007 10:17:36 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.13.8/8.13.8/Submit) id l57FHZdG098872; Thu, 7 Jun 2007 10:17:35 -0500 (CDT) (envelope-from brooks) Date: Thu, 7 Jun 2007 10:17:35 -0500 From: Brooks Davis To: Richard Noorlandt Message-ID: <20070607151735.GA98301@lor.one-eyed-alien.net> References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="uAKRQypu60I7Lcqm" Content-Disposition: inline In-Reply-To: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> User-Agent: Mutt/1.5.15 (2007-04-06) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Thu, 07 Jun 2007 10:17:36 -0500 (CDT) Cc: freebsd-fs@freebsd.org Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2007 15:17:37 -0000 --uAKRQypu60I7Lcqm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jun 07, 2007 at 05:04:09PM +0200, Richard Noorlandt wrote: > Hi everybody, >=20 > While reading a bit about tunefs I noticed that UFS reserves 8% of the d= rive > space for the root user and the system. However, I don't really understa= nd > what this space is actually used for. From the tunefs man page I underst= and > that it is primarily used to guard against fragmentation, and that's abo= ut > it. Is this the only thing that the reserved space is used for? >=20 > I'm building a large array for my fileserver, and it actually hurts a bi= t to > see that so much space is "wasted" without a very clear reason to do so. > Especially because the data on the array won't be modified very often, it > appears to be quite a lot of disk space just to prevent fragmentation. D= oes > anybody have some more information on this? >=20 > And while I'm at it: what is the effect of the expected average file size > option? What are the benefits and dangers of tweaking it? From the FreeB= SD > handbook I understand that the FS actually optimizes itself as time pass= es, > but that's about all that's said about it. The answer I've heard in the past is that the performance of the block allocation algorithms requires that most of the time an attempt to allocation from a cylinder group will succeed. If you reduce the free space too much that will not hold true anymore and thus your performance will suffer. 8% is actually a reduction from the historical 10%. In practice, the only way to decide what's right for your system is testing, but that's hard on a large system since it will take days to fill the disk. -- Brooks --uAKRQypu60I7Lcqm Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFGaCGPXY6L6fI4GtQRAoRDAJ9mU1veB2H4ilNGAZgNnyrCH0F8swCglWpo PIqaQsqQPgWjwKSp/Ke2o2I= =70yI -----END PGP SIGNATURE----- --uAKRQypu60I7Lcqm-- From owner-freebsd-fs@FreeBSD.ORG Thu Jun 7 16:41:18 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 22CEC16A46B for ; Thu, 7 Jun 2007 16:41:18 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id 859EC13C480 for ; Thu, 7 Jun 2007 16:41:17 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 96352 invoked by uid 2001); 7 Jun 2007 16:41:16 -0000 Date: Thu, 7 Jun 2007 11:41:16 -0500 From: "Rick C. Petty" To: Richard Noorlandt Message-ID: <20070607164116.GA95991@keira.kiwi-computer.com> References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Jun 2007 16:41:18 -0000 On Thu, Jun 07, 2007 at 05:04:09PM +0200, Richard Noorlandt wrote: > > While reading a bit about tunefs I noticed that UFS reserves 8% of the drive > space for the root user and the system. However, I don't really understand > what this space is actually used for. From the tunefs man page I understand > that it is primarily used to guard against fragmentation, and that's about > it. Is this the only thing that the reserved space is used for? It's used as the point where the filesystem changes over from a time-efficient algorithm to a space-efficient one. If you don't believe there's much of a difference, try writing past 100% full and witness the performance hit. There is probably a good reason why the default is 8%.. you could tune that down, but I'd be wary of doing so. > I'm building a large array for my fileserver, and it actually hurts a bit to > see that so much space is "wasted" without a very clear reason to do so. If you want to talk about wasted space, look at how many inodes are allocated on a newfs and how many you actually end up using. I have a patch to newfs which reports the amount of space "wasted" in metadata blocks.. Reducing the number of cylinder groups and number of inode blocks allocated per cylinder group can make this "wasted" space drop from 3% to 0.3%, in some circumstances. > Especially because the data on the array won't be modified very often, it > appears to be quite a lot of disk space just to prevent fragmentation. Does > anybody have some more information on this? If you don't care about fragmentation or speed, go ahead and fill up past 100%. If you need non-root writable access, go ahead and tune the 8% to a smaller number. Be warned-- you will likely be sacrificing speed just to fill that extra space. If your modifications are rare, you might be able to run some sort of offline defragmentation program. I was working on such a program at one point. Also, I wouldn't call this "a lot of disk space". Disks are cheap: http://www.newegg.com/Product/Product.asp?Item=N82E16822136073 $115 for 500GB drives. If you need more space, buy more disks. You may wish to consider the opportunity cost of the time you spend tweaking file systems to use that extra couple percent space versus the cost of additional storage. > And while I'm at it: what is the effect of the expected average file size > option? What are the benefits and dangers of tweaking it? From the FreeBSD > handbook I understand that the FS actually optimizes itself as time passes, > but that's about all that's said about it. Don't tweak average file size (or it's related avg. files per directory setting)! There are some reported bugs in UFS regarding the computation (integer overflow) that is performed on the average file size parameter. I first noticed this when tweaking it on a new filesystem that I knew the average filesize was high (ISO images) and my kernel panicked on the first write attempt. Subsequent reboots panicked during fsck. I've looked into patching the necessary UFS code but never got around to a complete fix. Instead, just modify the inode density (-i option to newfs) which means pretty much the same thing and doesn't have the bug. Tweaking this parameter only affects newfs.. actually it's used in the computation for number of inodes per cylinder group and indirectly affects total number of cylinder groups-- newfs uses a loop to increase number of cyl groups until the free block bitmap can fit into a single filesystem block (which defaults to 16384 bytes). Tweaking this option, I've been able to reduce the space allocated to metadata by tenfold. I recommend first running newfs with the -N option to print out the total number of allocated inodes. Make sure you won't go over this number (running out of inodes can be an aggravating problem), and if you're satisfied rerun newfs for real. If you're tweaking -i, you may wish to tweak the block size as well.. but make sure the fragment size is precisely 1/8 of the block size or all bets go out the window (particularly: performance hits). Aside from those three settings, I don't mess with newfs much because there is little to gain. If you know the precise files (i.e. total number of files + number of directories --> number of inodes, average filesize --> inode density), this helps you speeze more space without sacrificing anything. I do this often when transitioning a read-write filesystem to a read-only / archive filesystem. Set the time/space setting to space, set the reserved free space from 8% to 1%, fill up the filesystem, unmount and set it back to time & 8%, and mount it read-only for the remainder of its lifetime. This requires a total of five extra minutes of your admin time. Good luck, -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 02:45:37 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 59A7A16A421 for ; Fri, 8 Jun 2007 02:45:37 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by mx1.freebsd.org (Postfix) with ESMTP id EF51213C448 for ; Fri, 8 Jun 2007 02:45:36 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-235-248.carlnfd3.nsw.optusnet.com.au (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l582jV9P025622 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 8 Jun 2007 12:45:35 +1000 Date: Fri, 8 Jun 2007 12:45:33 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Richard Noorlandt In-Reply-To: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> Message-ID: <20070608120659.E75409@delplex.bde.org> References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 02:45:37 -0000 On Thu, 7 Jun 2007, Richard Noorlandt wrote: > While reading a bit about tunefs I noticed that UFS reserves 8% of the drive > space for the root user and the system. However, I don't really understand > what this space is actually used for. From the tunefs man page I understand > that it is primarily used to guard against fragmentation, and that's about > it. Is this the only thing that the reserved space is used for? Yes. Laying out files optimally is a hard problem, and leaving lots of unused space simplifies the problem. > And while I'm at it: what is the effect of the expected average file size > option? What are the benefits and dangers of tweaking it? Its main effect is to cause panics when it is set too high. Similarly for the average files per directory option -- set these so high that their product overflows to cause undefined behaviour including panics. Setting these options to non-overflowing values has subtle effects which seem to be limited to limiting the number of directories per cylinder group. IMO, they are mainly debugging options for determining if the subtle effects are ever large enough to be worth tuning for. They are documented mainly in a README or commit message. > From the FreeBSD > handbook I understand that the FS actually optimizes itself as time passes, > but that's about all that's said about it. No, ffs unoptimizes itself (becomes more fragmented) as time passes. The reserved space limits the extent of the fragmentation or at least the rate of growth of the fragmentation. Fragmentation of old, nearly full, ffs file systems can be very bad (maybe 10 times slower than a new one). Note that fsck only reports one sort of fragmentation which is not the main one that slows down old file systems. Bruce From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 08:58:13 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 169D216A41F for ; Fri, 8 Jun 2007 08:58:13 +0000 (UTC) (envelope-from micron@bglug.it) Received: from jack.tiscali.it (jack.tiscali.it [213.205.33.53]) by mx1.freebsd.org (Postfix) with ESMTP id 687E813C44B for ; Fri, 8 Jun 2007 08:58:12 +0000 (UTC) (envelope-from micron@bglug.it) Received: from [10.8.0.10] (84.220.184.158) by jack.tiscali.it (7.2.079) id 465A9B13001EE1A8 for freebsd-fs@freebsd.org; Fri, 8 Jun 2007 10:58:09 +0200 From: Flavio Castelli To: freebsd-fs@freebsd.org Date: Fri, 8 Jun 2007 10:58:01 +0200 User-Agent: KMail/1.9.5 References: <200706011557.13769.micron@bglug.it> <4661F340.6080009@net.utcluj.ro> In-Reply-To: <4661F340.6080009@net.utcluj.ro> X-Face: $Txq`+~>k4|^/-x\@oJeZW!JU; 1g78H("lLgtOyrD\&:=?iso-8859-1?q?uz=3AGv=7E=0A=09=5DHV/7cx=3DTaNUwE?=>(tllad(pN*f+LRB(:{k~/[R00SWX@=?iso-8859-1?q?=5DywcKa=0A=094=7DX?=(}gp/P"nfEAUQL(:G1a]n\'bQUC MIME-Version: 1.0 Content-Disposition: inline Message-Id: <200706081058.02229.micron@bglug.it> X-Length: 2641 X-UID: 2114 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Subject: Re: file system notifications X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 08:58:13 -0000 On Sunday 03 June 2007 00:46, you wrote: > I am curious. Have you tried fam? As far as I know, fam was created so > that developers won't have to bother about the mechanism they should use > to get file notifications. Yes I looked at fam and gamin (a project that extends fam) but I didn't cho= ose=20 them. =46am has lot of problems (first of all in some situations it can take lots= of=20 cpu cicles) and its development seems abandoned. Both fam and gamin use native file system notification structures when=20 available and use polling as a fall-back mechanism. Since kqueue has some=20 limitations (as exposed in my first post), it's really hard to develop a go= od=20 notification mechanism without falling back to polling or using a=20 mixed-approach. Just an example. Suppose you have told kqueue to watch dir /tmp/foo. If you=20 create/delete/rename/update a file/dir inside /tmp/foo, kqueue will tell yo= u=20 an event has occurred inside the watched directory. Kqueue's limitation is= =20 the lack of informations associated to the event since it _doesn't_ tell wh= at=20 happened. So a program has to walk across the watched dir (where the event= =20 took place) and find the changes that happened. Now, what I really like to know is if there is another system notification= =20 mechanism that works like inotify or fsevents. I would also use kqueue if there's a way to make it generate more=20 informations. Cheers Flavio =2D-=20 |=A7 micron<- ICQ #118796665 |=A7 GPG Key: |=A7 ~ Keyserver: pgp.mit.edu |=A7 ~ KeyID: 6D632BED ~ "Progress is merely a realisation of utopias" ~ From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 09:01:22 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 86D4A16A421 for ; Fri, 8 Jun 2007 09:01:22 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 7320F13C455 for ; Fri, 8 Jun 2007 09:01:22 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 38DC11A3C1A; Fri, 8 Jun 2007 02:01:21 -0700 (PDT) Received: from rot13.obsecurity.org (rot13.obsecurity.org [192.168.1.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id C7A575119F; Fri, 8 Jun 2007 05:01:21 -0400 (EDT) Received: by rot13.obsecurity.org (Postfix, from userid 1001) id C0B2FC219; Fri, 8 Jun 2007 05:01:21 -0400 (EDT) Date: Fri, 8 Jun 2007 05:01:21 -0400 From: Kris Kennaway To: Flavio Castelli Message-ID: <20070608090121.GA40524@rot13.obsecurity.org> References: <200706011557.13769.micron@bglug.it> <4661F340.6080009@net.utcluj.ro> <200706081058.02229.micron@bglug.it> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200706081058.02229.micron@bglug.it> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@freebsd.org Subject: Re: file system notifications X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 09:01:22 -0000 On Fri, Jun 08, 2007 at 10:58:01AM +0200, Flavio Castelli wrote: > Now, what I really like to know is if there is another system notification > mechanism that works like inotify or fsevents. Not currently. Kris From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 17:23:54 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0351316A468 for ; Fri, 8 Jun 2007 17:23:54 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id B91F313C458 for ; Fri, 8 Jun 2007 17:23:53 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 28523 invoked by uid 2001); 8 Jun 2007 17:23:53 -0000 Date: Fri, 8 Jun 2007 12:23:53 -0500 From: "Rick C. Petty" To: Fuzz Leonard Message-ID: <20070608172353.GB25954@keira.kiwi-computer.com> References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> <20070607164116.GA95991@keira.kiwi-computer.com> <30C58B28-CCA6-4A1A-B37A-53DEF55225BE@fuzzworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <30C58B28-CCA6-4A1A-B37A-53DEF55225BE@fuzzworks.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 17:23:54 -0000 On Thu, Jun 07, 2007 at 09:45:42PM -0500, Fuzz Leonard wrote: > But I digress. My question is "what is the benefit in setting the > file system back to 8% once it is full and read-only?" There is none. -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 20:05:12 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4061916A46B for ; Fri, 8 Jun 2007 20:05:12 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from ciao.gmane.org (main.gmane.org [80.91.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id BB02213C480 for ; Fri, 8 Jun 2007 20:05:11 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from list by ciao.gmane.org with local (Exim 4.43) id 1HwjpL-0000ho-Az for freebsd-fs@freebsd.org; Fri, 08 Jun 2007 21:08:54 +0200 Received: from 78-1-101-93.adsl.net.t-com.hr ([78.1.101.93]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 08 Jun 2007 21:08:51 +0200 Received: from ivoras by 78-1-101-93.adsl.net.t-com.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Fri, 08 Jun 2007 21:08:51 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Fri, 08 Jun 2007 21:08:08 +0200 Lines: 48 Message-ID: References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> <20070608120659.E75409@delplex.bde.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig71D83C7605536C43C6D8C0E2" X-Complaints-To: usenet@sea.gmane.org X-Gmane-NNTP-Posting-Host: 78-1-101-93.adsl.net.t-com.hr User-Agent: Thunderbird 1.5.0.12 (Windows/20070509) In-Reply-To: <20070608120659.E75409@delplex.bde.org> X-Enigmail-Version: 0.94.3.0 Sender: news Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 20:05:12 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig71D83C7605536C43C6D8C0E2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Bruce Evans wrote: > Fragmentation of old, nearly > full, ffs file systems can be very bad (maybe 10 times slower than a > new one). =20 Not only that, but it seems that sometimes (at least in 6.x) the file system can be subtly corrupted when filled to the end, and cannot be recovered even if the files that filled it are deleted and fsck is run. The symptom of this is random lockups when doing file system intensive operations. I'm not mentioning this only because of this thread: http://lists.freebsd.org/pipermail/freebsd-fs/2007-May/003272.html but I seem to have it on one of my servers in production since two weeks ago :( - since a file system has been filled (the "109% used" kind of filled) and freed a couple of times in the period of about a month, the system wedges about once a week (and it's been running fine for two years before this). So, leaving the 8% alone and making sure NOTHING can touch it (e.g. some deamon running under root, generating log files...) seems like a good strategy right now. Either that, or switch to ZFS, which is what I'm planning. --------------enig71D83C7605536C43C6D8C0E2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGaakYldnAQVacBcgRAm2jAJ9ASH+qHUqJnpO4NRxofLqa3U4+0gCfY8/6 ObfWojnFj5bvS9yQ/8DcXZE= =ObG7 -----END PGP SIGNATURE----- --------------enig71D83C7605536C43C6D8C0E2-- From owner-freebsd-fs@FreeBSD.ORG Fri Jun 8 22:30:07 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 47DA116A400 for ; Fri, 8 Jun 2007 22:30:07 +0000 (UTC) (envelope-from lists.freebsd@gmail.com) Received: from hu-out-0506.google.com (hu-out-0506.google.com [72.14.214.225]) by mx1.freebsd.org (Postfix) with ESMTP id BB02713C489 for ; Fri, 8 Jun 2007 22:30:06 +0000 (UTC) (envelope-from lists.freebsd@gmail.com) Received: by hu-out-0506.google.com with SMTP id 28so647767hub for ; Fri, 08 Jun 2007 15:30:05 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=g4z1OCmpnSRyDE6UONOoaWweVruNEOfMwlwc3YeVeXOSrcDol5OWhf4vdRTs3vg/qnntCWDCekjHsP4lu8IeU+cZVcb8KYGOIY9t5Jxj8n/1aeFjZMfMl/2g4174CXJpZx8jhg1aGdW1+uo60H8pCfjYQDN61jig5pijtOrBRuE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:references; b=JZ0UTPKaU5x3bAJAv43dJIAzW5Q3j/TNd4IYOjiJR1Ny/RgEmZllVadM8VST2KWxfPQRoyVJRMQySy9lExYaZP0XEw4NxUR5GHH2Q4m+VIapC45FgS0uBhqhwQXnpeRCgk6NrBZDnocSmDUseTTQYQNQ45wXXAXijFP7CAIR188= Received: by 10.143.4.16 with SMTP id g16mr168468wfi.1181341804353; Fri, 08 Jun 2007 15:30:04 -0700 (PDT) Received: by 10.143.155.17 with HTTP; Fri, 8 Jun 2007 15:30:04 -0700 (PDT) Message-ID: <99c92b5f0706081530t454ed9cfp4f95b9afd19e7ed5@mail.gmail.com> Date: Sat, 9 Jun 2007 00:30:04 +0200 From: "Richard Noorlandt" To: freebsd-fs@freebsd.org In-Reply-To: <20070607164116.GA95991@keira.kiwi-computer.com> MIME-Version: 1.0 References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> <20070607164116.GA95991@keira.kiwi-computer.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2007 22:30:07 -0000 Thanks for the reactions. They cleared up quite a bit, and my conclusion is that tweaking the FS isn't a very good idea. They're defaults for a reason, although I still have some doubts about the appropriateness of the defaults for large filesystems. Large filesystems don't seem to be very well supported at the moment. I hope (and believe) ZFS will settle this. It sounds promising :-) Unfortunately I don't think it's stable enough at the moment. 2007/6/7, Rick C. Petty : > > > If you know the precise files (i.e. total number of files + number of > directories --> number of inodes, average filesize --> inode density), > this > helps you speeze more space without sacrificing anything. I don't really understand what you're trying to say here. How do you exactly determine the number of inodes needed? And when you change the number of inodes at filesystem creation, what effect will it have when you run growfs later on? Will it expand the filesystem with an equal inode density, or is it expanded with the default density? Regards, Richard From owner-freebsd-fs@FreeBSD.ORG Sat Jun 9 02:02:10 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3043416A41F for ; Sat, 9 Jun 2007 02:02:10 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: from kiwi-computer.com (keira.kiwi-computer.com [63.224.10.3]) by mx1.freebsd.org (Postfix) with SMTP id AD98813C46A for ; Sat, 9 Jun 2007 02:02:09 +0000 (UTC) (envelope-from rick@kiwi-computer.com) Received: (qmail 40422 invoked by uid 2001); 9 Jun 2007 02:02:08 -0000 Date: Fri, 8 Jun 2007 21:02:08 -0500 From: "Rick C. Petty" To: Richard Noorlandt Message-ID: <20070609020208.GA38821@keira.kiwi-computer.com> References: <99c92b5f0706070804p42da0881kfc866b192be60ed5@mail.gmail.com> <20070607164116.GA95991@keira.kiwi-computer.com> <99c92b5f0706081530t454ed9cfp4f95b9afd19e7ed5@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <99c92b5f0706081530t454ed9cfp4f95b9afd19e7ed5@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@freebsd.org Subject: Re: tunefs question X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rick-freebsd@kiwi-computer.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jun 2007 02:02:10 -0000 On Sat, Jun 09, 2007 at 12:30:04AM +0200, Richard Noorlandt wrote: > Thanks for the reactions. They cleared up quite a bit, and my conclusion is > that tweaking the FS isn't a very good idea. They're defaults for a reason, Sure it is, there are just a few well-known bugs. > although I still have some doubts about the appropriateness of the defaults > for large filesystems. Large filesystems don't seem to be very well > supported at the moment. Where do you arrive at this conclusion? UFS2 supports large filesystems very well. The only known problems have to do with snapshots and full filesystems, and these affect large filesystems as well as small. > I hope (and believe) ZFS will settle this. It Then you are mistaken. ZFS has large memory requirements, and as you grow the filesystem you require more memory. UFS has a small memory footprint... it only requires memory for one cylinder group (superblock plus free bitmaps), plus all indirect blocks, in addition to the referenced blocks for each file open. This is not meant to critical to ZFS.. each file system has its own perks and problems. UFS has been tested and used for much longer, at least with the default newfs parameters. Also, it's called fast file system for a reason. > sounds promising :-) Unfortunately I don't think it's stable enough at the > moment. That's an understatement. However, it's receiving active attention and development. I doubt it will ever replace UFS, nor do I think that is the intention. > >If you know the precise files (i.e. total number of files + number of > >directories --> number of inodes, average filesize --> inode density), > >this > >helps you speeze more space without sacrificing anything. > > I don't really understand what you're trying to say here. How do you exactly > determine the number of inodes needed? Since my description is rather long, I've put it at the end of this email. Although I suggest leaving block & fragment size alone, I highly recommend specifying the inode density parameter!! At least if you know the average file size ahead of time. > And when you change the number of inodes at filesystem creation, what effect > will it have when you run growfs later on? Will it expand the filesystem > with an equal inode density, or is it expanded with the default density? growfs currently doesn't look at inode density. If you run growfs on a filesystem without modifying the size, it will allocate extra metadata blocks and could end up failing if the filesystem is nearly full. growfs wasn't developed at the same time as newfs, and the authors decided to ignore inode density as an option. IMO, this is a bug. Although the density is not stored as a parameter in the superblock, it is trivial to compute. Someday I'll probably get around to sending them a patch. If your plan is to use growfs on a filesystem someday, you shouldn't use anything but the default newfs options. However, then you're wasting all that space for metadata you will never use. For UFS2, to figure out how much space is spent on metadata, do the following calculation: ncg = number of cylinder groups (reported by newfs) ipg = number of inodes per group (reported by newfs) bsize = block size (specified to, reported by newfs) metadata = ncg * (65536 + 2 * bsize + 256 * ipg) bytes Using the newfs defaults, for a 70 GiB volume, 2.23 GiB (3.2%) is spent on metadata. NOTE: specifying inode density paramater to newfs greatly reduces the metadata size! -- When I'm migrating non-dynamic filesystems: find /path/to/current/mountpoint | wc -l This will tell you how many files and directories are on the current file system. Number of files plus number of directories == (approx) number of inodes used. With hard links, you may actually need fewer inodes than is shown here, but it's still a good estimate in most cases. Then: df -k /path/to/current/mountpoint will tell you how much storage is in use on said filesystem, in KiB. Divide this by the number of inodes and you have the average filesize and also an approximate inode density (inode density being defined as how much storage is required per inode, or how many inodes to create given a known storage size). I take this number and round it down to the nearest multiple of the block size (in favor of extra inodes instead of not enough). Then I create a new slice/volume of a specified size, greater than the total used space, including extra room if I think the filesystem will grow. I create the filesystem, specifying the -i option (and -b and -f options, if desired). I typically try a few settings and use the -N option as well, until I like the results. I look at the output of newfs: ... using _#_CG_ cylinder groups of _CG_SZ_, _CG_BLKS_, _I_ inodes. I check that the number of inodes per CG times the total number of CGs is greater than or equal to the number of inodes I need. For example, if I found 1000 files taking up 10 GB of space and I decide to allocate a 15 GB filesystem, I ensure there are at least 1500 inodes. As for tweaking the block/frag sizes, I wouldn't go less than 8192/1024 and you can't go above 65536/8192. Things to consider: Remember every file allocates space one block at a time. The last "block" of a file can be a fragment, which is 1/8 of a block. If you have lots of small files, you'll want a small frag size (and thus small blocks). If there are few files and they are all large, larger block sizes can be used. The defaults of 16384/2048 are generally sufficient. I only tweak these when I've calculated that I actually save space by tweaking them. Remember every directory will allocate at least one fragment. Also larger blocks means more inodes will fit into every cylinder group and that a cylinder group can describe more blocks since the free bitmap is part of the CG. -- Rick C. Petty From owner-freebsd-fs@FreeBSD.ORG Sat Jun 9 07:59:44 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C39E816A46D for ; Sat, 9 Jun 2007 07:59:44 +0000 (UTC) (envelope-from gergely.czuczy@harmless.hu) Received: from marvin.harmless.hu (marvin.harmless.hu [195.56.55.204]) by mx1.freebsd.org (Postfix) with ESMTP id 2DA5913C4B0 for ; Sat, 9 Jun 2007 07:59:43 +0000 (UTC) (envelope-from gergely.czuczy@harmless.hu) Received: from localhost (marvin-mail [192.168.0.2]) by marvin.harmless.hu (Postfix) with ESMTP id 2807E7C0AED for ; Sat, 9 Jun 2007 09:59:51 +0200 (CEST) X-Virus-Scanned: by amavisd-new-2.4.2 (20060627) (Debian) at harmless.hu Received: from marvin.harmless.hu ([192.168.0.2]) by localhost (marvin.harmless.hu [192.168.0.2]) (amavisd-new, port 10024) with ESMTP id NB1ITUeYvkPL for ; Sat, 9 Jun 2007 09:59:50 +0200 (CEST) Received: from marvin.harmless.hu (localhost [127.0.0.1]) by marvin.harmless.hu (Postfix) with ESMTP id BCE467C0ADC for ; Sat, 9 Jun 2007 09:59:35 +0200 (CEST) Date: Sat, 9 Jun 2007 09:59:35 +0200 From: Gergely CZUCZY To: freebsd-fs@freebsd.org Message-ID: <20070609075935.GA7863@harmless.hu> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=x-unknown; protocol="application/pgp-signature"; boundary="FCuugMFkClbJLl1L" Content-Disposition: inline User-Agent: mutt-ng/devel-r804 (FreeBSD) Subject: openafs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jun 2007 07:59:44 -0000 --FCuugMFkClbJLl1L Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, I've been googling again for distributed filesystem's state for freebsd, and I have found this so far: http://lists.openafs.org/pipermail/port-freebsd/2007-February/000199.html http://lists.openafs.org/pipermail/port-freebsd/2006-November/000198.html The guy seems to have done some progress, but it's still not functional. Could someone familiar enough with the VFS internals took a look at this? As I've googled the OpenAFS server runs stable on FreeBSD, but nobody is able to get the client working properly. It would be very nice to have some distributed filesystem support when 7.0 will be released. Bye, Gergely Czuczy mailto: gergely.czuczy@harmless.hu --=20 Weenies test. Geniuses solve problems that arise. --FCuugMFkClbJLl1L Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) owGdUz1rFUEUTQwWTrBIowgRbiMBzX7kSd5LnoSYD19Mo0WCCiIyu3t3d8zszDoz +54bBFsLCxGxERV/gJDC3j8QFGy1s7Cx8hd4d59JZSULO8yde8/ce86Z56enJk7M fD74ePfSsxevJz9MPY4uFpVzKvMKboZCeQthuOD1lrqXPfp6vU6H0z/FBd7thoMH 935uaOVQOW+3LrEPDh+5oJRcqCsQ59xYdCuVS70ldpS3KWyprXBCqz4IJYXC47Nd w5VN0XjXVKwTobI+PKy0w8QrjVCORxIZu45S6nnGtueGCBGigkzrjHAy4BldDKk2 kAjrjIgqqoVUSLS1dVjMWbCOO2xTUoMY2WQeuErYNuR82MQrlYDLBSVqSLnpM5Y7 V/aDQBKi9XWJiqe0miwoRYmm4EIGpTbO+4sXdMKw5w0wMhU3dRASfcvLfu4K+T9I Xe+GHmIRoRkjLY2R2G6OkFU1WMTCgtPj9hOtkBovEEqjM4PWzgNxAMK1kwspQWkH aaXihn8ufUbcVzJpi5rilBdCCm4Ala6yHEbC5cQHwq3BDqnl0FBVc6HeAw6yXVxL 2Cpbs9Bq0sqBSVt2k6Zco1KLZogGTKVaCUhI0AoGNOj6zua4SaUjndQgLGuPaaYM XQsSS0HugJE2e43KNBvRJWtqfruJNv1HCIRfgxIxHtPRMvFvJ4CtyoZrGOVkoJ4f slHDDsEYlMgtJoS+XiP5bAtNhrKGjf0q3q9ZI5PTfWquDftxG75KXi8I3Pp5xZjn rXRCdpu8KZDIQut82KJNZbExlhy2AtGUjXY5EciNsOizp6tTJyeaR3X0IGdObB1O vDl7Kvj2fvT7yY+viwfnDt++/D598GviXTz9pZw8v1qcWXx15/4nPnthZ3n2Dw== =cg+t -----END PGP SIGNATURE----- --FCuugMFkClbJLl1L--