From owner-freebsd-fs@FreeBSD.ORG Sun Jun 22 11:43:29 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F8DA37B401; Sun, 22 Jun 2003 11:43:29 -0700 (PDT) Received: from baerenklau.de.freebsd.org (baerenklau.de.freebsd.org [195.185.195.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC87243F3F; Sun, 22 Jun 2003 11:43:27 -0700 (PDT) (envelope-from wosch@panke.de.FreeBSD.org) Received: from baerenklau.de.freebsd.org (localhost.de.freebsd.org [127.0.0.1])h5MIhQCx005019; Sun, 22 Jun 2003 20:43:26 +0200 (CEST) Received: (from uucp@localhost)h5MIhQMk005017; Sun, 22 Jun 2003 20:43:26 +0200 (CEST) Received: (from wosch@localhost) by paula.panke.de.freebsd.org (8.12.9/8.12.6) id h5MIU1Ss000243; Sun, 22 Jun 2003 20:30:01 +0200 (CEST) (envelope-from wosch) Date: Sun, 22 Jun 2003 20:30:01 +0200 From: Wolfram Schneider To: Deepakala G Message-ID: <20030622203001.A207@paula.panke.de.freebsd.org> References: <20030620010217.51116.qmail@web8201.mail.in.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20030620010217.51116.qmail@web8201.mail.in.yahoo.com>; from g_deepakala@yahoo.co.in on Fri, Jun 20, 2003 at 02:02:17AM +0100 cc: freebsd-fs@freebsd.org cc: wosch@freebsd.org Subject: Re: wanting papers on fast file system for linux X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2003 18:43:29 -0000 On 2003-06-20 02:02:17 +0100, Deepakala G wrote: > I am a final year computer engineering student.My project title is "implementing fast file system for linux".Can u please send me sone related papers or the links to the papers(on fast file system)? > Deepakala Ganesan Hi Deepakala, I forwarded your E-Mail to the FreeBSD filesystem mailing list freebsd-fs@freebsd.org good luck, -Wolfram -- Wolfram Schneider http://wolfram.schneider.org From owner-freebsd-fs@FreeBSD.ORG Sun Jun 22 11:56:07 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5551937B401; Sun, 22 Jun 2003 11:56:07 -0700 (PDT) Received: from phoenix.infradead.org (phoenix.infradead.org [195.224.96.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E9F943FBD; Sun, 22 Jun 2003 11:56:06 -0700 (PDT) (envelope-from hch@infradead.org) Received: from hch by phoenix.infradead.org with local (Exim 4.10) id 19UA0S-0004Yu-00; Sun, 22 Jun 2003 19:56:04 +0100 Date: Sun, 22 Jun 2003 19:56:04 +0100 From: Christoph Hellwig To: Wolfram Schneider Message-ID: <20030622195604.A17500@infradead.org> References: <20030620010217.51116.qmail@web8201.mail.in.yahoo.com> <20030622203001.A207@paula.panke.de.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030622203001.A207@paula.panke.de.freebsd.org>; from wosch@freebsd.org on Sun, Jun 22, 2003 at 08:30:01PM +0200 cc: freebsd-fs@freebsd.org cc: Deepakala G Subject: Re: wanting papers on fast file system for linux X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Jun 2003 18:56:07 -0000 On Sun, Jun 22, 2003 at 08:30:01PM +0200, Wolfram Schneider wrote: > On 2003-06-20 02:02:17 +0100, Deepakala G wrote: > > I am a final year computer engineering student.My project title is "implementing fast file system for linux".Can u please send me sone related papers or the links to the papers(on fast file system)? > > Deepakala Ganesan There's already fs/ufs/ in the Linux source tree that supports about a dozend ufs/ffs variant. OTOH it's unmaintained and got horribly buggy over the time... From owner-freebsd-fs@FreeBSD.ORG Mon Jun 23 04:23:22 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A06737B401; Mon, 23 Jun 2003 04:23:22 -0700 (PDT) Received: from HAL9000.homeunix.com (ip114.bella-vista.sfo.interquest.net [66.199.86.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7209143F75; Mon, 23 Jun 2003 04:23:21 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.9) with ESMTP id h5NBNIJa026712; Mon, 23 Jun 2003 04:23:18 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.9/Submit) id h5NBNI9q026710; Mon, 23 Jun 2003 04:23:18 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Date: Mon, 23 Jun 2003 04:23:17 -0700 From: David Schultz To: Wolfram Schneider Message-ID: <20030623112317.GA12521@HAL9000.homeunix.com> Mail-Followup-To: Wolfram Schneider , Deepakala G , freebsd-fs@freebsd.org References: <20030620010217.51116.qmail@web8201.mail.in.yahoo.com> <20030622203001.A207@paula.panke.de.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030622203001.A207@paula.panke.de.freebsd.org> cc: freebsd-fs@FreeBSD.ORG cc: Deepakala G Subject: Re: wanting papers on fast file system for linux X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jun 2003 11:23:22 -0000 On Sun, Jun 22, 2003, Wolfram Schneider wrote: > On 2003-06-20 02:02:17 +0100, Deepakala G wrote: > > I am a final year computer engineering student.My project title is "implementing fast file system for linux".Can u please send me sone related papers or the links to the papers(on fast file system)? Off the top of my head, there's McKusick et al. A Fast Filesystem for UNIX. McKusick et al. The Design and Implementation of the 4.4 BSD Operating System. McKusick and Ganger. Soft Updates: A Technique for Eliminating Most Synchronous Writes in the Fast Filesystem. In Proc. FREENIX, 1999. The first two are very similar and give a good high-level overview of FFS. The third details softupdates for FFS, although neither technology is specific to the other. I believe one of the Solaris Internals books covers their lufs implementation. Other than that, it's a safe bet that any paper by Margo Seltzer will be interesting. Of course, there's always the source code, which you can find out how to obtain by checking out the FreeBSD website. It may be worthwhile to look at both the FreeBSD and NetBSD trees. For Linux, your best bet is probably to look at the ext2fs implementation, not the UFS code. ext2fs is essentially a simplified version of FFS. It mainly leaves out all the disk geometry-specific optimizations (which you don't want anyway) and support for file fragments (which you do want). The two filesystems also have different ways of dealing with certain problems (e.g. FFS's block reallocation vs. ext2fs's preallocation). From owner-freebsd-fs@FreeBSD.ORG Mon Jun 23 20:04:40 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 83A8E37B401 for ; Mon, 23 Jun 2003 20:04:40 -0700 (PDT) Received: from mta04ps.bigpond.com (mta04ps.bigpond.com [144.135.25.158]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE0D243FAF for ; Mon, 23 Jun 2003 20:04:38 -0700 (PDT) (envelope-from areilly@bigpond.net.au) Received: from areilly.bpc-users.org ([144.135.25.72]) by mta04ps.email.bigpond.com (iPlanet Messaging Server 5.2 HotFix 1.14 (built Mar 18 2003)) with SMTP id <0HGY00FWATTIX6@mta04ps.email.bigpond.com> for freebsd-fs@freebsd.org; Tue, 24 Jun 2003 13:03:18 +1000 (EST) Received: from cpe-144-132-191-61.nsw.bigpond.net.au ([144.132.191.61]) by psmam02bpa.bigpond.com(MAM $Name: REL_3_3_2b $ 80/3793867); Tue, 24 Jun 2003 13:03:20 +0000 Received: (qmail 63236 invoked from network); Tue, 24 Jun 2003 03:03:25 +0000 Received: from localhost (HELO ?127.0.0.1?) (andrew@127.0.0.1) by localhost with SMTP; Tue, 24 Jun 2003 03:03:25 +0000 Date: Tue, 24 Jun 2003 13:03:25 +1000 From: Andrew Reilly In-reply-to: To: Mohammad Nayyer Zubair Message-id: <1056423804.48266.54.camel@gurney.reilly.home> MIME-version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Content-type: text/plain Content-transfer-encoding: 7BIT References: cc: freebsd-fs@freebsd.org Subject: Re: ideas about a unioning file system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 03:04:40 -0000 Hi, On Fri, 2003-06-20 at 08:15, Mohammad Nayyer Zubair wrote: > Has anyone extensively used freebsd unionfs? From a system/network > administrator or from a kernel developer standpoint, what do you like > about it and what you dont like about it? I'm using unionfs thusly: # Device Mountpoint FStype Options Dump Pass# /dev/ad0s1a / ufs rw 0 1 /dev/vinum/mirror /home ufs rw 0 2 /dev/vinum/vinum0 /usr ufs rw,union 0 2 (sorry about the wrappage, cut and pasted from /etc/fstab.) There are other lines in /etc/fstab, of course, but these are the bulk of my workstation's file hierarchy. After years of tiny root partitions that eventually caused grief and breakage because of gradually expanding kernels/modules/cruft-in-etc, this time around I've made root about 500M and put the whole FreeBSD base system in there. So a buildworld/kernel/installworld touches everything in ad0s1a, and nothing else does (except /etc). This leaves the problem of what to do with /usr/ports, /usr/X11R6, /usr/local, /compat@->usr/compat, /tmp@->var/tmp, /var@->usr/var I could have mounted /dev/vinum/vinum0 somewhere like /usr1, and filled /usr with a bunch of symlinks, and I did that for a while. It's messy though, and quite a few things, like ports, bother to find their "true" path, so the /usr1 name leaks into config files and what-not. Ugly. Union mounting seems to be working for me. I do my backups, and it's not a super-heavily used system... Touch wood. > How should a unioning filesystem should behave? What specific features > would you like it to have? It should behave just the way it does: the stuff in /usr that was there before the mount stays there, and gets modified and all, just as it should be. Anything that wasn't in /usr before the mount gets written to the union partition. Reads see both. > Out of the previous efforts at a unioning file system like the Sun's TFS, > 3DFS, Plan 9 and FreeBSD unionfs itself, which fs do you think came close > to an ideal unioning file system? What's wrong with the one that we have? Cheers, -- Andrew Reilly From owner-freebsd-fs@FreeBSD.ORG Mon Jun 23 20:17:55 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52F3B37B401 for ; Mon, 23 Jun 2003 20:17:55 -0700 (PDT) Received: from mobile.hub.org (u153n214.eastlink.ca [24.224.153.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7AC6143F3F for ; Mon, 23 Jun 2003 20:17:54 -0700 (PDT) (envelope-from scrappy@hub.org) Received: by mobile.hub.org (Postfix, from userid 1001) id 744365DB; Tue, 24 Jun 2003 00:17:53 -0300 (ADT) Received: from localhost (localhost [127.0.0.1]) by mobile.hub.org (Postfix) with ESMTP id 6494E571; Tue, 24 Jun 2003 00:17:53 -0300 (ADT) Date: Tue, 24 Jun 2003 00:17:53 -0300 (ADT) From: The Hermit Hacker To: Andrew Reilly In-Reply-To: <1056423804.48266.54.camel@gurney.reilly.home> Message-ID: <20030624001138.R5387@hub.org> References: <1056423804.48266.54.camel@gurney.reilly.home> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org cc: Mohammad Nayyer Zubair Subject: Re: ideas about a unioning file system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 03:17:55 -0000 I missed the original message, so excuse me, but am piggy backing on Andrew's response :) On Tue, 24 Jun 2003, Andrew Reilly wrote: > Hi, > > On Fri, 2003-06-20 at 08:15, Mohammad Nayyer Zubair wrote: > > Has anyone extensively used freebsd unionfs? From a system/network > > administrator or from a kernel developer standpoint, what do you like > > about it and what you dont like about it? > > I'm using unionfs thusly: > > # Device Mountpoint FStype Options Dump > Pass# > /dev/ad0s1a / ufs rw 0 > 1 > /dev/vinum/mirror /home ufs rw 0 > 2 > /dev/vinum/vinum0 /usr ufs rw,union 0 > 2 > > (sorry about the wrappage, cut and pasted from /etc/fstab.) neptune# df -t union | wc -l 40 neptune# ssh jupiter df -t union | wc -l 41 pluto# ssh pluto df -t union | wc -l 40 we use it for two reasons: it *greatly* reduces the disk foot print of running jail'd environments (we figure we save ~40Gig per server), and it makes upgrading applications in ports alot easier when you only have to upgrade the "base" jail, instead of each and every one ... > > Out of the previous efforts at a unioning file system like the Sun's TFS, > > 3DFS, Plan 9 and FreeBSD unionfs itself, which fs do you think came close > > to an ideal unioning file system? > > What's wrong with the one that we have? Alot of things under the covers ... but at the 'visible' layer, I can't think of anything I'd expect it to do any differently ... there are alot of bugs still in unionfs, that are slowly being addresses as ppl are able to produce "good debug info" ... There was a recent thread on -arch that should be checked, that talked about alot of the "implementation deficiencies" in unionfs, but other then stability issues (ie. memory leaks, vnode leaks, etc), what unionfs *does* I've been most happy with ... From owner-freebsd-fs@FreeBSD.ORG Mon Jun 23 23:30:39 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8AD6F37B401 for ; Mon, 23 Jun 2003 23:30:39 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id D6D6643FA3 for ; Mon, 23 Jun 2003 23:30:38 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-38ldt1t.dialup.mindspring.com ([209.86.244.61] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19UhK7-0004I0-00; Mon, 23 Jun 2003 23:30:35 -0700 Message-ID: <3EF7EFCB.4A4BACB8@mindspring.com> Date: Mon, 23 Jun 2003 23:29:31 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Andrew Reilly References: <1056423804.48266.54.camel@gurney.reilly.home> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a42b293040c93906add70b55efaa797afc667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c cc: freebsd-fs@freebsd.org cc: Mohammad Nayyer Zubair Subject: Re: ideas about a unioning file system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 06:30:39 -0000 Andrew Reilly wrote: > On Fri, 2003-06-20 at 08:15, Mohammad Nayyer Zubair wrote: > > Has anyone extensively used freebsd unionfs? From a system/network > > administrator or from a kernel developer standpoint, what do you like > > about it and what you dont like about it? > > I'm using unionfs thusly: > > # Device Mountpoint FStype Options Dump > /dev/vinum/vinum0 /usr ufs rw,union 0 > 2 Actually, this is the union mount option, which isn't the same thing as unionfs. > > Out of the previous efforts at a unioning file system like the Sun's TFS, > > 3DFS, Plan 9 and FreeBSD unionfs itself, which fs do you think came close > > to an ideal unioning file system? > > What's wrong with the one that we have? It's a mount option, not a file system. 8-). -- Terry From owner-freebsd-fs@FreeBSD.ORG Mon Jun 23 23:52:12 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7A12637B401 for ; Mon, 23 Jun 2003 23:52:12 -0700 (PDT) Received: from mta04bw.bigpond.com (mta04bw.bigpond.com [139.134.6.87]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E95B43FA3 for ; Mon, 23 Jun 2003 23:52:11 -0700 (PDT) (envelope-from areilly@bigpond.net.au) Received: from areilly.bpc-users.org ([144.135.24.72]) by mta04bw.bigpond.com (Netscape Messaging Server 4.15 mta04bw Jul 16 2002 22:47:55) with SMTP id HGZ4DX00.J5B for ; Tue, 24 Jun 2003 16:51:33 +1000 Received: from cpe-144-132-191-61.nsw.bigpond.net.au ([144.132.191.61]) by bwmam02bpa.bigpond.com(MAM REL_3_3_2c 17/2523322); 24 Jun 2003 16:51:35 Received: (qmail 66005 invoked from network); 24 Jun 2003 06:51:41 -0000 Received: from localhost (HELO ?127.0.0.1?) (andrew@127.0.0.1) by localhost with SMTP; 24 Jun 2003 06:51:41 -0000 From: Andrew Reilly To: Terry Lambert In-Reply-To: <3EF7EFCB.4A4BACB8@mindspring.com> References: <1056423804.48266.54.camel@gurney.reilly.home> <3EF7EFCB.4A4BACB8@mindspring.com> Content-Type: text/plain Message-Id: <1056437500.48266.64.camel@gurney.reilly.home> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 24 Jun 2003 16:51:40 +1000 Content-Transfer-Encoding: 7bit cc: freebsd-fs@freebsd.org cc: Mohammad Nayyer Zubair Subject: Re: ideas about a unioning file system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 06:52:12 -0000 On Tue, 2003-06-24 at 16:29, Terry Lambert wrote: > Andrew Reilly wrote: > > On Fri, 2003-06-20 at 08:15, Mohammad Nayyer Zubair wrote: > > > Has anyone extensively used freebsd unionfs? From a system/network > > > administrator or from a kernel developer standpoint, what do you like > > > about it and what you dont like about it? > > > > I'm using unionfs thusly: > > > > # Device Mountpoint FStype Options Dump > > /dev/vinum/vinum0 /usr ufs rw,union 0 > > 2 > > Actually, this is the union mount option, which isn't the same > thing as unionfs. Aah. OK. I wondered a bit about that myself. So: what's unionfs? Does the union option in fstab invoke mount_union, or is union mounting something that mount manages for itself, independent of the FStype? Having looked at mount_union, I don't think that I like the copy-to-top-layer on write semantics for objects that exist in the bottom layer. I hope that's not happening to my system. > It's a mount option, not a file system. 8-). How could it be otherwise? -- Andrew Reilly From owner-freebsd-fs@FreeBSD.ORG Tue Jun 24 12:47:22 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 37DDB37B404; Tue, 24 Jun 2003 12:47:22 -0700 (PDT) Received: from TOMOYO.MyBSD.ORG.MY (router.ded2.com [202.157.183.149]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7189143F93; Tue, 24 Jun 2003 12:47:20 -0700 (PDT) (envelope-from skywizard@TOMOYO.MyBSD.ORG.MY) Received: from TOMOYO.MyBSD.ORG.MY (localhost [127.0.0.1]) by TOMOYO.MyBSD.ORG.MY (8.12.9/8.12.3) with ESMTP id h5OJnMBe062643; Wed, 25 Jun 2003 03:49:22 +0800 (MYT) (envelope-from skywizard@TOMOYO.MyBSD.ORG.MY) Received: (from skywizard@localhost) by TOMOYO.MyBSD.ORG.MY (8.12.9/8.12.3/Submit) id h5OJnMH4062642; Wed, 25 Jun 2003 03:49:22 +0800 (MYT) Date: Wed, 25 Jun 2003 03:49:22 +0800 (MYT) Message-Id: <200306241949.h5OJnMH4062642@TOMOYO.MyBSD.ORG.MY> To: FreeBSD-gnats-submit@freebsd.org From: skywizard@MyBSD.org.my X-send-pr-version: 3.113 X-GNATS-Notify: cc: freebsd-fs@freebsd.org cc: freebsd-current@freebsd.org cc: freebsd-stable@freebsd.org Subject: truncate operation on fat32 may corrupt the file system X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: skywizard@MyBSD.org.my List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jun 2003 19:47:22 -0000 >Submitter-Id: current-users >Originator: Ariff Abdullah >Organization: MyBSD >Confidential: no >Synopsis: truncate operation on fat32 may corrupt the file system >Severity: critical >Priority: high >Category: kern >Class: sw-bug >Release: FreeBSD 4.7-RELEASE i386 >Environment: System: 4.7-RELEASE, 5.1-RELEASE (GENERIC) >Description: Truncate operation involving truncate() or ftruncate() on FAT32 mounted as msdos either failed or silently corrupting the file or even worse, corrupting the neighbour file reiside in the same partition/file system. >How-To-Repeat: # cd /to/fat32/partition/ # dd if=/dev/zero of=XX bs=4099 count=1 # truncate -s 4097 XX truncate: XX: Argument list too long errno E2BIG >Fix: --- /usr/src/sys/msdosfs/msdosfs_denode.c.orig Tue Jun 24 06:01:09 2003 +++ /usr/src/sys/msdosfs/msdosfs_denode.c Tue Jun 24 05:53:41 2003 @@ -501,26 +501,19 @@ bn = cntobn(pmp, eofentry); error = bread(pmp->pm_devvp, bn, pmp->pm_bpcluster, NOCRED, &bp); - } else { - bn = de_blk(pmp, length); - error = bread(DETOV(dep), bn, pmp->pm_bpcluster, - NOCRED, &bp); - } - if (error) { - brelse(bp); + if (error) { + brelse(bp); #ifdef MSDOSFS_DEBUG - printf("detrunc(): bread fails %d\n", error); + printf("detrunc(): bread fails %d\n", error); #endif - return (error); + return (error); + } + bzero(bp->b_data + boff, pmp->pm_bpcluster - boff); + if (flags & IO_SYNC) + bwrite(bp); + else + bdwrite(bp); } - /* - * is this the right place for it? - */ - bzero(bp->b_data + boff, pmp->pm_bpcluster - boff); - if (flags & IO_SYNC) - bwrite(bp); - else - bdwrite(bp); } /* From owner-freebsd-fs@FreeBSD.ORG Thu Jun 26 07:51:39 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C74D937B415; Thu, 26 Jun 2003 07:51:39 -0700 (PDT) Received: from mother.ludd.luth.se (mother.ludd.luth.se [130.240.16.3]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1817C43FAF; Thu, 26 Jun 2003 07:51:38 -0700 (PDT) (envelope-from pb@ludd.luth.se) Received: from brother.ludd.luth.se (brother.ludd.luth.se [130.240.16.78]) by mother.ludd.luth.se (8.11.6+Sun/8.9.3) with ESMTP id h5QEpa820259; Thu, 26 Jun 2003 16:51:36 +0200 (MEST) From: Peter B Received: (from pb@localhost) by brother.ludd.luth.se (8.11.6+Sun/8.9.3) id h5QEpaP12720; Thu, 26 Jun 2003 16:51:36 +0200 (MEST) Message-Id: <200306261451.h5QEpaP12720@brother.ludd.luth.se> To: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org Date: Thu, 26 Jun 2003 16:51:36 +0200 (MEST) X-Mailer: ELM [version 2.4ME+ PL54 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Encrypted filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2003 14:51:40 -0000 I have searched for encrypted filesystems for un*x. Is there any better encrypted filesystems than the ones I have found for *bsd (+freebsd)..? Note that some comments are based on what others have said. I think it's important to keep in mind the different approches used, per-file vs disc-block aswell. I'm looking for something convinient to enrypt cdrom's. Which will also suit dvd-r media. It should preferable be portable and not require specific kernel hacks. To ensure feature stability & availability. The encrypted filesystems arena looks like a collection of software rather than a unified solution across platforms. Which operating systems manage to effectivly to use encrypted swap..? Openbsd seems to handle it nativly, while freebsd could possible use vncrypt in conjuction with swapon, or cfsd with swapon-file. Netbsd might use cgd? ==== Interesting encrypted filesystem projects ==== The following is directly usable on freebsd: cfs 2 GB limit (nfsv2), easy portable vncrypt Unstable? (and needs kernel module) geom(4) Modular disk I/O request transformation framework The following seems usable althought might require some work: loop-aes Only ported to linux so far http://sourceforge.net/projects/loop-aes/ cryptfs Port for freebsd available (btw, check out FiST!) http://www1.cs.columbia.edu/~ezk/research/cryptfs/index.html http://ftp.vit.edu.tw/pc/programming/hacktic/disk/ BestCrypt Source avail, 30day trial period. http://www.jetico.com/ Available, BUT not directly applicable: PPDD Linux specific, needs 100MHz+ pentium pgpdisk M$/win+Mac binary only http://mail.lab.net/lists/archive/cryptography-exploder/2003-February.txt PGPdisk + Linux ..? Janis Jagars, handle Disastry tcfs Alias cfs? (available for Linux,Netbsd,Openbsd) http://www.tcfs.it/ ncryptfs Follow up from cryptfs, not publicly released yet. http://www1.cs.columbia.edu/~ezk/research/ncryptfs/ncryptfs.html#sec:eval-feature From owner-freebsd-fs@FreeBSD.ORG Thu Jun 26 12:20:57 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 266DE37B404 for ; Thu, 26 Jun 2003 12:20:57 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 1114243FCB for ; Thu, 26 Jun 2003 12:20:54 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 46873 invoked from network); 26 Jun 2003 19:20:52 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 26 Jun 2003 19:20:52 -0000 Message-ID: <3EFB4792.5080805@tenebras.com> Date: Thu, 26 Jun 2003 12:20:50 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3.1) Gecko/20030425 X-Accept-Language: en-us, zh-tw, zh-cn, fr, en, de-de MIME-Version: 1.0 To: Peter B References: <200306261451.h5QEpaP12720@brother.ludd.luth.se> In-Reply-To: <200306261451.h5QEpaP12720@brother.ludd.luth.se> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-fs@freebsd.org cc: freebsd-hackers@freebsd.org Subject: Re: Encrypted filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2003 19:20:57 -0000 Peter B wrote: > I have searched for encrypted filesystems for un*x. Is there any better > encrypted filesystems than the ones I have found for *bsd (+freebsd)..? For per-file encryption, cryptfs/FiST is a good place to start. > I'm looking for something convinient to enrypt cdrom's. Which will also suit > dvd-r media. It should preferable be portable and not require specific kernel > hacks. To ensure feature stability & availability. Stackable virtual filesystems seem to be your friend. > Which operating systems manage to effectivly to use encrypted swap..? That's quite a different problem -- Poul-Henning Kamp's done work in GEOM based disk encryption which is directly applicable to encrypting swap. Key management is always interesting. -- "Well," Brahma said, "even after ten thousand explanations, a fool is no wiser, but an intelligent man requires only two thousand five hundred." - The Mahabharata From owner-freebsd-fs@FreeBSD.ORG Thu Jun 26 16:35:15 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 30B0937B401; Thu, 26 Jun 2003 16:35:15 -0700 (PDT) Received: from filer.fsl.cs.sunysb.edu (filer.fsl.cs.sunysb.edu [130.245.126.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E26F43FF5; Thu, 26 Jun 2003 16:35:13 -0700 (PDT) (envelope-from ezk@fsl.cs.sunysb.edu) Received: from agora.fsl.cs.sunysb.edu (IDENT:uH1vLzWqD2R6OGEBmLwuTiUjaF/wq1y5@agora.fsl.cs.sunysb.edu [130.245.126.12])h5QNYrcq011259; Thu, 26 Jun 2003 19:34:53 -0400 Received: from agora.fsl.cs.sunysb.edu (IDENT:cSvBBTTA/eVBdZqtrZBsgi4BVVp+0XC8@localhost.localdomain [127.0.0.1]) h5QNZCif003213; Thu, 26 Jun 2003 19:35:12 -0400 Received: (from ezk@localhost) by agora.fsl.cs.sunysb.edu (8.12.8/8.12.8/Submit) id h5QNZBNF003209; Thu, 26 Jun 2003 19:35:11 -0400 Date: Thu, 26 Jun 2003 19:35:11 -0400 Message-Id: <200306262335.h5QNZBNF003209@agora.fsl.cs.sunysb.edu> From: Erez Zadok To: Michael Sierchio In-reply-to: Your message of "Thu, 26 Jun 2003 12:20:50 PDT." <3EFB4792.5080805@tenebras.com> X-MailKey: Erez Zadok cc: freebsd-fs@freebsd.org cc: freebsd-hackers@freebsd.org cc: cwright@cs.sunysb.edu cc: Peter B Subject: Re: Encrypted filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2003 23:35:15 -0000 In message <3EFB4792.5080805@tenebras.com>, Michael Sierchio writes: > Peter B wrote: [...] > > Which operating systems manage to effectivly to use encrypted swap..? > > That's quite a different problem -- Poul-Henning Kamp's done work > in GEOM based disk encryption which is directly applicable to > encrypting swap. Key management is always interesting. You might check the work/papers by Niels Provos on encrypted swap. We've been working on this problem for our NCryptfs. To provide a comprehensive solution, we have to also handle swap. Luckily in Linux, there are specific VOPs and APIs that make it easy to hook a crypto f/s w/ the swap. Erez. From owner-freebsd-fs@FreeBSD.ORG Thu Jun 26 16:48:36 2003 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D3F837B401; Thu, 26 Jun 2003 16:48:36 -0700 (PDT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6CAEB44013; Thu, 26 Jun 2003 16:48:35 -0700 (PDT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.9/8.12.9) with ESMTP id h5QNmT1Y002584; Fri, 27 Jun 2003 01:48:30 +0200 (CEST) (envelope-from phk@phk.freebsd.dk) To: Erez Zadok From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 26 Jun 2003 19:35:11 EDT." <200306262335.h5QNZBNF003209@agora.fsl.cs.sunysb.edu> Date: Fri, 27 Jun 2003 01:48:29 +0200 Message-ID: <2583.1056671309@critter.freebsd.dk> cc: freebsd-fs@freebsd.org cc: freebsd-hackers@freebsd.org cc: cwright@cs.sunysb.edu cc: Peter B Subject: Re: Encrypted filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Jun 2003 23:48:36 -0000 In message <200306262335.h5QNZBNF003209@agora.fsl.cs.sunysb.edu>, Erez Zadok wr ites: >In message <3EFB4792.5080805@tenebras.com>, Michael Sierchio writes: >> Peter B wrote: >[...] >> > Which operating systems manage to effectivly to use encrypted swap..? >> >> That's quite a different problem -- Poul-Henning Kamp's done work >> in GEOM based disk encryption which is directly applicable to >> encrypting swap. Key management is always interesting. > >You might check the work/papers by Niels Provos on encrypted swap. > >We've been working on this problem for our NCryptfs. To provide a >comprehensive solution, we have to also handle swap. Luckily in Linux, >there are specific VOPs and APIs that make it easy to hook a crypto f/s w/ >the swap. I will present a paper on the GBDE encrypted disk facility at BSDcon2003. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.