From owner-freebsd-fs@FreeBSD.ORG Tue Jul 13 21:34:15 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E83D316A4CE for ; Tue, 13 Jul 2004 21:34:15 +0000 (GMT) Received: from porthos.compasstg.net (ns01.europenetsystems.net [207.13.208.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF3B043D3F for ; Tue, 13 Jul 2004 21:34:15 +0000 (GMT) (envelope-from mickonet@porthos.compasstg.net) Received: by porthos.compasstg.net (Postfix, from userid 1006) id C9FD541DE8; Tue, 13 Jul 2004 16:30:17 -0500 (CDT) Date: Tue, 13 Jul 2004 16:30:17 -0500 From: micko To: freebsd-fs@freebsd.org Message-ID: <20040713213017.GA38398@micko.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD porthos.compasstg.net 4.7-STABLE FreeBSD 4.7-STABLE Subject: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2004 21:34:16 -0000 Hi, I just installed fbsd 5.2.1 on a system which had fbsd 4.9 with a vinum ufs1 stripe. When I tried to mount the vinum stripe on fbsd 5.2.1 it went ok, but doing 'ls' gave me 'bad_dir' and a kernel panic. Doing fsck gave me 'unknown filesystem type'. Seems like there is no backwards compatibility. Can this somehow be saved or do I need to put the drives on a fbsd 4.x system to get the data? Please CC me when replying since I am not subscribed to the freebsd-fs list. thanks micko From owner-freebsd-fs@FreeBSD.ORG Tue Jul 13 22:13:07 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA9B216A4CE for ; Tue, 13 Jul 2004 22:13:07 +0000 (GMT) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6280243D1D for ; Tue, 13 Jul 2004 22:13:07 +0000 (GMT) (envelope-from grog@lemis.com) Received: from blackwater.lemis.com (blackwater.lemis.com [192.109.197.80]) by ozlabs.org (Postfix) with ESMTP id D95412BD43 for ; Wed, 14 Jul 2004 08:13:04 +1000 (EST) Received: by blackwater.lemis.com (Postfix, from userid 1004) id E9560511FE; Wed, 14 Jul 2004 07:43:02 +0930 (CST) Date: Wed, 14 Jul 2004 07:43:02 +0930 From: Greg 'groggy' Lehey To: micko Message-ID: <20040713221302.GA63062@wantadilla.lemis.com> References: <20040713213017.GA38398@micko.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZvS9be/3tNcYl/X" Content-Disposition: inline In-Reply-To: <20040713213017.GA38398@micko.net> User-Agent: Mutt/1.4.1i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: freebsd-fs@freebsd.org Subject: Re: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jul 2004 22:13:07 -0000 --LZvS9be/3tNcYl/X Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 13 July 2004 at 16:30:17 -0500, micko wrote: > Hi, I just installed fbsd 5.2.1 on a system which > had fbsd 4.9 with a vinum ufs1 stripe. When I tried > to mount the vinum stripe on fbsd 5.2.1 it went ok, but > doing 'ls' gave me 'bad_dir' and a kernel panic. Doing > fsck gave me 'unknown filesystem type'. Seems like there > is no backwards compatibility. That conclusion is incorrect. > Can this somehow be saved or do I need to put the drives on a fbsd > 4.x system to get the data? You need to find out what the problem is first. To do so, you should use the information at your disposal. See http://www.vinumvm.org/vinum/how-to-debug.html for more details. Greg -- Note: I discard all HTML mail unseen. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. --LZvS9be/3tNcYl/X Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQFA9F5uIubykFB6QiMRAtSAAJ4mbBwtrmPtRXNr7VX6ps21xlTQRQCeOZha d5KGaX36L5Z1SnOKETd5XNE= =eJUh -----END PGP SIGNATURE----- --LZvS9be/3tNcYl/X-- From owner-freebsd-fs@FreeBSD.ORG Wed Jul 14 01:10:49 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6721616A4CE; Wed, 14 Jul 2004 01:10:49 +0000 (GMT) Received: from porthos.compasstg.net (ns01.europenetsystems.net [207.13.208.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FEC243D45; Wed, 14 Jul 2004 01:10:49 +0000 (GMT) (envelope-from mickonet@porthos.compasstg.net) Received: by porthos.compasstg.net (Postfix, from userid 1006) id 41EC34113B; Tue, 13 Jul 2004 20:06:50 -0500 (CDT) Date: Tue, 13 Jul 2004 20:06:50 -0500 From: micko To: Greg 'groggy' Lehey Message-ID: <20040714010650.GA8692@micko.net> References: <20040713213017.GA38398@micko.net> <20040713221302.GA63062@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040713221302.GA63062@wantadilla.lemis.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD porthos.compasstg.net 4.7-STABLE FreeBSD 4.7-STABLE cc: freebsd-fs@freebsd.org Subject: Re: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 01:10:49 -0000 On Wed, Jul 14, 2004 at 07:43:02AM +0930, Greg 'groggy' Lehey wrote: > On Tuesday, 13 July 2004 at 16:30:17 -0500, micko wrote: > > Hi, I just installed fbsd 5.2.1 on a system which > > had fbsd 4.9 with a vinum ufs1 stripe. When I tried > > to mount the vinum stripe on fbsd 5.2.1 it went ok, but > > doing 'ls' gave me 'bad_dir' and a kernel panic. Doing > > fsck gave me 'unknown filesystem type'. Seems like there > > is no backwards compatibility. > > That conclusion is incorrect. > > > Can this somehow be saved or do I need to put the drives on a fbsd > > 4.x system to get the data? > > You need to find out what the problem is first. To do so, you should > use the information at your disposal. See > http://www.vinumvm.org/vinum/how-to-debug.html for more details. It turned out to be a failing drive, which now is totally dead. Is there anyway to get some data out of the 1st part of the stripe? thanks micko From owner-freebsd-fs@FreeBSD.ORG Wed Jul 14 03:04:52 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2721D16A4CE; Wed, 14 Jul 2004 03:04:52 +0000 (GMT) Received: from porthos.compasstg.net (ns01.europenetsystems.net [207.13.208.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id D717843D54; Wed, 14 Jul 2004 03:04:51 +0000 (GMT) (envelope-from mickonet@porthos.compasstg.net) Received: by porthos.compasstg.net (Postfix, from userid 1006) id 7D5714114E; Tue, 13 Jul 2004 22:00:52 -0500 (CDT) Date: Tue, 13 Jul 2004 22:00:52 -0500 From: micko To: Greg 'groggy' Lehey Message-ID: <20040714030052.GA9508@micko.net> References: <20040713213017.GA38398@micko.net> <20040713221302.GA63062@wantadilla.lemis.com> <20040714010650.GA8692@micko.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040714010650.GA8692@micko.net> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD porthos.compasstg.net 4.7-STABLE FreeBSD 4.7-STABLE cc: freebsd-fs@freebsd.org Subject: Re: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 03:04:52 -0000 On Tue, Jul 13, 2004 at 08:06:50PM -0500, micko wrote: > On Wed, Jul 14, 2004 at 07:43:02AM +0930, Greg 'groggy' Lehey wrote: > > On Tuesday, 13 July 2004 at 16:30:17 -0500, micko wrote: > > > Hi, I just installed fbsd 5.2.1 on a system which > > > had fbsd 4.9 with a vinum ufs1 stripe. When I tried > > > to mount the vinum stripe on fbsd 5.2.1 it went ok, but > > > doing 'ls' gave me 'bad_dir' and a kernel panic. Doing > > > fsck gave me 'unknown filesystem type'. Seems like there > > > is no backwards compatibility. > > > > That conclusion is incorrect. > > > > > Can this somehow be saved or do I need to put the drives on a fbsd > > > 4.x system to get the data? > > > > You need to find out what the problem is first. To do so, you should > > use the information at your disposal. See > > http://www.vinumvm.org/vinum/how-to-debug.html for more details. > > It turned out to be a failing drive, which now is totally dead. Is there > anyway to get some data out of the 1st part of the stripe? I found the problem. Originally when I was creating the partitions on the fbsd 4.x system I didn't slice up the drive, I used da0c/e. When using the drive under fbsd 5.x and devfs, the system brings up da0s1c/e. Is there anyway to override this behavior? thanks micko From owner-freebsd-fs@FreeBSD.ORG Wed Jul 14 04:37:09 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 37F9D16A4CE for ; Wed, 14 Jul 2004 04:37:09 +0000 (GMT) Received: from ozlabs.org (ozlabs.org [203.10.76.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id B86C543D2D for ; Wed, 14 Jul 2004 04:37:08 +0000 (GMT) (envelope-from grog@lemis.com) Received: from blackwater.lemis.com (blackwater.lemis.com [192.109.197.80]) by ozlabs.org (Postfix) with ESMTP id 8FBA42BD3D for ; Wed, 14 Jul 2004 14:37:06 +1000 (EST) Received: by blackwater.lemis.com (Postfix, from userid 1004) id A3078511FE; Wed, 14 Jul 2004 14:07:04 +0930 (CST) Date: Wed, 14 Jul 2004 14:07:04 +0930 From: Greg 'groggy' Lehey To: micko Message-ID: <20040714043704.GB79947@wantadilla.lemis.com> References: <20040713213017.GA38398@micko.net> <20040713221302.GA63062@wantadilla.lemis.com> <20040714010650.GA8692@micko.net> <20040714030052.GA9508@micko.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="St7VIuEGZ6dlpu13" Content-Disposition: inline In-Reply-To: <20040714030052.GA9508@micko.net> User-Agent: Mutt/1.4.1i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 9A1B 8202 BCCE B846 F92F 09AC 22E6 F290 507A 4223 cc: freebsd-fs@freebsd.org Subject: Re: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 04:37:09 -0000 --St7VIuEGZ6dlpu13 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tuesday, 13 July 2004 at 22:00:52 -0500, micko wrote: > On Tue, Jul 13, 2004 at 08:06:50PM -0500, micko wrote: >> On Wed, Jul 14, 2004 at 07:43:02AM +0930, Greg 'groggy' Lehey wrote: >>> On Tuesday, 13 July 2004 at 16:30:17 -0500, micko wrote: >>>> Hi, I just installed fbsd 5.2.1 on a system which >>>> had fbsd 4.9 with a vinum ufs1 stripe. When I tried >>>> to mount the vinum stripe on fbsd 5.2.1 it went ok, but >>>> doing 'ls' gave me 'bad_dir' and a kernel panic. Doing >>>> fsck gave me 'unknown filesystem type'. Seems like there >>>> is no backwards compatibility. >>> >>> That conclusion is incorrect. >>> >>>> Can this somehow be saved or do I need to put the drives on a fbsd >>>> 4.x system to get the data? >>> >>> You need to find out what the problem is first. To do so, you should >>> use the information at your disposal. See >>> http://www.vinumvm.org/vinum/how-to-debug.html for more details. >> >> It turned out to be a failing drive, which now is totally dead. Is there >> anyway to get some data out of the 1st part of the stripe? > > I found the problem. Originally when I was creating the partitions on the > fbsd 4.x system I didn't slice up the drive, I used da0c/e. When using > the drive under fbsd 5.x and devfs, the system brings up da0s1c/e. Is there > anyway to override this behavior? No. Why do you want to? And why is it a problem? Note the warning in the man page: partition ``c'' represents the whole disk and should not be used for any other purpose. I'm surprised you got it to work at all. *That*'s a bug. Greg -- Note: I discard all HTML mail unseen. Finger grog@FreeBSD.org for PGP public key. See complete headers for address and phone numbers. --St7VIuEGZ6dlpu13 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.0 (FreeBSD) iD8DBQFA9LhwIubykFB6QiMRAjCBAJ4373M/SlGAkKp9T5Qj+RzftguBeQCgnl0E 3nkFGHrxZOx4Ak6je9rTvD8= =qLgC -----END PGP SIGNATURE----- --St7VIuEGZ6dlpu13-- From owner-freebsd-fs@FreeBSD.ORG Wed Jul 14 13:00:57 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D5E116A4CE; Wed, 14 Jul 2004 13:00:57 +0000 (GMT) Received: from porthos.compasstg.net (ns01.europenetsystems.net [207.13.208.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1638243D3F; Wed, 14 Jul 2004 13:00:57 +0000 (GMT) (envelope-from mickonet@porthos.compasstg.net) Received: by porthos.compasstg.net (Postfix, from userid 1006) id D917641DB2; Wed, 14 Jul 2004 07:56:54 -0500 (CDT) Date: Wed, 14 Jul 2004 07:56:54 -0500 From: micko To: Greg 'groggy' Lehey Message-ID: <20040714125654.GA13080@micko.net> References: <20040713213017.GA38398@micko.net> <20040713221302.GA63062@wantadilla.lemis.com> <20040714010650.GA8692@micko.net> <20040714030052.GA9508@micko.net> <20040714043704.GB79947@wantadilla.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040714043704.GB79947@wantadilla.lemis.com> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD porthos.compasstg.net 4.7-STABLE FreeBSD 4.7-STABLE cc: freebsd-fs@freebsd.org Subject: Re: vinum ufs1 drives on freebsd 5.2.1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jul 2004 13:00:57 -0000 > > I found the problem. Originally when I was creating the partitions on the > > fbsd 4.x system I didn't slice up the drive, I used da0c/e. When using > > the drive under fbsd 5.x and devfs, the system brings up da0s1c/e. Is there > > anyway to override this behavior? > > No. Why do you want to? And why is it a problem? > > Note the warning in the man page: > > partition ``c'' represents the whole disk and should not be used > for any other purpose. > > I'm surprised you got it to work at all. *That*'s a bug. I was using daXe partition, and I had to put the drives back into a 4.x system since the 5.x was putting everything on daXs1. Can you point me to a page where I can more info on how fbsd 5.x devfs is pull the partition info from the drives? thanks micko From owner-freebsd-fs@FreeBSD.ORG Fri Jul 16 04:22:58 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6DFD816A4CE; Fri, 16 Jul 2004 04:22:58 +0000 (GMT) Received: from maui.ebi.ac.uk (maui.ebi.ac.uk [193.62.196.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5F0743D1D; Fri, 16 Jul 2004 04:22:56 +0000 (GMT) (envelope-from kreil@ebi.ac.uk) Received: from puffin.ebi.ac.uk (puffin.ebi.ac.uk [193.62.196.89]) by maui.ebi.ac.uk (8.11.7+Sun/8.11.7) with ESMTP id i6G4MrF27547; Fri, 16 Jul 2004 05:22:53 +0100 (BST) Received: from puffin.ebi.ac.uk (kreil@localhost) by puffin.ebi.ac.uk (8.11.6/8.11.6) with ESMTP id i6G4Mrs04821; Fri, 16 Jul 2004 05:22:53 +0100 Date: Fri, 16 Jul 2004 05:22:53 +0100 From: David Kreil Message-Id: <200407160422.i6G4Mrs04821@puffin.ebi.ac.uk> X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 To: freebsd-ports@freebsd.org, freebsd-questions@freebsd.org, freebsd-fs@freebsd.org, freebsd-geom@freebsd.org X-EBI-Information: This email is scanned using www.mailscanner.info. X-EBI: Found to be clean X-EBI-SpamCheck: not spam, SpamAssassin (score=3.003, required 5, SUSPICIOUS_RECIPS 3.00) X-EBI-SpamScore: sss cc: Kreil@ebi.ac.uk Subject: "sanitizing" disks: wiping swap, non-allocated space, and file-tails X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 04:22:58 -0000 to avoid leakage of sensitive information: any advice? X-Habeas-SWE-1: winter into spring X-Habeas-SWE-2: brightly anticipated X-Habeas-SWE-3: like Habeas SWE (tm) X-Habeas-SWE-4: Copyright 2002 Habeas (tm) X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-SWE-6: email in exchange for a license for this Habeas X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this X-Habeas-SWE-9: mark in spam to . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 16 Jul 2004 05:22:53 +0100 From: David Kreil Hi, (1) I was wondering whether anyone knew of packages/tools to aid in "sanitizing" a FreeBSD system, i.e., wiping + the swap slice + non-allocated space on volumes + "file-tails" (the part of the last allocated block of files not used) with random patterns to avoid leakage of sensitive information (plain text keys or decrypted texts). I am aware of the security limitations of any approach that does not involve dissolving the entire disk in acid etc but would be grateful for a pointer to a tool that at least + generates reasonably random data for its writes + ideally does a reasonable effort of turning off caching whereever it could (ideally in the file system, the disk driver, and the disk itself) or alternatively at least did the overwrites in such an order that the effect of caching would be minimized. If there are no "tools", would you know whether I can get FreeBSD on shutdown to stop using swap and access it as a raw disk device that I can write to, and how to hook into the shutdown process? (2) Related to this: I'm also interested in people's personal experiences in using partition or file system encryption options. For performance reasons I'd rather avoid having /tmp and swap and certain work space on an encrypted disk, hence the need for (1). If you feel that I'm asking this all in the wrong place, please let me know. With many thanks for your help, David. ------------------------------------------------------------------------ Dr David Philip Kreil ("`-''-/").___..--''"`-._ Research Fellow `6_ 6 ) `-. ( ).`-.__.`) University of Cambridge (_Y_.)' ._ ) `._ `. ``-..-' ++44 1223 764107, fax 333992 _..`--'_..-_/ /--'_.' ,' www.inference.phy.cam.ac.uk/dpk20 (il),-'' (li),' ((!.-' From owner-freebsd-fs@FreeBSD.ORG Fri Jul 16 04:51:42 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 986F416A4CE for ; Fri, 16 Jul 2004 04:51:42 +0000 (GMT) Received: from odin.ac.hmc.edu (Odin.AC.HMC.Edu [134.173.32.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7F54043D2D for ; Fri, 16 Jul 2004 04:51:42 +0000 (GMT) (envelope-from brdavis@odin.ac.hmc.edu) Received: from odin.ac.hmc.edu (IDENT:brdavis@localhost.localdomain [127.0.0.1]) by odin.ac.hmc.edu (8.12.10/8.12.10) with ESMTP id i6G4pZOF030001; Thu, 15 Jul 2004 21:51:35 -0700 Received: (from brdavis@localhost) by odin.ac.hmc.edu (8.12.10/8.12.3/Submit) id i6G4pZIo030000; Thu, 15 Jul 2004 21:51:35 -0700 Date: Thu, 15 Jul 2004 21:51:35 -0700 From: Brooks Davis To: David Kreil Message-ID: <20040716045134.GA28777@Odin.AC.HMC.Edu> References: <200407160422.i6G4Mrs04821@puffin.ebi.ac.uk> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="n8g4imXOkfNTN/H1" Content-Disposition: inline In-Reply-To: <200407160422.i6G4Mrs04821@puffin.ebi.ac.uk> User-Agent: Mutt/1.5.4i cc: freebsd-fs@freebsd.org Subject: Re: "sanitizing" disks: wiping swap, non-allocated space, and file-tails X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 04:51:42 -0000 --n8g4imXOkfNTN/H1 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Please don't cross post so much.] On Fri, Jul 16, 2004 at 05:22:53AM +0100, David Kreil wrote: > to avoid leakage of sensitive information: any advice? > From: David Kreil >=20 > (1) I was wondering whether anyone knew of packages/tools to aid in=20 > "sanitizing" > a FreeBSD system, i.e., wiping > + the swap slice > + non-allocated space on volumes > + "file-tails" (the part of the last allocated block of files not used) > with random patterns to avoid leakage of sensitive information (plain tex= t=20 > keys or decrypted texts). >=20 > I am aware of the security limitations of any approach that does not invo= lve=20 > dissolving the entire disk in acid etc but would be grateful for a pointe= r to=20 > a tool that at least > + generates reasonably random data for its writes > + ideally does a reasonable effort of turning off caching whereever it c= ould > (ideally in the file system, the disk driver, and the disk itself) > or alternatively at least did the overwrites in such an order that the > effect of caching would be minimized. >=20 > If there are no "tools", would you know whether I can get FreeBSD on shut= down=20 > to stop using swap and access it as a raw disk device that I can write to= , and=20 > how to hook into the shutdown process? The only way to clean a file system I can think of is, to dump the fs, scrub the disk, restore the fs. That will keep only real data and remove all the junk. You should be able to scrub swap at shutdown if you use swapoff to disable it, but you've got to be sure it's all unused first. You can't trust this though because you don't know the system will shut down cleanly. There is a currently null operation in the disk code that acts when a block is no longer used. It should be possible to hook into that code to implement some sort of scrubber. It wouldn't be reliable for the same reason that a swap scrubber can't be, but it would be OK most of the time. The easiest way to scrub a disk is: dd if=3D/dev/random of=3D/dev/ bs=3D dd if=3D/dev/zero of=3D/dev/ bs=3D This doesn't meet DoD requirements, but only for obsolete reasons. > (2) Related to this: >=20 > I'm also interested in people's personal experiences in using partition o= r=20 > file system encryption options. >=20 > For performance reasons I'd rather avoid having /tmp and swap and certain= work=20 > space on an encrypted disk, hence the need for (1). If you swap, your performance will be suck enough that encrypting it won't hurt much, especially with modern CPUs. I wouldn't worry at all about that cost. /tmp is probably similar for most applications. -- Brooks --=20 Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 --n8g4imXOkfNTN/H1 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (GNU/Linux) iD8DBQFA917WXY6L6fI4GtQRAkRFAJ9LHCkYPU3K6kpl332aJALe6rGmyACfe7CG u/Miuf5kB7TSTdGYBLu5mqU= =rIvU -----END PGP SIGNATURE----- --n8g4imXOkfNTN/H1-- From owner-freebsd-fs@FreeBSD.ORG Fri Jul 16 18:13:42 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B194416A4CE for ; Fri, 16 Jul 2004 18:13:42 +0000 (GMT) Received: from afields.ca (afields.ca [216.194.67.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 744BC43D39 for ; Fri, 16 Jul 2004 18:13:42 +0000 (GMT) (envelope-from afields@afields.ca) Received: from afields.ca (localhost.afields.ca [127.0.0.1]) by afields.ca (8.12.11/8.12.11) with ESMTP id i6GIDem9020654; Fri, 16 Jul 2004 14:13:40 -0400 (EDT) (envelope-from afields@afields.ca) Received: (from afields@localhost) by afields.ca (8.12.11/8.12.11/Submit) id i6GIDdwu020653; Fri, 16 Jul 2004 14:13:39 -0400 (EDT) (envelope-from afields) Date: Fri, 16 Jul 2004 14:13:39 -0400 From: Allan Fields To: Brooks Davis Message-ID: <20040716181339.GA18056@afields.ca> References: <200407160422.i6G4Mrs04821@puffin.ebi.ac.uk> <20040716045134.GA28777@Odin.AC.HMC.Edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040716045134.GA28777@Odin.AC.HMC.Edu> User-Agent: Mutt/1.4i cc: freebsd-fs@freebsd.org cc: David Kreil Subject: Re: "sanitizing" disks: wiping swap, non-allocated space, and file-tails X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 18:13:42 -0000 On Thu, Jul 15, 2004 at 09:51:35PM -0700, Brooks Davis wrote: > There is a currently null operation in the disk code that acts when > a block is no longer used. It should be possible to hook into that > code to implement some sort of scrubber. It wouldn't be reliable for > the same reason that a swap scrubber can't be, but it would be OK most > of the time. > > The easiest way to scrub a disk is: > > dd if=/dev/random of=/dev/ bs= > > dd if=/dev/zero of=/dev/ bs= > > This doesn't meet DoD requirements, but only for obsolete reasons. > > > (2) Related to this: > > > > I'm also interested in people's personal experiences in using partition or > > file system encryption options. > > > > For performance reasons I'd rather avoid having /tmp and swap and certain work > > space on an encrypted disk, hence the need for (1). > > If you swap, your performance will be suck enough that encrypting it > won't hurt much, especially with modern CPUs. I wouldn't worry at all > about that cost. /tmp is probably similar for most applications. Agreed, the simplest approach for base-level storage security is to encrypt it all. Hardware is cheap and fast enough. Trying to sweep-up afterward is more difficult, any way you look at it. Assessing leakage: it's quite possible users could inadvertently copy an important document to /tmp or a program could write to swap things that would be encrypted in on-disk counterparts. This is rightly identified as a weak-link here. Another thing to note is /var can contain sensitive data, the locate database and mail/print spools to name a few are potential areas of significance. Some also consider logs sensitive. > > -- Brooks > > -- > Any statement of the form "X is the one, true Y" is FALSE. > PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4 -- Allan Fields, AFRSL - http://afields.ca 2D4F 6806 D307 0889 6125 C31D F745 0D72 39B4 5541 From owner-freebsd-fs@FreeBSD.ORG Fri Jul 16 23:06:06 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 598F616A4CE for ; Fri, 16 Jul 2004 23:06:06 +0000 (GMT) Received: from snowhite.cis.uoguelph.ca (snowhite.cis.uoguelph.ca [131.104.48.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBA9543D39 for ; Fri, 16 Jul 2004 23:06:05 +0000 (GMT) (envelope-from rick@snowhite.cis.uoguelph.ca) Received: (from rick@localhost) by snowhite.cis.uoguelph.ca (8.9.3/8.9.3) id TAA25106; Fri, 16 Jul 2004 19:06:43 -0400 (EDT) Date: Fri, 16 Jul 2004 19:06:43 -0400 (EDT) From: rick@snowhite.cis.uoguelph.ca Message-Id: <200407162306.TAA25106@snowhite.cis.uoguelph.ca> To: nfsv4@ietf.org cc: tech@openbsd.org cc: fs@freebsd.org cc: deicher@sandia.gov Subject: beta test NFSv4 server for BSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jul 2004 23:06:06 -0000 I've just put up my 3rd beta release of my NFSv4 (also does v2 and 3) server for FreeBSD5.2 and OpenBSD3.4. (It seems pretty solid now, although work is still required on ACLs and sysadmin tools.) If you are interested, you can anonymous ftp it from snowhite.cis.uoguelph.ca (131.104.48.1) in pub/nfsv4. Have a nice weekend, rick From owner-freebsd-fs@FreeBSD.ORG Sat Jul 17 01:54:34 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C81B016A4CE for ; Sat, 17 Jul 2004 01:54:34 +0000 (GMT) Received: from maui.ebi.ac.uk (maui.ebi.ac.uk [193.62.196.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 704B443D1F for ; Sat, 17 Jul 2004 01:54:33 +0000 (GMT) (envelope-from kreil@ebi.ac.uk) Received: from puffin.ebi.ac.uk (puffin.ebi.ac.uk [193.62.196.89]) by maui.ebi.ac.uk (8.11.7+Sun/8.11.7) with ESMTP id i6H1sUF10035; Sat, 17 Jul 2004 02:54:30 +0100 (BST) Received: from puffin.ebi.ac.uk (kreil@localhost) by puffin.ebi.ac.uk (8.11.6/8.11.6) with ESMTP id i6H1sTm14158; Sat, 17 Jul 2004 02:54:30 +0100 Message-Id: <200407170154.i6H1sTm14158@puffin.ebi.ac.uk> X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 To: Brooks Davis In-Reply-To: Your message of "Thu, 15 Jul 2004 21:51:35 PDT." <20040716045134.GA28777@Odin.AC.HMC.Edu> X-Habeas-SWE-1: winter into spring X-Habeas-SWE-2: brightly anticipated X-Habeas-SWE-3: like Habeas SWE (tm) X-Habeas-SWE-4: Copyright 2002 Habeas (tm) X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-SWE-6: email in exchange for a license for this Habeas X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this X-Habeas-SWE-9: mark in spam to . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 17 Jul 2004 02:54:29 +0100 From: David Kreil X-EBI-Information: This email is scanned using www.mailscanner.info. X-EBI: Found to be clean X-EBI-SpamCheck: not spam, SpamAssassin (score=-8, required 5, HABEAS_SWE -8.00) cc: freebsd-fs@freebsd.org cc: David Kreil Subject: Re: "sanitizing" disks: wiping swap, non-allocated space, and file-tails X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2004 01:54:35 -0000 Dear Brooks, Many thanks for your fast and helpful reply! > [Please don't cross post so much.] I'm sorry. I had sent it to "ports" two weeks ago, assuming that external programs might be required and then didn't quite know where to turn next. I guess I should have tried individual groups at a time. > The only way to clean a file system I can think of is, to dump the fs, > scrub the disk, restore the fs. That will keep only real data and > remove all the junk. Ok, I can do that once to start from a clean slate but it would not be practical to do this regularly. > You should be able to scrub swap at shutdown if > you use swapoff to disable it, but you've got to be sure it's all unused > first. You can't trust this though because you don't know the system > will shut down cleanly. > > There is a currently null operation in the disk code that acts when > a block is no longer used. It should be possible to hook into that > code to implement some sort of scrubber. It wouldn't be reliable for > the same reason that a swap scrubber can't be, but it would be OK most > of the time. Yes, both approaches would already improve my situation considerably. The handbook says that a shutdown "will attempt to run the script /etc/rc.shutdown, and then proceed to send all processes the TERM signal, and subsequently the KILL signal to any that do not terminate timely". Would turning off swap and scrubbing in rc.shutdown be sufficient, or do I need a way of doing this *after* all other processes have been terminated (and if so, what would be a good approach)? You mentioned the possibility of hooking a scrub routine into the fs code that releases a block. My main worry with such an approach would be that repeated random writes of an individual block would probably never really be written to disk due to drive caching etc. Is there a way around this problem? > The easiest way to scrub a disk is: > > dd if=3D/dev/random of=3D/dev/ bs=3D > > dd if=3D/dev/zero of=3D/dev/ bs=3D > > This doesn't meet DoD requirements, but only for obsolete reasons. Interesting - can you say what type of reasons these are? :-) > > (2) Related to this: > > I'm also interested in people's personal experiences in using partition > > or file system encryption options. > > For performance reasons I'd rather avoid having /tmp and swap and certain= > > work space on an encrypted disk, hence the need for (1). > > If you swap, your performance will be suck enough that encrypting it > won't hurt much, especially with modern CPUs. I wouldn't worry at all > about that cost. /tmp is probably similar for most applications. Really? A loss in disk performance of factor of four should be noticable I had thought. Ah well, since Allan Fields and you seem to agree on this I suppose I should try to dump/restore my system to a gbde partition. With many thanks again for your help and best regards, David. ------------------------------------------------------------------------ Dr David Philip Kreil ("`-''-/").___..--''"`-._ Research Fellow `6_ 6 ) `-. ( ).`-.__.`) University of Cambridge (_Y_.)' ._ ) `._ `. ``-..-' ++44 1223 764107, fax 333992 _..`--'_..-_/ /--'_.' ,' www.inference.phy.cam.ac.uk/dpk20 (il),-'' (li),' ((!.-' From owner-freebsd-fs@FreeBSD.ORG Sat Jul 17 02:04:42 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D622016A4CE for ; Sat, 17 Jul 2004 02:04:42 +0000 (GMT) Received: from maui.ebi.ac.uk (maui.ebi.ac.uk [193.62.196.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20D9E43D31 for ; Sat, 17 Jul 2004 02:04:42 +0000 (GMT) (envelope-from kreil@ebi.ac.uk) Received: from puffin.ebi.ac.uk (puffin.ebi.ac.uk [193.62.196.89]) by maui.ebi.ac.uk (8.11.7+Sun/8.11.7) with ESMTP id i6H24eF16231; Sat, 17 Jul 2004 03:04:40 +0100 (BST) Received: from puffin.ebi.ac.uk (kreil@localhost) by puffin.ebi.ac.uk (8.11.6/8.11.6) with ESMTP id i6H24eU16729; Sat, 17 Jul 2004 03:04:40 +0100 Message-Id: <200407170204.i6H24eU16729@puffin.ebi.ac.uk> X-Mailer: exmh version 2.4 06/23/2000 with nmh-1.0.4 To: Allan Fields In-Reply-To: Your message of "Fri, 16 Jul 2004 14:13:39 EDT." <20040716181339.GA18056@afields.ca> X-Habeas-SWE-1: winter into spring X-Habeas-SWE-2: brightly anticipated X-Habeas-SWE-3: like Habeas SWE (tm) X-Habeas-SWE-4: Copyright 2002 Habeas (tm) X-Habeas-SWE-5: Sender Warranted Email (SWE) (tm). The sender of this X-Habeas-SWE-6: email in exchange for a license for this Habeas X-Habeas-SWE-7: warrant mark warrants that this is a Habeas Compliant X-Habeas-SWE-8: Message (HCM) and not spam. Please report use of this X-Habeas-SWE-9: mark in spam to . Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 17 Jul 2004 03:04:40 +0100 From: David Kreil X-EBI-Information: This email is scanned using www.mailscanner.info. X-EBI: Found to be clean X-EBI-SpamCheck: not spam, SpamAssassin (score=-8, required 5, HABEAS_SWE -8.00) cc: freebsd-fs@freebsd.org cc: David Kreil Subject: Re: "sanitizing" disks: wiping swap, non-allocated space, and file-tails X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jul 2004 02:04:42 -0000 Dear Allan, Thank you very much for your helpful comments! > > [Brooks Davis] > > If you swap, your performance will be suck enough that encrypting it > > won't hurt much, especially with modern CPUs. I wouldn't worry at all > > about that cost. /tmp is probably similar for most applications. > > Agreed, the simplest approach for base-level storage security is > to encrypt it all. Hardware is cheap and fast enough. I still somewhat worry about the factor four in performance lost that is mentioned in the gdbe paper. This is no problem for a set of sensitive private files but at the system level it does cause me worry. As you seem to be so confident about this, however, (or have I misunderstood you?) I'll be happy to give it a go. > Trying to sweep-up afterward is more difficult, any way you look at it. Yes, I completely agree. > Another thing to note is /var can contain sensitive data, the locate > database and mail/print spools to name a few are potential > areas of significance. Some also consider logs sensitive. Thanks for pointing this out. The Handbook describes a basic gdbe setup but mentions that getting other volumes (like /home) onto a gdbe partition was trickier. Can you tell me which volumes you have successfully put onto a gdbe partition and what was required to get this working? I wonder, in particular, what issues I have to expect in wanting to keep system relevant directories like /var on a gdbe partition. With many thanks again for your help and best regards, David. ------------------------------------------------------------------------ Dr David Philip Kreil ("`-''-/").___..--''"`-._ Research Fellow `6_ 6 ) `-. ( ).`-.__.`) University of Cambridge (_Y_.)' ._ ) `._ `. ``-..-' ++44 1223 764107, fax 333992 _..`--'_..-_/ /--'_.' ,' www.inference.phy.cam.ac.uk/dpk20 (il),-'' (li),' ((!.-'