From owner-freebsd-fs@FreeBSD.ORG Sun Oct 22 18:10:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DE12C16A407 for ; Sun, 22 Oct 2006 18:10:30 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.191]) by mx1.FreeBSD.org (Postfix) with ESMTP id E854A43DB8 for ; Sun, 22 Oct 2006 18:10:05 +0000 (GMT) (envelope-from r.c.ladan@gmail.com) Received: by nf-out-0910.google.com with SMTP id p77so2162956nfc for ; Sun, 22 Oct 2006 11:09:53 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:user-agent:mime-version:to:subject:x-enigmail-version:content-type:content-transfer-encoding; b=W6QGShDbZ5/f4DMoN10d7wMkD4F87HasE02gBaE5XIH4aZzrfKxcM1SLnRdGCt2wAZ8PbdmWJoUx7Ho+2+XoJh64CsRRXsi+/cBZa3fkGEYiXpeMfQM1LGg8+HWX1iGavA8LyQvdF70ZcrY+ZQfpg5K71N9Viw2MpOzGx6Nwiu4= Received: by 10.48.48.18 with SMTP id v18mr12826311nfv; Sun, 22 Oct 2006 11:09:53 -0700 (PDT) Received: from ?192.168.123.106? ( [195.241.221.201]) by mx.google.com with ESMTP id y24sm1906727nfb.2006.10.22.11.09.52; Sun, 22 Oct 2006 11:09:53 -0700 (PDT) Message-ID: <453BB3E4.1010803@gmail.com> Date: Sun, 22 Oct 2006 20:09:40 +0200 From: Rene Ladan User-Agent: Thunderbird 1.5.0.7 (X11/20061018) MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: First XTAF implementation available X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Oct 2006 18:10:30 -0000 Hi, I put a first implementation of the XTAF (xbox 360) file system online at http://home.tiscali.nl/freebsd/xtaf-20061022.tbz This files in this tarball are relative to /usr/src (RELENG_6 as of 2006-10-22 14:18 UTC). Apart from new files, it also changes some existing files, mostly to connect XTAF to the build. 'make buildkernel' and 'make buildworld' succeed on RELENG_6 (CURRENT is untested), but this code is still highly experimental / unusable. The todo-list includes, but is not limited to: * remove remaining Win95 filename code (XTAF uses 42.0 filenames, just like DOS uses 8.3 filenames; XTAF filenames are case-preserving and case-insensitive) * remove remaining locale code (XTAF filenames are _probably_ non-localized) * change endianness of the FAT code (XTAF is big-endian, DOS is little endian) * adapt the BPB (boot parameter block) to match that of Xbox 360 drives * find out which FAT size the memory units use (12/16/32 bits), requires soldering * fix the locking code (!), I probably broke it while removing code irrelevant to XTAF * find out if the FAT is mirrored * find out if the root directory has . and .. entries If you still like to experiment with it and have the possibility to attach an Xbox 360 drive to your PC, then _please_ only play with an image of it ( 'dd if=/dev/daX of=image.dump bs=1m' ), unless you want to spend US $100 on each bug in the code :/ Regards, Rene -- GPG fingerprint = E738 5471 D185 7013 0EE0 4FC8 3C1D 6F83 12E1 84F6 (subkeys.pgp.net) "It won't fit on the line." -- me, 2001 From owner-freebsd-fs@FreeBSD.ORG Sun Oct 22 18:17:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0636716A416 for ; Sun, 22 Oct 2006 18:17:30 +0000 (UTC) (envelope-from adamsch1@yahoo.com) Received: from web31806.mail.mud.yahoo.com (web31806.mail.mud.yahoo.com [68.142.207.69]) by mx1.FreeBSD.org (Postfix) with SMTP id BFB7E43DB8 for ; Sun, 22 Oct 2006 18:17:12 +0000 (GMT) (envelope-from adamsch1@yahoo.com) Received: (qmail 60842 invoked by uid 60001); 22 Oct 2006 18:17:12 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=1Px8GG7vo7t0YXJoMvsULwV1QW1eVmMbBxAWfLJmDPmPj1nBW5UdB+0JRycS1ordh+AyM8d3FcgUPri8gFfTAtPcKMJ7cTsQzR1GUkxrTEp8bimyjsGyL256DmWpRkSrq4sS9uBh26fG8gYBCWQLFDbDPztu1vvARJl2RKs7nXw= ; Message-ID: <20061022181712.60840.qmail@web31806.mail.mud.yahoo.com> Received: from [69.236.66.48] by web31806.mail.mud.yahoo.com via HTTP; Sun, 22 Oct 2006 11:17:12 PDT Date: Sun, 22 Oct 2006 11:17:12 -0700 (PDT) From: Shane Adams To: Rene Ladan , freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: First XTAF implementation available X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Oct 2006 18:17:30 -0000 Link isn't working for me, get the 404. Would be interested in checking th= e code out.=0A=0ACheers,=0A Shane=0A=0A=0A----- Original Message ----=0AFr= om: Rene Ladan =0ATo: freebsd-fs@freebsd.org=0ASent: S= unday, October 22, 2006 11:09:40 AM=0ASubject: First XTAF implementation av= ailable=0A=0AHi,=0A=0AI put a first implementation of the XTAF (xbox 360) f= ile system online=0Aat http://home.tiscali.nl/freebsd/xtaf-20061022.tbz=0A= =0AThis files in this tarball are relative to /usr/src (RELENG_6 as of=0A20= 06-10-22 14:18 UTC). Apart from new files, it also changes some=0Aexisting= files, mostly to connect XTAF to the build.=0A=0A'make buildkernel' and 'm= ake buildworld' succeed on RELENG_6 (CURRENT is=0Auntested), but this code = is still highly experimental / unusable. The=0Atodo-list includes, but is = not limited to:=0A=0A* remove remaining Win95 filename code (XTAF uses 42.0= filenames, just=0Alike DOS uses 8.3 filenames; XTAF filenames are case-pre= serving and=0Acase-insensitive)=0A* remove remaining locale code (XTAF file= names are _probably_ non-localized)=0A* change endianness of the FAT code (= XTAF is big-endian, DOS is little=0Aendian)=0A* adapt the BPB (boot paramet= er block) to match that of Xbox 360 drives=0A* find out which FAT size the = memory units use (12/16/32 bits), requires=0Asoldering=0A* fix the locking = code (!), I probably broke it while removing code=0Airrelevant to XTAF=0A* = find out if the FAT is mirrored=0A* find out if the root directory has . an= d .. entries=0A=0AIf you still like to experiment with it and have the poss= ibility to=0Aattach an Xbox 360 drive to your PC, then _please_ only play w= ith an=0Aimage of it ( 'dd if=3D/dev/daX of=3Dimage.dump bs=3D1m' ), unless= you want to=0A spend US $100 on each bug in the code :/=0A=0ARegards,=0ARe= ne=0A-- =0AGPG fingerprint =3D E738 5471 D185 7013 0EE0 4FC8 3C1D 6F83 12E= 1 84F6=0A(subkeys.pgp.net)=0A=0A"It won't fit on the line."=0A -- me= , 2001=0A=0A_______________________________________________=0Afreebsd-fs@fr= eebsd.org mailing list=0Ahttp://lists.freebsd.org/mailman/listinfo/freebsd-= fs=0ATo unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"= =0A=0A=0A From owner-freebsd-fs@FreeBSD.ORG Sun Oct 22 18:23:09 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A184316A403 for ; Sun, 22 Oct 2006 18:23:09 +0000 (UTC) (envelope-from r.c.ladan@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.186]) by mx1.FreeBSD.org (Postfix) with ESMTP id C9F8143D68 for ; Sun, 22 Oct 2006 18:23:08 +0000 (GMT) (envelope-from r.c.ladan@gmail.com) Received: by nf-out-0910.google.com with SMTP id p77so2166740nfc for ; Sun, 22 Oct 2006 11:23:07 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:user-agent:mime-version:to:cc:subject:references:in-reply-to:x-enigmail-version:content-type:content-transfer-encoding; b=m5/+kC7bD+zp/n0xusne1jeQYPLdcNrd9L+ialsdKW+yynNdHWbXXVTGxI5YXcefylAi9qWXnZe7Dnae0Z1Kabpw3uijmt+PEEVF4znRaj0Qp5yJnl0bdAllcO4ff01EOGreTfDsQYcAyzHMtq4lfWEI6VZfA0F1i7f5Fx6BFi0= Received: by 10.48.230.18 with SMTP id c18mr12800013nfh; Sun, 22 Oct 2006 11:23:06 -0700 (PDT) Received: from ?192.168.123.106? ( [195.241.221.201]) by mx.google.com with ESMTP id g1sm1947363nfe.2006.10.22.11.23.06; Sun, 22 Oct 2006 11:23:06 -0700 (PDT) Message-ID: <453BB6FD.5070403@gmail.com> Date: Sun, 22 Oct 2006 20:22:53 +0200 From: Rene Ladan User-Agent: Thunderbird 1.5.0.7 (X11/20061018) MIME-Version: 1.0 To: Shane Adams References: <20061022181712.60840.qmail@web31806.mail.mud.yahoo.com> In-Reply-To: <20061022181712.60840.qmail@web31806.mail.mud.yahoo.com> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: First XTAF implementation available X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Oct 2006 18:23:09 -0000 Shane Adams schreef: > Link isn't working for me, get the 404. Would be interested in checking the code out. > Whoops, the link should be http://home.tiscali.nl/rladan/freebsd/xtaf-20061022.tbz Regards, Rene > Cheers, > Shane > > > ----- Original Message ---- > From: Rene Ladan > To: freebsd-fs@freebsd.org > Sent: Sunday, October 22, 2006 11:09:40 AM > Subject: First XTAF implementation available > > Hi, > > I put a first implementation of the XTAF (xbox 360) file system online > at http://home.tiscali.nl/freebsd/xtaf-20061022.tbz > > This files in this tarball are relative to /usr/src (RELENG_6 as of > 2006-10-22 14:18 UTC). Apart from new files, it also changes some > existing files, mostly to connect XTAF to the build. > > 'make buildkernel' and 'make buildworld' succeed on RELENG_6 (CURRENT is > untested), but this code is still highly experimental / unusable. The > todo-list includes, but is not limited to: > > * remove remaining Win95 filename code (XTAF uses 42.0 filenames, just > like DOS uses 8.3 filenames; XTAF filenames are case-preserving and > case-insensitive) > * remove remaining locale code (XTAF filenames are _probably_ non-localized) > * change endianness of the FAT code (XTAF is big-endian, DOS is little > endian) > * adapt the BPB (boot parameter block) to match that of Xbox 360 drives > * find out which FAT size the memory units use (12/16/32 bits), requires > soldering > * fix the locking code (!), I probably broke it while removing code > irrelevant to XTAF > * find out if the FAT is mirrored > * find out if the root directory has . and .. entries > > If you still like to experiment with it and have the possibility to > attach an Xbox 360 drive to your PC, then _please_ only play with an > image of it ( 'dd if=/dev/daX of=image.dump bs=1m' ), unless you want to > spend US $100 on each bug in the code :/ > > Regards, > Rene -- GPG fingerprint = E738 5471 D185 7013 0EE0 4FC8 3C1D 6F83 12E1 84F6 (subkeys.pgp.net) "It won't fit on the line." -- me, 2001 From owner-freebsd-fs@FreeBSD.ORG Mon Oct 23 13:09:45 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.ORG Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF3A816A492 for ; Mon, 23 Oct 2006 13:09:45 +0000 (UTC) (envelope-from fjoe@neo.samodelkin.net) Received: from neo.samodelkin.net (samodelkin.net [195.62.0.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6411543D5E for ; Mon, 23 Oct 2006 13:09:45 +0000 (GMT) (envelope-from fjoe@neo.samodelkin.net) Received: by neo.samodelkin.net (Postfix, from userid 1000) id 293F3170B9; Mon, 23 Oct 2006 20:09:43 +0700 (NOVST) Date: Mon, 23 Oct 2006 20:09:43 +0700 From: Max Khon To: Oliver Fromme Message-ID: <20061023130942.GA73780@samodelkin.net> References: <451AA52F.2020408@centtech.com> <200609271744.k8RHipTS032655@lurza.secnetix.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200609271744.k8RHipTS032655@lurza.secnetix.de> User-Agent: Mutt/1.4.2i Cc: freebsd-fs@FreeBSD.ORG Subject: Re: Hi: Porting Cramfs on FreeBSD X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Oct 2006 13:09:45 -0000 Hi! On Wed, Sep 27, 2006 at 07:44:51PM +0200, Oliver Fromme wrote: > > I'm currently working on a tarfs, that will do something similar to > > this, except it allows you to use a regular tar file as the file system > > image. > > That's cool. I'm looking forward to it. > > > I don't have compression working yet, but that is on the > > roadmap. [...] large file system sizes (based on available > > memory) > > Hm. Does that mean that the whole (uncompressed) FS image > will have to fit into memory? That would be a disadvantage > compared to cramfs. Cramfs doesn't compress the whole FS > as one object (like .tar.gz), but it compresses it page-by- > page, so every page can be uncompressed independently, and > memory usage is very low, which is good for small embedded > applications. cloop (geom_uzip) works exactly like this, but on the block device level. /fjoe From owner-freebsd-fs@FreeBSD.ORG Mon Oct 23 13:44:32 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1866E16A4F0; Mon, 23 Oct 2006 13:44:32 +0000 (UTC) (envelope-from fjoe@neo.samodelkin.net) Received: from neo.samodelkin.net (samodelkin.net [195.62.0.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED01F43D97; Mon, 23 Oct 2006 13:22:13 +0000 (GMT) (envelope-from fjoe@neo.samodelkin.net) Received: by neo.samodelkin.net (Postfix, from userid 1000) id E605A170C5; Mon, 23 Oct 2006 20:22:04 +0700 (NOVST) Date: Mon, 23 Oct 2006 20:22:04 +0700 From: Max Khon To: Joerg Schilling Message-ID: <20061023132204.GB73780@samodelkin.net> References: <45223d3d.5Il+WW9HwkAuGvSH%Joerg.Schilling@fokus.fraunhofer.de> <4522AB6D.9050309@elischer.org> <4522aec5.4ZbRSHkZMuJ7r3BT%Joerg.Schilling@fokus.fraunhofer.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4522aec5.4ZbRSHkZMuJ7r3BT%Joerg.Schilling@fokus.fraunhofer.de> User-Agent: Mutt/1.4.2i Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, julian@elischer.org Subject: Re: Reliable hard links with mkisofs/ISO-9660/RR X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Oct 2006 13:44:32 -0000 Hi! On Tue, Oct 03, 2006 at 08:41:09PM +0200, Joerg Schilling wrote: > > > I just started to implement a long planned extension to mkisofs > > > and Solaris hsfs that will allow hard links to work correctly. > > > > > > Is there any interest to also support this on FreeBSD? > > > > > > > Sorry to be obtuse, but if you write it in mkisofs then won't it > > automatically be supported > > on FreeBSD when we use mkisofs? > > > > Is it possible that you are asking about our kernel isofs support? > > FreeBSD currently seems to bascally implement the same fake inode algorithm > as SunOS does since ~ 1989. > > I did extend mkisofs during the past days and started to extend hsfs from > Solaris. Once I am ready, I could present the results in case there is someone > who is willing to work on the FreeBSD filesystem module, I could explain what > needs to be done. Please do. /fjoe From owner-freebsd-fs@FreeBSD.ORG Tue Oct 24 12:10:08 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5275F16A492; Tue, 24 Oct 2006 12:10:08 +0000 (UTC) (envelope-from schilling@fokus.fraunhofer.de) Received: from mailhub.fokus.fraunhofer.de (mailhub.fokus.fraunhofer.de [193.174.154.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id E1B2843DB2; Tue, 24 Oct 2006 12:09:06 +0000 (GMT) (envelope-from schilling@fokus.fraunhofer.de) Received: from burner.fokus.fraunhofer.de (burner [10.147.65.166]) by mailhub.fokus.fraunhofer.de (8.11.6p2/8.11.6) with ESMTP id k9OC7Ud14841; Tue, 24 Oct 2006 14:07:30 +0200 (MEST) Received: (from jes@localhost) by burner.fokus.fraunhofer.de (8.12.9+Sun/8.12.9/Submit) id k9OC5khE005162; Tue, 24 Oct 2006 14:05:46 +0200 (CEST) Date: Tue, 24 Oct 2006 14:05:46 +0200 From: Joerg.Schilling@fokus.fraunhofer.de (Joerg Schilling) To: fjoe@samodelkin.net Message-ID: <453e019a.0bXjcrt8pIRVg4RY%Joerg.Schilling@fokus.fraunhofer.de> References: <45223d3d.5Il+WW9HwkAuGvSH%Joerg.Schilling@fokus.fraunhofer.de> <4522AB6D.9050309@elischer.org> <4522aec5.4ZbRSHkZMuJ7r3BT%Joerg.Schilling@fokus.fraunhofer.de> <20061023132204.GB73780@samodelkin.net> In-Reply-To: <20061023132204.GB73780@samodelkin.net> User-Agent: nail 11.22 3/20/05 MIME-Version: 1.0 Content-Type: text/plain; charset=ISO8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org, julian@elischer.org Subject: Re: Reliable hard links with mkisofs/ISO-9660/RR X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Oct 2006 12:10:08 -0000 Max Khon wrote: > > FreeBSD currently seems to bascally implement the same fake inode algorithm > > as SunOS does since ~ 1989. > > > > I did extend mkisofs during the past days and started to extend hsfs from > > Solaris. Once I am ready, I could present the results in case there is someone > > who is willing to work on the FreeBSD filesystem module, I could explain what > > needs to be done. > > Please do. Mkisofs now supports correct link counts for files and directories and implements an inode number algorithm. If the sector that immediately follows the VD-sectors and (in case available the related UDF sectors) starts with "MKI " and if the last 3 bytes of this sector contain a checksum from the PVD, a new mkisofs that is in the right mode did create the image. In this case, the starting LBA numbers in the ISO-9660 directory entries are useful as inode numbers. If the SUS-RRIP "PX" entry has a size of 44 bytes, it contains the same number as RR inode number. If you switch to this new inode scheme, NFS exports are expeted to work no longer unless you change the NFS FID algorithm too. I implemented Solaris hsfs the following way: - if hsfs sees a non-inode aware filesystem, it uses inode number 16 for all zero sized files. The files are diistinguished internally from the LBA/off values for the Directory entry. - if hsfs sees a inode aware filesystem, it uses the numbers. - NFS FIDs use LBA/off + inode number. Jörg -- EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin js@cs.tu-berlin.de (uni) schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily From owner-freebsd-fs@FreeBSD.ORG Tue Oct 24 15:24:35 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BD89816A4D4; Tue, 24 Oct 2006 15:24:35 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C60643D8C; Tue, 24 Oct 2006 15:23:55 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id A30B6456B1; Tue, 24 Oct 2006 17:23:43 +0200 (CEST) Received: from localhost (pjd.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4FCBE45681; Tue, 24 Oct 2006 17:23:35 +0200 (CEST) Date: Tue, 24 Oct 2006 17:23:08 +0200 From: Pawel Jakub Dawidek To: freebsd-geom@FreeBSD.org Message-ID: <20061024152308.GG75746@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Rn7IEEq3VEzCw+ji" Content-Disposition: inline X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org Subject: Updated gjournal patches [20061024]. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Oct 2006 15:24:35 -0000 --Rn7IEEq3VEzCw+ji Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi. Patches are at: HEAD: http://people.freebsd.org/~pjd/patches/gjournal_20061024.patch RELENG_6: http://people.freebsd.org/~pjd/patches/gjournal6_20061024.patch Those patches mostly contain build fixes for gconcat and gstripe. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --Rn7IEEq3VEzCw+ji Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFFPi/cForvXbEpPzQRAsQ8AJ4vcJCxQQokxiJd8qCcP4U3zMoEgACg3htt AhjQSPyM28vy4+uLFLjPYOE= =FlYh -----END PGP SIGNATURE----- --Rn7IEEq3VEzCw+ji-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 03:57:34 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 764F216A403 for ; Wed, 25 Oct 2006 03:57:34 +0000 (UTC) (envelope-from aronesimi@yahoo.com) Received: from web58414.mail.re3.yahoo.com (web58414.mail.re3.yahoo.com [68.142.236.182]) by mx1.FreeBSD.org (Postfix) with SMTP id DF36B43D46 for ; Wed, 25 Oct 2006 03:57:33 +0000 (GMT) (envelope-from aronesimi@yahoo.com) Received: (qmail 30753 invoked by uid 60001); 25 Oct 2006 03:57:24 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=bueNBJ/PK+0TqXHb6f7yMtqpQKFQKUv/yD3MVoPMlDaH98FCzrFwTFYAeDEZYI6SeRlsnMTATnrJBwz0XhAe7wtuj1Vo7f70goTV9RbSXGcVFlPGL1zrAPyOFQsCfO3vkryxFxzxX3a2CAKaa+UBSX02Q+Yun+R1zLnBT6fh/ck= ; Message-ID: <20061025035724.30751.qmail@web58414.mail.re3.yahoo.com> Received: from [75.72.230.91] by web58414.mail.re3.yahoo.com via HTTP; Tue, 24 Oct 2006 20:57:24 PDT Date: Tue, 24 Oct 2006 20:57:24 -0700 (PDT) From: Arone Silimantia To: freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Mailman-Approved-At: Wed, 25 Oct 2006 05:18:44 +0000 Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 03:57:34 -0000 > BTW, use of snapshots with stock 6.1 is not very attractive idea, better > to update to the 6-STABLE (many important fixes in that area were made). Funny, that's what was said about 5.1-RELEASE (and every release after it). Snapshots have never been useful, or safe, and as an added bonus, two years later, now _quotas_ are dangerous.[1] Hey great new logo though! Very pretty! [1] The quota command appeared in 4.2BSD. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 05:20:34 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B76F416A407 for ; Wed, 25 Oct 2006 05:20:34 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7ADD543D46 for ; Wed, 25 Oct 2006 05:20:34 +0000 (GMT) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 59B331A4D84; Tue, 24 Oct 2006 22:20:34 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id 73AC551404; Wed, 25 Oct 2006 01:20:31 -0400 (EDT) Date: Wed, 25 Oct 2006 01:20:31 -0400 From: Kris Kennaway To: Arone Silimantia Message-ID: <20061025052031.GA8958@xor.obsecurity.org> References: <20061025035724.30751.qmail@web58414.mail.re3.yahoo.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: <20061025035724.30751.qmail@web58414.mail.re3.yahoo.com> User-Agent: Mutt/1.4.2.2i Cc: freebsd-fs@freebsd.org Subject: Re: Snapshot corruption on 6.1/amd64 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 05:20:34 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Oct 24, 2006 at 08:57:24PM -0700, Arone Silimantia wrote: > and as an > added bonus, two > years later, now _quotas_ are dangerous.[1] Incorrect. Kris --G4iJoqBmSsgzjUCe Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFFPvQfWry0BWjoQKURAuigAKCoWZAPA7zK5SdNOh9Ct+jcXUQsbQCeMJlO PGP/4QXP3qCYr8+HvXdLr2s= =L/r0 -----END PGP SIGNATURE----- --G4iJoqBmSsgzjUCe-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 16:43:30 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FBDB16A548 for ; Wed, 25 Oct 2006 16:43:30 +0000 (UTC) (envelope-from root@parse.com) Received: from amd64.ott.parse.com (ottawa-hs-206-191-28-202.s-ip.magma.ca [206.191.28.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1CED943DAE for ; Wed, 25 Oct 2006 16:43:01 +0000 (GMT) (envelope-from root@parse.com) Received: from amd64.ott.parse.com (localhost.parse.com [127.0.0.1]) by amd64.ott.parse.com (8.13.4/8.13.1) with ESMTP id k9PGgrVY054537 for ; Wed, 25 Oct 2006 12:42:53 -0400 (EDT) (envelope-from root@parse.com) Received: (from root@localhost) by amd64.ott.parse.com (8.13.4/8.13.1/Submit) id k9PGgr4t054536 for freebsd-fs@freebsd.org; Wed, 25 Oct 2006 12:42:53 -0400 (EDT) (envelope-from root) From: Robert Krten Message-Id: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> To: freebsd-fs@freebsd.org Date: Wed, 25 Oct 2006 12:42:53 -0400 (EDT) X-Mailer: ELM [version 2.5 PL6] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: Naive question about encrypted disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 16:43:30 -0000 I've read a few articles and papers on both encryption and the encrypted filesystems available under FreeBSD, and have what probably amounts to a naive question :-) I've read that if you know the plaintext, or parts of it, then obtaining the key is possible (maybe not "trivial", but "possible"). Assuming the above is true, then the question I have is, when you encrypt the entire disk, aren't there bits of plaintext that you can derive? I'm thinking of meta data like what newfs leaves behind -- wouldn't it be possible to assume/guess the location and content of at least some of that meta data, and thus be able to then obtain the key? Or are the pieces of meta data that you can reliably guess at too small to be of use? Or... ? Like I said, I'm not an expert on crypto or filesystems by any stretch :-) Thanks in advance, -RK -- Robert Krten, PARSE Software Devices Realtime Systems Architecture, Consulting, Books and Training at www.parse.com Looking for Digital Equipment Corp. PDP-1 through PDP-15 minicomputers! From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 17:09:07 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4221516A417 for ; Wed, 25 Oct 2006 17:09:07 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 272D243DA1 for ; Wed, 25 Oct 2006 17:08:39 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id C9C5046E87; Wed, 25 Oct 2006 13:08:38 -0400 (EDT) Date: Wed, 25 Oct 2006 18:08:38 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Robert Krten In-Reply-To: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> Message-ID: <20061025180112.P33725@fledge.watson.org> References: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Naive question about encrypted disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 17:09:07 -0000 On Wed, 25 Oct 2006, Robert Krten wrote: > I've read a few articles and papers on both encryption and the encrypted > filesystems available under FreeBSD, and have what probably amounts to a > naive question :-) > > I've read that if you know the plaintext, or parts of it, then obtaining the > key is possible (maybe not "trivial", but "possible"). > > Assuming the above is true, then the question I have is, when you encrypt > the entire disk, aren't there bits of plaintext that you can derive? I'm > thinking of meta data like what newfs leaves behind -- wouldn't it be > possible to assume/guess the location and content of at least some of that > meta data, and thus be able to then obtain the key? Or are the pieces of > meta data that you can reliably guess at too small to be of use? Or... ? > > Like I said, I'm not an expert on crypto or filesystems by any stretch :-) Deriving the key when you have examples of plaintext and ciphertext for that plaintext is known as a "known-plaintext attack". Resistence to known-plaintext attacks is one of the most important properties required of modern crypto algorithms. Other examples of cases where resistance to known-plaintext attacks is critical include: - IPSEC, where it's often the case that a potential attacker can trigger known plaintext to appear in the plaintext, and also through a packet sniffer gain access to the ciphertext, but is not permitted to know the secret key. - SSL web servers, where a customer of an ISP may be able to provide content delivered using SSL, and can gain access to the ciphertext, but should not be able to derive the key. There are attacks that reduce the computational cost of deriving keying materials against known crypto algorithms; however, those attacks typically do not signifcantly weaken the cipher. Where they do, we have a special term we can use to describe the algorithm: "broken". Many crypto protocols (that is to say, conventions involving the use of crypto) include "salt" or "initial vectors" (IVs) to limit the effectiveness of dictionary attacks and known-plaintext attacks by causing the same plaintext to be encrypted differently each time it is encrypted. These are typically pseudo-random values, or in the case of chained crypto modes, earlier data in the ciphertext or cleartext, or in the case of counter mode, a incrementing counter. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 17:15:01 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C968416A403 for ; Wed, 25 Oct 2006 17:15:01 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 17A6043D66 for ; Wed, 25 Oct 2006 17:15:01 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id A5F8A46E6C; Wed, 25 Oct 2006 13:15:00 -0400 (EDT) Date: Wed, 25 Oct 2006 18:15:00 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Robert Krten In-Reply-To: <20061025180112.P33725@fledge.watson.org> Message-ID: <20061025181040.K33725@fledge.watson.org> References: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> <20061025180112.P33725@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org Subject: Re: Naive question about encrypted disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 17:15:01 -0000 On Wed, 25 Oct 2006, Robert Watson wrote: > Deriving the key when you have examples of plaintext and ciphertext for that > plaintext is known as a "known-plaintext attack". Resistence to > known-plaintext attacks is one of the most important properties required of > modern crypto algorithms. Other examples of cases where resistance to > known-plaintext attacks is critical include: FYI, there are a number of quite good books on cryptography and the use of cryptography available. Some of them even manage to get across the key concepts to be aware of as a consumer of cryptography without losing the reader in the details of how the algorithms are implemented. :-) Ross Anderson's Security Engineering is now available online: http://www.cl.cam.ac.uk/~rja14/book.html His crypto chapter (5) is both accessible and informative. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 19:37:18 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56F3516A403; Wed, 25 Oct 2006 19:37:18 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id E055943D45; Wed, 25 Oct 2006 19:37:17 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([165.236.175.187]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k9PJbACK092184; Wed, 25 Oct 2006 13:37:16 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <453FBCDF.5050807@samsco.org> Date: Wed, 25 Oct 2006 13:37:03 -0600 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> <20061025180112.P33725@fledge.watson.org> In-Reply-To: <20061025180112.P33725@fledge.watson.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-fs@freebsd.org, Robert Krten Subject: Re: Naive question about encrypted disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 19:37:18 -0000 Robert Watson wrote: > > On Wed, 25 Oct 2006, Robert Krten wrote: > >> I've read a few articles and papers on both encryption and the >> encrypted filesystems available under FreeBSD, and have what probably >> amounts to a naive question :-) >> >> I've read that if you know the plaintext, or parts of it, then >> obtaining the key is possible (maybe not "trivial", but "possible"). >> >> Assuming the above is true, then the question I have is, when you >> encrypt the entire disk, aren't there bits of plaintext that you can >> derive? I'm thinking of meta data like what newfs leaves behind -- >> wouldn't it be possible to assume/guess the location and content of at >> least some of that meta data, and thus be able to then obtain the >> key? Or are the pieces of meta data that you can reliably guess at >> too small to be of use? Or... ? >> >> Like I said, I'm not an expert on crypto or filesystems by any stretch >> :-) > > > Deriving the key when you have examples of plaintext and ciphertext for > that plaintext is known as a "known-plaintext attack". Resistence to > known-plaintext attacks is one of the most important properties required > of modern crypto algorithms. Other examples of cases where resistance > to known-plaintext attacks is critical include: > > - IPSEC, where it's often the case that a potential attacker can trigger > known > plaintext to appear in the plaintext, and also through a packet > sniffer gain > access to the ciphertext, but is not permitted to know the secret key. > > - SSL web servers, where a customer of an ISP may be able to provide > content > delivered using SSL, and can gain access to the ciphertext, but should > not > be able to derive the key. > > There are attacks that reduce the computational cost of deriving keying > materials against known crypto algorithms; however, those attacks > typically do not signifcantly weaken the cipher. Where they do, we have > a special term we can use to describe the algorithm: "broken". > > Many crypto protocols (that is to say, conventions involving the use of > crypto) include "salt" or "initial vectors" (IVs) to limit the > effectiveness of dictionary attacks and known-plaintext attacks by > causing the same plaintext to be encrypted differently each time it is > encrypted. These are typically pseudo-random values, or in the case of > chained crypto modes, earlier data in the ciphertext or cleartext, or in > the case of counter mode, a incrementing counter. > So, if you know that multiple superblock copies are going to be written at predictable places on the disk, and you know that the these copies are identical, unchanging, and have predictable contents, does that give you a starting point for a known-plaintext attack? I believe that is the question here. Even if the block granularity of GBDE ensures that the superblocks will be encrypted with other less-predictable data, could you still predict that the outter cylinder groups of the disk might be unused, and therefore have lots of predictable data on them? Scott From owner-freebsd-fs@FreeBSD.ORG Wed Oct 25 21:42:25 2006 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A44C416A407 for ; Wed, 25 Oct 2006 21:42:25 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5582443D45 for ; Wed, 25 Oct 2006 21:42:25 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 0A25C46D96; Wed, 25 Oct 2006 17:42:25 -0400 (EDT) Date: Wed, 25 Oct 2006 22:42:24 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Scott Long In-Reply-To: <453FBCDF.5050807@samsco.org> Message-ID: <20061025223246.G33725@fledge.watson.org> References: <200610251642.k9PGgr4t054536@amd64.ott.parse.com> <20061025180112.P33725@fledge.watson.org> <453FBCDF.5050807@samsco.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-fs@freebsd.org, Robert Krten Subject: Re: Naive question about encrypted disks X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Oct 2006 21:42:25 -0000 On Wed, 25 Oct 2006, Scott Long wrote: >> Deriving the key when you have examples of plaintext and ciphertext for >> that plaintext is known as a "known-plaintext attack". Resistence to >> known-plaintext attacks is one of the most important properties required of >> modern crypto algorithms. Other examples of cases where resistance to >> known-plaintext attacks is critical include: >> >> - IPSEC, where it's often the case that a potential attacker can trigger >> known >> plaintext to appear in the plaintext, and also through a packet sniffer >> gain >> access to the ciphertext, but is not permitted to know the secret key. >> >> - SSL web servers, where a customer of an ISP may be able to provide >> content >> delivered using SSL, and can gain access to the ciphertext, but should >> not >> be able to derive the key. >> >> There are attacks that reduce the computational cost of deriving keying >> materials against known crypto algorithms; however, those attacks typically >> do not signifcantly weaken the cipher. Where they do, we have a special >> term we can use to describe the algorithm: "broken". >> >> Many crypto protocols (that is to say, conventions involving the use of >> crypto) include "salt" or "initial vectors" (IVs) to limit the >> effectiveness of dictionary attacks and known-plaintext attacks by causing >> the same plaintext to be encrypted differently each time it is encrypted. >> These are typically pseudo-random values, or in the case of chained crypto >> modes, earlier data in the ciphertext or cleartext, or in the case of >> counter mode, a incrementing counter. > > So, if you know that multiple superblock copies are going to be written at > predictable places on the disk, and you know that the these copies are > identical, unchanging, and have predictable contents, does that give you a > starting point for a known-plaintext attack? I believe that is the question > here. Even if the block granularity of GBDE ensures that the superblocks > will be encrypted with other less-predictable data, could you still predict > that the outter cylinder groups of the disk might be unused, and therefore > have lots of predictable data on them? A detailed description of the gbde design can be found here: http://phk.freebsd.dk/pubs/bsdcon-03.gbde.paper.pdf While I don't know too much about the details, the paper indicates that there are a few elements of the gbde design to counter the known plaintext nature of the superblocks, cylinder groups, etc: (1) Random sector relocation. Without the keying material necessary to unlock the master key sector, you can't trivially locate the block in question. A portion of the master key is used to cryptographically generate the actual physical sector storage location for the logical sector. (2) A generated per-sector key, stored in key storage sectors, is used to encrypt each sector. So while the IV paramater to the crypto algorithm is zero, the keying material is different, so the same superblock contents will be encrypted to different ciphertet. The first portion addresses an initial concern about locating sectors believed to have predictable content, and the second portion addresses the encryption of the same data to the same plaintext. In the administrative directions in the paper, it's also recommended that the disk surface be completely randomized before beginning so that there isn't a usefully identifiable difference between "never written" and "in use or deleted" blocks when initialized on a disk with arbitrary (but possibly non-random) contents. A quick look at the gbde(8) admin tool suggests that it does not randomize disk contents, but I've not read closely. Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-fs@FreeBSD.ORG Fri Oct 27 03:42:22 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0670016A40F for ; Fri, 27 Oct 2006 03:42:22 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (arm132.internetdsl.tpnet.pl [83.17.198.132]) by mx1.FreeBSD.org (Postfix) with ESMTP id 487B943D45 for ; Fri, 27 Oct 2006 03:42:20 +0000 (GMT) (envelope-from pjd@garage.freebsd.pl) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7BFE1456B1; Fri, 27 Oct 2006 05:42:19 +0200 (CEST) Received: from localhost (dkd232.neoplus.adsl.tpnet.pl [83.24.7.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 8289045683; Fri, 27 Oct 2006 05:42:14 +0200 (CEST) Date: Fri, 27 Oct 2006 05:41:49 +0200 From: Pawel Jakub Dawidek To: freebsd-fs@FreeBSD.org Message-ID: <20061027034149.GE9491@garage.freebsd.pl> References: <20060822104516.GB16033@garage.freebsd.pl> <20060905084911.GB16045@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6e7ZaeXHKrTJCxdu" Content-Disposition: inline In-Reply-To: <20060905084911.GB16045@garage.freebsd.pl> X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 7.0-CURRENT i386 User-Agent: mutt-ng/devel-r804 (FreeBSD) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.5 required=3.0 tests=BAYES_00,RCVD_IN_NJABL_DUL, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: zfs-discuss@opensolaris.org Subject: Re: Porting ZFS file system to FreeBSD. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 03:42:22 -0000 --6e7ZaeXHKrTJCxdu Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Sep 05, 2006 at 10:49:11AM +0200, Pawel Jakub Dawidek wrote: > On Tue, Aug 22, 2006 at 12:45:16PM +0200, Pawel Jakub Dawidek wrote: > > Hi. > >=20 > > I started porting the ZFS file system to the FreeBSD operating system. > [...] >=20 > Just a quick note about progress in my work. I needed slow down a bit, > but: Here is another update: After way too much time spend on fighting the buffer cache I finally made mmap(2)ed reads/writes to work and (which is also very important) keep regular reads/writes working. Now I'm able to build FreeBSD's kernel and userland with both sources and objects placed on ZFS file system. I also tried to crash it with fsx, fsstress and postmark, but no luck, it works stable. On the other hand I'm quite sure there are many problems in ZPL still, but fixing mmap(2) allows me to move forward. As a said note - ZVOL seems to be full functional. I need to find a way to test ZIL, so if you guys at SUN have some ZIL tests like uncleanly stopped file system, which at mount time will exercise entire ZIL functionality where we can verify that my FS was fixed properly that would be great. PS. There is still a lot to do, so please, don't ask me for patches yet. --=20 Pawel Jakub Dawidek http://www.wheel.pl pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --6e7ZaeXHKrTJCxdu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.4 (FreeBSD) iD8DBQFFQX/8ForvXbEpPzQRApwXAKDAOromyth2vlXnoCgzqv+0qEeTuQCeLTc1 ijpBwUTvUxdDijdoQADoXOQ= =tu+o -----END PGP SIGNATURE----- --6e7ZaeXHKrTJCxdu-- From owner-freebsd-fs@FreeBSD.ORG Fri Oct 27 12:25:39 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C907816A412; Fri, 27 Oct 2006 12:25:39 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from auriate.fluffles.net (a83-68-3-169.adsl.cistron.nl [83.68.3.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6738043D45; Fri, 27 Oct 2006 12:25:39 +0000 (GMT) (envelope-from etc@fluffles.net) Received: from destiny ([10.0.0.21]) by auriate.fluffles.net with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GdQmI-000CgR-9f; Fri, 27 Oct 2006 14:25:38 +0200 Message-ID: <4541FAC1.1000601@fluffles.net> Date: Fri, 27 Oct 2006 14:25:37 +0200 From: Fluffles User-Agent: Thunderbird 1.5.0.7 (X11/20060917) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <20061024152308.GG75746@garage.freebsd.pl> In-Reply-To: <20061024152308.GG75746@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-geom@FreeBSD.org Subject: Re: Updated gjournal patches [20061024]. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 12:25:39 -0000 Hi Pawel, I've tried your recent (20061024) gjournal patches on my test fileserver, which is an amd64 FreeBSD7 box with 8 SATA disks. I'm currently using graid5 on it, with success. I've also tried using gjournal, but with little success. The compilation/installation and creation of the journal device goes as planned, but i get kernel panics not long after i start writing to it. A "newfs -b 65536 /dev/raid5/sophia.journal" quickly panicked the system. After that i tried without the -b parameter; that appeared to work but not long after i got a panic again; same panic as before. Please look at the screenshot i made of the panic message: http://dev.fluffles.net/images/gjournal-panic1.png Do you know what could be causing the panic? The panic message itself is not very explanatory. ;-) Perhaps gjournal doesn't play nice with graid5? I haven't tried it on a single disk, yet. Also i have a question about it's performance. You mentioned earlier that writing big files takes about twice as long with gjournal, i wonder if this is inherit to journaling itself or due to the current implementation. Windows' journaling NTFS, for example, isn't slower than FAT32 with big files if i remember correctly. What major differences in the journaling process causes this? Also, in your earlier post you explained the advantages of a journal with regard to significantly reduces fsck times at boot. But my major concern is dataloss: on my testserver i've had many kernel panics/freezes due to the experimental graid5 module being tested by Arne. This has resulted in the system not being able to boot because the ad0s2a (read: a!) partition has lost files. And it won't be the first time a lockup or power failure caused dataloss on my systems. That's why i want to use gjournal: to protect from dataloss. Am i correct in my assumption that gjournal addresses my needs in this regard? Greetings, Veronica From owner-freebsd-fs@FreeBSD.ORG Fri Oct 27 13:27:16 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7066516A4D2; Fri, 27 Oct 2006 13:27:16 +0000 (UTC) (envelope-from etc@fluffles.net) Received: from auriate.fluffles.net (a83-68-3-169.adsl.cistron.nl [83.68.3.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id 512A043DC4; Fri, 27 Oct 2006 13:25:57 +0000 (GMT) (envelope-from etc@fluffles.net) Received: from destiny ([10.0.0.21]) by auriate.fluffles.net with esmtpa (Exim 4.63 (FreeBSD)) (envelope-from ) id 1GdRiR-000D1O-Bb; Fri, 27 Oct 2006 15:25:43 +0200 Message-ID: <454208D6.7000006@fluffles.net> Date: Fri, 27 Oct 2006 15:25:42 +0200 From: Fluffles User-Agent: Thunderbird 1.5.0.7 (X11/20060917) MIME-Version: 1.0 To: Fluffles References: <20061024152308.GG75746@garage.freebsd.pl> <4541FAC1.1000601@fluffles.net> In-Reply-To: <4541FAC1.1000601@fluffles.net> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , freebsd-geom@FreeBSD.org Subject: Re: Updated gjournal patches [20061024]. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 13:27:16 -0000 Addendum: i tried gjournal on a single disk now without any GEOM layers on it, but it still crashes with the same error. - Veronica Fluffles wrote: > Hi Pawel, > > I've tried your recent (20061024) gjournal patches on my test > fileserver, which is an amd64 FreeBSD7 box with 8 SATA disks. I'm > currently using graid5 on it, with success. > I've also tried using gjournal, but with little success. The > compilation/installation and creation of the journal device goes as > planned, but i get kernel panics not long after i start writing to it. A > "newfs -b 65536 /dev/raid5/sophia.journal" quickly panicked the system. > After that i tried without the -b parameter; that appeared to work but > not long after i got a panic again; same panic as before. > > Please look at the screenshot i made of the panic message: > http://dev.fluffles.net/images/gjournal-panic1.png > > Do you know what could be causing the panic? The panic message itself is > not very explanatory. ;-) > Perhaps gjournal doesn't play nice with graid5? I haven't tried it on a > single disk, yet. > > Also i have a question about it's performance. You mentioned earlier > that writing big files takes about twice as long with gjournal, i wonder > if this is inherit to journaling itself or due to the current > implementation. Windows' journaling NTFS, for example, isn't slower than > FAT32 with big files if i remember correctly. What major differences in > the journaling process causes this? > > Also, in your earlier post you explained the advantages of a journal > with regard to significantly reduces fsck times at boot. But my major > concern is dataloss: on my testserver i've had many kernel > panics/freezes due to the experimental graid5 module being tested by > Arne. This has resulted in the system not being able to boot because the > ad0s2a (read: a!) partition has lost files. And it won't be the first > time a lockup or power failure caused dataloss on my systems. That's why > i want to use gjournal: to protect from dataloss. Am i correct in my > assumption that gjournal addresses my needs in this regard? > > Greetings, > > Veronica > _______________________________________________ > freebsd-geom@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-geom > To unsubscribe, send any mail to "freebsd-geom-unsubscribe@freebsd.org" > > > From owner-freebsd-fs@FreeBSD.ORG Fri Oct 27 16:18:33 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D3E4216A510; Fri, 27 Oct 2006 16:18:33 +0000 (UTC) (envelope-from eschrock@zion.eng.sun.com) Received: from nwkea-mail-4.sun.com (nwkea-mail-4.sun.com [192.18.42.26]) by mx1.FreeBSD.org (Postfix) with ESMTP id F126B43D8D; Fri, 27 Oct 2006 16:18:21 +0000 (GMT) (envelope-from eschrock@zion.eng.sun.com) Received: from engmail3mpk.sfbay.Sun.COM ([129.146.11.26]) by nwkea-mail-4.sun.com (8.13.6+Sun/8.12.9) with ESMTP id k9RGILWX021684; Fri, 27 Oct 2006 09:18:21 -0700 (PDT) Received: from zion.eng.sun.com (zion.SFBay.Sun.COM [129.146.17.75]) by engmail3mpk.sfbay.Sun.COM (8.13.6+Sun/8.13.6/ENSMAIL,v2.2) with ESMTP id k9RGILNn014308; Fri, 27 Oct 2006 09:18:21 -0700 (PDT) Received: from zion.eng.sun.com (localhost [127.0.0.1]) by zion.eng.sun.com (8.13.7+Sun/8.13.7) with ESMTP id k9RGIL5P016124; Fri, 27 Oct 2006 09:18:21 -0700 (PDT) Received: (from eschrock@localhost) by zion.eng.sun.com (8.13.7+Sun/8.13.7/Submit) id k9RGILVq016123; Fri, 27 Oct 2006 09:18:21 -0700 (PDT) Date: Fri, 27 Oct 2006 09:18:21 -0700 From: Eric Schrock To: Pawel Jakub Dawidek Message-ID: <20061027161820.GA15305@eng.sun.com> References: <20060822104516.GB16033@garage.freebsd.pl> <20060905084911.GB16045@garage.freebsd.pl> <20061027034149.GE9491@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061027034149.GE9491@garage.freebsd.pl> User-Agent: Mutt/1.4.2.1i Cc: freebsd-fs@FreeBSD.org, zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Re: Porting ZFS file system to FreeBSD. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 16:18:33 -0000 Congrats, Pawel. This is truly an impressive piece of work. As you're probably aware, Noel integrated the patches your provided us into build 51. Hopefully that got rid of some spurious differences between the code bases. We do have a program called 'ziltest' that Neil can probably provide for you that does a good job stressing the ZIL. We also have a complete test suite (functional and stress), but it would be non-trivial to port, and I don't know what the current status is for open sourcing the test suites in general. Let us know if there's anything else we can help with. - Eric On Fri, Oct 27, 2006 at 05:41:49AM +0200, Pawel Jakub Dawidek wrote: > > Here is another update: > > After way too much time spend on fighting the buffer cache I finally > made mmap(2)ed reads/writes to work and (which is also very important) > keep regular reads/writes working. > > Now I'm able to build FreeBSD's kernel and userland with both sources > and objects placed on ZFS file system. > > I also tried to crash it with fsx, fsstress and postmark, but no luck, > it works stable. > > On the other hand I'm quite sure there are many problems in ZPL still, > but fixing mmap(2) allows me to move forward. > > As a said note - ZVOL seems to be full functional. > > I need to find a way to test ZIL, so if you guys at SUN have some ZIL > tests like uncleanly stopped file system, which at mount time will > exercise entire ZIL functionality where we can verify that my FS was > fixed properly that would be great. > > PS. There is still a lot to do, so please, don't ask me for patches yet. > > -- > Pawel Jakub Dawidek http://www.wheel.pl > pjd@FreeBSD.org http://www.FreeBSD.org > FreeBSD committer Am I Evil? Yes, I Am! -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock From owner-freebsd-fs@FreeBSD.ORG Fri Oct 27 17:17:35 2006 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F276816A407; Fri, 27 Oct 2006 17:17:34 +0000 (UTC) (envelope-from Neil.Perrin@Sun.COM) Received: from brmea-mail-1.sun.com (brmea-mail-1.Sun.COM [192.18.98.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3D8843D7E; Fri, 27 Oct 2006 17:16:40 +0000 (GMT) (envelope-from Neil.Perrin@Sun.COM) Received: from fe-amer-02.sun.com ([192.18.108.176]) by brmea-mail-1.sun.com (8.13.6+Sun/8.12.9) with ESMTP id k9RHGYXM002279; Fri, 27 Oct 2006 11:16:34 -0600 (MDT) Received: from conversion-daemon.mail-amer.sun.com by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) id <0J7T007010SNW100@mail-amer.sun.com> (original mail from Neil.Perrin@Sun.COM); Fri, 27 Oct 2006 11:16:34 -0600 (MDT) Received: from [129.147.9.15] by mail-amer.sun.com (Sun Java System Messaging Server 6.2-6.01 (built Apr 3 2006)) with ESMTPSA id <0J7T009DK1BCWLU3@mail-amer.sun.com>; Fri, 27 Oct 2006 11:16:34 -0600 (MDT) Date: Fri, 27 Oct 2006 11:16:24 -0600 From: Neil Perrin In-reply-to: <20061027161820.GA15305@eng.sun.com> Sender: Neil.Perrin@Sun.COM To: Pawel Jakub Dawidek Message-id: <45423EE8.609@Sun.COM> MIME-version: 1.0 Content-type: multipart/mixed; boundary="Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw)" X-Accept-Language: en-us, en References: <20060822104516.GB16033@garage.freebsd.pl> <20060905084911.GB16045@garage.freebsd.pl> <20061027034149.GE9491@garage.freebsd.pl> <20061027161820.GA15305@eng.sun.com> User-Agent: Mozilla/5.0 (X11; U; SunOS sun4u; en-US; rv:1.7) Gecko/20060120 Cc: freebsd-fs@FreeBSD.org, zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Re: Porting ZFS file system to FreeBSD. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Neil.Perrin@Sun.COM List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 17:17:35 -0000 This is a multi-part message in MIME format. --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw) Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT Pawel, I second that praise. Well done! Attached is a copy of ziltest. You will have to adapt this a bit to your environment. In particular it uses bringover to pull a subtree of our source and then builds and later runs it. This tends to create a fair number of transactions with various dependencies. You'll obviously have to update the paths and tools. However, at least initially, I'd recommend you simplify things by perhaps jhaving the only test as a creation of a file. The basic flow behind ziltest is: 1. Create an empty file system FS1 2. Freeze FS1 3. Perform various user commands that create files, directories, etc 4. Copy FS1 to FS2 5. Unmount and unfreeze FS1 6. Remount FS1 (resulting in replay of log) 7. Compare FS1 & FS2 and complain if not equal Hope this helps and good luck: Neil. Eric Schrock wrote On 10/27/06 10:18,: > Congrats, Pawel. This is truly an impressive piece of work. As you're > probably aware, Noel integrated the patches your provided us into build > 51. Hopefully that got rid of some spurious differences between the > code bases. > > We do have a program called 'ziltest' that Neil can probably provide for > you that does a good job stressing the ZIL. We also have a complete > test suite (functional and stress), but it would be non-trivial to port, > and I don't know what the current status is for open sourcing the test > suites in general. > > Let us know if there's anything else we can help with. > > - Eric > > On Fri, Oct 27, 2006 at 05:41:49AM +0200, Pawel Jakub Dawidek wrote: > >>Here is another update: >> >>After way too much time spend on fighting the buffer cache I finally >>made mmap(2)ed reads/writes to work and (which is also very important) >>keep regular reads/writes working. >> >>Now I'm able to build FreeBSD's kernel and userland with both sources >>and objects placed on ZFS file system. >> >>I also tried to crash it with fsx, fsstress and postmark, but no luck, >>it works stable. >> >>On the other hand I'm quite sure there are many problems in ZPL still, >>but fixing mmap(2) allows me to move forward. >> >>As a said note - ZVOL seems to be full functional. >> >>I need to find a way to test ZIL, so if you guys at SUN have some ZIL >>tests like uncleanly stopped file system, which at mount time will >>exercise entire ZIL functionality where we can verify that my FS was >>fixed properly that would be great. >> >>PS. There is still a lot to do, so please, don't ask me for patches yet. >> >>-- >>Pawel Jakub Dawidek http://www.wheel.pl >>pjd@FreeBSD.org http://www.FreeBSD.org >>FreeBSD committer Am I Evil? Yes, I Am! > > > -- > Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw) Content-type: text/plain; name=ziltest Content-transfer-encoding: 7BIT Content-disposition: inline; filename=ziltest #!/bin/ksh -x # # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License, Version 1.0 only # (the "License"). You may not use this file except in compliance # with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at usr/src/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # # Copyright 2006 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # ident "@(#)ziltest 1.2 06/01/30 SMI" # # - creates a 150MB pool in /tmp # - Should take about a minute (depends on access to the gate for bringover). # - You can change the gate to local by setting and exporting ZILTEST_GATE # PATH=/usr/bin PATH=$PATH:/usr/sbin PATH=$PATH:/usr/ccs/bin #PATH=$PATH:/net/slug.eng/opt/export/`uname -p`/opt/SUNWspro/SOS8/bin #PATH=$PATH:/net/anthrax.central/export/tools/onnv-tools/SUNWspro/SOS8/bin PATH=$PATH:/net/haulass.central/export/tools/onnv-tools/SUNWspro/SOS8/bin #PATH=$PATH:/net/slug.eng/opt/onbld/bin PATH=$PATH:/opt/onbld/bin export PATH # # SETUP # ZILTEST_GATE=${ZILTEST_GATE-/net/haulass.central/export/clones/onnv} CMD=`basename $0` POOL=ziltestpool.$$ DEVSIZE=${DEVSIZE-150m} POOLDIR=/tmp POOLFILE=$POOLDIR/ziltest_poolfile.$$ FS=$POOL/fs ROOT=/$FS COPY=/tmp/${POOL} KEEP=no cleanup() { zfs destroy $FS zpool iostat $POOL print zpool status $POOL zpool destroy $POOL rm -rf $COPY rm $POOLFILE } bail() { test $KEEP = no && cleanup print $1 exit 1 } test $# -eq 0 || bail "usage: $CMD" mkfile $DEVSIZE $POOLFILE || bail "can't make $POOLFILE" zpool create $POOL $POOLFILE || bail "can't create pool $POOL" zfs set compression=on $POOL || bail "can't enable compression on $POOL" zfs create $FS || bail "can't create $FS" mkdir -p $COPY || bail "can't create $COPY" touch $ROOT/touched lockfs -f $ROOT zpool freeze $POOL || bail "can't freeze $POOL" # ==================================================================== # TESTS # Put tests below. # Use $ROOT for all file name prefixes # ==================================================================== touch $ROOT/a mv $ROOT/a $ROOT/b touch $ROOT/c ln -s $ROOT/c $ROOT/d touch $ROOT/e ln $ROOT/e $ROOT/f mkdir $ROOT/dir_to_delete rmdir $ROOT/dir_to_delete bringover -p $ZILTEST_GATE -w $ROOT/tws usr/src/cmd/date >/dev/null ws $ROOT/tws >/dev/null << EOF cd usr/src/cmd/date make EOF cp /etc/project $ROOT/small_file cp /etc/auto_home $ROOT/small_file cp -R /kernel/exec $ROOT rm -rf $ROOT/kernel touch $ROOT/setattr chmod 567 $ROOT/setattr # # Write to an open but removed file # (sleep 2; date) > $ROOT/date & sleep 1; rm $ROOT/date; wait #Large file dd if=/usr/share/lib/termcap of=/ROOT/large bs=128 # # Write zeroes, which compresss to holes, in the middle of a file # dd if=$POOLFILE of=$ROOT/holes.1 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.1 bs=128k count=2 dd if=$POOLFILE of=$ROOT/holes.2 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.2 bs=128k count=2 oseek=2 dd if=$POOLFILE of=$ROOT/holes.3 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.3 bs=128k count=2 oseek=2 conv=notrunc # # Test extended attributes # mkdir $ROOT/xattr.dir runat $ROOT/xattr.dir mkfile 1k fileattr runat $ROOT/xattr.dir mkfile 1k tmpattr runat $ROOT/xattr.dir rm tmpattr touch $ROOT/xattr.file runat $ROOT/xattr.file mkfile 1k fileattr runat $ROOT/xattr.file mkfile 1k tmpattr runat $ROOT/xattr.file rm tmpattr rm $ROOT/xattr.file # ==================================================================== # CLEANUP # ==================================================================== KEEP=yes # keep stuff around if we fail, so we can look at it cd $ROOT find . | cpio -pdmuv $COPY cd / zfs unmount $FS || bail "can't unmount $FS" print $CMD: transactions to replay: zdb -ivv $FS || bail "can't run zdb on $POOL" # # Export and reimport the pool to unfreeze it and claim log blocks. # It has to be import -f because we can't write a frozen pool's labels! # zpool export $POOL || bail "can't export $POOL" zpool import -f -d $POOLDIR $POOL || bail "can't import $POOL" # ==================================================================== # PLAYBACK # # Exit here if you want to peruse the log before playback. # ==================================================================== print $CMD: current block usage: zdb -bcv $POOL || bail "blocks were leaked!" runat $ROOT/xattr.dir ls -la || bail "can't list xattr directory" diff -r $ROOT $COPY > /dev/null || diff -r $ROOT $COPY || bail "replay diffs!" $ROOT/tws/usr/src/cmd/date/date || bail "can't run the date(1) binary we built" cleanup exit 0 --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw)--