Date: Fri, 27 Oct 2006 11:16:24 -0600 From: Neil Perrin <Neil.Perrin@Sun.COM> To: Pawel Jakub Dawidek <pjd@FreeBSD.org> Cc: freebsd-fs@FreeBSD.org, zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] Re: Porting ZFS file system to FreeBSD. Message-ID: <45423EE8.609@Sun.COM> In-Reply-To: <20061027161820.GA15305@eng.sun.com> References: <20060822104516.GB16033@garage.freebsd.pl> <20060905084911.GB16045@garage.freebsd.pl> <20061027034149.GE9491@garage.freebsd.pl> <20061027161820.GA15305@eng.sun.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw) Content-type: text/plain; format=flowed; charset=us-ascii Content-transfer-encoding: 7BIT Pawel, I second that praise. Well done! Attached is a copy of ziltest. You will have to adapt this a bit to your environment. In particular it uses bringover to pull a subtree of our source and then builds and later runs it. This tends to create a fair number of transactions with various dependencies. You'll obviously have to update the paths and tools. However, at least initially, I'd recommend you simplify things by perhaps jhaving the only test as a creation of a file. The basic flow behind ziltest is: 1. Create an empty file system FS1 2. Freeze FS1 3. Perform various user commands that create files, directories, etc 4. Copy FS1 to FS2 5. Unmount and unfreeze FS1 6. Remount FS1 (resulting in replay of log) 7. Compare FS1 & FS2 and complain if not equal Hope this helps and good luck: Neil. Eric Schrock wrote On 10/27/06 10:18,: > Congrats, Pawel. This is truly an impressive piece of work. As you're > probably aware, Noel integrated the patches your provided us into build > 51. Hopefully that got rid of some spurious differences between the > code bases. > > We do have a program called 'ziltest' that Neil can probably provide for > you that does a good job stressing the ZIL. We also have a complete > test suite (functional and stress), but it would be non-trivial to port, > and I don't know what the current status is for open sourcing the test > suites in general. > > Let us know if there's anything else we can help with. > > - Eric > > On Fri, Oct 27, 2006 at 05:41:49AM +0200, Pawel Jakub Dawidek wrote: > >>Here is another update: >> >>After way too much time spend on fighting the buffer cache I finally >>made mmap(2)ed reads/writes to work and (which is also very important) >>keep regular reads/writes working. >> >>Now I'm able to build FreeBSD's kernel and userland with both sources >>and objects placed on ZFS file system. >> >>I also tried to crash it with fsx, fsstress and postmark, but no luck, >>it works stable. >> >>On the other hand I'm quite sure there are many problems in ZPL still, >>but fixing mmap(2) allows me to move forward. >> >>As a said note - ZVOL seems to be full functional. >> >>I need to find a way to test ZIL, so if you guys at SUN have some ZIL >>tests like uncleanly stopped file system, which at mount time will >>exercise entire ZIL functionality where we can verify that my FS was >>fixed properly that would be great. >> >>PS. There is still a lot to do, so please, don't ask me for patches yet. >> >>-- >>Pawel Jakub Dawidek http://www.wheel.pl >>pjd@FreeBSD.org http://www.FreeBSD.org >>FreeBSD committer Am I Evil? Yes, I Am! > > > -- > Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock > _______________________________________________ > zfs-discuss mailing list > zfs-discuss@opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw) Content-type: text/plain; name=ziltest Content-transfer-encoding: 7BIT Content-disposition: inline; filename=ziltest #!/bin/ksh -x # # CDDL HEADER START # # The contents of this file are subject to the terms of the # Common Development and Distribution License, Version 1.0 only # (the "License"). You may not use this file except in compliance # with the License. # # You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE # or http://www.opensolaris.org/os/licensing. # See the License for the specific language governing permissions # and limitations under the License. # # When distributing Covered Code, include this CDDL HEADER in each # file and include the License file at usr/src/OPENSOLARIS.LICENSE. # If applicable, add the following below this CDDL HEADER, with the # fields enclosed by brackets "[]" replaced with your own identifying # information: Portions Copyright [yyyy] [name of copyright owner] # # CDDL HEADER END # # # Copyright 2006 Sun Microsystems, Inc. All rights reserved. # Use is subject to license terms. # # ident "@(#)ziltest 1.2 06/01/30 SMI" # # - creates a 150MB pool in /tmp # - Should take about a minute (depends on access to the gate for bringover). # - You can change the gate to local by setting and exporting ZILTEST_GATE # PATH=/usr/bin PATH=$PATH:/usr/sbin PATH=$PATH:/usr/ccs/bin #PATH=$PATH:/net/slug.eng/opt/export/`uname -p`/opt/SUNWspro/SOS8/bin #PATH=$PATH:/net/anthrax.central/export/tools/onnv-tools/SUNWspro/SOS8/bin PATH=$PATH:/net/haulass.central/export/tools/onnv-tools/SUNWspro/SOS8/bin #PATH=$PATH:/net/slug.eng/opt/onbld/bin PATH=$PATH:/opt/onbld/bin export PATH # # SETUP # ZILTEST_GATE=${ZILTEST_GATE-/net/haulass.central/export/clones/onnv} CMD=`basename $0` POOL=ziltestpool.$$ DEVSIZE=${DEVSIZE-150m} POOLDIR=/tmp POOLFILE=$POOLDIR/ziltest_poolfile.$$ FS=$POOL/fs ROOT=/$FS COPY=/tmp/${POOL} KEEP=no cleanup() { zfs destroy $FS zpool iostat $POOL print zpool status $POOL zpool destroy $POOL rm -rf $COPY rm $POOLFILE } bail() { test $KEEP = no && cleanup print $1 exit 1 } test $# -eq 0 || bail "usage: $CMD" mkfile $DEVSIZE $POOLFILE || bail "can't make $POOLFILE" zpool create $POOL $POOLFILE || bail "can't create pool $POOL" zfs set compression=on $POOL || bail "can't enable compression on $POOL" zfs create $FS || bail "can't create $FS" mkdir -p $COPY || bail "can't create $COPY" touch $ROOT/touched lockfs -f $ROOT zpool freeze $POOL || bail "can't freeze $POOL" # ==================================================================== # TESTS # Put tests below. # Use $ROOT for all file name prefixes # ==================================================================== touch $ROOT/a mv $ROOT/a $ROOT/b touch $ROOT/c ln -s $ROOT/c $ROOT/d touch $ROOT/e ln $ROOT/e $ROOT/f mkdir $ROOT/dir_to_delete rmdir $ROOT/dir_to_delete bringover -p $ZILTEST_GATE -w $ROOT/tws usr/src/cmd/date >/dev/null ws $ROOT/tws >/dev/null << EOF cd usr/src/cmd/date make EOF cp /etc/project $ROOT/small_file cp /etc/auto_home $ROOT/small_file cp -R /kernel/exec $ROOT rm -rf $ROOT/kernel touch $ROOT/setattr chmod 567 $ROOT/setattr # # Write to an open but removed file # (sleep 2; date) > $ROOT/date & sleep 1; rm $ROOT/date; wait #Large file dd if=/usr/share/lib/termcap of=/ROOT/large bs=128 # # Write zeroes, which compresss to holes, in the middle of a file # dd if=$POOLFILE of=$ROOT/holes.1 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.1 bs=128k count=2 dd if=$POOLFILE of=$ROOT/holes.2 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.2 bs=128k count=2 oseek=2 dd if=$POOLFILE of=$ROOT/holes.3 bs=128k count=8 dd if=/dev/zero of=$ROOT/holes.3 bs=128k count=2 oseek=2 conv=notrunc # # Test extended attributes # mkdir $ROOT/xattr.dir runat $ROOT/xattr.dir mkfile 1k fileattr runat $ROOT/xattr.dir mkfile 1k tmpattr runat $ROOT/xattr.dir rm tmpattr touch $ROOT/xattr.file runat $ROOT/xattr.file mkfile 1k fileattr runat $ROOT/xattr.file mkfile 1k tmpattr runat $ROOT/xattr.file rm tmpattr rm $ROOT/xattr.file # ==================================================================== # CLEANUP # ==================================================================== KEEP=yes # keep stuff around if we fail, so we can look at it cd $ROOT find . | cpio -pdmuv $COPY cd / zfs unmount $FS || bail "can't unmount $FS" print $CMD: transactions to replay: zdb -ivv $FS || bail "can't run zdb on $POOL" # # Export and reimport the pool to unfreeze it and claim log blocks. # It has to be import -f because we can't write a frozen pool's labels! # zpool export $POOL || bail "can't export $POOL" zpool import -f -d $POOLDIR $POOL || bail "can't import $POOL" # ==================================================================== # PLAYBACK # # Exit here if you want to peruse the log before playback. # ==================================================================== print $CMD: current block usage: zdb -bcv $POOL || bail "blocks were leaked!" runat $ROOT/xattr.dir ls -la || bail "can't list xattr directory" diff -r $ROOT $COPY > /dev/null || diff -r $ROOT $COPY || bail "replay diffs!" $ROOT/tws/usr/src/cmd/date/date || bail "can't run the date(1) binary we built" cleanup exit 0 --Boundary_(ID_5A7NZo4MhgDSyXwDZ9jxDw)--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?45423EE8.609>