From owner-freebsd-fs@FreeBSD.ORG Tue May 22 12:06:49 2007 Return-Path: X-Original-To: freebsd-fs@freebsd.org Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 41FEF16A469 for ; Tue, 22 May 2007 12:06:49 +0000 (UTC) (envelope-from bv@bilver.wjv.com) Received: from wjv.com (fl-65-40-24-38.sta.embarqhsd.net [65.40.24.38]) by mx1.freebsd.org (Postfix) with ESMTP id A8CF013C4B7 for ; Tue, 22 May 2007 12:06:48 +0000 (UTC) (envelope-from bv@bilver.wjv.com) Received: from bilver.wjv.com (localhost.wjv.com [127.0.0.1]) by wjv.com (8.13.8/8.13.1) with ESMTP id l4MC6XHP089580; Tue, 22 May 2007 08:06:33 -0400 (EDT) (envelope-from bv@bilver.wjv.com) Received: (from bv@localhost) by bilver.wjv.com (8.13.8/8.13.1/Submit) id l4MC6Rsw089579; Tue, 22 May 2007 08:06:27 -0400 (EDT) (envelope-from bv) Date: Tue, 22 May 2007 08:06:27 -0400 From: Bill Vermillion To: Dag-Erling Sm??rgrav Message-ID: <20070522120627.GB89056@wjv.com> References: <475187.33232.qm@web63006.mail.re1.yahoo.com> <86r6p9xf2c.fsf@dwp.des.no> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <86r6p9xf2c.fsf@dwp.des.no> User-Agent: Mutt/1.4.2.2i Organization: W.J.Vermillion / Orlando - Winter Park ReplyTo: bv@wjv.com X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED,BAYES_00, SPF_HELO_PASS,SPF_PASS autolearn=ham version=3.1.7 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on bilver.wjv.com Cc: freebsd-fs@freebsd.org, Brooks Davis , Gore Jarold Subject: Re: VERY frustrated with FreeBSD/UFS stability - please help or comment... X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: bv@wjv.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 May 2007 12:06:49 -0000 The time has come the Dag-Erling Sm??rgrav said, to talk of many things but all that was heard on Tue, May 22, 2007 at 13:39 was whether pigs have wings - or: > Gore Jarold writes: > > Specifically, I have private departmental fileservers that other > > fileservers rsync to using Mike Rubel-style rsync snapshots: > > http://www.mikerubel.org/computers/rsync_snapshots/ > > This means that the remote system runs a script like this: > > ssh user@host rm -rf backup.2 > > ssh user@host mv backup.1 backup.2 > > ssh user@host cp -al backup.0 backup.1 > > rsync /files user@host:/backup.0 > This is extremely inefficient, as you have discovered. > Speaking in the abstract, what you want to do every day is the > following: > client1% rsync --archive --delete /vol server:/backup/client1 > client2% rsync --archive --delete /vol server:/backup/client2 > server% for vol in /backup/* ; do mksnap_ffs $vol $vol/.snap/`date` ; done > No copying or deleting; you take a snapshot when the rsync job > is done, and the next day you rsync again to the same directory; > only what has actually changed will be transferred, and there > is no need to create and populate full copies of each directory > tree every time. And one other way to 'copy' files/directories >>IF<< they are on the same file system, is to use cpio with the -pdlm option. All that does is build another directory with all files in the first liked statically to the second. Then you just 'rm' the files in the first. Since there is NO COPYING - this is quick, won't scatter files around as they remain where they were originally but with just a new directory pointing to them. Probably not the best way for something done regularly but a very quick and efficient way to move files around in a system with very low overhead and is quite fast. It also has the plus for the new directory is that it gets you a clean directory, which can be a perfomance gain if the original had lots of file, many of which were deleted. > The easiest way to do this (if you're not afraid to run > experimental code) is to use ZFS on the server, as it lets you > easily create separate file systems for each client, and creates > and maintains snapshots far more cheaply than FFS. I've been watching the ZFS and I'm going to wait awhile, but it will be awhile before I put in on my servers. Bill -- Bill Vermillion - bv @ wjv . com