From owner-freebsd-questions@FreeBSD.ORG Tue Aug 30 07:32:11 2005 Return-Path: X-Original-To: questions@freebsd.org Delivered-To: freebsd-questions@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40B1816A41F for ; Tue, 30 Aug 2005 07:32:11 +0000 (GMT) (envelope-from svein-freebsd-questions@theloosingend.net) Received: from fri.itea.ntnu.no (fri.itea.ntnu.no [129.241.7.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8AFAB43D46 for ; Tue, 30 Aug 2005 07:32:10 +0000 (GMT) (envelope-from svein-freebsd-questions@theloosingend.net) Received: from localhost (localhost [127.0.0.1]) by fri.itea.ntnu.no (Postfix) with ESMTP id 369FC825B for ; Tue, 30 Aug 2005 09:32:09 +0200 (CEST) Received: from maren.thelosingend.net (maren.math.ntnu.no [129.241.211.48]) by fri.itea.ntnu.no (Postfix) with SMTP for ; Tue, 30 Aug 2005 09:32:08 +0200 (CEST) Received: (qmail 14042 invoked by uid 1001); 30 Aug 2005 09:32:08 +0200 Received: from localhost (sendmail-bs@127.0.0.1) by localhost with SMTP; 30 Aug 2005 09:32:08 +0200 Date: Tue, 30 Aug 2005 09:32:08 +0200 (CEST) From: Svein Halvor Halvorsen X-X-Sender: sveinhal@maren.thelosingend.net To: Greg Barniskis In-Reply-To: <43133BA5.2010608@scls.lib.wi.us> Message-ID: <20050830091919.J13913@maren.thelosingend.net> References: <20050828234043.H22315@maren.thelosingend.net> <20050829161506.E2522@maren.thelosingend.net> <43131C85.1070100@meijome.net> <20050829170053.M3014@maren.thelosingend.net> <43133BA5.2010608@scls.lib.wi.us> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Content-Scanned: with sophos and spamassassin at mailgw.ntnu.no. Cc: questions@freebsd.org, Norberto Meijome Subject: Re: rsync and moving files [Re: backup w/ snapshots] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: questions@freebsd.org List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Aug 2005 07:32:11 -0000 * Greg Barniskis [2005-08-29 11:45 -0500] > Eh? Bad assumptions about snapshots, I think. If a snapshot occupied even a > tenth of the space of the data that it represented, we would quickly fill all > our disks and the snapshot technology would be almost as painful as useful. > > A snapshot is essentially only an index of occupied disk space, not a copy of > the actual data, and a snapshot is therefore much, much, much, much smaller > than the data files that have changed. Read the relevant man pages and > handbook sections again, and test your assumptions by measuring the actual > change in snapshot size. I don't think your perceived problem really exists. Yes, that's correct! But let's say I keep more than one snapshot around. I maybe didn't mention this, but this the sole purpose of using snapshots; for me to have more full backups laying around. If I change the disk alot between snapshots. Eg. I rsync moved files (yes, within tha same fs), this will result in alot of file deletion and creation. Next, when I make the snapshot, a new list of occupied diskspace will be made, and all of these blocks will be marked "in use", and therefore take up alot of diskspace. In reality the information change between the two snapshots, didn't change much at all, but the effect remains: my disk cannot longer store two snapshots (unless the backup disk is twice as large, which it is not). The solution: Somehow, I need to mirror all the move ops on the remote system before doing the rsync. This could probably be done by making a hash table of inodes/filenames pairs (or triplets, etc) each time i sync. Then the next time, I could compare the old table with the new, to find out which files are the same only with new names, then find those names on the remote system, change them to the new ones, and then rsyncing. If the inodes are recycled for brand new files between syncs, I don't think that would be a problem. The following rsync-job would recognize the diffs and sync that, which it would have done anyway, if the file is new. What do you think?