From owner-freebsd-questions@FreeBSD.ORG Wed Dec 17 10:35:11 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93C261065673 for ; Wed, 17 Dec 2008 10:35:11 +0000 (UTC) (envelope-from bernard@dugas-family.org) Received: from smtp11.ispronet.com (smtp11.ispronet.com [81.28.196.233]) by mx1.freebsd.org (Postfix) with ESMTP id 4F2C38FC16 for ; Wed, 17 Dec 2008 10:35:10 +0000 (UTC) (envelope-from bernard@dugas-family.org) Received: from localhost (unknown [127.0.0.1]) by smtp11.ispronet.com (Postfix) with ESMTP id C3E93B846; Wed, 17 Dec 2008 11:35:09 +0100 (CET) Received: from smtp11.ispronet.com ([127.0.0.1]) by localhost (smtp11.ispronet.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 64081-04; Wed, 17 Dec 2008 11:35:00 +0100 (CET) Received: from [192.168.1.59] (unknown [81.28.194.61]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp11.ispronet.com (Postfix) with ESMTP; Wed, 17 Dec 2008 11:35:00 +0100 (CET) Message-ID: <4948D650.8040603@dugas-family.org> Date: Wed, 17 Dec 2008 11:37:04 +0100 From: Bernard Dugas User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; fr-FR; rv:1.7) Gecko/20040616 X-Accept-Language: fr,en MIME-Version: 1.0 To: Mel References: <4947B8AB.7000304@dugas-family.org> <200812170947.12794.fbsd.questions@rachie.is-a-geek.net> In-Reply-To: <200812170947.12794.fbsd.questions@rachie.is-a-geek.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at ispronet.com Cc: freebsd-questions@freebsd.org Subject: Re: Extracting changed files list from snapshot X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 17 Dec 2008 10:35:11 -0000 Mel wrote: > On Tuesday 16 December 2008 15:18:19 Bernard Dugas wrote: >>I want to extract the list of files changed between 2 snapshots, to be >>able to do efficient backups. > > Just use dump(8)? Yes in first step, may be. But for fast replication of changes ? Imagine i have a 1TB drive with 1 partition on serverA, and i want it duplicated on another 1TB drive serverB, with 1Gbps network between them, so network speed is not a problem. I begin with dump or any copy mean to have the 2 drives having the partitions in the same state, from a first "snapshot image 0" which is the reference on serverA, made with mksnap_ffs. A user change only 1 small file on serverA. After 1mn, i create "snapshot0+1" file on serverA. To find the files changed on serverA between snapshot0 and snapshot0+1, any current dump or equivalent will have to read all file records of 1TB partition, inside snapshot0+1 complete view of partition, to find 1 changed file. Even on nice day, it may take more than 1mn. But the beauty and efficiency of snapshot concept is that snapshot0 file (internal structure) contains a reference to all data blocks changed between snapshot0 and snapshot0+1 : this is why a snapshot file is so small and efficient. It is certainly possible to find in snapshot0 file inside structure the list of blocks changed until snapshot0+1 was made, and then find which file is the owner of the changed blocks. Then it will appear that only 1 file has been changed on whole partition and we just have to send this changed file from snapshot0+1 view to serverB. So using the internal structure of snapshots file is far more efficient than reading all directory records for last change time... Please note that i try to use the expressions "snapshot file" and "snapshot view" very cautiosly, because they don't mean the same at all :-) > And what snapshots do you mean? As in mksnap_ffs? Cause that's described > in /usr/src/sys/ufs/ffs/ffs_snapshot.c: > 127 TAILQ_HEAD(snaphead, inode); > 128 > 129 struct snapdata { > 130 struct snaphead sn_head; > 131 daddr_t sn_listsize; > 132 daddr_t *sn_blklist; > 133 struct lock sn_lock; > 134 }; > 135 > > and not exposed to userland. Thanks, this is a good hint ! Just needing some doc and help to understand how to use it : i'm more in design now, programmation skills are far away :-( Best regards, -- Bernard DUGAS Mobile +33 615 333 770