From owner-freebsd-fs@FreeBSD.ORG Mon May 5 13:21:37 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D6F81065683 for ; Mon, 5 May 2008 13:21:37 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 2A6178FC18 for ; Mon, 5 May 2008 13:21:37 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.storspeed.com (209-163-168-124.static.tenantsolutions.net [209.163.168.124] (may be forged)) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id m45DLTwB075638; Mon, 5 May 2008 08:21:30 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-Id: From: Eric Anderson To: ticso@cicely.de In-Reply-To: <20080503125050.GG40730@cicely12.cicely.de> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v919.2) Date: Mon, 5 May 2008 08:21:29 -0500 References: <48070DCF.9090902@fsn.hu> <4CA7BA82-E95C-45FF-9B94-8EF27B6DB024@freebsd.org> <20080503125050.GG40730@cicely12.cicely.de> X-Mailer: Apple Mail (2.919.2) X-Spam-Status: No, score=-2.2 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: freebsd-fs@freebsd.org Subject: Re: Consistent inodes between distinct machines X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 May 2008 13:21:37 -0000 On May 3, 2008, at 7:50 AM, Bernd Walter wrote: > On Fri, May 02, 2008 at 03:40:11PM -0500, Eric Anderson wrote: >> On Apr 17, 2008, at 3:43 AM, Attila Nagy wrote: >> >>> Hello, >>> >>> I have several NFS servers, where the service must be available >>> 0-24. The servers are mounted read only on the clients and I've >>> solved the problem of maintaining consistent inodes between them by >>> rsyncing an UFS image and mounting it via md on the NFS servers. >>> The machines have a common IP address with CARP, so if one of them >>> falls out, the other(s) can take over. >>> >>> This works nice, but rsyncing multi gigabyte files are becoming more >>> and more annoying, so I've wondered whether it would be possible to >>> get constant inodes between machines via alternative ways. >> >> >> Why not avoid syncing multi-gigabyte files by splitting your huge FS >> image into many smaller say 512MB files, then use md and geom concat/ >> stripe/etc to make them all one image that you mount? > > Where would be the positive effect by doing this? > FFS distributes data over the media, so all the small files changes > in almost every case and you have to checksum-compare the whole > virtual > disk anyway. > With multiple files the syncing is more complex. For example a normal > rsync run can garantie that you get a complete file synced or none > at all, but this doesn't work out of the box with multiple files, so > you risk half updated data. The positive effect is when your image size is smaller than the cylinder group size, so every image is not getting changes. The smaller your image, the better the efficiency, but the more difficult the concat becomes. Possibly another way is to mirror devices over a ggate or iscsi link. Eric