From owner-freebsd-hackers@FreeBSD.ORG Thu Jul 10 13:10:18 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F1E2D37B401 for ; Thu, 10 Jul 2003 13:10:17 -0700 (PDT) Received: from cirb503493.alcatel.com.au (c20257.belrs2.nsw.optusnet.com.au [198.142.180.81]) by mx1.FreeBSD.org (Postfix) with ESMTP id 690C243F75 for ; Thu, 10 Jul 2003 13:10:16 -0700 (PDT) (envelope-from peterjeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1])h6AKAE1J004203 for ; Fri, 11 Jul 2003 06:10:14 +1000 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.8/8.12.8/Submit) id h6AKADLE004202 for freebsd-hackers@freebsd.org; Fri, 11 Jul 2003 06:10:13 +1000 (EST) Date: Fri, 11 Jul 2003 06:10:13 +1000 From: Peter Jeremy To: freebsd-hackers@freebsd.org Message-ID: <20030710201013.GA1892@cirb503493.alcatel.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i Subject: Mirroring using vinum+NFS X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2003 20:10:18 -0000 I'm looking at building a fileserver and want to mirror the data across two systems (if one fails, I can use the other). Consider system A as the server and system B as the mirror. In theory, on system A I should be able to: mount B:/big/data/blob /remote vnconfig /big/data/block as vn0 vnconfig /remote as vn1 vinum mirror vn0+vn1 as serverdata mount /dev/vinum/serverdata /serverdata export /serverdata (The reason for running the local filesystem through a vnode mount is to make system A and system B interchangeable). Does anyone see any problems with this? Any ideas on how much overhead a 50-100GB vnode mount adds over a raw device? Can anyone suggest the behaviour if system B fails? Can vinum treat this as a "failed disk" or will it get caught by the NFS I/O blocking until the failed server returns? Does anyone have any alternative suggestions? It would probably be enough to watch all local writes and mirror them onto the remote system. This approach might even be cleaner than a true mirror but I don't believe I can do this without a fair amount of local development (watching writes either using kqueue or a custom geom module and queueing them into a second filesystem via some protocol). The volume of data (size and number of files) makes an rsync approach fairly impractical. Peter