From owner-freebsd-current@FreeBSD.ORG Fri Jul 18 16:38:18 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1DB1F1065681 for ; Fri, 18 Jul 2008 16:38:18 +0000 (UTC) (envelope-from mvolaski@aecom.yu.edu) Received: from mx1.aecom.yu.edu (mx1.aecom.yu.edu [129.98.1.51]) by mx1.freebsd.org (Postfix) with ESMTP id E6EC98FC0C for ; Fri, 18 Jul 2008 16:38:17 +0000 (UTC) (envelope-from mvolaski@aecom.yu.edu) Received: from draco.aecom.yu.edu (draco.aecom.yu.edu [129.98.1.160]) by mx1.aecom.yu.edu (Postfix) with ESMTP id 70E689F00E0; Fri, 18 Jul 2008 12:06:46 -0400 (EDT) X-AuditID: 816201a0-ab8c9bb0000015ac-f8-4880bf961803 Received: from smtp1.aecom.yu.edu (smtp1.aecom.yu.edu [129.98.1.61]) by draco.aecom.yu.edu (Symantec Mail Security) with ESMTP id 06B83718002; Fri, 18 Jul 2008 12:06:46 -0400 (EDT) Received: from [129.98.90.227] (usseinstein.aecom.yu.edu [129.98.90.227]) by smtp1.aecom.yu.edu (Postfix) with ESMTP id B30AFB6CD; Fri, 18 Jul 2008 12:06:45 -0400 (EDT) Mime-Version: 1.0 Message-Id: Date: Fri, 18 Jul 2008 12:06:15 -0400 To: freebsd-current@freebsd.org From: Maurice Volaski Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Brightmail-Tracker: AAAAAA== Cc: pjd@FreeBSD.org Subject: Would ZFS and gmirror work well together in a two-node failover cluster? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2008 16:38:18 -0000 I am looking to put together a two-node high-availability cluster where each node has identical data storage consisting of a set of internal data drives (separate from the boot drive). I want ZFS to manage the drives as a JDBOD in a RAIDZ2 configuration. Thus, if an individual drive misbehaves or fails, ZFS detects and handles the fault. But I'm also looking to mirror this entire setup in real time to a second identical server. Basically, my question is can this work well on FreeBSD while taking full advantage of ZFS? Specifically, my understanding is that the only way to handle the real time mirror is with gmirror and ggated, but it's not clear how gmirror would interact with ZFS. I am assuming that gmirror operates only on individual drives, so if I had a set of 24 drives on each server, there would be 24 mirrored drive pairs. One concern I have is that this setup could run into trouble with gmirror's potentially sabotaging ZFS's RAIDZ2. For example, when a drive starts failing, won't gmirror see it before ZFS does and take the unfavorable action of substituting the corresponding drive in the failover server in subsequent I/O, leaving ZFS's RAIDZ2 out of the loop? This is just one particular scenario, but in general, it's not entirely clear that it's possible to have fine-grained control of when, how much and in what direction gmirror manages synchronization among drive pairs. -Maurice -- Maurice Volaski, mvolaski@aecom.yu.edu Computing Support, Rose F. Kennedy Center Albert Einstein College of Medicine of Yeshiva University