From owner-freebsd-hackers@FreeBSD.ORG  Sat Apr  9 21:19:51 2005
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id BF98D16A4CE
	for <freebsd-hackers@freebsd.org>;
	Sat,  9 Apr 2005 21:19:51 +0000 (GMT)
Received: from ox.eicat.ca (ox.eicat.ca [66.96.30.35])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 7DB4B43D2F
	for <freebsd-hackers@freebsd.org>;
	Sat,  9 Apr 2005 21:19:51 +0000 (GMT)
	(envelope-from dgilbert@daveg.ca)
Received: by ox.eicat.ca (Postfix, from userid 66)
	id 04FA9DA00; Sat,  9 Apr 2005 17:19:50 -0400 (EDT)
Received: by canoe.dclg.ca (Postfix, from userid 101)
	id C5BD41A08C0; Sat,  9 Apr 2005 17:19:43 -0400 (EDT)
From: David Gilbert <dgilbert@dclg.ca>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <16984.18159.732608.234243@canoe.dclg.ca>
Date: Sat, 9 Apr 2005 17:19:43 -0400
To: freebsd-hackers@freebsd.org
X-Mailer: VM 7.17 under 21.4 (patch 17) "Jumbo Shrimp" XEmacs Lucid
Subject: ggate failures.
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 09 Apr 2005 21:19:51 -0000

I have two systems, each with 4 300 gig SATA disks.  Let's call them
m0 and m1.  M1 exports it's disks with ggated ... on two private GigE
networks.  M0, on those same two GigE networks, imports them with
ggatec.  M0, then does the following:

Mirror	 Disks
======	 =====
s0	 ggate0 da0s1g
s1	 ggate1 da1s1g
s2	 ggate2 da2s1g
s3	 ggate3 da3s1g

And then:

concat	Disks
======	=====
v0	s0 s1 s2 s3

(so v0 is a concatination of 4 mirrors that consist of a local and
remote disk, each)

Now... This all works, and we create a filesystem on v0.  The problem
arises that whenever a lot of activity occurs on v0 (untaring a copy
of /usr is sufficient), the ggate links break down.  An example
message from the dmesg:

GEOM_MIRROR: Request failed (error=5). ggate2[WRITE(offset=259891840000, length=8192)]

Now... I don't know a lot about ggate, but this appears trivial to
trigger.  Has anyone tried similar configurations and is there any
wisdom about ggate configurations?

Dave.

-- 
============================================================================
|David Gilbert, Independent Contractor.       | Two things can only be     |
|Mail:       dave@daveg.ca                    |  equal if and only if they |
|http://daveg.ca                              |   are precisely opposite.  |
=========================================================GLO================