From owner-freebsd-fs@FreeBSD.ORG  Mon Feb  6 11:52:09 2012
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 67BA9106564A
	for <freebsd-fs@freebsd.org>; Mon,  6 Feb 2012 11:52:09 +0000 (UTC)
	(envelope-from daniel@digsys.bg)
Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.3.230])
	by mx1.freebsd.org (Postfix) with ESMTP id E8DBE8FC1A
	for <freebsd-fs@freebsd.org>; Mon,  6 Feb 2012 11:52:08 +0000 (UTC)
Received: from dcave.digsys.bg (dcave.digsys.bg [192.92.129.5])
	(authenticated bits=0)
	by smtp-sofia.digsys.bg (8.14.5/8.14.5) with ESMTP id q16Bpw48034817
	(version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO)
	for <freebsd-fs@freebsd.org>; Mon, 6 Feb 2012 13:52:04 +0200 (EET)
	(envelope-from daniel@digsys.bg)
Message-ID: <4F2FBEDE.5020403@digsys.bg>
Date: Mon, 06 Feb 2012 13:51:58 +0200
From: Daniel Kalchev <daniel@digsys.bg>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
	rv:10.0) Gecko/20120201 Thunderbird/10.0
MIME-Version: 1.0
To: freebsd-fs@freebsd.org
References: <CA+dUSypg_3uNYMtU2tnvrvAPFw8MjM596tDZ=R_eqpE=GL1-=A@mail.gmail.com>
	<FAE6B79D-6791-4B1F-8E0D-79BEB2765B3B@digsys.bg>
	<4F2FBCD2.6000603@my.gd>
In-Reply-To: <4F2FBCD2.6000603@my.gd>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Subject: Re: HAST considarations
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 11:52:09 -0000



On 06.02.12 13:43, Damien Fleuriot wrote:
> This issue is due to a bug in OpenBSD 3.8's implementation of CARP.
>
> It triggers if you have net.inet.carp.preempt=1 on the node.
>
> If the sysctl is set, the interface assumes MASTERship immediately upon
> being brought up, then yields in the presence of a better master.

I know about this patch, but on my systems

net.inet.carp.preempt=0

I was running 8-stable, now 9-stable on these servers and observe the 
same behavior.

George Kontostanos:

My setup has 1G interfaces for the CARP/Internet and 10G interfaces for the backend/HAST. I am doing hast over the 10G interfaces. For a system with part of 10k SAS drives, and ZFS mirror (each element of the mirror is an HAST provider), running bonnie++ I see about 100MB/sec flow to the secondary HAST and that about saturates the disks as well (50-60MB/sec.. should have been better)
I had earlier experiment with 4 drives in each system and that replicated at up to 230 MB/sec.

Daniel