From owner-freebsd-stable@FreeBSD.ORG Fri Oct 22 14:51:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BB971065674; Fri, 22 Oct 2010 14:51:11 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-ew0-f54.google.com (mail-ew0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 001338FC14; Fri, 22 Oct 2010 14:51:10 +0000 (UTC) Received: by ewy28 with SMTP id 28so492697ewy.13 for ; Fri, 22 Oct 2010 07:51:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:to:cc:subject:references :x-comment-to:date:in-reply-to:message-id:user-agent:mime-version :content-type; bh=qkmL2QPQYfrKCZDDr/DiHE1A+2+4S6V77b6pm5+T2PU=; b=b2+4Po+7hmkg9PRm+0hh+5s5WHPcNBrQ5faDBoYepA8z1789FO31zRgqv9SaEibrJB 1nI4NCJLEU4DVbMBcfQC1mRS/bxGJak5i/vMxlef+i6JjnS2oDi4Crm8cyJoIH9y95zs rBtIXk7qUN7ZRtsh85gMTpRjb2P8DLANu95EA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:to:cc:subject:references:x-comment-to:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=B3W14HAyMoe+KE42QpJUd0578K3cJwb45dFFSYndiz2NBCKmygUAlMRrO5MW31uyfJ dab+DgMwLkB2OLVo4WBwx6AT1ZxtR+xNujeX5o8dSDIlb351L02Nz5i1GApkzSKX8o/s Hu0dCCXInEDmXQrNmGh/LK2vWAcQOEo8OT9eU= Received: by 10.213.82.11 with SMTP id z11mr9014041ebk.29.1287759068172; Fri, 22 Oct 2010 07:51:08 -0700 (PDT) Received: from localhost ([95.69.174.185]) by mx.google.com with ESMTPS id u8sm1470404fah.12.2010.10.22.07.51.04 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 22 Oct 2010 07:51:06 -0700 (PDT) From: Mikolaj Golub To: Pete French References: X-Comment-To: Pete French Date: Fri, 22 Oct 2010 17:51:03 +0300 In-Reply-To: (Pete French's message of "Thu, 21 Oct 2010 13:25:34 +0100") Message-ID: <861v7ii8mg.fsf@kopusha.home.net> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Pawel Jakub Dawidek , freebsd-stable@freebsd.org Subject: Re: hast vs ggate+gmirror sychrnoisation speed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Oct 2010 14:51:11 -0000 On Thu, 21 Oct 2010 13:25:34 +0100 Pete French wrote: PF> Well, I bit the bullet and moved to using hast - all went beautifully, PF> and I migrated the pool with no downtime. The one thing I do notice, PF> however, is that the synchronisation with hast is much slower PF> than the older ggate+gmirror combination. It's about half the PF> speed in fact. PF> When I orginaly setup my ggate configuration I did a lot of tweaks to PF> get the speed good - these copnsisted of expanding the send and PF> receive space for the sockets using sysctl.conf, and then providing PF> large buffers to ggate. Is there a way to control this with hast ? PF> I still have the sysctls set (as the machines have not rebooted) PF> but I cant see any options in hast.conf which are equivalent to the PF> "-S 262144 -R 262144" which I use with ggate PF> Any advice, or am I barking up the wrong tree here ? Currently there are no options in hast.conf to change send and receive buffer size. They are hardcoded in sbin/hastd/proto_tcp4.c: val = 131072; if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_SNDBUF, &val, sizeof(val)) == -1) { pjdlog_warning("Unable to set send buffer size on %s", addr); } val = 131072; if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_RCVBUF, &val, sizeof(val)) == -1) { pjdlog_warning("Unable to set receive buffer size on %s", addr); } You could change the values and recompile hastd :-). It would be interesting to know about the results of your experiment (if you do). Also note there is another hardcoded value in sbin/hastd/proto_common.c /* Maximum size of packet we want to use when sending data. */ #define MAX_SEND_SIZE 32768 that looks like might affect synchronization speed too. Previously we had 128kB here but this has been changed to 32Kb because it was reported about slow synchronization with MAX_SEND_SIZE=128kB. http://svn.freebsd.org/viewvc/base?view=revision&revision=211452 I wonder couldn't slow synchronization with MAX_SEND_SIZE=131072 be due to SO_SNDBUF/SO_RCVBUF be equal to this size? May be increasing SO_SNDBUF/SO_RCVBUF we could reach better performance with MAX_SEND_SIZE=128kB? -- Mikolaj Golub