Date: Fri, 22 Oct 2010 20:13:01 +0200 From: Pawel Jakub Dawidek <pjd@FreeBSD.org> To: Mikolaj Golub <to.my.trociny@gmail.com> Cc: freebsd-stable@freebsd.org, Pete French <petefrench@ticketswitch.com> Subject: Re: hast vs ggate+gmirror sychrnoisation speed Message-ID: <20101022181301.GA2014@garage.freebsd.pl> In-Reply-To: <861v7ii8mg.fsf@kopusha.home.net> References: <E1P8uD8-000Hn3-Ae@dilbert.ticketswitch.com> <861v7ii8mg.fsf@kopusha.home.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--MGYHOYXEY6WxJCY8
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Fri, Oct 22, 2010 at 05:51:03PM +0300, Mikolaj Golub wrote:
>=20
> On Thu, 21 Oct 2010 13:25:34 +0100 Pete French wrote:
>=20
> PF> Well, I bit the bullet and moved to using hast - all went beautifull=
y,
> PF> and I migrated the pool with no downtime. The one thing I do notice,
> PF> however, is that the synchronisation with hast is much slower
> PF> than the older ggate+gmirror combination. It's about half the
> PF> speed in fact.
What speed do you expect? IIRC from my tests, I was able to saturate
1Gbit link with initial synchronization. Also note, that hast
synchronize only differences, and not the entire thing after crash or
power failure.
> PF> When I orginaly setup my ggate configuration I did a lot of tweaks to
> PF> get the speed good - these copnsisted of expanding the send and
> PF> receive space for the sockets using sysctl.conf, and then providing
> PF> large buffers to ggate. Is there a way to control this with hast ?
> PF> I still have the sysctls set (as the machines have not rebooted)
> PF> but I cant see any options in hast.conf which are equivalent to the
> PF> "-S 262144 -R 262144" which I use with ggate
>=20
> PF> Any advice, or am I barking up the wrong tree here ?
>=20
> Currently there are no options in hast.conf to change send and receive bu=
ffer
> size. They are hardcoded in sbin/hastd/proto_tcp4.c:
>=20
> val =3D 131072;
> if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_SNDBUF, &val,
> sizeof(val)) =3D=3D -1) {
> pjdlog_warning("Unable to set send buffer size on %s", ad=
dr);
> }
> val =3D 131072;
> if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_RCVBUF, &val,
> sizeof(val)) =3D=3D -1) {
> pjdlog_warning("Unable to set receive buffer size on %s",=
addr);
> }
>=20
> You could change the values and recompile hastd :-). It would be interest=
ing
> to know about the results of your experiment (if you do).
I'd also like to know if it improves anything.
> Also note there is another hardcoded value in sbin/hastd/proto_common.c
>=20
> /* Maximum size of packet we want to use when sending data. */
> #define MAX_SEND_SIZE 32768
>=20
> that looks like might affect synchronization speed too. Previously we had=
128kB
> here but this has been changed to 32Kb because it was reported about slow
> synchronization with MAX_SEND_SIZE=3D128kB.
The same restriction exists in ggate.
--=20
Pawel Jakub Dawidek http://www.wheelsystems.com
pjd@FreeBSD.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
--MGYHOYXEY6WxJCY8
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (FreeBSD)
iEYEARECAAYFAkzB1CwACgkQForvXbEpPzR0+wCg1uu30r442bGwo+N3Vm5VUbVI
6xoAoKUUQsXZc1aaOFmzwQzr9fNdHCAo
=2gQ7
-----END PGP SIGNATURE-----
--MGYHOYXEY6WxJCY8--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101022181301.GA2014>