Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 22 Oct 2010 20:13:01 +0200
From:      Pawel Jakub Dawidek <pjd@FreeBSD.org>
To:        Mikolaj Golub <to.my.trociny@gmail.com>
Cc:        freebsd-stable@freebsd.org, Pete French <petefrench@ticketswitch.com>
Subject:   Re: hast vs ggate+gmirror sychrnoisation speed
Message-ID:  <20101022181301.GA2014@garage.freebsd.pl>
In-Reply-To: <861v7ii8mg.fsf@kopusha.home.net>
References:  <E1P8uD8-000Hn3-Ae@dilbert.ticketswitch.com> <861v7ii8mg.fsf@kopusha.home.net>

next in thread | previous in thread | raw e-mail | index | archive | help

--MGYHOYXEY6WxJCY8
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Oct 22, 2010 at 05:51:03PM +0300, Mikolaj Golub wrote:
>=20
> On Thu, 21 Oct 2010 13:25:34 +0100 Pete French wrote:
>=20
>  PF> Well, I bit the bullet and moved to using hast - all went beautifull=
y,
>  PF> and I migrated the pool with no downtime. The one thing I do notice,
>  PF> however, is that the synchronisation with hast is much slower
>  PF> than the older ggate+gmirror combination. It's about half the
>  PF> speed in fact.

What speed do you expect? IIRC from my tests, I was able to saturate
1Gbit link with initial synchronization. Also note, that hast
synchronize only differences, and not the entire thing after crash or
power failure.

>  PF> When I orginaly setup my ggate configuration I did a lot of tweaks to
>  PF> get the speed good - these copnsisted of expanding the send and
>  PF> receive space for the sockets using sysctl.conf, and then providing
>  PF> large buffers to ggate. Is there a way to control this with hast ?
>  PF> I still have the sysctls set (as the machines have not rebooted)
>  PF> but I cant see any options in hast.conf which are equivalent to the
>  PF> "-S 262144 -R 262144" which I use with ggate
>=20
>  PF> Any advice, or am I barking up the wrong tree here ?
>=20
> Currently there are no options in hast.conf to change send and receive bu=
ffer
> size. They are hardcoded in sbin/hastd/proto_tcp4.c:
>=20
>         val =3D 131072;
>         if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_SNDBUF, &val,
>             sizeof(val)) =3D=3D -1) {
>                 pjdlog_warning("Unable to set send buffer size on %s", ad=
dr);
>         }
>         val =3D 131072;
>         if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_RCVBUF, &val,
>             sizeof(val)) =3D=3D -1) {
>                 pjdlog_warning("Unable to set receive buffer size on %s",=
 addr);
>         }
>=20
> You could change the values and recompile hastd :-). It would be interest=
ing
> to know about the results of your experiment (if you do).

I'd also like to know if it improves anything.

> Also note there is another hardcoded value in sbin/hastd/proto_common.c
>=20
>  /* Maximum size of packet we want to use when sending data. */
> #define MAX_SEND_SIZE   32768
>=20
> that looks like might affect synchronization speed too. Previously we had=
 128kB
> here but this has been changed to 32Kb because it was reported about slow
> synchronization with MAX_SEND_SIZE=3D128kB.

The same restriction exists in ggate.

--=20
Pawel Jakub Dawidek                       http://www.wheelsystems.com
pjd@FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!

--MGYHOYXEY6WxJCY8
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.14 (FreeBSD)

iEYEARECAAYFAkzB1CwACgkQForvXbEpPzR0+wCg1uu30r442bGwo+N3Vm5VUbVI
6xoAoKUUQsXZc1aaOFmzwQzr9fNdHCAo
=2gQ7
-----END PGP SIGNATURE-----

--MGYHOYXEY6WxJCY8--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20101022181301.GA2014>