From owner-freebsd-stable@FreeBSD.ORG Fri Oct 22 18:13:42 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 21415106566C for ; Fri, 22 Oct 2010 18:13:42 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 88DD78FC1D for ; Fri, 22 Oct 2010 18:13:41 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 3596645C99; Fri, 22 Oct 2010 20:13:39 +0200 (CEST) Received: from localhost (chello089073192049.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 3855A45C9F; Fri, 22 Oct 2010 20:13:33 +0200 (CEST) Date: Fri, 22 Oct 2010 20:13:01 +0200 From: Pawel Jakub Dawidek To: Mikolaj Golub Message-ID: <20101022181301.GA2014@garage.freebsd.pl> References: <861v7ii8mg.fsf@kopusha.home.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MGYHOYXEY6WxJCY8" Content-Disposition: inline In-Reply-To: <861v7ii8mg.fsf@kopusha.home.net> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-stable@freebsd.org, Pete French Subject: Re: hast vs ggate+gmirror sychrnoisation speed X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Oct 2010 18:13:42 -0000 --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 22, 2010 at 05:51:03PM +0300, Mikolaj Golub wrote: >=20 > On Thu, 21 Oct 2010 13:25:34 +0100 Pete French wrote: >=20 > PF> Well, I bit the bullet and moved to using hast - all went beautifull= y, > PF> and I migrated the pool with no downtime. The one thing I do notice, > PF> however, is that the synchronisation with hast is much slower > PF> than the older ggate+gmirror combination. It's about half the > PF> speed in fact. What speed do you expect? IIRC from my tests, I was able to saturate 1Gbit link with initial synchronization. Also note, that hast synchronize only differences, and not the entire thing after crash or power failure. > PF> When I orginaly setup my ggate configuration I did a lot of tweaks to > PF> get the speed good - these copnsisted of expanding the send and > PF> receive space for the sockets using sysctl.conf, and then providing > PF> large buffers to ggate. Is there a way to control this with hast ? > PF> I still have the sysctls set (as the machines have not rebooted) > PF> but I cant see any options in hast.conf which are equivalent to the > PF> "-S 262144 -R 262144" which I use with ggate >=20 > PF> Any advice, or am I barking up the wrong tree here ? >=20 > Currently there are no options in hast.conf to change send and receive bu= ffer > size. They are hardcoded in sbin/hastd/proto_tcp4.c: >=20 > val =3D 131072; > if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_SNDBUF, &val, > sizeof(val)) =3D=3D -1) { > pjdlog_warning("Unable to set send buffer size on %s", ad= dr); > } > val =3D 131072; > if (setsockopt(tctx->tc_fd, SOL_SOCKET, SO_RCVBUF, &val, > sizeof(val)) =3D=3D -1) { > pjdlog_warning("Unable to set receive buffer size on %s",= addr); > } >=20 > You could change the values and recompile hastd :-). It would be interest= ing > to know about the results of your experiment (if you do). I'd also like to know if it improves anything. > Also note there is another hardcoded value in sbin/hastd/proto_common.c >=20 > /* Maximum size of packet we want to use when sending data. */ > #define MAX_SEND_SIZE 32768 >=20 > that looks like might affect synchronization speed too. Previously we had= 128kB > here but this has been changed to 32Kb because it was reported about slow > synchronization with MAX_SEND_SIZE=3D128kB. The same restriction exists in ggate. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --MGYHOYXEY6WxJCY8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkzB1CwACgkQForvXbEpPzR0+wCg1uu30r442bGwo+N3Vm5VUbVI 6xoAoKUUQsXZc1aaOFmzwQzr9fNdHCAo =2gQ7 -----END PGP SIGNATURE----- --MGYHOYXEY6WxJCY8--