From owner-freebsd-current@FreeBSD.ORG Sat May 21 16:26:23 2011 Return-Path: Delivered-To: current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6E490106566B for ; Sat, 21 May 2011 16:26:23 +0000 (UTC) (envelope-from pawel@dawidek.net) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 1B9488FC12 for ; Sat, 21 May 2011 16:26:22 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7F67F45EA4; Sat, 21 May 2011 17:58:25 +0200 (CEST) Received: from localhost (89-73-195-149.dynamic.chello.pl [89.73.195.149]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5CAC345E5C; Sat, 21 May 2011 17:58:19 +0200 (CEST) Date: Sat, 21 May 2011 17:57:55 +0200 From: Pawel Jakub Dawidek To: Maxim Sobolev Message-ID: <20110521155755.GC3326@garage.freebsd.pl> References: <4DD2CEE7.1070805@sippysoft.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="p2kqVDKq5asng8Dg" Content-Disposition: inline In-Reply-To: <4DD2CEE7.1070805@sippysoft.com> X-OS: FreeBSD 9.0-CURRENT amd64 User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: "current@freebsd.org" Subject: Re: Randomization in hastd(8) synchronization thread X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 May 2011 16:26:23 -0000 --p2kqVDKq5asng8Dg Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, May 17, 2011 at 12:39:19PM -0700, Maxim Sobolev wrote: > Hi Pawel, >=20 > I am trying to use hastd(8) over slow links and one problem is > apparent right now - current approach with synchronizing content > sequentially is not working in this case. What happens is that hastd > hits the first frequently updated block and cannot make any progress > anymore. In my case I have 30GB of dirty space to be synchronized > over just 1mbps uplink. >=20 > The quick fix that I've applied is randomization in the block > selection code. This way eventually all least used blocks will be > synchronized, leaving only hot ones dirty. More effective approach > would be to use some kind of LRU selection algorithm, but > statistical approach would work just as good in this case. >=20 > Please review the patch below: >=20 > http://sobomax.sippysoft.com/activemap.c.diff Hmm, hastd keeps separate bitmap for synchronization. It is stored in am_syncmap field. Blocks that are dirtied during regular writes should not effect on synchronization bitmap and synchronization progress. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com FreeBSD committer http://www.FreeBSD.org Am I Evil? Yes, I Am! http://yomoli.com --p2kqVDKq5asng8Dg Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk3X4QMACgkQForvXbEpPzREzQCfc7d0K2hj9txaPeZs1XpRdAQH UioAoNJx0Oee2AZL37etZ+LD4ukNPAS5 =4ZL3 -----END PGP SIGNATURE----- --p2kqVDKq5asng8Dg--