From owner-freebsd-toolchain@freebsd.org Wed Jan 24 18:07:05 2018 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A342BEB93A0 for ; Wed, 24 Jan 2018 18:07:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47B6F6D150; Wed, 24 Jan 2018 18:07:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (unknown [127.0.1.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 16CC2623A; Wed, 24 Jan 2018 18:07:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 4911FA270; Wed, 24 Jan 2018 18:07:04 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id FiBD4TIPz86r; Wed, 24 Jan 2018 18:07:01 +0000 (UTC) Subject: Re: parallel_run start_builder and clonefs for UFS in ports-mgmt/poudriere-devel: avoiding elapsed-time spent in the getblk wait channel, sleep a little DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com 70CBEA26B To: Mark Millard , FreeBSD Toolchain References: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net> From: Bryan Drewery Openpgp: id=F9173CB2C3AAEA7A5C8A1F0935D771BB6E4697CF; url=http://www.shatow.net/bryan/bryan2.asc Organization: FreeBSD Message-ID: <1882357c-bdfc-31dd-6590-5730e0fbdace@FreeBSD.org> Date: Wed, 24 Jan 2018 10:07:01 -0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jan 2018 18:07:05 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA Content-Type: multipart/mixed; boundary="I5UFMREXw2R7dESlInDRhiWGJKANpKr7L"; protected-headers="v1" From: Bryan Drewery To: Mark Millard , FreeBSD Toolchain Message-ID: <1882357c-bdfc-31dd-6590-5730e0fbdace@FreeBSD.org> Subject: Re: parallel_run start_builder and clonefs for UFS in ports-mgmt/poudriere-devel: avoiding elapsed-time spent in the getblk wait channel, sleep a little References: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net> In-Reply-To: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net> --I5UFMREXw2R7dESlInDRhiWGJKANpKr7L Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 12/9/2017 3:15 PM, Mark Millard wrote: > I've been experimenting with using devel/poudriere-devel > in the contexts: >=20 > rpi2, bpi-m3 (Cortex-A7's, 4 core effective, 1 GiByte and 2 GiByte RAM)= > rpi3, Pine64+ 2GB (Cortex-A53's, 4 core, 1 GiByte and 2 GiByte RAM) > powerpc64 (32-bit) powerpc (so-called Quad-Core PowerMacs, 16GiBYte and= 2 GiByte effective RAM) > amd64 (Ryzen Threadripper 16-core/32-thread-total, 128 GiByte RAM) >=20 > As part of this even the large configurations are > currently set up for UFS so that aspect is uniform. >=20 > One thing that I've noted (using the amd64 context > as an example) is a non-linear relationship between > builders and starting/cloning time: >=20 >=20 > [00:00:26] Building 47 packages using 4 builders > [00:00:26] Starting/Cloning builders > [00:00:32] Hit CTRL+t at any time to see build progress and stats >=20 > [00:00:22] Building 49 packages using 14 builders > [00:00:22] Starting/Cloning builders > [00:05:07] Hit CTRL+t at any time to see build progress and stats >=20 > [00:00:33] Building 38 packages using 29 builders > [00:00:33] Starting/Cloning builders > [00:17:22] Hit CTRL+t at any time to see build progress and stats >=20 > Monitoring the longer running ones I see lots of: >=20 > getblk=20 > (Waiting to access file data that is being accessed by another thread) >=20 > So I tried a crude form of staggering the cloning > by adding a 1 second sleep in start_builders > ( /usr/local/share/poudriere/common.sh ): >=20 > start_builders() { > local arch=3D$(injail uname -p) >=20 > msg "Starting/Cloning builders" > bset status "starting_jobs:" > run_hook start_builders start >=20 > bset builders "${JOBS}" > bset status "starting_builders:" > parallel_start > for j in ${JOBS}; do > parallel_run start_builder ${j} ${arch} > sleep 1 > done > parallel_stop >=20 > run_hook start_builders stop > } >=20 > The result for 14 was: >=20 > [00:00:14] Building 23 packages using 14 builders > [00:00:14] Starting/Cloning builders > [00:00:53] Hit CTRL+t at any time to see build progress and stats >=20 > For 23 (I no longer had more to build) the result was: >=20 > [00:00:12] Building 23 packages using 23 builders > [00:00:12] Starting/Cloning builders > [00:04:49] Hit CTRL+t at any time to see build progress and stats >=20 > Trying 5 seconds instead of 1 second for the > sleep time: >=20 > The result for 14 was: >=20 > [00:00:12] Building 23 packages using 14 builders > [00:00:12] Starting/Cloning builders > [00:01:25] Hit CTRL+t at any time to see build progress and stats >=20 > For 23 the result was: >=20 > [00:00:12] Building 23 packages using 23 builders > [00:00:12] Starting/Cloning builders > [00:02:11] Hit CTRL+t at any time to see build progress and stats >=20 > It appears that a few second sleep is of significant > help for having lots of builders. >=20 Interesting. I wonder why exactly... > =3D=3D=3D > Mark Millard > markmi at dsl-only.net >=20 --=20 Regards, Bryan Drewery --I5UFMREXw2R7dESlInDRhiWGJKANpKr7L-- --7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJaaMtFAAoJEDXXcbtuRpfPDeYIAMvJMoKGKsJXhz6y58laToPO HCWO72p3oGL1ThNN0nD2ZwfKUBTuliBB3tSBYXOivSNVVHV6cWhJpHY6zRFIhzQ0 4YjOjNX5sG0pfnLZCGSO0eukN8E751qjizcNCG5u7pG1JEYYECt6FlGumqy6X1Sa oB2wwzalxvLcOmGkhlNwBbSaSsGx+/COsHm6UJv65Oc1Yv16xfATn5gD6mqdjJcg UkKIrgzlnHpOkM+jnR8PU5dNFpIEzanw3tU1PYeZ2jxgQBOre1JZi5WK24bCmKz7 GjznPHwNeJXitR0Rpk/W65kXYooBsgjfmlvOcRsOMMKgyrEVk9l9lO/ZlUyminI= =7JwS -----END PGP SIGNATURE----- --7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA--