Date: Wed, 12 May 2021 15:02:49 -0700 From: Bryan Drewery <bdrewery@FreeBSD.org> To: Mark Millard <marklmi@yahoo.com>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Cc: FreeBSD ports <freebsd-ports@freebsd.org> Subject: Re: ports-mgmt/poudriere-devel, lang/rust (for example), and USE_TMPFS that includes wrkdir (or yes) Message-ID: <1ed96ea6-3886-7a9d-e54b-d47aebfbe5ac@FreeBSD.org> In-Reply-To: <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> References: <8534A936-B9E6-4BF1-AF26-7F214D359E5C.ref@yahoo.com> <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v Content-Type: multipart/mixed; boundary="FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9"; protected-headers="v1" From: Bryan Drewery <bdrewery@FreeBSD.org> To: Mark Millard <marklmi@yahoo.com>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Cc: FreeBSD ports <freebsd-ports@freebsd.org> Message-ID: <1ed96ea6-3886-7a9d-e54b-d47aebfbe5ac@FreeBSD.org> Subject: Re: ports-mgmt/poudriere-devel, lang/rust (for example), and USE_TMPFS that includes wrkdir (or yes) References: <8534A936-B9E6-4BF1-AF26-7F214D359E5C.ref@yahoo.com> <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> In-Reply-To: <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> --FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable There is no solution at the moment and is a common complaint (about concurrent large builds). I had TMPFS_LIMIT=3D15 and had rust fail on me from that. Quite large... Bryan On 5/10/2021 10:19 PM, Mark Millard wrote: > I've been using USE_TMPFS=3Dyes (so "wrkdir data") on > various systems, both ZFS (recently) and UFS > (generally, even now). Only one system builds rust > (in order for something else to be built), at least > so far. >=20 > An example of the wrkdirs tmpfs use for rust is > (UFS context): >=20 > # df -m | grep tmpfs > Filesystem 1M-blocks Used Avail Capacity Mounted on > . . . > tmpfs 301422 17859 283563 6% /usr/local/poudriere/data/= =2Em/FBSDFSSDjail-default/01/wrkdirs > . . . >=20 > This was near the end but the maximum figure was probably > somewhat higher than the 17 GiByte+ figure above. The > context the example is from is for the only large capacity > build machine that I have access to, an amd64 context. I > have other build contexts as well, but, so far, none have > had to deal with building rust. >=20 > Rust likely would fit the 8 GiByte RAM + 24 GiByte swap > aarch64 build context with USE_TMPFS including wrkdir if > it was the only builder running at the time. But the > existing builds for the context allow 4 builders in > parallel, one per core. [This deals just fine with > llvm10, llvm11, llvm12, and, gcc10 (no bootstrap) being > what happens to build in parallel, even with USE_TMPFS > that includes wrkdir. Rust is just uses more space all > by itself.] >=20 > If I end up with something that requires rust for the > aarch64 builder context, is there a different technique > to deal with the tradeoff other than giving up on > USE_TMPFS spanning wrkdir for all other other > ports/builder-instances as well, presuming the same > media and partitioning (such as total swap space)? >=20 > Imaginary examples could be: >=20 > A) Tell poudriere that lang/rust is to be built by itself > despite the general 4-builder context. >=20 > B) Tell poudriere that USE_TMPFS excludes wrkdir for > lang/rust's specific builder. >=20 > C) . . . (good question) . . . >=20 > So far all I've come up with is explicitly building > lang/rust by itself first, a form of (A): >=20 > # poudriere bulk -jNAME -w lang/rust > # poudriere bulk -jNAME -w -f ~/origins/CA72-origins.txt >=20 > (Hopefully, reliably remembering to do so.) >=20 > Is there any better technique that I've not noticed? >=20 > To some extent here, lang/rust is being used an example > of a more general issue: Other ports could have similar > issues with attempted wrkdir-included USE_TMPFS use. >=20 > Note: If I build using WITH_DEBUG, the one system that > I have access to that can build such a lang/rust with > workdir included in USE_TMPFS shows over 130 GiBytes > in the tmpfs earn the end of the builder's activity. > (This is a amd64 context with 128 GiBytes of RAM and > 192 GiBytes of swapping/paging space.) >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) >=20 --=20 Regards, Bryan Drewery --FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9-- --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEE+Rc8ssOq6npcih8JNddxu25Gl88FAmCcUIkFAwAAAAAACgkQNddxu25Gl8+j JQgAzzE7R9yf5z+A/aA6+2Tr5P0oGkjXPVJ1FGd4+lOW90EVbL/fbAW91OSkh+rAdLAzG/yLm05J J6OAfuIpcdIEWYiU+Bv0DI8qgOqAmK7dSUWFA5hbm5fZZQ58Vp8DVv4NWnFErNtJQj0Blo+wPCre h1pP1E99kjnPawb6BsiCE2GrkmGN/oqBCp6Fjl0KKXVOkQ8Ogxj4nvm4WLZEtFRHoHUTVJMZdr95 kImLdGf7/nmmTsJgAlZMXtwRDKWovSHYfojS9gPHkPCdPol3HvvL+ak8yMhcKcytBqyuc0q/Rkag 8ygQHR4tnoU+iRHTvoD9KOLkHeUbUjkoMsWMeM1ENQ== =IL1t -----END PGP SIGNATURE----- --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1ed96ea6-3886-7a9d-e54b-d47aebfbe5ac>