Date: Thu, 20 May 2021 11:48:40 -0700 From: Mark Millard <marklmi@yahoo.com> To: ml@netfence.it, FreeBSD ports <freebsd-ports@freebsd.org> Subject: Re: Large builds with poudriere Message-ID: <4F3B2B9B-1B55-4500-A93A-2AC765BE4770@yahoo.com> In-Reply-To: <6307B6F4-87CB-4C8C-A3E9-11089B13FDAF@yahoo.com> References: <6307B6F4-87CB-4C8C-A3E9-11089B13FDAF@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-May-20, at 11:11, Mark Millard <marklmi at yahoo.com> wrote: > Andrea Venturoli ml at netfence.it wrote on > Thu May 20 14:54:50 UTC 2021 : >=20 >> On 5/20/21 4:11 PM, Alexis Praga wrote: >>>=20 >>> Thanks for the suggestions. >>>=20 >>>> At least, if your machine is powerful enough, you can edit your >>>> /usr/local/etc/poudriere.conf and list these big ports in >>>> ALLOW_MAKE_JOBS_PACKAGES. >>> I'm using ALLOW_MAKE_JOBS=3Dyes already so it won't help = unfortunately, I think. >>=20 >> Those are not the same things. Check the docs. >>=20 >> Briefly, ALLOW_MAKE_JOBS will allow building several packages at once=20= >> (one per core) with one core dedicated to each. >=20 > Nope: /usr/local/etc/poudriere.conf.sample reports that it enables > more than one process per cpu: >=20 > # By default MAKE_JOBS is disabled to allow only one process per cpu > # Use the following to allow it anyway =20 > # ALLOW_MAKE_JOBS=3Dyes My wording was poor by being incomplete: ALLOW_MAKE_JOBS allows each builder to have as many processes as there are cores/processors. So if multiple builders are allowed (and happen to be in use), then the total number of processes ready to run for building can be bigger than the number of cores/processors, something that shows up in the load = averages. How many builders are allowed can be controlled with: # parallel build support. # =20 # By default poudriere uses hw.ncpu to determine the number of builders. # You can override this default by changing PARALLEL_JOBS here, or # by specifying the -J flag to bulk/testport. #=20 # Example to define PARALLEL_JOBS to one single job # PARALLEL_JOBS=3D1 Looking in /usr/ports/Mk/bsd.port.mk there is also the following, that is part of where some of the "MAKE_JOBS" terminology comes from: # MAKE_JOBS_UNSAFE # - Disallow multiple jobs even when user = set a global override. # To be used with known bad ports. # DISABLE_MAKE_JOBS # - Set to disable the multiple jobs = feature. User settable. # MAKE_JOBS_NUMBER # - Override the number of make jobs to be = used. User settable. # MAKE_JOBS_NUMBER_LIMIT # - Set a limit for maximum number of make = jobs allowed to be # used. >> ALLOW_MAKE_JOBS_PACKAGES will let a single package build using more = cores. >=20 >=20 > Even when MAKE_JOBS is otherwise disabled: >=20 > # List of packages that will always be allowed to use MAKE_JOBS =20= > # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports=20 > # which holdup the rest of the queue to build more quickly. > #ALLOW_MAKE_JOBS_PACKAGES=3D"pkg ccache py*" =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4F3B2B9B-1B55-4500-A93A-2AC765BE4770>