Date: Sat, 12 Jun 2021 13:26:16 -0700 From: Jose Quinteiro <freebsd@quinteiro.org> To: ports@freebsd.org Subject: Re: Restraining poudriere Message-ID: <0e365b53-eb19-e751-f688-518d799e0f81@quinteiro.org> In-Reply-To: <20210612175704.GC71089@www.zefox.net> References: <20210612172957.GA71089@www.zefox.net> <F2BE7E84-8290-443C-8C71-D61095139D14@grem.de> <20210612175704.GC71089@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 6/12/21 10:57 AM, bob prohaska wrote: > On Sat, Jun 12, 2021 at 07:36:48PM +0200, Michael Gmelin wrote: >> >> >>> On 12. Jun 2021, at 19:31, bob prohaska <fbsd@www.zefox.net> wrote: >>> >>> ???In playing with poudriere on raspberry pi 3 and 4 it seems to >>> work well on the 8 GB Pi4 but is over-optimistic on the 1 GB Pi3. >>> >>> Can poudriere be configured to tackle packages one at a time? >> >> Yes, see poudriere.conf: >> >> # parallel build support. >> # >> # By default poudriere uses hw.ncpu to determine the number of builders. >> # You can override this default by changing PARALLEL_JOBS here, or >> # by specifying the -J flag to bulk/testport. >> # >> # Example to define PARALLEL_JOBS to one single job >> # PARALLEL_JOBS=1 >> >> -m >> > > I perhaps misunderstood what was meant by "builders", confusing it > with threads. Or maybe cores.... > > Trying it now, hoping to see parallel core use. You won't. Setting PARALLEL_JOBS=1 means exactly one Poudriere worker will run, and that make will not build in parallel. You have to decide whether you want multiple Poudriere jobs, each using a single core at most, one Poudriere job with make(1) potentially using several cores, or some combination of both. The three variables that control this are: PARALLEL_JOBS, as you already know. This is the maximum number of workers Poudriere will launch. It can be less than this if there are multiple dependent packages waiting for a big package to build. This happens a lot with Llvm and the like. Not having ALLOW_MAKE_JOBS set is very frustrating in this case, because the big pig of a build will use one core at most and take forever. ALLOW_MAKE_JOBS and MAKE_JOBS_NUMBER (make.conf). The first one allows make(1) to launch parallel jobs, and the second controls the maximum number of make jobs that will be launched. The problem here is that many builds have lengthy steps that are not concurrent (I'm looking at you, autotools!) A lot of times the actual compile and link steps take a tiny fraction of the total build time. I chose to use a combination of both, and had to experiment with different numbers for PARALLEL_JOBS mad MAKE_JOBS_NUMBER so that a majority of my cores were used for the build, but I did not run out of memory: https://forums.FreeBSD.org/threads/poudriere-never-completes-first-run-without-errors.78000/post-486707 > > Thanks for writing! > > bob prohaska > You're welcome! Jose
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0e365b53-eb19-e751-f688-518d799e0f81>