Date: Mon, 9 Sep 2019 23:46:03 +0100 From: Matthew Seaman <matthew@FreeBSD.org> To: freebsd-questions@freebsd.org Subject: Re: Poudriere tips Message-ID: <9a3aecb9-704e-f195-9db7-526f11b75841@FreeBSD.org> In-Reply-To: <a91ce358-d2de-3ade-3ebe-64349ef2cae7@ifdnrg.com> References: <a91ce358-d2de-3ade-3ebe-64349ef2cae7@ifdnrg.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --YHh7Y2hdUlL34ixMcwGXj73otZuJyWNhV Content-Type: multipart/mixed; boundary="pGKfeCicIXLcwSwejtmFvCx2WK3e8GmyH"; protected-headers="v1" From: Matthew Seaman <matthew@FreeBSD.org> To: freebsd-questions@freebsd.org Message-ID: <9a3aecb9-704e-f195-9db7-526f11b75841@FreeBSD.org> Subject: Re: Poudriere tips References: <a91ce358-d2de-3ade-3ebe-64349ef2cae7@ifdnrg.com> In-Reply-To: <a91ce358-d2de-3ade-3ebe-64349ef2cae7@ifdnrg.com> --pGKfeCicIXLcwSwejtmFvCx2WK3e8GmyH Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: quoted-printable On 09/09/2019 20:58, Paul Macdonald via freebsd-questions wrote: > After many years of procrastination, i finally have a poudriere system > setup That's some expert level procrastination there... > It wasn't actually that hard to do, and i already wonder why i didn't d= o > it ages ago. >=20 > I'd be grateful if anyone in the group had any tips to share that i can= > benefit from, before learning the hard way? You're using poudriere to build your own package repo, rather than eg. as a testing stage in port maintenance? Well: tips. Try these for size. * Create your ports tree based on your pre-existing checked out version of the ports in /usr/ports: ``` % poudriere ports -l PORTSTREE METHOD TIMESTAMP PATH default svn 2019-09-08 11:26:04 /usr/ports ``` (You don't have to use svn as the method -- any available method will wor= k) * Then create a link from /usr/local/etc/poudriere.d like so: ``` % ls -l /usr/local/etc/poudriere.d/options lrwxr-xr-x 1 root wheel 13 Dec 24 2012 /usr/local/etc/poudriere.d/options@ -> /var/db/ports ``` This means that instead of using `poudriere options` to set the build options for your local ports, you can just change to the appropriate directory under /usr/ports and `make config` * If you're building ports for a number of machines of varying versions, you'll need a poudriere jail for each major version of FreeBSD your machines are running, and that jail should be running a release version as old as (or older) than the earliest version you have on each major branch. * Contrary-wise, the version of the OS you run on your poudriere build box must be newer than (or at least as new as) the most modern poudriere jail you have. So a recent 12-STABLE machine could have poudriere jails for 12.0-RELEASE and 11.2-RELEASE, but not HEAD. Packages built on an 11.2 jail will work just fine on an 11.3 system, but not necessarily the converse: packages built in an 11.3 jail may not work properly on an 11.2 system(*). * Of course, if you're building packages for a single machine, then just make the poudriere jail the same version as your machine. * Use ccache. * ccache defaults to a 5GB maximum cache size. Depending on how many packages you're building this may not be enough. Keep an eye on your ccache stats over a few weeks of package building to see if enlarging the cache would be useful. * Use CHECK_CHANGED_OPTIONS=3Dverbose in poudriere.conf * Use CHECK_CHANGED_DEPS=3Dyes in poudriere.conf * Given the above, you will rarely need to do a 'poudriere bulk -c' -- the vast majority of the time poudriere will upgrade just the packages it needs to with an incremental 'poudriere bulk'. Even if there are big changes like the recent switch of the default version of python from 2.7 to 3.6. You may end up with some older (eg. python27) packages still in your repo, but that's generally not a problem. * One thing that will always trigger a complete rebuild of all packages (effectively a bulk -c) is applying any updates to the poudriere jail, even if those are eg. kernel security patches (which are irrelevant for jails). You don't need to be religious about patching your poudriere jails since they aren't an exposed attack surface. Unless, that is, the security patches apply to system libraries /and/ you are building software with static linkage. * Use ATOMIC_PACKAGE_REPOSITORY=3Dyes and COMMIT_PACKAGES_ON_FAILURE=3Dye= s so that you can still benefit from all the packages that succeeded in building even if some of them did fail. * Even if your poudriere jail is several patch-levels older than it might be, the ability to keep all your installed packages up to date easily will still pay dividends in helping you keep all your servers properly secured. * If you aren't building many (ie. less than about a thousand) packages, I find that enabling the display of packages still to be built in the poudriere web interface (HTML_TRACK_REMAINING=3Dyes in poudriere.conf) is= useful, and with the relatively small number of packages it doesn't have a significant effect on performance. * Watch out for packages that have BUILD/RUN depends on llvm*, openjdk or a number of other monster packages. Those can easily take as long to build as all of your other packages put together, and may suck up great gobs of system resources if you try and make them build quicker by allowing them to use multiple make jobs. Just be patient -- they'll get there in the end. Cheers, Matthew (*) As a general rule. It could well be that the two specific versions mentioned /don't/ have any such problem -- I haven't tested so I can't be certain. --pGKfeCicIXLcwSwejtmFvCx2WK3e8GmyH-- --YHh7Y2hdUlL34ixMcwGXj73otZuJyWNhV Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEEGfFU7L8RLlBUTj8wAFE/EOCp5OcFAl121itfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDE5 RjE1NEVDQkYxMTJFNTA1NDRFM0YzMDAwNTEzRjEwRTBBOUU0RTcACgkQAFE/EOCp 5OeURQ//aNx6gqECVWangqUQiuIvOhEPsjrrVbAhyyCqemAwUbqWD/t0XQhQjO7g q5p6coL6Rtl/4FAg1ZpBOGmuN37khbVfSRbBfxZFPTpgajdKXgYODSSykWidesPL zliimp3wSHp3NnGz09Z7UVlSxDHYGOR3LKMip6EkL7xxci6Z/5EkrnFixaXFvZhN Fvae5m4LJOlBRkD6BeU3mTNSgYlNpoE6Mah8i0UVmt4+trYi2eRbgToyFrJ8+1Lo PD7rREHQSbA5uTaLsadt3IgthqX7UO6iqxgGvgzgm7LD1+WexEKIgLBisfqMfie+ N8f2Yhi5N0TNA3P436FtU4ruZXtTSfwv7zmoAwZY9xrvfRh4rX1PBC2Jio/MxitH CE2G5om7rifDvgcGnPbrKpo281ji/3z84kWd4OreWlEpr/+lBP02uLJ+jwjmSryc i7n1slRcJ3dT2fys6VyCTJD30/wnjpLcW2tJN1tNiqb2FE41SkjZ69JTvHnQ/UuW U0ePlvIEcnX7cYfd8tjhY8uq0nnO9QySJo54rOLUI+SfGQLd4/A4Y4aNsQKT2QAv XUjEmZ8FICfQaUtfXTxHJ2u1oY5Qu8I1HPrKRUSDAbuOkek1/DEW1QVZmgUWnhrB qifUX3AFdWvE8EBeHLyhi2Slf7OFdsno1VKcnnqDqeb7APraFAU= =mnrM -----END PGP SIGNATURE----- --YHh7Y2hdUlL34ixMcwGXj73otZuJyWNhV--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9a3aecb9-704e-f195-9db7-526f11b75841>