Date: Tue, 5 Oct 2021 22:41:38 +0200 From: Piotr Kubaj <pkubaj@anongoth.pl> To: Matthias Andree <mandree@freebsd.org> Cc: ports-committers@freebsd.org, dev-commits-ports-all@freebsd.org, dev-commits-ports-main@freebsd.org, Mathieu Arnold <mat@freebsd.org> Subject: Re: git: fb5f03a87cf4 - main - Mk/bsd.lto.mk: add global LTO support for ports Message-ID: <YVy4gsubg0nsG1Nq@KGPE-D16> In-Reply-To: <44365771-0a1d-7bcb-2cb6-85f592624a3b@FreeBSD.org> References: <202109301834.18UIYKrL013410@gitrepo.freebsd.org> <20211004133056.z7e2iyrhlwprvvvp@aching.in.mat.cc> <YVx84wGVlSr2nmBX@KGPE-D16> <20211005163152.gv2vwvg4nuqwga7q@aching.in.mat.cc> <YVyCEPlZJvdce4Ak@KGPE-D16> <44365771-0a1d-7bcb-2cb6-85f592624a3b@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--tp+hag3RKUfqxW3m Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 21-10-05 22:04:32, Matthias Andree wrote: >=20 > Am 05.10.21 um 18:49 schrieb Piotr Kubaj: > > On 21-10-05 18:31:52, Mathieu Arnold wrote: > >> On Tue, Oct 05, 2021 at 06:27:15PM +0200, Piotr Kubaj wrote: > >>> On 21-10-04 15:30:56, Mathieu Arnold wrote: > >>>> On Thu, Sep 30, 2021 at 06:34:20PM +0000, Piotr Kubaj wrote: > >>>>> The branch main has been updated by pkubaj: > >>>>> > >>>>> URL: https://cgit.FreeBSD.org/ports/commit/?id=3Dfb5f03a87cf432751f= ae1f0ae7f29c9d4fc65917 > >>>>> > >>>>> commit fb5f03a87cf432751fae1f0ae7f29c9d4fc65917 > >>>>> Author: Piotr Kubaj <pkubaj@FreeBSD.org> > >>>>> AuthorDate: 2021-09-30 18:27:50 +0000 > >>>>> Commit: Piotr Kubaj <pkubaj@FreeBSD.org> > >>>>> CommitDate: 2021-09-30 18:27:50 +0000 > >>>>> > >>>>> Mk/bsd.lto.mk: add global LTO support for ports > >>>>> =20 > >>>>> It's well known that LTO provides both performance and size ben= efits for > >>>>> binaries. > >>>>> =20 > >>>>> Add preliminary, opt-in support for global LTO enforcement to p= orts. Ports that > >>>>> provide LTO option on their own and the ones that don't work wi= th LTO will need > >>>>> to set LTO_UNSAFE in the future. > >>>>> =20 > >>>>> PR: 258536 > >>>> > >>>> Not to be picky about approval and all, but this was added to the > >>>> framework, and the framework is maintained by portmgr. When you wan= t to > >>>> add something to it, you must consult with portmgr before anything g= ets > >>>> committed. > >>>> > >>>> In that case, we would have told you not to do it this way, but to m= ake > >>>> this a Mk/Uses/lto.mk. > >>>> > >>>> So please, turn this into a USES=3Dlto. > >>> > >>> I did consult, but no one replied. > >> > >> There is absolutely no maintainer timeout for the framework, you cannot > >> just add code there without explicit approval. > >=20 > > And this is a port of a bigger issue, where portmgr ignores emails unti= l numerously asked for (if one is lucky). > > As one of users wrote in https://bugs.freebsd.org/bugzilla/show_bug.cgi= ?id=3D251117, for which portmgr is assigned, "portmgr@ more and more feels = to me like a black hole, or /dev/null: Anything sent there seems to disappe= ar without effect." > >=20 > > Since it was a change that doesn't change anything out-of-the-box, I de= cided to commit it. > >=20 > >> > >>> IMO adding it to USES is not a good idea, since USES are supposed to = be used per port and my idea was to force LTO for all ports, same way that = SSP already does. > >> > >> All I see in the patch is a USE_LTO knob, and a LTO_UNSAFE one, without > >> any documentation of what it is for, what it does, what it might do, > >> what it is about, or anything else. > > Neither has SSP, I don't see any documentation for it (including commit= er handbook which has just one line regarding USES=3Dkmod at https://docs.f= reebsd.org/en/books/porters-handbook/book/#uses-kmod). >=20 > Piotr, >=20 > While I sympathize with your findings about portmgr@ 'responsiveness' > having had my shares of ignores and brushes, I would tend to agree that > we should not add undocumented knobs anywhere, so: >=20 > 1. please add documentation including motivation OK, just take note that there are several issues with it right now, so it's= definitely not ready for use. E.g. libffi, perl and pkgconf don't build. Other than that, what is the best place for documentation? Do you mean port= er's handbook? >=20 > 2. I wonder about a deployment plan. LTO linking incurs several issues, > such as memory use, disk-space use with fat objects, and concurrency > issues that cause a massive prolongation of linking, which I've often > seen to run a single thread. I think these items need consideration. That's what we have package builders for. AFAIK they run Poudriere without = ALLOW_MAKE_JOBS, so mostly single-threaded. If LTO is ever enabled for everyone, we should still keep a knob to disable= it globally. But I agree that building everything with LTO may require memory upgrades f= or builders. The extreme example is mongodb (has an LTO option enabled by d= efault) which allocates close to 30GB RAM during linking. >=20 > Other than that, I appreciate the initiative to support LTO inside the > framework, to avoid each and every port implementing it by itself, which > will cause a maintenance mess down the road. Thanks. >=20 > Regards, > Matthias --tp+hag3RKUfqxW3m Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEycyIeNkkgohzsoorelmbhSCDnJ0FAmFcuIIACgkQelmbhSCD nJ2wPg//Z5WPf0KRXPNx54hlNJ/tMZJg1Rbr30fhvn8XtuQnQEBdlP07ch8O3KBA QAIB+SYlh1IKnP6A5Pv1OnNk7GSBycaki2fuzZ0VSWP6Kijki+1e/Phm2AgiQLRx wtpJNVyJXyfbe54BpFYDHOkgsa4jjnfU9sQU/irJAivMiL3S418GZdGxBaZ0t6BJ ARhL2tbpwillOh8/lL1KfywNDo29w7sRxSDp2Mdzvj/LYxGFbYBNATxH3FJR1C5F L4JoWpWoLvtfkTegvVuPx8S9fyjYLshklIghbZA+a9BuIY5P9yk7jyWUstrxOSkC OrZsB9uWzwta2Dnmi2ZuCRjWQ/qp3H7khA0gePk8wLJG6vSK7bprlzxtvC3obF8a ZQGeAvY2lL+kcx/eEZUq+AwiMUJEgop3gf5N8zN4HngGZ8WJpuFiwaWGXODo5DkQ UNzIBEKTEIaIwx+YXyEdFEBvXL3m6PNE8Vu1fyXeowUVRtmZYNgXrugk9azUJ+hz 6ZMIZXpnXNbrTYD+XDW+JZMSbTj5gwvfCjUBUSkSjUo5uV0BS/meTNbD6OitWn2E uRsLIbmRUgHVB0cPCJ2+F565X/iLxUZAPAUwoTJUgZyrevooqWw6ke13pQegtiKV p2s05x7n/0qa+FfXx79dlOykoegx+k0klfMJvIElhoS4Z5GDUkc= =XzJI -----END PGP SIGNATURE----- --tp+hag3RKUfqxW3m--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YVy4gsubg0nsG1Nq>