Date: Tue, 4 Nov 2014 09:14:59 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Warner Losh <imp@bsdimp.com> Cc: "freebsd-arch@freebsd.org" <arch@freebsd.org> Subject: Re: Why do we have @ in modules builds? Message-ID: <20141104082203.J2282@besplex.bde.org> In-Reply-To: <3285BC54-05D8-41DB-88FE-BAD681A3E45B@bsdimp.com> References: <3285BC54-05D8-41DB-88FE-BAD681A3E45B@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 3 Nov 2014, Warner Losh wrote: > Does anybody recall why we have @ symlink in our module builds? > I=92m constantly working around issues that this creates. Maybe it is tim= e > to eliminate it? It is probably because I finished implementing '@' for modules better than for kernels. wollman introduced '@', but didn't change config(8) to generate it. My version of config does generate it, and I always use it for kernels, but I never committed this. My kernel object tree is under /usr/obj, and my source tree is hacked to point to it. E.g., /sys/i386/compile -> /usr/obj/usr/src/sys/compile (should be to .../sys/i386/compile). '@' then points back to /usr/src/sys. Modules have more varied paths than kernels, so @ is even more useful for them. -current (ref11-i386) also uses /usr/obj, but instead of '@' it uses: % S=3D/usr/src/sys % ... % .if !defined(S) % .if exists(./@/.) % S=3D=09./@ % .else % S=3D=09../../.. % .endif % .endif % .include "$S/conf/kern.pre.mk" %=20 % INCLUDES+=3D -I$S/contrib/libfdt The default S wouldn't work at all. Config doesn't support '@', but suppor= ts 'S' instead. (It seems to only do this when necessary -- it doesn't do it for my source-relative object trees on the same machine.) The generated Makefile then uses $S throughout as before. This hasn't chan= ged since at least wollman's change in ~1993. Using $S instead of its expansio= n keeps the makefile fairly readable. The main advantage of '@' is that it reduces the unreadability of make output and .depend files. When '@' is used, $S expands to just "./@". This keeps individual pathnames fairly short and makes it easier to see the basenames. $S might be much longer than the above, depending on the depth of the source tree and on realpath-like expansion of symlinks. make output remains fairly unreadable due to other spam in it. The spam that it used to have for -D options has been adequately replaced by spam from -W options. Without automatic generation of the paths, finding the path to the sources is fragile. To find the top level directory, kmod.mk still seems to just search .., ../.., /sys and /usr/src/sys. No trace of SRCDIR in the Aug 6 RELENG_11 version. I think I wrote most of that. It is better than nothing. Modularity is also very broken for programs. Most programs now depend on src.opts.mk, which doesn't exist for programs outside of the source tree. src.opts.mk sort of goes in the opposite and worse direction than SRCDIR. With a single source tree, you can hard-code SRCDIR in a single file near src.opts.mk and not have to guess it. Actually, the mk directory is still hard to find. Bruce From owner-freebsd-arch@FreeBSD.ORG Wed Nov 5 09:02:21 2014 Return-Path: <owner-freebsd-arch@FreeBSD.ORG> Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3828C19A for <freebsd-arch@freebsd.org>; Wed, 5 Nov 2014 09:02:21 +0000 (UTC) Received: from mail-wi0-x22c.google.com (mail-wi0-x22c.google.com [IPv6:2a00:1450:400c:c05::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B7A9B8B5 for <freebsd-arch@freebsd.org>; Wed, 5 Nov 2014 09:02:20 +0000 (UTC) Received: by mail-wi0-f172.google.com with SMTP id bs8so11714047wib.11 for <freebsd-arch@freebsd.org>; Wed, 05 Nov 2014 01:02:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=p/5ceOF7Rpm8/x4XQOCwn2j6Ttsr3V6lwsfpS9ItldA=; b=d/RiW/M6fX+We3ICxIUU2cyOzIhfxNtSH2B/iH6hQ71nAHP51GkycawgDgsbZmhTb4 6ZHTuYYo4EOsLlMr6jpcdrwRaQ6IeJIt1FIRJgIs2q24iRYDI0yHQ1szywGZCubHq1kw KYGjVsi0nod2BH7dtadUMnZTDKcb22RAyLPg1WAkJIVkRtZjkFJCORSweuqIY81J2vrk H7gCAgV+6QTkfQL1dbs3uGuldyNbrqyltsB4yokp/Gg+N5r26ljYgv9xekuGeAoAOgn4 LJUNo4RwCLAOJ6+uoUY1lwIgoZ5me4NCThDwAU/8ln/Rz5M5a66/4I+i7eyTdsLrfxCO 2ing== X-Received: by 10.180.94.70 with SMTP id da6mr3930083wib.69.1415178138951; Wed, 05 Nov 2014 01:02:18 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id cv7sm3334849wjc.3.2014.11.05.01.02.17 for <multiple recipients> (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Nov 2014 01:02:17 -0800 (PST) Sender: Baptiste Daroussin <baptiste.daroussin@gmail.com> Date: Wed, 5 Nov 2014 10:02:15 +0100 From: Baptiste Daroussin <bapt@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Subject: Re: PIE/PIC support on base Message-ID: <20141105090215.GF10388@ivaldir.etoilebsd.net> References: <CAMe1fxaYn+JaKzGXx+ywv8F0mKDo72g=W23KUWOKZzpm8wX4Tg@mail.gmail.com> <CAGSa5y3s9r0DRyinfqV=PJc_BT=Em-SLfwhD25nP0=6ki9pHWw@mail.gmail.com> <CAMe1fxaBEc5T77xjpRsMi_kkc5LXwPGooLWTO9C1FJcLSPnO8w@mail.gmail.com> <CAGSa5y2=bKpaeLO_S5W+1YGq02WMgCZn_5bbEMw+x3j-MYDOoA@mail.gmail.com> <CADt0fhzg5G1cLEBNfHXSEi9iP7mCP=8sSwpXbFobig=pm=QsFQ@mail.gmail.com> <CAGSa5y1LBxkUNSgKkw=F9_uykXDeBV7_WL0a7Wt++GgMTSULEQ@mail.gmail.com> <CADt0fhweiymn2D09+e7f44AreWe+8cmAtDVeec0NfmuWuOOhbg@mail.gmail.com> <315B4DC5-0E04-4F6B-BBB0-477D049025BF@bsdimp.com> <CADt0fhyCBa3PTnZ3dpc-hpysyC9V0MXR16s-e10V0ioAfaWHuw@mail.gmail.com> <C7C48B02-E65C-4F90-A503-1FDDCB590B7D@bsdimp.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="udcq9yAoWb9A4FsZ" Content-Disposition: inline In-Reply-To: <C7C48B02-E65C-4F90-A503-1FDDCB590B7D@bsdimp.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: PaX Team <pageexec@freemail.hu>, FreeBSD Arch <freebsd-arch@freebsd.org>, Shawn Webb <lattera@gmail.com> X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture <freebsd-arch.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-arch>, <mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch/>; List-Post: <mailto:freebsd-arch@freebsd.org> List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>, <mailto:freebsd-arch-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 05 Nov 2014 09:02:21 -0000 --udcq9yAoWb9A4FsZ Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Oct 17, 2014 at 08:05:57AM -0600, Warner Losh wrote: > [[cc trimmed ]] >=20 > On Oct 17, 2014, at 7:46 AM, Shawn Webb <lattera@gmail.com> wrote: >=20 > > On Fri, Oct 17, 2014 at 9:41 AM, Warner Losh <imp@bsdimp.com> wrote: > >=20 > > On Oct 17, 2014, at 2:05 AM, Shawn Webb <lattera@gmail.com> wrote: > >=20 > > > On Fri, Oct 17, 2014 at 3:53 AM, Jeremie Le Hen <jlh@freebsd.org> wro= te: > > > > > >> On Fri, Oct 17, 2014 at 12:15 AM, Shawn Webb <lattera@gmail.com> wro= te: > > >>> > > >>> > > >>> On Thu, Oct 16, 2014 at 5:59 PM, Jeremie Le Hen <jlh@freebsd.org> w= rote: > > >>>> > > >>>> On Thu, Oct 16, 2014 at 8:21 PM, David Carlier > > >>>> <david.carlier@hardenedbsd.org> wrote: > > >>>>> > > >>>>> I chose the "atomic" approach, at the moment very few binaries are > > >>>>> concerned at the moment. So I applied INCLUDE_PIC_ARCHIVE in the > > >> needed > > >>>>> libraries plus created WITH_PIE which add fPIE/fpie -pie flags on= ly if > > >>>>> you > > >>>>> include <bsd.prog.pie.mk> (which include <bsd.prog.mk>...) otherw= ise > > >>>>> other > > >>>>> binaries include <bsd.prog.mk> as usual hence does not apply. Look > > >>>>> reasonable approach ? > > >>>> > > >>>> I think I understand what you mean. But I think PIE is commonplace > > >>>> nowadays and I don't understand what you win by not enabling it for > > >>>> the whole system. Is it a performance concern? Is it to preserve > > >>>> conservative minds from to much change? :) > > >>> > > >>> > > >>> Looping in Kostik, Bryan Drewery, the PaX team, Hunger, and Sean Br= uno. > > >>> > > >>> On i386, there is a performance cost due to not having an extra reg= ister > > >>> available for the relocation work that has to happen. PIE doesn't c= arry > > >> much > > >>> of a performance penalty on amd64, though it still does carry some = on > > >> first > > >>> resolution of functions (due to the extra relocation step the RTLD = has to > > >>> worry about). On amd64, after symbol resolution has taken place, th= ere > > >> is no > > >>> further performance penalty due to amd64 having an extra register t= o use > > >> for > > >>> PIE/PIC. I'm unsure what, if any, performance penalty PIE carries o= n ARM, > > >>> AArch64, and sparc64. > > >>> > > >>> Certain folk would prefer to see PIE enabled only in certain > > >> applications. > > >>> /bin/ls can't really make much use of PIE. But sshd can. I personal= ly > > >> would > > >>> like to see all of base's applications compiled as PIEs, but that's= a > > >> long > > >>> ways off. It took OpenBSD several years to accomplish that. Having > > >> certain > > >>> high-visibility applications (like sshd, inetd, etc) is a great sta= rt. > > >>> Providing a framework for application developers to opt their appli= cation > > >>> into PIE is another great start. > > >>> > > >>> Those are my two cents. > > >> > > >> OK. As long as i386 is still an important architecture, it can make > > >> sense to enable this on a per-binary basis if we don't want to have a > > >> discrepancy between archs. Also I buy your argument on /bin/ls but I > > >> was challenging to enable for the whole system because I wonder if > > >> there aren't some unexpected attack surfaces, besides the obvious on= es > > >> (servers). > > >> > > >> Do you know what took so much time to OpenBSD? > > > > > > > > > In a private conversation with Theo, I realized that my recollection = of the > > > time it took OpenBSD to compile all of base as PIEs was wrong. Quotin= g him: > > > > > > "It took 5 people approximately 3 months to debug it, activate it, and > > > start shipping it the next release. That was on amd64, for all > > > dynamically linked binaries, except one (a gcc bug took some time to > > > find). The next architectures followed about 1 or 2 per 6-month > > > release." > > > > > > Given that only one person has worked on this in the past (me) and no= w the > > > task has been delegated to another (David Carlier), I think we're doi= ng > > > okay on our end. There's a lot of moving parts, and neither of us ful= ly > > > understand all of them completely. We're working on it in HardenedBSD= , in > > > the hardened/current/pie branch. > > > > > > I'm thinking we might try for a WITH_PIE knob (and *not* use USE_PIE)= and > > > have certain high-profile applications opt-in to PIE until we work ou= t all > > > the details for everything en masse. Baptiste did bring up a good poi= nt > > > with INTERNALLIB and I'm unsure of how we should handle that. > >=20 > > WITH_PIE or WITHOUT_PIE controls, on a global basis, via the MK_PIE > > variable, whether or not the user wants to turn on this feature for tho= se > > program that can do PIE. Designating which programs do, or don=E2=80=99= t, > > use PIE simply must be done with a different variable. I posted a bit o= f a > > rant about the current state of things that suggested a couple of > > alternatives as well as giving some history as to why some options > > aren=E2=80=99t to be used and the history behind some of my reactions. = :) > >=20 > > For this reason, I think WITH_PIE, as I understand your proposal, > > likely isn=E2=80=99t a good fit with other WITH_xxx variables used in t= he src > > tree today. > >=20 > > Gotcha. To be honest, I found your email a tad bit confusing. Are you s= uggesting we create an ENABLE_feature framework? Or are you suggesting we h= ave a USE_PIE flag? Or are you suggesting something different entirely (and= if you are, what?)? >=20 > I=E2=80=99m saying we don=E2=80=99t have a good framework at the moment t= o do this. We > have several bad ones that all have their pitfalls. This is one reason I = had > the fast reaction to NO_PIE, then a minute later said =E2=80=9Cgo ahead a= nd use > it and I=E2=80=99ll fix it.=E2=80=9D I=E2=80=99m still cool with that pos= ition, btw. >=20 > As for a name, that can be debated a lot, but I=E2=80=99d like to see so= mething > new, easy to use and unambiguous. If you are looking for a suggestion > for that name, let=E2=80=99s go with WANTS_PIE. Only Makefiles can set it. >=20 > WANTS_PIE undefined means do the default behavior as defined by the > current MK_PIE setting and perhaps system policy. =E2=80=9CGo with this f= low." >=20 > WANTS_PIE=3Dyes means that if MK_PIE is =E2=80=9Cyes=E2=80=9D, then do PI= E things for > this thing we=E2=80=99re building. If MK_PIE is =E2=80=9Cno=E2=80=9D, tho= ugh PIE is disabled for > everything. >=20 > WANTS_PIE=3Dno means that if MK_PIE is =E2=80=9Cyes=E2=80=9D, then disabl= e doing PIE > things for this component. If MK_PIE is no, it is also disabled. >=20 > This could also be extended to NEEDS_foo, which says =E2=80=9CI need foo = to > build, and if MK_foo is set to no, don=E2=80=99t build me.=E2=80=9D I don= =E2=80=99t think anything > that you are doing falls into this category though. >=20 > WANTS/NEEDS also avoids the historical use of USE in the ports tree > possibly creating confusion.=20 >=20 > If you go with WANTS_PIE, then you wouldn=E2=80=99t need bad.*.pie.mk. >=20 > Comments? >=20 > Warner On amd64 WANTS_PIE will be useless as we can easily activate PIE on every p= laces For i386 we would propably prefer cherry picking the what we want to see bu= ilt with PIE. Don't know for other arches. So here is what I do propose: if MK_PIE=3Dno: no PIE at all if MK_PIE=3Dyes: - on amd64/(platforms without performance penalty): build everything with P= IE from libs to prog - on i386/(platforms with performance penalty): build with PIE if WANTS_PIE is defined. So the difference with the previous approach are: - No way to opt out PIE for a single binary either totally disable or enabl= e (I have encountered no binary so far in the base system which fails with PIE enabled - again only tested on amd64) - Activate PIE for both binaries and libraries (no reason not to include libraries) If we are ok with this I will publish a patch with does it in the next coup= le of week. We can start by only supporting amd64 as fully PIE first then slowly add ot= her platforms. What do you think? regards, Bapt --udcq9yAoWb9A4FsZ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlRZ55cACgkQ8kTtMUmk6EyougCgoIiULNL3x49jBDJKcCOK5M4z QQsAn00ZLWfRKv7A+87qLiM92jOvfbIn =m1vr -----END PGP SIGNATURE----- --udcq9yAoWb9A4FsZ--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20141104082203.J2282>