From owner-svn-src-all@FreeBSD.ORG Mon May 26 15:53:53 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16A45CBF for ; Mon, 26 May 2014 15:53:53 +0000 (UTC) Received: from mail-ig0-f171.google.com (mail-ig0-f171.google.com [209.85.213.171]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD9C721B2 for ; Mon, 26 May 2014 15:53:52 +0000 (UTC) Received: by mail-ig0-f171.google.com with SMTP id c1so112721igq.16 for ; Mon, 26 May 2014 08:53:45 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=i64PCHAqA1/MwLnjtKycQ7ENrGwOKEMZYwEMk8zFrXI=; b=YRqX9VEP8hiculdtPMgGCS1a/QVysirOvRc+ebVg+ZvYMDjSdp8F48LwxAKvsc5Fc5 422VU7ZgFm7NobbxzxApxbj1IZRQrCuUkcfM49JP8SaBE+2HvBI3ceFS+Rzt+nFUCxU+ d8i9wTyFfbba5JBRqyMGF8t7JJcy8hPD1MGWq1F0FjrQ5zi9mkMBIc31YS/xzrn+l5J1 9rK0koABV72js4d+hn3B4esPOD3Iz4ap9noBbAFd8yDH/BYt6XqQy/CXc/gIVx+WEXAv RrMhShKrTnNJqk9ULsFONNYgO2Bax/iqCUg4ZyXNVPzMvg66EWu1iQO77AUV1U1VDmfB l7EA== X-Gm-Message-State: ALoCoQl/dZeDX+Qa11u0ksYEwz61SQHS9Za16rbYR1qy80GbvEUxyKimpVLw4qM4tKRa5uzoL/8L X-Received: by 10.42.48.147 with SMTP id s19mr3403720icf.88.1401119625790; Mon, 26 May 2014 08:53:45 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id w5sm715182igk.9.2014.05.26.08.53.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 26 May 2014 08:53:45 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: svn commit: r266553 - head/release/scripts From: Warner Losh In-Reply-To: <5383522F.30108@freebsd.org> Date: Mon, 26 May 2014 09:53:57 -0600 Message-Id: References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <537F6706.6070509@freebsd.org> <20140523153619.GF72340@ivaldir.etoilebsd.net> <537F6EBC.3080008@freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org> To: Nathan Whitehorn X-Mailer: Apple Mail (2.1878.2) Cc: Baptiste Daroussin , src-committers@freebsd.org, Ian Lepore , svn-src-all@freebsd.org, Glen Barber , svn-src-head@freebsd.org, Tijl Coosemans X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 May 2014 15:53:53 -0000 --Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1253 On May 26, 2014, at 8:39 AM, Nathan Whitehorn = wrote: > On 05/26/14 02:35, Tijl Coosemans wrote: >> On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: >>> On May 24, 2014, at 5:53 PM, Warner Losh wrote: >>>> On May 24, 2014, at 5:13 PM, Tijl Coosemans = wrote: >>>>> There isn't necessarily any chroot environment. There's one = kernel, >>>>> two equally valid ABIs (ILP32 and LP64) and any binary like uname = might >>>>> use either of them. If uname -p returns a different result = depending on >>>>> which of these two ABIs it was compiled for that could be a = problem for >>>>> any script that uses it. >>>> Well, it depends on what you want to do with the script, eh? If you = want >>>> to know the ABI of the native binary uname, that=92s one thing. But = if you >>>> want to know the supported ABIs, you are doing it wrong by using = uname. >>>> You should be using sysctl kern.supported_abi. That will tell you = all the >>>> ABIs that you can install packages for on this machine, which is = what you >>>> really want to know. So I=92m having trouble connecting the dots = between >>>> this and what you are saying here. >>>>=20 >>>> I still am absolutely flabbergasted why the MACHINE_ARCH names = aren=92t >>>> necessary and sufficient for packaging. I=92ve yet to see any = coherent >>>> reason to not use them. >>> Why do I care that they match? Good question. When I was doing = FreeNAS, I >>> looked at integrating pkgng into nanobsd. At the time this was quite >>> difficult because every single architecture name was different = between >>> pkgng and MACHINE_ARCH. This would mean I=92d have to drag around a = huge >>> table to know how to translate one to the other (there was no simple = regex >>> either, and things like mipsn32 wouldn=92t have fit into the scheme = at the >>> time). I would very much like us to see us keep these names in sync = and >>> avoid large translation tables that are difficult to maintain. >>>=20 >>> Now, do you need to get it from uname -p? No. If you want to parse = elf >>> files to get it, that=92s fine, so long as the names map directly to = the >>> MACHINE_ARCH names that we=92ve been using for years. They = completely >>> describe the universe of supported platforms. Are they perfect? No, = around >>> the edge there may be an odd-ball that=92s possible to build, but is >>> unsupported and likely doesn=92t work at all. Have we learned from = these >>> mistakes? Yes. Anything that=92s actively supported has a proper = name. This >>> name is needed, btw, so that any machine can self-host, a nice = feature of >>> the /usr/src system. >> ABI consists of the following elements: >>=20 >> - OS >> - OS ABI version (major version number in FreeBSD) These two are encoded in FreeBSD and major version. There=92s no problem = encoding these in the package architecture string. They are easily = scriptable and totally obvious to FreeBSD users and pose no problems. = Nobody is opposed to these, and actually they are rather a good idea. >> - instruction set >> - programming model (ILP32 or LP64) >> - byte order (little/big endian) These three are encoded in MACHINE_ARCH and have been for quite some = time. And you forgot several things as well: register conventions, = calling conventions, stack alignment, struct alignment, pointer = conversion conventions, address space layout, page size constraints, = etc. There are simply far too many to try to break down like you are = trying to do. And that=92s even before we get into shared library = conventions... >> These are almost orthogonal dimensions in the sense that almost any >> combination is possible. (A combination that isn't possible is a >> 32-bit instruction set with LP64.) All of these items are encoded in MACHINE_ARACH and have been for at = least a decade. There=92s no new argument here. If they were actually = orthogonal, then that would be one thing. But they aren=92t. They are = all closely interrelated and we only support a vanishingly small number = of possible conventions. Combinatorically, it can be hundreds. = Practically, it is usually only a handful. >> What you are asking for now is to combine two dimensions into one and >> combination in this case means multiplication so if you have 3 >> instruction sets and 2 programming models, the combined dimension = needs >> 6 different values. You need to make the case for why you think this >> is a good idea. Because uanme has to be 6 different things so the right binaries are = built. It is really that simple. And we=92ve already made the case, and = have been using this convention for a very long time. It works. I=92m = not sure that the burden is on us to justify why a convention that=92s = been in use since FreeBSD 6 needs to not change. As weird as you might = think it is, it is a convention that our users understand. >> For the past 20 years we got away with this because >> on every installation of FreeBSD we only used one programming model = at >> a time. This is still the case for byte order of course. This isn=92t true. For the past 15 years we=92ve supported two = programming models on amd64 at the same time. For longer than that we=92ve= supported linux emulation on i386. The project has known about these = things for a long long time, and has settled on MACHINE_ARCH to = represent all possible builds. We=92ve had mixed MIPS for about a = decade, though the support has varied in quality and execution. We = learned that TARGET_BIG_ENDIAN was bad, really bad, and we had to have a = separate name for each ABI we supported with no external info apart from = that name. We could have easily picked the convention you are proposing = here, but we didn=92t. We picked another one. Also, the =93for the past 20 years=94 argument cuts both ways. Look at = NetBSD. There, they have the same convention we have here of having a = separate MACHINE_ARCH for each ABI. They have been even more successful = at it that we have, and have avoided the pitfalls of TARGET_BIG_ENDIAN = much better than we have. pkgsrc ties nicely into that. so for 20 years = people have successfully used the current model, not just in FreeBSD, = but also elsewhere. >> What I'm saying is to keep the option open for installations with >> multiple programming models, where most binaries could use ILP32 and >> only the ones that actually need a 64-bit address space use LP64. >> You query the instruction set using uname and the programming models >> using getconf. What I=92m saying is I don=92t see any benefit at all to our users to = having an additional, arbitrary sting they have to deal with. There=92s = actually quite a few other details that you need to know before you can = even call getconf. >> I suppose you could replace the "x86" in the pkg scheme with = i386/amd64, >> but then you'd still be talking about i386:32, amd64:32 and amd64:64 >> instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by =93i386=94, =93x32=94 (or = =93amd64x32=94) and =93amd64=94 respectively. Just like we did with = mips. As a users, how the heck am I to know what all these strange = strings map to? I have an amd64 machine, what package do I install? = x86:64 WTF is up with that? How am I supposed to message that to users? = How am I supposed to write a sane script that ties together packages and = base system when there are two different systems to describe the same = thing? I=92ve yet to see any benefit that is so huge that it trumps the = ease of use for our users and the eases of script writing for the = nanobsd and crochets of the world. >=20 > No. We support multiple "models" now and have for ten years. That's = what MACHINE_ARCH is for: it defines the choice of the last three things = you list above. Specifically, a shared value of MACHINE_ARCH guarantees = and OS version guarantees, in FreeBSD-land, complete binary = compatibility of executables. Kernels support multiple ones, in general = (e.g. i386 binaries on amd64, powerpc binaries on powerpc64). They may = support more in the future (x32 on amd64, potentially even cross-endian = binaries). We have a nice flexible scheme in FreeBSD for supporting = this. If you want to find out the list of the things the installed = kernel can run, check the kern.supported_archs sysctl. Simple. Don=92t forget we=92ve supported linux emulation for 18 years, and let=92s= not forget about IBCS and SYSV emulation, present from the very start = as well. If we look historically at BSD, I know that BSD4.2 on the VAXen = ran pdp11 binaries, which pushes back the time horizon another 10 years. > These strings are just as expressive as the ones in pkg. They are the = standard. They're what external build systems test against, what the = src, doc, and ports trees use to define what to do universally. It's = what users and code expect. The wheel we've had for 20 years is = perfectly good -- why invent a new, incompatible one? Exactly. What is the huge benefit that justifies the huge pain this is = going to cause? The FreeBSD project already has some pain because it = chose amd64 as its arch name (spread through about 2 dozen makefiles in = the base that do s/amd64/x86_64/ in places. Do you want to multiply this = times 6 architectures with arbitrary and difficult to explain = differences to our users? So rather than a repetition of the arguments that aren=92t very strong, = and certainly don=92t come close to justifying the extra pain our users = will feel, perhaps an argument for that future pain would be useful. Warner --Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTg2OVAAoJEGwc0Sh9sBEA3A4QAM5XIi3nUSbfmpfSWniQbR6v qv4c9GKyv6UhRRVccXd6QGpfcS+beRpwqIi/LKhmFzkQQTfTG15Ao5mTKLlZqUTw fvb4IA4XdKoSuSqxJ0OwYPeXGdO4nYfMnbcBiP1QOWi7ZjGW4roC4fvaRClLRo59 y9ajXzRuB6yoFfphWNOSjFfCkVdEa22O+XLEhiCS8HB4/qZzyXuwnX8usIoeWFLM 4FKdBWEYxmO3UWv9HoZWFoU7LCWcgXxklVTlaVJKkh3JJUKssJNnRe7MFW9VG2qx eTuTXN0qq6IKnbLNzXnY3eiGV23TAlVDi1Nxxfs0poRPcSVWmP9SMN4sUvJ4pj8r CjAHF4MnPA94CUp0xRH6xiIl1vaqYkwr/NwiD9CEa6XrcywBvQ1vIjdqZBk+cZ+p uTvKlB/RSEwsF+9q75soLG3wcu2Bj6o2E/A18nJGOWMsiIsNG23wtfA5xaZKFvM0 bXCs6rWPXe2ZxTFZa98sQpddrB+00kMtTB3RsW5d2LBlaVJUe3ofUCmA8c5X4Mcz mYdtCAu9vBhkP5Y4LgKQaQGJxlwfr/CqhTT/67hNsLxCUd6Zwcs9scPjfUPDK6np 5MNRyDoSpCvhptG8Sxp4RayjvrQDWlDgP7bgKLYbACz3aKssuhDKt/oU+hkC+QT3 Dg6D88yAPgak5hfp0xTh =rcPk -----END PGP SIGNATURE----- --Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5--