Date: Mon, 26 May 2014 09:53:57 -0600 From: Warner Losh <imp@bsdimp.com> To: Nathan Whitehorn <nwhitehorn@freebsd.org> Cc: Baptiste Daroussin <bapt@freebsd.org>, src-committers@freebsd.org, Ian Lepore <ian@freebsd.org>, svn-src-all@freebsd.org, Glen Barber <gjb@freebsd.org>, svn-src-head@freebsd.org, Tijl Coosemans <tijl@freebsd.org> Subject: Re: svn commit: r266553 - head/release/scripts Message-ID: <DAD3E386-6555-4C43-9BBA-F3BFD28CC19B@bsdimp.com> In-Reply-To: <5383522F.30108@freebsd.org> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <537F6706.6070509@freebsd.org> <20140523153619.GF72340@ivaldir.etoilebsd.net> <537F6EBC.3080008@freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <C5A59513-AF58-4749-BCD7-F54BB6F56E90@gmail.com> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1253 On May 26, 2014, at 8:39 AM, Nathan Whitehorn <nwhitehorn@freebsd.org> = wrote: > On 05/26/14 02:35, Tijl Coosemans wrote: >> On Sat, 24 May 2014 19:00:18 -0600 Warner Losh wrote: >>> On May 24, 2014, at 5:53 PM, Warner Losh <imp@bsdimp.com> wrote: >>>> On May 24, 2014, at 5:13 PM, Tijl Coosemans <tijl@freebsd.org> = wrote: >>>>> There isn't necessarily any chroot environment. There's one = kernel, >>>>> two equally valid ABIs (ILP32 and LP64) and any binary like uname = might >>>>> use either of them. If uname -p returns a different result = depending on >>>>> which of these two ABIs it was compiled for that could be a = problem for >>>>> any script that uses it. >>>> Well, it depends on what you want to do with the script, eh? If you = want >>>> to know the ABI of the native binary uname, that=92s one thing. But = if you >>>> want to know the supported ABIs, you are doing it wrong by using = uname. >>>> You should be using sysctl kern.supported_abi. That will tell you = all the >>>> ABIs that you can install packages for on this machine, which is = what you >>>> really want to know. So I=92m having trouble connecting the dots = between >>>> this and what you are saying here. >>>>=20 >>>> I still am absolutely flabbergasted why the MACHINE_ARCH names = aren=92t >>>> necessary and sufficient for packaging. I=92ve yet to see any = coherent >>>> reason to not use them. >>> Why do I care that they match? Good question. When I was doing = FreeNAS, I >>> looked at integrating pkgng into nanobsd. At the time this was quite >>> difficult because every single architecture name was different = between >>> pkgng and MACHINE_ARCH. This would mean I=92d have to drag around a = huge >>> table to know how to translate one to the other (there was no simple = regex >>> either, and things like mipsn32 wouldn=92t have fit into the scheme = at the >>> time). I would very much like us to see us keep these names in sync = and >>> avoid large translation tables that are difficult to maintain. >>>=20 >>> Now, do you need to get it from uname -p? No. If you want to parse = elf >>> files to get it, that=92s fine, so long as the names map directly to = the >>> MACHINE_ARCH names that we=92ve been using for years. They = completely >>> describe the universe of supported platforms. Are they perfect? No, = around >>> the edge there may be an odd-ball that=92s possible to build, but is >>> unsupported and likely doesn=92t work at all. Have we learned from = these >>> mistakes? Yes. Anything that=92s actively supported has a proper = name. This >>> name is needed, btw, so that any machine can self-host, a nice = feature of >>> the /usr/src system. >> ABI consists of the following elements: >>=20 >> - OS >> - OS ABI version (major version number in FreeBSD) These two are encoded in FreeBSD and major version. There=92s no problem = encoding these in the package architecture string. They are easily = scriptable and totally obvious to FreeBSD users and pose no problems. = Nobody is opposed to these, and actually they are rather a good idea. >> - instruction set >> - programming model (ILP32 or LP64) >> - byte order (little/big endian) These three are encoded in MACHINE_ARCH and have been for quite some = time. And you forgot several things as well: register conventions, = calling conventions, stack alignment, struct alignment, pointer = conversion conventions, address space layout, page size constraints, = etc. There are simply far too many to try to break down like you are = trying to do. And that=92s even before we get into shared library = conventions... >> These are almost orthogonal dimensions in the sense that almost any >> combination is possible. (A combination that isn't possible is a >> 32-bit instruction set with LP64.) All of these items are encoded in MACHINE_ARACH and have been for at = least a decade. There=92s no new argument here. If they were actually = orthogonal, then that would be one thing. But they aren=92t. They are = all closely interrelated and we only support a vanishingly small number = of possible conventions. Combinatorically, it can be hundreds. = Practically, it is usually only a handful. >> What you are asking for now is to combine two dimensions into one and >> combination in this case means multiplication so if you have 3 >> instruction sets and 2 programming models, the combined dimension = needs >> 6 different values. You need to make the case for why you think this >> is a good idea. Because uanme has to be 6 different things so the right binaries are = built. It is really that simple. And we=92ve already made the case, and = have been using this convention for a very long time. It works. I=92m = not sure that the burden is on us to justify why a convention that=92s = been in use since FreeBSD 6 needs to not change. As weird as you might = think it is, it is a convention that our users understand. >> For the past 20 years we got away with this because >> on every installation of FreeBSD we only used one programming model = at >> a time. This is still the case for byte order of course. This isn=92t true. For the past 15 years we=92ve supported two = programming models on amd64 at the same time. For longer than that we=92ve= supported linux emulation on i386. The project has known about these = things for a long long time, and has settled on MACHINE_ARCH to = represent all possible builds. We=92ve had mixed MIPS for about a = decade, though the support has varied in quality and execution. We = learned that TARGET_BIG_ENDIAN was bad, really bad, and we had to have a = separate name for each ABI we supported with no external info apart from = that name. We could have easily picked the convention you are proposing = here, but we didn=92t. We picked another one. Also, the =93for the past 20 years=94 argument cuts both ways. Look at = NetBSD. There, they have the same convention we have here of having a = separate MACHINE_ARCH for each ABI. They have been even more successful = at it that we have, and have avoided the pitfalls of TARGET_BIG_ENDIAN = much better than we have. pkgsrc ties nicely into that. so for 20 years = people have successfully used the current model, not just in FreeBSD, = but also elsewhere. >> What I'm saying is to keep the option open for installations with >> multiple programming models, where most binaries could use ILP32 and >> only the ones that actually need a 64-bit address space use LP64. >> You query the instruction set using uname and the programming models >> using getconf. What I=92m saying is I don=92t see any benefit at all to our users to = having an additional, arbitrary sting they have to deal with. There=92s = actually quite a few other details that you need to know before you can = even call getconf. >> I suppose you could replace the "x86" in the pkg scheme with = i386/amd64, >> but then you'd still be talking about i386:32, amd64:32 and amd64:64 >> instead of x86:32, x86:x32 and x86:64. I suppose you could replace these by =93i386=94, =93x32=94 (or = =93amd64x32=94) and =93amd64=94 respectively. Just like we did with = mips. As a users, how the heck am I to know what all these strange = strings map to? I have an amd64 machine, what package do I install? = x86:64 WTF is up with that? How am I supposed to message that to users? = How am I supposed to write a sane script that ties together packages and = base system when there are two different systems to describe the same = thing? I=92ve yet to see any benefit that is so huge that it trumps the = ease of use for our users and the eases of script writing for the = nanobsd and crochets of the world. >=20 > No. We support multiple "models" now and have for ten years. That's = what MACHINE_ARCH is for: it defines the choice of the last three things = you list above. Specifically, a shared value of MACHINE_ARCH guarantees = and OS version guarantees, in FreeBSD-land, complete binary = compatibility of executables. Kernels support multiple ones, in general = (e.g. i386 binaries on amd64, powerpc binaries on powerpc64). They may = support more in the future (x32 on amd64, potentially even cross-endian = binaries). We have a nice flexible scheme in FreeBSD for supporting = this. If you want to find out the list of the things the installed = kernel can run, check the kern.supported_archs sysctl. Simple. Don=92t forget we=92ve supported linux emulation for 18 years, and let=92s= not forget about IBCS and SYSV emulation, present from the very start = as well. If we look historically at BSD, I know that BSD4.2 on the VAXen = ran pdp11 binaries, which pushes back the time horizon another 10 years. > These strings are just as expressive as the ones in pkg. They are the = standard. They're what external build systems test against, what the = src, doc, and ports trees use to define what to do universally. It's = what users and code expect. The wheel we've had for 20 years is = perfectly good -- why invent a new, incompatible one? Exactly. What is the huge benefit that justifies the huge pain this is = going to cause? The FreeBSD project already has some pain because it = chose amd64 as its arch name (spread through about 2 dozen makefiles in = the base that do s/amd64/x86_64/ in places. Do you want to multiply this = times 6 architectures with arbitrary and difficult to explain = differences to our users? So rather than a repetition of the arguments that aren=92t very strong, = and certainly don=92t come close to justifying the extra pain our users = will feel, perhaps an argument for that future pain would be useful. Warner --Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTg2OVAAoJEGwc0Sh9sBEA3A4QAM5XIi3nUSbfmpfSWniQbR6v qv4c9GKyv6UhRRVccXd6QGpfcS+beRpwqIi/LKhmFzkQQTfTG15Ao5mTKLlZqUTw fvb4IA4XdKoSuSqxJ0OwYPeXGdO4nYfMnbcBiP1QOWi7ZjGW4roC4fvaRClLRo59 y9ajXzRuB6yoFfphWNOSjFfCkVdEa22O+XLEhiCS8HB4/qZzyXuwnX8usIoeWFLM 4FKdBWEYxmO3UWv9HoZWFoU7LCWcgXxklVTlaVJKkh3JJUKssJNnRe7MFW9VG2qx eTuTXN0qq6IKnbLNzXnY3eiGV23TAlVDi1Nxxfs0poRPcSVWmP9SMN4sUvJ4pj8r CjAHF4MnPA94CUp0xRH6xiIl1vaqYkwr/NwiD9CEa6XrcywBvQ1vIjdqZBk+cZ+p uTvKlB/RSEwsF+9q75soLG3wcu2Bj6o2E/A18nJGOWMsiIsNG23wtfA5xaZKFvM0 bXCs6rWPXe2ZxTFZa98sQpddrB+00kMtTB3RsW5d2LBlaVJUe3ofUCmA8c5X4Mcz mYdtCAu9vBhkP5Y4LgKQaQGJxlwfr/CqhTT/67hNsLxCUd6Zwcs9scPjfUPDK6np 5MNRyDoSpCvhptG8Sxp4RayjvrQDWlDgP7bgKLYbACz3aKssuhDKt/oU+hkC+QT3 Dg6D88yAPgak5hfp0xTh =rcPk -----END PGP SIGNATURE----- --Apple-Mail=_7567707D-397E-4D83-8565-B4A06EA951E5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DAD3E386-6555-4C43-9BBA-F3BFD28CC19B>