From owner-svn-src-all@FreeBSD.ORG Sun May 25 01:00:11 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4ABB79B for ; Sun, 25 May 2014 01:00:11 +0000 (UTC) Received: from mail-ig0-f179.google.com (mail-ig0-f179.google.com [209.85.213.179]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9077E2CE5 for ; Sun, 25 May 2014 01:00:11 +0000 (UTC) Received: by mail-ig0-f179.google.com with SMTP id hn18so2010267igb.6 for ; Sat, 24 May 2014 18:00:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:message-id:references:to; bh=Z+IMEZekvr4N7gJpIdIXzUdNTXHaCJ+2b6fYytx0xXo=; b=Sc7who1nCXRtDs1IvfrC3TkAICi4Dwdzx50By0liRvaKOGoXjeMNMqChUxgIYYwopu 3KB0pwmefrbUpQb0KcDv3+TOw5yK1h1J/J2Da+GI4kHmx4R35tQE12y1CfYocshsxkng rp+8a16w/qGJMUtMAGUwWXaFgcXCutG6zS3KRbxwHwNkMJp5DURBiYmncK++JASHSkns uOOd1HoFaJuQMZO9K3D1NX3lpxT46wvRRPtvX0M5KsCX5/yCbUnpgF+yFw9cg2jaIiJI AMAu9xiTZACOOUebL42pm/ByQo2cCw6Dzao2E59rOgw5bvpJuFzfDd4wKK2rgIeB9rAU OVHA== X-Gm-Message-State: ALoCoQm/x9/+yFQTxMnmwhWGbu/x26YoO3emEuXBIlcJyl9PuxbwM8DjYhQ7dEJ1aV0eQyLFOX/9 X-Received: by 10.42.214.143 with SMTP id ha15mr13317518icb.59.1400979610414; Sat, 24 May 2014 18:00:10 -0700 (PDT) Received: from [10.0.0.119] (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPSA id z4sm15730969igl.13.2014.05.24.18.00.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 24 May 2014 18:00:09 -0700 (PDT) Sender: Warner Losh Content-Type: multipart/signed; boundary="Apple-Mail=_D96245BB-20CC-4BA9-B917-447FFECDDD08"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: svn commit: r266553 - head/release/scripts From: Warner Losh In-Reply-To: <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> Date: Sat, 24 May 2014 19:00:18 -0600 Message-Id: <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <537F6706.6070509@freebsd.org> <20140523153619.GF72340@ivaldir.etoilebsd.net> <537F6EBC.3080008@freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> To: Tijl Coosemans X-Mailer: Apple Mail (2.1878.2) Cc: Baptiste Daroussin , src-committers@freebsd.org, Ian Lepore , svn-src-all@freebsd.org, Glen Barber , Nathan Whitehorn , svn-src-head@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 May 2014 01:00:11 -0000 --Apple-Mail=_D96245BB-20CC-4BA9-B917-447FFECDDD08 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-7 On May 24, 2014, at 5:53 PM, Warner Losh wrote: >=20 > On May 24, 2014, at 5:13 PM, Tijl Coosemans wrote: >=20 >> On Sat, 24 May 2014 11:57:44 -0700 Nathan Whitehorn wrote: >>> On 05/24/14 11:23, Ian Lepore wrote: >>>> On Sat, 2014-05-24 at 18:53 +0200, Tijl Coosemans wrote: >>>>> On Sat, 24 May 2014 09:04:33 -0700 Nathan Whitehorn wrote: >>>>>> On 05/24/14 07:59, Tijl Coosemans wrote: >>>>>>> On Fri, 23 May 2014 17:29:48 -0600 Warner Losh wrote: >>>>>>>> On May 23, 2014, at 10:20 AM, Baptiste Daroussin = wrote: >>>>>>>>> On Fri, May 23, 2014 at 08:52:28AM -0700, Nathan Whitehorn = wrote: >>>>>>>>>> On 05/23/14 08:36, Baptiste Daroussin wrote: >>>>>>>>>>> On Fri, May 23, 2014 at 08:19:34AM -0700, Nathan Whitehorn = wrote: >>>>>>>>>>>> Is there any chance of finally switching the pkg abi = identifiers to just >>>>>>>>>>>> be uname -p? >>>>>>>>>>>> -Nathan >>>>>>>>>>> Keeping asking won't make it happen, I have explained a = large number of time why it >>>>>>>>>>> happened, why it is not easy for compatibility and why uname = -p is still not >>>>>>>>>>> representing the ABI we do support, and what flexibility we = need that the >>>>>>>>>>> current string offers to us. >>>>>>>>>>>=20 >>>>>>>>>>> if one is willing to do the work, please be my guess, just = dig into the archives >>>>>>>>>>> and join the pkg development otherwise: no it won't happen = before a while >>>>>>>>>>> because we have way too much work on the todo and this item = is stored at the >>>>>>>>>>> very end of this todo. >>>>>>>>>>>=20 >>>>>>>>>>> regards, >>>>>>>>>>> Bapt >>>>>>>>>> I'm happy to do the work, and have volunteered now many = times. If uname >>>>>>>>>> -p does not describe the ABI fully, then uname -p needs = changes on the >>>>>>>>>> relevant platforms. Which are they? What extra flexibility = does the >>>>>>>>>> string give you if uname -p describes the ABI completely? >>>>>>>>>> -Nathan >>>>>>>>> just simple examples in armv6: >>>>>>>>> - eabi vs oabi >>>>>>>>> - The different float abi (even if only one is supported for = now others are >>>>>>>>> being worked on) >>>>>>>>> - little endian vs big endian >>>>>>>> All of those are encoded in the MACHINE_ARCH + freebsd version, = no exceptions >>>>>>>> on supported architectures that are tier 2 or higher. This = seems like a weak reason. >>>>>>>>=20 >>>>>>>>> the extras flexibilit is being able to say this binary do = support freebsd i386 >>>>>>>>> and amd64 in one key, freebsd:9:x86:*, or or all arches = freebsd:10:* >>>>>>>> Will there be a program to convert this new, special invention = to the standard >>>>>>>> that we=A2ve used for the past 20 years? If you need the = flexibility, which I=A2m not >>>>>>>> entirely sure I=A2ve seen a good use case for. When would you = have a x86 binary >>>>>>>> package? Wouldn=A2t it be either i386 or amd64? >>>>>>> ABI isn't just about the instruction set. It's also about the = sizes of C >>>>>>> types (like pointers). If I remember correctly, the pkg scheme = was chosen >>>>>>> to allow for ABIs like x32 which use the 64 bit instruction set = with 32 >>>>>>> bit pointers. MACHINE_ARCH would also be amd64 in this case. >>>>>> No, it wouldn't. MACHINE_ARCH would be something else (x32, = probably) in >>>>>> such cases. MACHINE_ARCH (and uname -p, which reports it) is the = FreeBSD >>>>>> ABI identifier and encodes 100% of the ABI information. This = would be >>>>>> true even if there is never an x32 kernel. >>>>> No, there's no such thing as an x32 kernel. It's an amd64 kernel = that >>>>> supports a second userland ABI. In C preprocessor terms they are >>>>> distinguished by (__amd64__ && _LP64) and (__amd64__ && !_LP64). >>>>> uname -p gives you the processor architecture (the __amd64__ bit) = but >>>>> then you can still choose the sizes of standard C types (the _LP64 = bit). >>>>> So far we've always had one ABI per processor architecture but = this >>>>> is not strictly necessary. >>>>=20 >>>> All you have to do is look at the plethora of ARM ABIs we support = (and >>>> the corresponding separate kernel for each) to see the falseness of = that >>>> last sentence. ARM variations include v4 vs v6, OABI vs EABI = (calling >>>> and register usage standards), hard vs soft float, little vs big = endian. >>>> Virtually all combinations of those are possible (there are a few = combos >>>> we don't support), and each one has its own MACHINE_ARCH. >>>=20 >>> Exactly. This doesn't rely on the kernel either. The hw.machine_arch=20= >>> sysctl (what uname -p returns) gives the ABI of the calling binary=20= >>> rather than the kernel. So if you use a 32-bit uname (e.g. in a = chroot)=20 >>> on an amd64 host, you get i386. The same will be true if and when we=20= >>> support a 32-bit amd64 userland -- even if there is no x32 kernel, = an=20 >>> x32 uname will return "x32" (or "amd32" or whatever it ends up being=20= >>> called). That string will also appear in kern.supported_archs. >>=20 >> There isn't necessarily any chroot environment. There's one kernel, >> two equally valid ABIs (ILP32 and LP64) and any binary like uname = might >> use either of them. If uname -p returns a different result depending = on >> which of these two ABIs it was compiled for that could be a problem = for >> any script that uses it. >=20 > Well, it depends on what you want to do with the script, eh? If you = want to know the ABI of the native binary uname, that=A2s one thing. But = if you want to know the supported ABIs, you are doing it wrong by using = uname. You should be using sysctl kern.supported_abi. That will tell you = all the ABIs that you can install packages for on this machine, which is = what you really want to know. So I=A2m having trouble connecting the = dots between this and what you are saying here. >=20 > I still am absolutely flabbergasted why the MACHINE_ARCH names aren=A2t = necessary and sufficient for packaging. I=A2ve yet to see any coherent = reason to not use them. Why do I care that they match? Good question. When I was doing FreeNAS, = I looked at integrating pkgng into nanobsd. At the time this was quite = difficult because every single architecture name was different between = pkgng and MACHINE_ARCH. This would mean I=A2d have to drag around a = huge table to know how to translate one to the other (there was no = simple regex either, and things like mipsn32 wouldn=A2t have fit into = the scheme at the time). I would very much like us to see us keep these = names in sync and avoid large translation tables that are difficult to = maintain. Now, do you need to get it from uname -p? No. If you want to parse elf = files to get it, that=A2s fine, so long as the names map directly to the = MACHINE_ARCH names that we=A2ve been using for years. They completely = describe the universe of supported platforms. Are they perfect? No, = around the edge there may be an odd-ball that=A2s possible to build, but = is unsupported and likely doesn=A2t work at all. Have we learned from = these mistakes? Yes. Anything that=A2s actively supported has a proper = name. This name is needed, btw, so that any machine can self-host, a = nice feature of the /usr/src system. Warner --Apple-Mail=_D96245BB-20CC-4BA9-B917-447FFECDDD08 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJTgUCiAAoJEGwc0Sh9sBEA7EAQAOJpSr5bhzRhSXaeSx4tKVxw r5y8nq/QGaHCnHQy9G8y9ocFV2u5EPmanyjXnSL/4ALpy+q27L96lY62fhMr/Fst vlvRT7qsbc3XTyCXQnSHyENnrS4ksZvf8Rmei0feLMUUdkk6tjb9Ps3w1wfeQyrf 8oZnQmJUxJnQ+rlNTTvBfGgR2sjSER9wCsQzNonFIgnTQoLuIcnlD5WKiljzacbl JUE1Tbp1xgHVwHKEpV93M166pY/U3D4a1d9FOu7Ba/XlkGgkjhF7Z3C0PJKaU1bE N16dTegu3/k4jfS2fuUqoclZ4B0OTAahwZONhk8uMfSamZLzFhxe4wvWnBhaQo/q Coas142KZsVV1TJqINalcM6FH1i6Dfb2q4d+9NnVl/Ft+8G1t9+UTXDjoXPGajtr OEzfYNr+Og4ln3CM6HF6hspQJCh+W35cLX5ON5XFLlGTpUylxZ3yrByxaecOA/6v STcBFTsTk3bhfCvU8uKzEz+4sgIP8SpdFuG5U5vtGQuilihkfcb3unpxMdRZAxka sLCJ4z1FWNvpTUU7+iW6OWV6S3ZKbEjZ1v//en6C8209BEc1Eez8hgIZRMnqnMl+ zHC26KmKo+vhYMkJInZMq9dV4iE2itls1pPqhJ6/VmYjA5HCj+HXH34s/GqO3Rue CnYZlGyLll9n95puzc9L =Yv0I -----END PGP SIGNATURE----- --Apple-Mail=_D96245BB-20CC-4BA9-B917-447FFECDDD08--