Date: Tue, 27 May 2014 14:31:44 -0600 From: Warner Losh <imp@bsdimp.com> To: Tijl Coosemans <tijl@FreeBSD.org> Cc: Baptiste Daroussin <bapt@freebsd.org>, src-committers@freebsd.org, Ian Lepore <ian@freebsd.org>, svn-src-all@freebsd.org, Glen Barber <gjb@freebsd.org>, Nathan Whitehorn <nwhitehorn@freebsd.org>, svn-src-head@freebsd.org Subject: Re: svn commit: r266553 - head/release/scripts Message-ID: <13EB325C-3882-46AA-9B17-3BF19997C978@bsdimp.com> In-Reply-To: <20140527214038.17d00369@kalimero.tijl.coosemans.org> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <20140523153619.GF72340@ivaldir.etoilebsd.net> <537F6EBC.3080008@freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <C5A59513-AF58-4749-BCD7-F54BB6F56E90@gmail.com> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org> <DAD3E386-6555-4C43-9BBA-F3BFD28CC19B@bsdimp.com> <20140527001811.3e9d3e8d@kalimero.tijl.coosemans.org> <05D1A11D-5985-42EA-84AD-209A8B51D391@bsdimp.com> <20140527093633.0a922e13@kalimero.tijl.coosemans.org> <85FABD2B-81BB-4E1A-B61E-4216A144A9DB@bsdimp.com> <20140527214038.17d00369@kalimero.tijl.coosemans.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=windows-1252 On May 27, 2014, at 1:40 PM, Tijl Coosemans <tijl@FreeBSD.org> wrote: > On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: >> On May 27, 2014, at 1:36 AM, Tijl Coosemans <tijl@FreeBSD.org> wrote: >>> On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: >>>> On May 26, 2014, at 4:18 PM, Tijl Coosemans <tijl@FreeBSD.org> = wrote:=20 >>>>> On Mon, 26 May 2014 09:53:57 -0600 Warner Losh wrote: >>>>>> On May 26, 2014, at 8:39 AM, Nathan Whitehorn = <nwhitehorn@freebsd.org> wrote: >>>>>>> On 05/26/14 02:35, Tijl Coosemans wrote: >>>>>>>> I suppose you could replace the "x86" in the pkg scheme with = i386/amd64, >>>>>>>> but then you'd still be talking about i386:32, amd64:32 and = amd64:64 >>>>>>>> instead of x86:32, x86:x32 and x86:64. =20 >>>>>>=20 >>>>>> I suppose you could replace these by ?i386?, ?x32? (or = ?amd64x32?) and >>>>>> ?amd64? respectively. >>>>>=20 >>>>> So you're on an amd64 or mips64 system (as indicated by uname) but = you >>>>> want to use the 32-bit package if possible. How does your script = know >>>>> about the magic "x32", "amd64x32" or "mipsn32" strings? Wouldn't = it be >>>>> easier if you could just use "`uname -p`:32?? >>>>=20 >>>> Oh give me a break. You know it because you know you are building = for >>>> mipsn32 because that?s what you?ve set MACHINE_ARCH or TARGET_ARCH = to, >>>=20 >>> No, MACHINE_ARCH or TARGET_ARCH is "amd64" or "mips64". You are = building >>> the 64-bit OS and then decide separately per package whether you = want the >>> ILP32 one or the LP64 one. >>=20 >> I think I understand why we?re talking past each other. This bit is = wrong. >> The LP64 one has a MACHINE_ARCH of ?amd64? or ?mips64?. The ILP32 = one will >> have MACHINE_ARCH of ?i386? or ?mips?. The weird ones (ILP32 with = 64-bit >> registers) will have a different MACHINE_ARCH of ?x32? or ?mipsn32?. >> Selection can be done on a case by case basis, but this will be = validated >> against the supported_abis sysctl. There?s no need to have different = names >> here, the current standard ones do just fine, are completely = sufficient and >> all inclusive. This is rather by definition, and your understanding = of the >> definition sounds flawed. >>=20 >> While we have limited support for building 32-bit binaries, it is to = build >> 32-bit binaries for a different MACHINE_ARCH. -m32 on amd64 creates = i386 >> binaries, not amd64:32 binaries. -m32 on powerpc64 creates powerpc = binaries, >> not powerpc64:32 binaries. On mips, -mabi-n32 (I think the option is) = is >> required to create the mipsn32 binaries. In every single one of these = cases, >> there exists a MACHINE_ARCH that completely describes the binary. >=20 > amd64:32 is provided by the -mx32 compiler flag. True, but not relevant. >> So I?m still waiting for a use case that requires the new names. One = has >> not been articulated, and I don?t think one actually exists. >=20 > Imagine you've built a system with MACHINE_ARCH amd64 and one with > MACHINE_ARCH mips64. Now you want to populate these systems with a = list > of packages for which you wrote a script. These systems each support = 2 > ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one = (x32 > and mipsn32). Both are native in the sense that they make full use of = the > instruction set. This is not like i386 on amd64 or mips o32 on mips64 > because those are more like compat shims that operate under special > (crippled) cpu modes that nobody uses unless they're stuck with old = code. > Both our ABIs on the other hand are native and equally valid and which = one > to use for a particular package depends entirely on the use case. If = your > use case requires more than 4G of address space you'll have to use the > 64-bit package, otherwise you can use the 32-bit package which, = depending > on how pointer heavy the code is, may give a performance benefit. You > make this choice for each of the packages in your list and add that > information to your script. Long hypothetical, but so what? In such a case, you=92d pick one of two = different MACHINE_ARCH values depending on the package. This this is a = fairly atypical use case, it would not be unreasonable for the person = wanting to do this to know the proper companion ABI. In both cases, you = have the choice of two other values to use. Which one you use will = depend on a variety of factors, and what might be right for one = application may be wrong for others. And in both cases, there=92s actually two choices: for amd64, you=92d = have i386 and x32. Both of these are fine choices, and it would depend = on the workload which one is better (i386 has better toolchain support = and maturity, x32 offers some interesting theoretical wins, but doesn=92t = have the same maturity). Same with mips64, you=92d have two choices as = well. In both cases, you can=92t just take uname/MACHINE_ARCH and slap = :32 on the end. > Now let's work with Nathan's patch which uses the following string to > identify the pkg repository to fetch packages from: > `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` >=20 > On our two systems that would be FreeBSD:11:amd64 and = FreeBSD:11:mips64. > Now if your script has to install the 32-bit version of a package how > can it go from those two strings to FreeBSD:11:x32 or = FreeBSD:11:mipsn32 > without a lookup table? You couldn=92t. Which is the whole reason I want to have them have a = standard name so you don=92t need the lookup table for the common case. = This is an =93off in the weeds=94 case, and optimizing for it doesn=92t = make sense. Especially because in each case, you have two different = 32-bit ABIs to choose from. You=92d have to have some kind of table in = either case. Also, the proper name for n32, in your current system, is = mips:32:n32, which (a) is wrong and (b) isn=92t regular. > I say, you can more easily indicate whether you > want the 32-bit or 64-bit package by appending :32 or :64 to the = original > strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. Except there=92s no such thing as mips64:32 in the current system. = There=92s two different ABIs that could mean. It could be o32 or n32, = with the same sort of trade offs. There=92d need to be a person in the = loop to know, so there=92s already a need to have special knowledge. You = can=92t get there by just adding :32 to the existing thing... > Like I said in a reply to Nathan's patch, pkg could default to :32 or = :64 > for every arch so it can be left out in many cases. FreeBSD:11:i386 > would then be equivalent to FreeBSD:11:i386:32, FreeBSD:11:amd64 to > FreeBSD:11:amd64:64, etc. I don=92t see what value that adds to have the extra :32 or :64. = Nathan=92s patches make it possible to have an automated build system = with the typical use case (I want to build all my binaries, packages, = etc the same). But adding a :32 isn=92t going to even work for your = hypothetical example because people actually building such systems will = need to specify which 32-bit ABI they want to use anyway. And the = degenerate case of :32 just doesn=92t work with mips=85=20 Consider too that we=92re planning a new i386 ABI, which is identical to = the current i386 ABI, except time_t is 64 bit. We=92ve penciled in the = name i386t64 for this ABI/MACHINE_ARCH. This will be much less painful, = it is thought, than finding all the current places that take time_t as = an arg and shimming=85 So if we were to do that, then your current = scheme wouldn=92t be able to encompass that eventuality and we=92d be = back to this argument... > This also does not preclude the existence of a mipsn32 MACHINE_ARCH in > case you want to build a pure mipsn32 system (including the kernel), = but > I wonder how many people would use that if it is possible to have a > mips64 system run n32 binaries. I don't think many people would run a > pure x32 system (with x32 kernel) so I don't see the need to have an > "x32" value for MACHINE_ARCH (or TARGET_ARCH). Actually, you absolutely must have a x32 MACHINE_ARCH if you want to be = able to run x32 binaries. Otherwise, how are you going to build the = libraries that use that API? Our build system simply isn=92t setup to = build them any other way. Although the usr/lib32 stuff could grow extra = goo for that, you are still building them the same way you=92d build. We = have a stylized way to create a sys root, which is needed for the = compilers to work. We=92d likely have to grow better multi lib support = than we have now as well. You=92d need some way to identify these = binaries, segregate their ld.so, etc. Even if no kernel ever is linked = this way, it is still an absolute requirement. Warner --Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJThPYwAAoJEGwc0Sh9sBEA9EAQANrGMeeK9cJHf4BLpU61flVT HmdMcrdFzKWzEUgkzGsPgOX13a83FnNAGrROdgGANKxUqS21g0VdsqhOgLgyHS3d u6gKCJtxfe4JA6bTO8zO+uiUqiyAv3+4qrZEbH3g5kCMjM+TQjj5na4+/odq0dYx 0Ud3Ff7nu7uMlK6SojoHYU78Nvb7M3oLmvKvc0clEX9RS47hdmm8CJOjO8eMv3Gw qFrHjJwo+S4pxxuYzUOG4CHiKckJ/hlkChkWV0R0TLfxwpNZTocwpiM9rPxPfMzH qsw0Ifu9xhN9lBbXsdhKR5lAFfqtPLFh/Jb52aFl4bDQMtYHb6PG9cJDuFezFOeY L6Ga3SPXvr7tNCpUETFwmhDgxruDedsNLyfEGsat5T4/64UnkwHVM1U/9ZcGwJQ+ 3YOb9oeJpf0Js8dw+734QDVU/RQqJYHyq7N3e+EwnF1lVcgnt+YAG2ISetj+NYN9 J33Hp2Txe7qonvGpI93WdvG3+oDhAYadIO5KoPXFbrZo2nv7aVVWnF0SXb1Wkt1R 2XENnDCvkg+TgCbmxQGJpsBa9gUkhnRTGIelwWDr843jE+umtbgEIQAzykTnt85d ajlBFjqlN3v3RipxOUTk978OGw+XCCZeh/gizFhf2Zf/XCVNPkKWEtEnLNfYLCnW eGizCocprieENZyv+bKw =YhAC -----END PGP SIGNATURE----- --Apple-Mail=_F9B225B6-1C99-4233-A917-99CE5519C109--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?13EB325C-3882-46AA-9B17-3BF19997C978>