Date: Wed, 28 May 2014 12:50:27 +0200 From: Tijl Coosemans <tijl@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Cc: Baptiste Daroussin <bapt@freebsd.org>, src-committers@freebsd.org, Ian Lepore <ian@freebsd.org>, svn-src-all@freebsd.org, Glen Barber <gjb@freebsd.org>, Nathan Whitehorn <nwhitehorn@freebsd.org>, svn-src-head@freebsd.org Subject: Re: svn commit: r266553 - head/release/scripts Message-ID: <20140528125027.6d0cc4fb@kalimero.tijl.coosemans.org> In-Reply-To: <13EB325C-3882-46AA-9B17-3BF19997C978@bsdimp.com> References: <201405221922.s4MJM4Y9025265@svn.freebsd.org> <20140523162020.GG72340@ivaldir.etoilebsd.net> <C5A59513-AF58-4749-BCD7-F54BB6F56E90@gmail.com> <20140524165940.3c687553@kalimero.tijl.coosemans.org> <5380C311.60201@freebsd.org> <20140524185345.263f230d@kalimero.tijl.coosemans.org> <1400955835.1152.323.camel@revolution.hippie.lan> <5380EBA8.1030200@freebsd.org> <20140525011307.142b41ab@kalimero.tijl.coosemans.org> <3CCAFAD3-FABE-40EF-ABF9-815FE5826349@bsdimp.com> <9FE34CE4-C71F-4806-9EF6-30CB1051C62F@bsdimp.com> <20140526113502.239db74d@kalimero.tijl.coosemans.org> <5383522F.30108@freebsd.org> <DAD3E386-6555-4C43-9BBA-F3BFD28CC19B@bsdimp.com> <20140527001811.3e9d3e8d@kalimero.tijl.coosemans.org> <05D1A11D-5985-42EA-84AD-209A8B51D391@bsdimp.com> <20140527093633.0a922e13@kalimero.tijl.coosemans.org> <85FABD2B-81BB-4E1A-B61E-4216A144A9DB@bsdimp.com> <20140527214038.17d00369@kalimero.tijl.coosemans.org> <13EB325C-3882-46AA-9B17-3BF19997C978@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 27 May 2014 14:31:44 -0600 Warner Losh wrote: > On May 27, 2014, at 1:40 PM, Tijl Coosemans <tijl@FreeBSD.org> wrote: >> On Tue, 27 May 2014 07:18:06 -0600 Warner Losh wrote: >>> On May 27, 2014, at 1:36 AM, Tijl Coosemans <tijl@FreeBSD.org> wrote: >>>> On Mon, 26 May 2014 16:31:21 -0600 Warner Losh wrote: >>> So I?m still waiting for a use case that requires the new names. One has >>> not been articulated, and I don?t think one actually exists. >> >> Imagine you've built a system with MACHINE_ARCH amd64 and one with >> MACHINE_ARCH mips64. Now you want to populate these systems with a list >> of packages for which you wrote a script. These systems each support 2 >> ABIs: a native 64-bit one (amd64 and mips64) and a native 32-bit one (x32 >> and mipsn32). Both are native in the sense that they make full use of the >> instruction set. This is not like i386 on amd64 or mips o32 on mips64 >> because those are more like compat shims that operate under special >> (crippled) cpu modes that nobody uses unless they're stuck with old code. >> Both our ABIs on the other hand are native and equally valid and which one >> to use for a particular package depends entirely on the use case. If your >> use case requires more than 4G of address space you'll have to use the >> 64-bit package, otherwise you can use the 32-bit package which, depending >> on how pointer heavy the code is, may give a performance benefit. You >> make this choice for each of the packages in your list and add that >> information to your script. > > Long hypothetical, but so what? In such a case, you’d pick one of two > different MACHINE_ARCH values depending on the package. This this is a > fairly atypical use case, it would not be unreasonable for the person > wanting to do this to know the proper companion ABI. In both cases, you > have the choice of two other values to use. Which one you use will depend > on a variety of factors, and what might be right for one application may > be wrong for others. > > And in both cases, there’s actually two choices: for amd64, you’d have > i386 and x32. Both of these are fine choices, and it would depend on the > workload which one is better (i386 has better toolchain support and > maturity, x32 offers some interesting theoretical wins, but doesn’t have > the same maturity). Same with mips64, you’d have two choices as well. In > both cases, you can’t just take uname/MACHINE_ARCH and slap :32 on the end. As far as I can see nobody uses i386 on amd64 except in cases where you know you're dealing with i386 like an i386 jail, or you need to run an old i386 binary that you're stuck with, or with compat shims like Wine. The choice of i386 is not comparable with the generic choice between ILP32 or LP64 on a 64-bit instruction set, which is why I'm ok with changing the "x86" in the pkg scheme to either "i386" or "amd64" like you are proposing. If pkg would keep using instruction set families like "x86" or "mips" then you could distinguish between old 32-bit, new 32-bit and new 64-bit using 3 generic suffixes to avoid lookup tables. The pattern of an old 32-bit instruction set that got extended to 64-bit on which you can define a new 32-bit ABI is common to many instruction set families, but I'm ok with old 32-bit and new 64-bit having separate MACHINE_ARCH values. >> Now let's work with Nathan's patch which uses the following string to >> identify the pkg repository to fetch packages from: >> `uname -s`:`uname -r | cut -f 1 -d .`:`uname -p` >> >> On our two systems that would be FreeBSD:11:amd64 and FreeBSD:11:mips64. >> Now if your script has to install the 32-bit version of a package how >> can it go from those two strings to FreeBSD:11:x32 or FreeBSD:11:mipsn32 >> without a lookup table? > > You couldn’t. Which is the whole reason I want to have them have a > standard name so you don’t need the lookup table for the common case. > This is an “off in the weeds” case, and optimizing for it doesn’t make > sense. Especially because in each case, you have two different 32-bit > ABIs to choose from. You’d have to have some kind of table in either > case. Also, the proper name for n32, in your current system, is > mips:32:n32, which (a) is wrong and (b) isn’t regular. Well, it may not make sense to you and be off in the weeds, but it does to me. Very few processes actually need more than 4G of virtual address space. I cannot immediately find anything in /bin or /usr/bin for instance (maybe clang when compiling a very large C++ file?). I'm not sure where you're getting mips:32:n32 from. We're talking in the context of Nathan's patch here. >> I say, you can more easily indicate whether you >> want the 32-bit or 64-bit package by appending :32 or :64 to the original >> strings, so FreeBSD:11:amd64:32 and FreeBSD:11:mips64:32. > > Except there’s no such thing as mips64:32 in the current system. There’s > two different ABIs that could mean. It could be o32 or n32, with the same > sort of trade offs. There’d need to be a person in the loop to know, so > there’s already a need to have special knowledge. You can’t get there by > just adding :32 to the existing thing... Yes, I am assuming that o32 on mips64 is like i386 on amd64, i.e. only used in cases where you know you're dealing with mips o32, and that normally n32 is the 32-bit ABI of choice on mips64 because it provides better performance. >> Like I said in a reply to Nathan's patch, pkg could default to :32 or :64 >> for every arch so it can be left out in many cases. FreeBSD:11:i386 >> would then be equivalent to FreeBSD:11:i386:32, FreeBSD:11:amd64 to >> FreeBSD:11:amd64:64, etc. > > I don’t see what value that adds to have the extra :32 or :64. Nathan’s > patches make it possible to have an automated build system with the > typical use case (I want to build all my binaries, packages, etc the > same). But adding a :32 isn’t going to even work for your hypothetical > example because people actually building such systems will need to > specify which 32-bit ABI they want to use anyway. And the degenerate case > of :32 just doesn’t work with mips… > > Consider too that we’re planning a new i386 ABI, which is identical to > the current i386 ABI, except time_t is 64 bit. We’ve penciled in the > name i386t64 for this ABI/MACHINE_ARCH. This will be much less painful, > it is thought, than finding all the current places that take time_t as > an arg and shimming… So if we were to do that, then your current scheme > wouldn’t be able to encompass that eventuality and we’d be back to this > argument... Nobody will use this new ABI on amd64 when x32 is supported. X32 has more registers available and a far better calling convention. >> This also does not preclude the existence of a mipsn32 MACHINE_ARCH in >> case you want to build a pure mipsn32 system (including the kernel), but >> I wonder how many people would use that if it is possible to have a >> mips64 system run n32 binaries. I don't think many people would run a >> pure x32 system (with x32 kernel) so I don't see the need to have an >> "x32" value for MACHINE_ARCH (or TARGET_ARCH). > > Actually, you absolutely must have a x32 MACHINE_ARCH if you want to be > able to run x32 binaries. Otherwise, how are you going to build the > libraries that use that API? Our build system simply isn’t setup to > build them any other way. Although the usr/lib32 stuff could grow extra > goo for that, you are still building them the same way you’d build. We > have a stylized way to create a sys root, which is needed for the > compilers to work. We’d likely have to grow better multi lib support > than we have now as well. You’d need some way to identify these binaries, > segregate their ld.so, etc. Even if no kernel ever is linked this way, > it is still an absolute requirement. I'm kind of hoping we can close this discussion now and agree to disagree on the points that we still disagree, so I don't really want to discuss our build system. We'd be off for another week. I understand why we build compat32 libraries the way we do now. It was the only way at the time. In my opinion the choice between 32-bit or 64-bit should be like choosing between static or shared, pic or non-pic, or with or without profiling. It's just a compiler flag while our current build system treats it almost like a full blown cross compilation problem.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140528125027.6d0cc4fb>