Date: Mon, 25 Nov 2019 23:26:46 +0500 From: Ruslan Garipov <ruslanngaripov@gmail.com> To: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-current@freebsd.org Subject: Re: buildworld on CPU-A, installworld on CPU-B ends up with SIGILL Message-ID: <a03e4c56-1c20-9b68-2a27-b3f6f91da18b@gmail.com> In-Reply-To: <bf490832-9433-f5a4-afe2-2463652f453b@quip.cz> References: <5596338e-134c-9849-de9e-710d3106687f@gmail.com> <bf490832-9433-f5a4-afe2-2463652f453b@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/25/2019 10:30 PM, Miroslav Lachman wrote: > Ruslan Garipov wrote on 2019/11/25 15:06: >> Hello. >> >> I want to build kernel and world (of FreeBSD 13.0-CURRENT) on a fast >> virtual machine for other ones (all the virtual machines are hosted on >> VMware EXSi hypervisors, which have different physical CPUs). >> >> After `make -j16 buildworld` has finished successfully on the build >> machine, I get there, for example, >> /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install program having the >> shlxq instruction (one from the BMI2 instruction set extensions). This >> eventually causes make installkernel and make installworld to fail with >> SIGILL on a virtual machine which must consume built world and kernel, >> and which is hosted on another ESXi instance, with older physical CPU >> (read: with CPU not implementing shlxq). >> >> The build machine (FreeBSD 13.0-CURRENT r354802) builds (x)install using >> the following commands (a part of buildworld): >> >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD >> -MF.depend.xinstall.o -MTxinstall.o -std=gnu99 -Wno-format-zero-length >> -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include >> -c /usr/src/usr.bin/xinstall/xinstall.c -o xinstall.o >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -MD >> -MF.depend.getid.o -MTgetid.o -std=gnu99 -Wno-format-zero-length >> -Qunused-arguments -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include >> -c /usr/src/contrib/mtree/getid.c -o getid.o >> $ cc -O2 -pipe -O2 -march=x86-64 -pipe -I/usr/src/contrib/mtree >> -I/usr/src/lib/libnetbsd -DHAVE_STRUCT_STAT_ST_FLAGS=1 -std=gnu99 >> -Wno-format-zero-length -Qunused-arguments >> -I/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/include -static >> -L/usr/obj/usr/src/amd64.amd64/tmp/legacy/usr/lib -o xinstall xinstall.o >> getid.o -L/usr/obj/usr/src/amd64.amd64/tmp/obj-tools/lib/libmd -lmd -legacy >> >> This produces xintstall with `shlxq`s: >> >> $ llvm-objdump --disassemble xinstall | grep -c shlxq >> 135 >> >> I believe statically linked /usr/lib/libmd.a is a stuff which brings >> `shlxq`s into the xinstall. I didn't examine it further, sorry... >> >> My question is: is it possible to buildworld without issuing >> instructions which are native for the host CPU? I have neither >> /etc/make.conf, nor /etc/src.conf on the build machine. Is it possible >> at all for FreeBSD CURRENT to be built outside a host and installed on >> the host later? >> >> Just as a reference: >> >> My build machine has Intel(R) Xeon(R) Gold 6150 CPU that supports BMI2: >> >> # cpucontrol -i 7 /dev/cpuctl0 >> cpuid level 0x7: 0x00000000 0xd19f6ffb 0x00000018 0xbc000000 >> >> (Bit 08 in EBX is set) >> >> , and a consuming machine has Intel(R) Xeon(R) CPU E5-4617 CPU that >> doesn't support BMI2: >> >> # cpucontrol -i 7 /dev/cpuctl0 >> cpuid level 0x7: 0x00000000 0x00000002 0x00000000 0xbc000000 >> >> (Bit 08 in EBX is unset). >> >> Both machines install kernel and world without any problem, if they were >> built locally. > > I didn't tried this with current but I am using it with stable (11.3 at > this time). Building on Xeon E3-1240v3 and installing on many different > machines. Some of them are 10+ years old AMD Opteron, some Xeon E5649, > some 10 years old Intel Pentium. > So at least it worked in the past (11.3 amd64). Did you use this > workflow in the past / did it work? No, unfortunately I didn't. Always built world/kernel on target host. > I remember some issue in the past which was (accidentaly?) fixed by > running "make buildworld && make builkernel && make installkernel && > make installworld" on the build host (to some different DESTDIR) and > then "make installkernel && make installworld" on the target host (build > machine is shared via NFS) Therefore, this trick somehow "fixes" /usr/obj shared on the build machine? I'll try this later. Thanks! > > Miroslav Lachman >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a03e4c56-1c20-9b68-2a27-b3f6f91da18b>