Date: Mon, 18 Jun 2018 15:33:56 -0700 From: Bryan Drewery <bdrewery@FreeBSD.org> To: Li-Wen Hsu <lwhsu@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: A head buildworld race visible in the ci.freebsd.org build history Message-ID: <f99c267b-d884-2007-5074-78d6e3176acb@FreeBSD.org> In-Reply-To: <CAKBkRUzdSyN-xVBs0G%2BQWZ5v%2BeGKfBbgKD931pvR3QE_5=fRdw@mail.gmail.com> References: <74EAD684-0E0B-453A-B746-156777CF604A@yahoo.com> <1884103f-d1fb-aca6-2edd-062e11d05617@FreeBSD.org> <20180618204517.GD2430@kib.kiev.ua> <068108ab-76f2-0f2d-fd92-11c838a4d391@FreeBSD.org> <CAKBkRUzdSyN-xVBs0G%2BQWZ5v%2BeGKfBbgKD931pvR3QE_5=fRdw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qOqODywPw7sr1YLQU9do3M2p18zqeWgUz Content-Type: multipart/mixed; boundary="qlTVijLjr7Vess76ZBjTCf4WFmsuLr5Od"; protected-headers="v1" From: Bryan Drewery <bdrewery@FreeBSD.org> To: Li-Wen Hsu <lwhsu@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Message-ID: <f99c267b-d884-2007-5074-78d6e3176acb@FreeBSD.org> Subject: Re: A head buildworld race visible in the ci.freebsd.org build history References: <74EAD684-0E0B-453A-B746-156777CF604A@yahoo.com> <1884103f-d1fb-aca6-2edd-062e11d05617@FreeBSD.org> <20180618204517.GD2430@kib.kiev.ua> <068108ab-76f2-0f2d-fd92-11c838a4d391@FreeBSD.org> <CAKBkRUzdSyN-xVBs0G+QWZ5v+eGKfBbgKD931pvR3QE_5=fRdw@mail.gmail.com> In-Reply-To: <CAKBkRUzdSyN-xVBs0G+QWZ5v+eGKfBbgKD931pvR3QE_5=fRdw@mail.gmail.com> --qlTVijLjr7Vess76ZBjTCf4WFmsuLr5Od Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 6/18/2018 3:31 PM, Li-Wen Hsu wrote: > On Mon, Jun 18, 2018 at 6:27 PM Bryan Drewery <bdrewery@freebsd.org> wr= ote: >> >> On 6/18/2018 1:45 PM, Konstantin Belousov wrote: >>> On Mon, Jun 18, 2018 at 12:42:46PM -0700, Bryan Drewery wrote: >>>> On 6/15/2018 10:55 PM, Mark Millard wrote: >>>>> In watching ci.freebsd.org builds I've seen a notable >>>>> number of one time failures, such as (example from >>>>> powerpc64): >>>>> >>>>> --- all_subdir_lib/libufs --- >>>>> ranlib -D libufs.a >>>>> ranlib: fatal: Failed to open 'libufs.a' >>>>> *** [libufs.a] Error code 70 >>>>> >>>>> where the next build works despite the change being >>>>> irrelevant to whatever ranlib complained about. >>>>> >>>>> Other builds failed similarly: >>>>> >>>>> --- all_subdir_lib/libbsm --- >>>>> ranlib -D libbsm_p.a >>>>> ranlib: fatal: Failed to open 'libbsm_p.a' >>>>> *** [libbsm_p.a] Error code 70 >>>>> >>>>> and: >>>>> >>>>> --- kerberos5/lib__L --- >>>>> ranlib -D libgssapi_spnego_p.a >>>>> --- libgssapi_spnego.a --- >>>>> ranlib -D libgssapi_spnego.a >>>>> --- libgssapi_spnego_p.a --- >>>>> ranlib: fatal: Failed to open 'libgssapi_spnego_p.a' >>>>> *** [libgssapi_spnego_p.a] Error code 70 >>>>> >>>>> and so on. >>>>> >>>>> >>>>> It is not limited to powerpc64. For example, for aarch64 >>>>> there are: >>>>> >>>>> --- libpam_exec.a --- >>>>> building static pam_exec library >>>>> ar -crD libpam_exec.a `NM=3D'nm' NMFLAGS=3D'' lorder pam_exec.o |= tsort -q` >>>>> ranlib -D libpam_exec.a >>>>> ranlib: fatal: Failed to open 'libpam_exec.a' >>>>> *** [libpam_exec.a] Error code 70 >>>>> >>>>> and: >>>>> >>>>> --- all_subdir_lib/libusb --- >>>>> ranlib -D libusb.a >>>>> ranlib: fatal: Failed to open 'libusb.a' >>>>> *** [libusb.a] Error code 70 >>>>> >>>>> and: >>>>> >>>>> --- all_subdir_lib/libbsnmp --- >>>>> ranlib: fatal: Failed to open 'libbsnmp.a' >>>>> --- all_subdir_lib/ncurses --- >>>>> --- all_subdir_lib/ncurses/panelw --- >>>>> --- panel.pico --- >>>>> --- all_subdir_lib/libbsnmp --- >>>>> *** [libbsnmp.a] Error code 70 >>>>> >>>>> >>>>> Even amd64 gets such: >>>>> >>>>> --- libpcap.a --- >>>>> ranlib -D libpcap.a >>>>> ranlib: fatal: Failed to open 'libpcap.a' >>>>> *** [libpcap.a] Error code 70 >>>>> >>>>> and: >>>>> >>>>> >>>>> --- libkafs5.a --- >>>>> ranlib: fatal: Failed to open 'libkafs5.a' >>>>> --- libkafs5_p.a --- >>>>> ranlib: fatal: Failed to open 'libkafs5_p.a' >>>>> --- cddl/lib__L --- >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua/lbaseli= b.c:60:26: note: include the header <ctype.h> or explicitly provide a dec= laration for 'toupper' >>>>> --- kerberos5/lib__L --- >>>>> *** [libkafs5_p.a] Error code 70 >>>>> >>>>> make[5]: stopped in /usr/src/kerberos5/lib/libkafs5 >>>>> --- libkafs5.a --- >>>>> *** [libkafs5.a] Error code 70 >>>>> >>>>> and: >>>>> >>>>> >>>>> --- lib__L --- >>>>> ranlib -D libclang_rt.asan_cxx-i386.a >>>>> ranlib: fatal: Failed to open 'libclang_rt.asan_cxx-i386.a' >>>>> *** [libclang_rt.asan_cxx-i386.a] Error code 70 >>>>> >>>>> >>>>> (Notice the variability in what .a the ranlib's fail for.) >>>>> >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> I looked at this a few days ago and don't believe it's actually a bu= ild >>>> race. I think there is something wrong with the ar/ranlib on that sy= stem >>>> or something else. I've found no evidence of concurrent building of = the >>>> .a files in question. >>> >>> FWIW, I got the similar failure when I did last checks for the OFED >>> commit. For me, it was libgcc.a. >>> >> >> If it was -lgcc_s then it's a known rare build race due to >> tools/install.sh not handling -S. >=20 > It seems a more general problem, this one: >=20 > https://ci.freebsd.org/job/FreeBSD-head-aarch64-build/8190/console >=20 > calls for libcuse_p.a, while this one: >=20 > https://ci.freebsd.org/job/FreeBSD-head-mips-build/2919/console >=20 > calls for libfifolog.a >=20 Well why is ar -> ranlib so special? Nothing else is failing. What filesystem are these using for objdirs? What revision is the host kernel? --=20 Regards, Bryan Drewery --qlTVijLjr7Vess76ZBjTCf4WFmsuLr5Od-- --qOqODywPw7sr1YLQU9do3M2p18zqeWgUz Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJbKDNUAAoJEDXXcbtuRpfPizUIALxtnZDaono3T4sjzUTN9qIM GqoVNd6mb+d61ntQ+UKp6hdgKni7tikPsqFiSf4aAC0NSj2FurbB4sgKKjdsCl8R WtzerM8WpPEy/ro2Ow5hxntGfW+F308W5dePXGS+ugkTqz3FMuJ0tGfNre/cEfHX f91iVrouAW45Bfg8IRIqdk2Py7aaDTRXUbcSYyiFb9jwQf0EWS5seR9SEu30Yjck MStso/CDuNBUpEzPfObww2lr7TaJFdg51KfshQ6cFkby6mzlylRMigsUlXcyR3Z7 Xzb7qvL3XPsBxgPYmFhemEqCIlX94lEvxBGtVzGrPT0k55BEB/DYoxVj5ym98N8= =O4Hw -----END PGP SIGNATURE----- --qOqODywPw7sr1YLQU9do3M2p18zqeWgUz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?f99c267b-d884-2007-5074-78d6e3176acb>