From owner-freebsd-toolchain@freebsd.org Sun Jan 15 14:09:23 2017 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2054ACB16D5 for ; Sun, 15 Jan 2017 14:09:23 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-17.reflexion.net [208.70.210.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DAC251EF9 for ; Sun, 15 Jan 2017 14:09:22 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 29980 invoked from network); 15 Jan 2017 14:09:16 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 15 Jan 2017 14:09:16 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.20.1) with SMTP; Sun, 15 Jan 2017 09:09:16 -0500 (EST) Received: (qmail 30581 invoked from network); 15 Jan 2017 14:09:15 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 15 Jan 2017 14:09:15 -0000 Received: from [192.168.1.111] (c-67-170-167-181.hsd1.or.comcast.net [67.170.167.181]) by iron2.pdx.net (Postfix) with ESMTPSA id 270CDEC7B11; Sun, 15 Jan 2017 06:09:15 -0800 (PST) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: qemu-arm-static appears to have problems with signal delivery during (at least) poudrirer-devel based cross builds of some ports with ALLOW_MAKE_JOBS=yes Date: Sun, 15 Jan 2017 06:09:14 -0800 References: To: freebsd-arm , FreeBSD Toolchain In-Reply-To: Message-Id: <7AF92A3C-3563-4B2E-B14A-D6BAF30A16A2@dsl-only.net> X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 15 Jan 2017 14:09:23 -0000 On 2017-Jan-14, at 10:53 PM, Mark Millard = wrote: > [Context: head (12) -r312009 and ports head -r431413.] >=20 > I've been experimenting on amd64 with poudriere-devel with -x > for -a arm.armv6 and I ran into: >=20 >> TCG temporary leak before 00021826 >> qemu: uncaught target signal 4 (Illegal instruction) - core dumped >=20 > in 3 of the 31 ports for the build, but 4 skipped so 3 of 27 > attempted. The 00021826 is the same number in all the examples > so far (whatever its base). >=20 > These seem to be the only TCG messages and each failure starts with > one and then reports the qemu message. (Also true for the below.) > As far as I can tell the TCG notice is the report of an internal > qemu problem that is then translated into an Illegal instruction. >=20 > This was with ALLOW_MAKE_JOBS=3Dyes but -J 1 for poudriere. >=20 > For 2 of the problem ports retries worked, still using > ALLOW_MAKE_JOBS=3Dyes and -J 1 . >=20 > But the 3rd port failed each time tried with ALLOW_MAKE_JOBS=3Dyes > --but in a different step each time. >=20 > In all failure cases it was gmake that got the "illegal instruction". >=20 > But disabling ALLOW_MAKE_JOBS=3Dyes appears (so far) to avoid the > issue. For example, that 3rd failing port built fine. (I've > been doing more ports since, with ALLOW_MAKE_JOBS=3Dyes repeatedly > failing and lack of it working.) >=20 > My guess is SIGCHLD delivery sometimes touches something (or a timing) > that is not handled well in qemu-arm-static. I've had not problems > on an rpi2 or bpim3 in the past. >=20 > (I have seen some analogous "soemtimes" issues on powerpc under > and version of lang that mishandled the stack part of the ABI > FreeBSD uses, SIGCHLD sometimes getting on the stack at a bad-time > for the messed up code generation, leading to stack corruption. Code > not getting signals had no problems.) >=20 > Note: The amd64 context is FreeBSD under VirtualBox under macOS > and it has had no problem for native builds of world, kernel, > or ports. Avoiding ALLOW_MAKE_JOBS=3Dyes is not sufficient to guarantee builds will work. Here is one that got near the end before failing the same way: . . . install -m 0644 = /wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/gcc-6.3.0/gcc/cp/type-util= s.h = /wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/stage/usr/local/lib/gcc/ar= m-none-eabi/6.3.0/plugin/include/cp/type-utils.h install: DONTSTRIP set - will not strip installed binaries TCG temporary leak before 00021826 qemu: uncaught target signal 4 (Illegal instruction) - core dumped gmake[1]: *** [Makefile:4176: install-gcc] Illegal instruction gmake[1]: Leaving directory = '/wrkdirs/usr/ports/devel/arm-none-eabi-gcc/work/.build' *** Error code 2 Stop. make: stopped in /usr/ports/devel/arm-none-eabi-gcc =3D=3D=3D=3D>> Cleaning up wrkdir =3D=3D=3D> Cleaning for arm-none-eabi-gcc-6.3.0 build of devel/arm-none-eabi-gcc ended at Sun Jan 15 00:04:02 PST 2017 build time: 02:52:28 !!! build failure encountered !!! Going back to the earlier initial problem (that I happen to have the material for handy): expanding the .tbz of the failed build and finding the core showed: # find . -name "*.core" -exec file {} \; = = = = ./work/binutils-2.27/ld/qemu_gmake.core: ELF 32-bit LSB core file ARM, = version 1 (FreeBSD), FreeBSD-style, from 'ke' [I've not figured out what I can do with that --or how.] One thing unusual on my part is that I use -mcpu=3Dcortex-a7 . That matches how I historically buildworld buildkernel for installation on the rpi2 and bpim3. I've never had problems like this with builds on the rpi2 or the bpim3 (buildworld, buildkernel, port builds). It might be that qemu-arm-static has a problem with -mcpu=3Dcortex-a7 code that is generated --but not always. Using the make.conf as an example: # more /usr/local/etc/poudriere.d/head-cortex-a7-make.conf WANT_QT_VERBOSE_CONFIGURE=3D1 # DEFAULT_VERSIONS+=3Dperl5=3D5.24 WITH_DEBUG=3D WITH_DEBUG_FILES=3D MALLOC_PRODUCTION=3D # #system clang 3.8+ (gcc6 rejects -march=3Darmv7a): #CFLAGS+=3D -march=3Darmv7-a -mcpu=3Dcortex-a7 #CXXFLAGS+=3D -march=3Darmv7-a -mcpu=3Dcortex-a7 #CPPFLAGS+=3D -march=3Darmv7-a -mcpu=3Dcortex-a7 # #lang/gcc6's xgcc stage considers the above conflicting so use just: CFLAGS+=3D -mcpu=3Dcortex-a7 CXXFLAGS+=3D -mcpu=3Dcortex-a7 CPPFLAGS+=3D -mcpu=3Dcortex-a7 For my context poudriere with -x for -a arm.armv6 and the use of qemu-arm-static does not look reliable enough to depend on. It is not obvious that the -x use contributes to the problem: it may well not. =3D=3D=3D Mark Millard markmi at dsl-only.net