From owner-freebsd-arch@freebsd.org Fri Feb 26 23:30:37 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1754AB5D9E for ; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9DF2F5FB for ; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 9CA88AB5D9D; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C412AB5D9C for ; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 8DB225FA; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [IPv6:::1]) by freefall.freebsd.org (Postfix) with ESMTP id 7E2D610C2; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 38455143AB; Fri, 26 Feb 2016 23:30:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id BVngh4SXjGib; Fri, 26 Feb 2016 23:30:29 +0000 (UTC) Subject: Re: Build -j target tags and command output DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com 40B5C143A5 To: "Simon J. Gerraty" References: <56D0CD68.606@FreeBSD.org> <77472.1456528808@kaos.jnpr.net> Cc: arch@freebsd.org From: Bryan Drewery X-Enigmail-Draft-Status: N1110 Organization: FreeBSD Message-ID: <56D0E017.4070305@FreeBSD.org> Date: Fri, 26 Feb 2016 15:30:31 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <77472.1456528808@kaos.jnpr.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2016 23:30:37 -0000 On 2/26/16 3:20 PM, Simon J. Gerraty wrote: > Bryan Drewery wrote: >> I'm looking for opinions on whether we should keep or remove the -- >=20 > Keep. >=20 > Without them it is virtually impossible to identify which job produced > certain output. >=20 > Even with them, the log from a parallel hirerarchical build can get > confusing - especially when multiple makes are writing at the same time= . > But you are still better off with some clues. >=20 >> Removing them would yield potentially hard-to-debug failures since the >=20 > Yes, pls don't do that. >=20 >> failed command could be anywhere. At least the 'make stopped' error >> that follows would print the directory. There is an interesting featu= re >=20 > Yes, but generally the exciting stuff that caused the stoppage is > earlier in the log. > The --- lines allow use of scripts to demux the output. >=20 >> in the meta mode build that will keep a log and print a more detailed >> error message on failures so you really know which directory failed an= d >> what environment it had, but it can be spammy as well since it prints >> the same error information on the 'another make hit an error, dying' c= ases. >=20 > Yes, the issue is largely moot with dirdeps/meta mode build. > As your example clearly demonstrates. Yes, though I would still suggest disabling .MAKE.MODE.PREFIX in that mode since it is not needed when a meta file is created due to the full path to object. Though we would need to resolve the non-meta targets not printing the 'Building' line somehow. Perhaps a way to make .MAKE.JOB.PREFIX fully changeable (not forcing the --- on the end) and supporting ${.TARGET} like .MAKE.META.PREFIX and then only showing it when not creating a meta file when in meta mode. But *still* show the command output since there is no .meta file to track the command for debugging. The point is we could get Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/clean rm -f *.o vs --- all_subdir --- --- all_subdir_bin --- --- all_subdir_bin/sh_clean --- --- clean --- rm -f *.o Of course the recursive printing by sub-makes would have to be disabled somehow in this mode. Backing up to simpler changes now though... I thought I asked you this before but I couldn't find the mail. Just having a '.JOBSILENT' on a target would be nice to hide subdir printing but not hide the command output (requiring .SILENT on clean: obj:, etc), and then also allowing ${.TARGET} to work in .MAKE.JOB.PREFIX so that we can get something like the following, where the job is only the final one and not any of the subdir ones. --- /usr/obj/root/git/freebsd/amd64.amd64/bin/sh (clean) --- rm -f *.o >=20 > Not all targets produce a .meta file, and for these you may still get > --- job lines, but they can hardly be considered an issue. >=20 > All the gory details - that you need to debug issues saved in > in the .meta file with no contamination from other jobs >=20 >> Longterm, I think a merge between DIRDEPS_BUILD output and the NetBSD >> build output makes sense and removes these --- lines entirely. It wou= ld >=20 > With dirdeps build the noise that NetBSD outputs is just noise: >=20 > Checking /tank/home/sjg/work/NetBSD/current/src/lib/csu for i386 ... > Building /tank/home/sjg/work/NetBSD/current/obj/i386/lib/csu/gcrt0.o > Building /tank/home/sjg/work/NetBSD/current/obj/i386/lib/csu/sysident_a= ssym.h > Building /tank/home/sjg/work/NetBSD/current/obj/i386/lib/csu/crtn.o > --- gcrt0.o --- > # compile csu/gcrt0.o > --- sysident_assym.h --- > # create csu/sysident_assym.h > --- crtn.o --- > # compile csu/crtn.o >=20 Well I didn't mean literally a merge, I meant more picking from both. Granted, the NetBSD method is very hard to implement fully and won't work for custom build targets and downstream build changes since it requires adding an echo into every target. >=20 >> DIRDEPS_BUILD output: >> >>> ~/git/freebsd # WITH_DIRDEPS_BUILD=3Dyes make -C bin/sh -j15 >> ... >>> --- /root/git/freebsd/bin/sh.amd64,amd64 1334 --- >>> @ 1456524548 [2016-02-26 14:09:08] Checking /root/git/freebsd/bin/sh = for amd64,amd64 ... >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/.dirdep >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/builtins.c >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/mknodes.o >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/mksyntax.o >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/token.h >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/stage_incs >=20 >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/stage_symlinks.= man1 >>> Building /usr/obj/root/git/freebsd/amd64.amd64/bin/sh/stage_links.man= 1 >>> Checking /root/git/freebsd/bin/sh/Makefile.depend: .dirdep.meta built= ins.c.meta mknodes.o.meta mksyntax.o.meta token.h.meta stage_incs.meta mk= syntax.meta mknodes.meta >>> @ 1456524550 [2016-02-26 14:09:10] Finished bin/sh.amd64,amd64 second= s=3D2 meta=3D49 created=3D49 --=20 Regards, Bryan Drewery