From owner-freebsd-hackers@FreeBSD.ORG Fri Apr 29 07:09:31 2011 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 248FB106566C for ; Fri, 29 Apr 2011 07:09:31 +0000 (UTC) (envelope-from rdivacky@vlakno.cz) Received: from vlakno.cz (lev.vlakno.cz [46.28.110.116]) by mx1.freebsd.org (Postfix) with ESMTP id A23438FC0C for ; Fri, 29 Apr 2011 07:09:30 +0000 (UTC) Received: by vlakno.cz (Postfix, from userid 1002) id EE5E67F3B02; Fri, 29 Apr 2011 09:09:28 +0200 (CEST) Date: Fri, 29 Apr 2011 09:09:28 +0200 From: Roman Divacky To: Hartmut Brandt Message-ID: <20110429070928.GA83618@freebsd.org> References: <20110427193946.GA41659@freebsd.org> <20110428174523.I61666@beagle.kn.op.dlr.de> <20110428173613.GA31077@freebsd.org> <20110428203709.M62691@beagle.kn.op.dlr.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110428203709.M62691@beagle.kn.op.dlr.de> User-Agent: Mutt/1.4.2.3i Cc: hackers@freebsd.org Subject: Re: make question X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2011 07:09:31 -0000 On Thu, Apr 28, 2011 at 08:50:27PM +0200, Hartmut Brandt wrote: > On Thu, 28 Apr 2011, Roman Divacky wrote: > > RD>On Thu, Apr 28, 2011 at 05:52:58PM +0200, Hartmut Brandt wrote: > RD>> Hi Roman, > RD>> > RD>> On Wed, 27 Apr 2011, Roman Divacky wrote: > RD>> > RD>> RD>You seem to have messed with bsd make so I have a question for you :) > RD>> > RD>> Yeah, that was some time ago ... > RD>> > RD>> RD>When a job is about to be executed in JobStart() a pipe is created with > RD>> RD>its ends connected to job->inPipe/job->outPipe. When the job is actually > RD>> RD>created in JobExec() the ps.out is set to job->outPipe so that in > RD>> RD>JobDoOutput() we can read from that pipe and basically just parse the output > RD>> RD>for shell->noPrint and leaving it out from the output. This is meant (I think) > RD>> RD>for supressing the "filter" thing. Ie. that if we do some @command the > RD>> RD>restoration of setting of quiet mode is filtered out. > RD>> RD> > RD>> RD> > RD>> RD>In -B mode we do it differently, as we invoke one shell per command we don't > RD>> RD>have to insert quiet/verbose commands and thus avoid all the piping/parsing > RD>> RD>dance. > RD>> RD> > RD>> RD>So my question is - why don't we invoke one shell per command by default > RD>> RD>and avoid the piping/parsing? Is this because of performance? I think that > RD>> RD>the piping/parsing of the output can have worse impact than invoking a shell > RD>> RD>for every command. Especially given that most targets consists of just one > RD>> RD>command. > RD>> > RD>> The answer is in /usr/share/doc/psd/12.make. This is so one can write > RD>> something like > RD>> > RD>> debug: > RD>> DEBUG_FLAGS=-g > RD>> for i in $(SUBDIR); do > RD>> $(MAKE) -C $$i all > RD>> done > RD>> > RD>> instead of: > RD>> > RD>> debug: > RD>> DEBUG_FLAGS=-g \ > RD>> for i in $(SUBDIR); do \ > RD>> $(MAKE) -C $$i all ; \ > RD>> done > RD>> > RD>> -B means 'backward compatible' and does what the original v7 make did: one > RD>> shell per command. This means you don't have to write the backslashes and > RD>> the shell variable will be seen in the sub-makes and programs. > RD>> > RD>> I think we can change this, because it would break makefiles that assume > RD>> that the entire script is given to the shell in one piece. > RD> > RD>I think you answered the question why we parse the target. But I asked why > RD>we parse the output from it. > > My intention was to say why we use one shell for all commands for a given > rule. If we'd use one shell per line the above would not work, because the > first shell would see just the environment variable assignment (which > would be completly useless). The next shell would see a partial 'for' > statement and complain, and would not have the environment variable and so > on. So this is not so much about parsing, but about execution. > > I suppose that the tricky point is with @-lines in the middle of a > multi-line script. Unless I am reading the code wrong the "one shell per command" is the default mode. see in main.c: /* * Be compatible if user did not specify -j and did not explicitly * turned compatibility on */ if (!compatMake && !forceJobs) compatMake = TRUE; You have to specify -j to turn off the compat mode. > RD>Anyway, so you think it would be ok to change it to one shell per command and > RD>avoid the shell output parsing or not? > > Unless I misunderstand the question I would say no, because this would > certainly render makefiles invalid that rely on the multi-line scripts > beeing handled by a single shell. I think the chances of this breakage are pretty low as this is the default mode. > RD>I am interested in this so that "make -j*" lets the command know that the > RD>output is a TTY, eg. clang can emit coloured warnings. > > Hmm. I see. Just a wild guess: couldn't we use a pty to talk to the shell? > If that could work the question is of course what one would expect from > something like: make 2>&1 >make.out I looked at what gnu make does and I think they do pretty much the same what I suggested. Ie. one shell per command with no parsing of output by printing directly to stdout. Thus they have no problem with the process detecting stdout being a tty.