From owner-freebsd-standards@FreeBSD.ORG Sun Sep 28 14:50:21 2003 Return-Path: Delivered-To: freebsd-standards@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B576316A4B3 for ; Sun, 28 Sep 2003 14:50:21 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24FE144029 for ; Sun, 28 Sep 2003 14:50:21 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h8SLoKFY009742 for ; Sun, 28 Sep 2003 14:50:20 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h8SLoKKG009740; Sun, 28 Sep 2003 14:50:20 -0700 (PDT) (envelope-from gnats) Date: Sun, 28 Sep 2003 14:50:20 -0700 (PDT) Message-Id: <200309282150.h8SLoKKG009740@freefall.freebsd.org> To: freebsd-standards@FreeBSD.org From: "James E. Flemer" Subject: Re: standards/57295: [patch] make does not include cmd line variables in MAKEFLAGS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "James E. Flemer" List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Sep 2003 21:50:21 -0000 The following reply was made to PR standards/57295; it has been noted by GNATS. From: "James E. Flemer" To: FreeBSD-gnats-submit@freebsd.org Cc: Subject: Re: standards/57295: [patch] make does not include cmd line variables in MAKEFLAGS Date: Sun, 28 Sep 2003 17:42:45 -0400 (EDT) The patch I submitted does not do necessary quoting when values contain spaces etc. It turns out that OpenBSD has a fix for the MAKEFLAGS issue that fits into FreeBSD make with a few modifications. Here is a patch based on the OpenBSD solution. Some changes were needed to match our data structures, but the functionality is directly from the OpenBSD commit. Obtained From: OpenBSD $OpenBSD: var.c,v 1.46 2000/07/18 20:17:20 espie Exp $ $OpenBSD: main.c,v 1.39 2000/07/18 20:17:20 espie Exp $ --- make.diff begins here --- Index: main.c =================================================================== RCS file: /home/ncvs/src/usr.bin/make/main.c,v retrieving revision 1.84 diff -u -r1.84 main.c --- main.c 14 Sep 2003 12:31:33 -0000 1.84 +++ main.c 28 Sep 2003 21:17:09 -0000 @@ -616,6 +616,10 @@ MainParseArgs(argc, argv); +#ifdef POSIX + Var_AddCmdline(MAKEFLAGS); +#endif + /* * Find where we are... * All this code is so that we know where we are when we start up Index: nonints.h =================================================================== RCS file: /home/ncvs/src/usr.bin/make/nonints.h,v retrieving revision 1.19 diff -u -r1.19 nonints.h --- nonints.h 28 Oct 2002 23:33:57 -0000 1.19 +++ nonints.h 28 Sep 2003 21:17:09 -0000 @@ -135,6 +135,7 @@ void Var_Append(char *, char *, GNode *); Boolean Var_Exists(char *, GNode *); char *Var_Value(char *, GNode *, char **); +void Var_AddCmdLine(char *); char *Var_Parse(char *, GNode *, Boolean, int *, Boolean *); char *Var_Subst(char *, char *, GNode *, Boolean); char *Var_GetTail(char *); Index: var.c =================================================================== RCS file: /home/ncvs/src/usr.bin/make/var.c,v retrieving revision 1.43 diff -u -r1.43 var.c --- var.c 18 Sep 2003 03:15:57 -0000 1.43 +++ var.c 28 Sep 2003 21:17:09 -0000 @@ -832,6 +832,42 @@ } +#ifdef POSIX + + +/* In POSIX mode, variable assignments passed on the command line are + * propagated to sub makes through MAKEFLAGS. + */ +void +Var_AddCmdline(char *name) +{ + const Var *v; + LstNode ln; + unsigned int i; + Buffer buf; + static const char quotable[] = " \t\n\\'\""; + char *s; + + buf = Buf_Init (MAKE_BSIZE); + + for (ln = Lst_First(VAR_CMD->context); ln != NULL; + ln = Lst_Succ(ln)) { + /* We assume variable names don't need quoting */ + v = (Var *)Lst_Datum(ln); + Buf_AddBytes(buf, strlen(v->name), v->name); + Buf_AddByte(buf, '='); + for (s = Buf_GetAll(v->val, (int *)NULL); *s != '\0'; s++) { + if (strchr(quotable, *s)) + Buf_AddByte(buf, '\\'); + Buf_AddByte(buf, *s); + } + Buf_AddByte(buf, ' '); + } + Var_Append(name, Buf_GetAll(buf, (int *)NULL), VAR_GLOBAL); + Buf_Destroy(buf, 1); +} +#endif + /*- *----------------------------------------------------------------------- * Var_Parse -- --- make.diff ends here --- From owner-freebsd-standards@FreeBSD.ORG Mon Sep 29 12:10:19 2003 Return-Path: Delivered-To: freebsd-standards@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BBDA716A4BF for ; Mon, 29 Sep 2003 12:10:19 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 222B643FDD for ; Mon, 29 Sep 2003 12:10:19 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h8TJAIFY039101 for ; Mon, 29 Sep 2003 12:10:19 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h8TJAIWs039100; Mon, 29 Sep 2003 12:10:18 -0700 (PDT) (envelope-from gnats) Date: Mon, 29 Sep 2003 12:10:18 -0700 (PDT) Message-Id: <200309291910.h8TJAIWs039100@freefall.freebsd.org> To: freebsd-standards@FreeBSD.org From: Dan Nelson Subject: Re: standards/57295: [patch] make does not include cmd linevariables in MAKEFLAGS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Dan Nelson List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2003 19:10:19 -0000 The following reply was made to PR standards/57295; it has been noted by GNATS. From: Dan Nelson To: freebsd-gnats-submit@FreeBSD.org, jflemer@alum.rpi.edu Cc: Subject: Re: standards/57295: [patch] make does not include cmd line variables in MAKEFLAGS Date: Mon, 29 Sep 2003 14:03:59 -0500 Doesn't POSIX allow our current behaviour? MAKEFLAGS The characters are formatted in a manner similar to a portion of the make utility command line: options are preceded by hyphens and -separated as described in the Base Definitions volume of IEEE Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines. The macro=value macro definition operands can also be included. The difference between the contents of MAKEFLAGS and the make utility command line is that the contents of the variable shall not be subjected to the word expansions (see Word Expansions) associated with parsing the command line values. It says that macro=value can be included, but doesn't have to be. From owner-freebsd-standards@FreeBSD.ORG Mon Sep 29 15:30:18 2003 Return-Path: Delivered-To: freebsd-standards@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3FA1416A4B3 for ; Mon, 29 Sep 2003 15:30:18 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2EE5B4400D for ; Mon, 29 Sep 2003 15:30:17 -0700 (PDT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h8TMUGFY059502 for ; Mon, 29 Sep 2003 15:30:16 -0700 (PDT) (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h8TMUG6o059501; Mon, 29 Sep 2003 15:30:16 -0700 (PDT) (envelope-from gnats) Date: Mon, 29 Sep 2003 15:30:16 -0700 (PDT) Message-Id: <200309292230.h8TMUG6o059501@freefall.freebsd.org> To: freebsd-standards@FreeBSD.org From: "James E. Flemer" Subject: Re: standards/57295: [patch] make does not include cmd linevariables in MAKEFLAGS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "James E. Flemer" List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2003 22:30:18 -0000 The following reply was made to PR standards/57295; it has been noted by GNATS. From: "James E. Flemer" To: freebsd-gnats-submit@FreeBSD.org Cc: Subject: Re: standards/57295: [patch] make does not include cmd line variables in MAKEFLAGS Date: Mon, 29 Sep 2003 18:19:31 -0400 Dan Nelson wrote: > Doesn't POSIX allow our current behaviour? > > MAKEFLAGS > > The characters are formatted in a manner similar to a portion of the > make utility command line: options are preceded by hyphens and > -separated as described in the Base Definitions volume of IEEE > Std 1003.1-2001, Section 12.2, Utility Syntax Guidelines. The > macro=value macro definition operands can also be included. The > difference between the contents of MAKEFLAGS and the make utility > command line is that the contents of the variable shall not be subjected > to the word expansions (see Word Expansions) associated with parsing the > command line values. > > It says that macro=value can be included, but doesn't have to be. I believe the paragraph you quote is referring to a user setting MAKEFLAGS in the environment before running make. In otherwords, if a user sets MAKEFLAGS="-D FOO BAR=baz", make should accept it. The paragraph of interest on what make sets MAKEFLAGS to before makefiles are processed is: ``Before the makefile(s) are read, all of the make utility command line options (except -f and -p) and make utility command line macro definitions (except any for the MAKEFLAGS macro), not already included in the MAKEFLAGS macro, shall be added to the MAKEFLAGS macro, quoted in an implementation-defined manner such that when MAKEFLAGS is read by another instance of the make command, the original macro's value is recovered.''[1] [1] IEEE Std 1003.1, 2003 Edition, http://www.opengroup.org/onlinepubs/007904975/utilities/make.html \ #tag_04_84_13_04 From owner-freebsd-standards@FreeBSD.ORG Wed Oct 1 08:33:44 2003 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E777616A4B3 for ; Wed, 1 Oct 2003 08:33:44 -0700 (PDT) Received: from colossus.systems.pipex.net (colossus.systems.pipex.net [62.241.160.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BE2243FF5 for ; Wed, 1 Oct 2003 08:33:41 -0700 (PDT) (envelope-from mark@thuvia.org) Received: from dotar.thuvia.org (81-86-228-29.dsl.pipex.com [81.86.228.29]) by colossus.systems.pipex.net (Postfix) with ESMTP id 7EDF716000867 for ; Wed, 1 Oct 2003 16:33:39 +0100 (BST) Received: from dotar.thuvia.org (localhost [127.0.0.1]) by dotar.thuvia.org (8.12.9/8.12.9) with ESMTP id h91FXdPA016654 for ; Wed, 1 Oct 2003 16:33:39 +0100 (BST) (envelope-from mark@dotar.thuvia.org) Received: (from mark@localhost) by dotar.thuvia.org (8.12.9/8.12.9/Submit) id h91FXdI1016653 for freebsd-standards@freebsd.org; Wed, 1 Oct 2003 16:33:39 +0100 (BST) (envelope-from mark) Message-Id: <200310011533.h91FXdI1016653@dotar.thuvia.org> From: Mark Valentine Date: Wed, 1 Oct 2003 16:33:39 +0000 X-Mailer: Mail User's Shell (7.2.6 beta(5) 10/07/98) To: freebsd-standards@freebsd.org Subject: sorting out the echo(1) mess X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2003 15:33:45 -0000 I've been working on revamping our rather broken printf(1), and since echo(1) is related and has fewer issues I thought I'd take a wee detour and see if there isn't just a tiny chance of getting a concensus on sorting that out first. My goal is to try to reach an agreement on what we should fix and what we should document in the way of standards conformance, backwards and sideways compatibility, and consistency within our own code base. A few outstanding issues I can see (please feel free to add any more you're aware of): (A) echo(1) has a minor compatibility issue (both with traditional implementations and current BSD implementations), by way of its half-baked \c handling; (B) echo(1) behaves like neither csh(1) nor sh(1) echo, thanks to \c and/or -e; (C) sh(1) echo is not POSIX-compliant (though our POSIX 2001 Utility Compliance states that it is), due to the -e option; the standard has changed wording from "need not" to "do not" to "shall not" [support any options] in recent revisions, though the current standard *does* special case -n (non-XSI profiles only); (D) our echo(1) and printf(1) %s/%b implementations are inconsistent, no matter which way you read the standard. Here are some proposals to resolve these issues: (1) (a) Remove the \c glitch in echo(1), thus restoring backwards compatibility (A) and [partly] consistency (B) - if this is unpalatable, then (b) document it as a compatibility bug and see (3b); (2) I'd really like to (a) ditch sh(1) echo -e altogether for compliance (C) and consistency (B), and tell people to use printf %b to get XSI echo behaviour (an informative section of the standard spells this out in detail); however, -e has been around too long and the other BSD implementations (as well as bash, zsh and so on) have the same non-conformance problem, so (b) document it as such instead (but otherwise keep echo(1) and sh(1) echo aligned, possibly even sharing code (not that there's much of it), as with test(1) and printf(1). See (3a). NOTE: printf(1) doesn't currently work well enough to serve as XSI echo - but watch this space... (3) optional: (a) build an XSI echo(1) variant (strictly conforming) and install it somewhere like /usr/xopen/echo; also, if we do (1b) rather than (1a), provide (b) a strictly-conforming BSD variant echo(1) - without -e - somewhere like /usr/posix/echo. I've thrown the idea of /usr/posix and /usr/xopen around in the past as ways of controlling the user's environment so far as standards conformance in utilities is concerned: PATH=/bin:/usr/bin # native BSD env (backwards compat) PATH=/usr/posix:/bin:/usr/bin # standard POSIX env + BSD PATH=/usr/xopen:/usr/posix:/bin:/usr/bin # alternate XSI environment (As a comparison, consider the following environments in Solaris: PATH=/usr/bin:/usr/ucb # native System V env (backwards compat) PATH=/usr/xpg4/bin:/usr/bin:/usr/ucb # standard POSIX+XSI env PATH=/usr/ucb:/usr/bin # alternate BSD environment Well, roughly!) /usr/posix is only necessary if we can't/won't make all utilities conform to POSIX (even without XSI) - but that can only be achieved - as we've already seen with stuff like sed(1) and expr(1) - by breaking backwards compatibility (but it's still an option, as painful as it is for some of us). /usr/xopen is only relevant if we care about any subset of XSI extensions which are incompatible with BSD (such as backslash escapes in echo(1)!). Other options are available, such as moving old compatible versions of incompatibly conformance-fixed utilities to a compat bin (I dread to see the day we need /usr/ucb on a BSD system, though!), or by way of environment variables (fragile) or system tunables (inflexible), but I've had experience of all of these variants over the years and an approach like the above is the only one that's worked well for me. (4) I'll address printf(1) issues seperately, but just be aware that there is a large overlap with echo(1), and so opting against fixing an echo(1) conformance issue may adversely affect the ability to fix a consistency issue between the two or to fix a printf(1) conformance issue. Anyway, my main issues right now can be summarised thus: * echo(1) can and should be both backwards compatible with traditional BSD behaviour and conform to IEEE Std 1003.1-2001; sh(1) echo should behave the same. * If we can't get away with implementing this, let's make sure the issues are documented in the right place. * If the default behaviour can't be made standards compliant for whatever reason, we should provide a way to create an environment that *does* conform to IEEE Std 1003.1-2001 (ISO/IEC 9945:2003), at least for the subset of the standard which is relevant to us. I volunteer both code and documentation (that's the easy bit, it's getting a concensus that's usually impossible! Do I get out the grey or the multi- coloured paint? ;-) Comments? I'll be back in a couple of days to to feel the apathy... Cheers, Mark. -- "Tigers will do ANYTHING for a tuna fish sandwich." "We're kind of stupid that way." *munch* *munch* --