From owner-freebsd-standards@FreeBSD.ORG Mon Mar 16 11:07:03 2009 Return-Path: Delivered-To: freebsd-standards@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AFC681065686 for ; Mon, 16 Mar 2009 11:07:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 9CD828FC15 for ; Mon, 16 Mar 2009 11:07:03 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id n2GB73ac043409 for ; Mon, 16 Mar 2009 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id n2GB738a043405 for freebsd-standards@FreeBSD.org; Mon, 16 Mar 2009 11:07:03 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 16 Mar 2009 11:07:03 GMT Message-Id: <200903161107.n2GB738a043405@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-standards@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-standards@FreeBSD.org X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Mar 2009 11:07:04 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o stand/130067 standards Wrong numeric limits in system headers? o stand/129554 standards lp(1) [patch] Implement -m and -t options o stand/129524 standards FreeBSD 7.0 isnt detecting my hardrives with raid5 o stand/128546 standards ls -p does not follow symlinks o bin/125855 standards sh(1) allows for multiline, non-escaped control struct o stand/124860 standards flockfile(3) doesn't work when the memory has been exh o stand/123688 standards POSIX standard changes in unistd.h and grp.h o stand/121921 standards [patch] Add leap second support to at(1), atrun(8) o stand/121568 standards [patch] ln(1): wrong "ln -s" behaviour o stand/120947 standards xsm ignores system.xsm and .xsmstartup o stand/119804 standards [locale] [patch] Invalid (long)date format in pl_PL.IS o stand/116826 standards [patch] sh support for POSIX character classes o stand/116477 standards rm(1): rm behaves unexpectedly when using -r and relat o bin/116413 standards incorrect getconf(1) handling of unsigned constants gi o stand/116081 standards make does not work with the directive sinclude p stand/107561 standards [libc] [patch] [request] Missing SUS function tcgetsid o stand/104743 standards [headers] [patch] Wrong values for _POSIX_ minimal lim o stand/100017 standards [Patch] Add fuser(1) functionality to fstat(1) o stand/96236 standards [patch] [posix] sed(1) incorrectly describes a functio o stand/96016 standards [headers] clock_getres et al should be in o stand/94729 standards [libc] fcntl() throws undocumented ENOTTY o kern/93705 standards [headers] [patch] ENODATA and EGREGIOUS (for glibc com o stand/92362 standards [headers] [patch] Missing SIGPOLL in kernel headers a stand/86484 standards [patch] mkfifo(1) uses wrong permissions o stand/83845 standards [libm] [patch] add log2() and log2f() support for libm o stand/82654 standards C99 long double math functions are missing o stand/81287 standards [patch] fingerd(8) might send a line not ending in CRL a stand/80293 standards sysconf() does not support well-defined unistd values o stand/79056 standards [feature request] [atch] regex(3) regression tests o stand/70813 standards [patch] ls(1) not Posix compliant o stand/66357 standards make POSIX conformance problem ('sh -e' & '+' command- s kern/64875 standards [libc] [patch] [request] add a system call: fdatasync( s stand/62858 standards malloc(0) not C99 compliant o stand/56476 standards cd9660 unicode support simple hack p stand/55112 standards glob.h, glob_t's gl_pathc should be "size_t", not "int o stand/54839 standards [pcvt] pcvt deficits o stand/54833 standards [pcvt] more pcvt deficits o stand/54410 standards one-true-awk not POSIX compliant (no extended REs) o stand/46119 standards Priority problems for SCHED_OTHER using pthreads o stand/44425 standards getcwd() succeeds even if current dir has perm 000. p stand/41576 standards POSIX compliance of ln(1) o stand/39256 standards snprintf/vsnprintf aren't POSIX-conformant for strings s stand/36076 standards Implementation of POSIX fuser command o kern/27835 standards [libc] execve() doesn't conform to execve(2) spec in s a docs/26003 standards getgroups(2) lists NGROUPS_MAX but not syslimits.h o stand/25777 standards [kernel] [patch] atime not updated on exec o bin/25542 standards sh(1) null char in quoted string s stand/24590 standards timezone function not compatible witn Single Unix Spec o bin/24390 standards ln(1) Replacing old dir-symlinks when using /bin/ln o stand/21519 standards sys/dir.h should be deprecated some more s bin/14925 standards getsubopt isn't poisonous enough 51 problems total. From owner-freebsd-standards@FreeBSD.ORG Wed Mar 18 12:13:50 2009 Return-Path: Delivered-To: freebsd-standards@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A046A1065C41 for ; Wed, 18 Mar 2009 12:13:50 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 1DDFF8FC18 for ; Wed, 18 Mar 2009 12:13:49 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id n2ICDPPX042351; Wed, 18 Mar 2009 13:13:48 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id n2ICDPpT042350; Wed, 18 Mar 2009 13:13:25 +0100 (CET) (envelope-from olli) Date: Wed, 18 Mar 2009 13:13:25 +0100 (CET) Message-Id: <200903181213.n2ICDPpT042350@lurza.secnetix.de> From: Oliver Fromme To: freebsd-standards@FreeBSD.ORG X-Newsgroups: list.freebsd-standards User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Wed, 18 Mar 2009 13:13:49 +0100 (CET) Cc: Subject: Suspecting bug in /bin/sh's IFS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Mar 2009 12:13:53 -0000 Hello, According to the sh(1) manpage and the SUSv3 wording, the order of characters in the IFS variable should not matter for field splitting. However, I get strange results with our /bin/sh ... I'm trying to parse config files that look like this: ip = 10.1.2.3 host = some.name desc = some description here That is, the format of lines is this: * "=" * Of course that should be simple: Just add "=" to $IFS and read the lines in a loop, like the following (the echo command is for debugging; the real shell code uses the setvar command instead): IFS="=$IFS" while read key val; do echo "'$key' -- '$val'" done < config However, the "=" characters and some spaces are included in $val, which is wrong behaviour, I think. I get this output: 'ip' -- ' = 10.1.2.3' 'host' -- '= some.name' 'desc' -- '= some description here' If I change the IFS line like this (diff -u format): - IFS="=$IFS" + IFS="$IFS=" then I get the correct and expected output: 'ip' -- '10.1.2.3' 'host' -- 'some.name' 'desc' -- 'some description here' Note that the characters in IFS are exactly the same, only the order is different. On Solaris, the same shell script produces correct output in both cases, independent from the order of characters in IFS. I've also tested zsh and bash: They also produce correct output in both cases. So this really seems to be a bug in FreeBSD's /bin/sh. I'm about to file a PR, but I suppose it's better to ask the standards mailinglist first, so here I am. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd From owner-freebsd-standards@FreeBSD.ORG Thu Mar 19 14:13:04 2009 Return-Path: Delivered-To: freebsd-standards@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5907810656D0 for ; Thu, 19 Mar 2009 14:13:04 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id CDAF48FC20 for ; Thu, 19 Mar 2009 14:13:03 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id n2JECdDM011121; Thu, 19 Mar 2009 15:13:02 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id n2JECdbl011120; Thu, 19 Mar 2009 15:12:39 +0100 (CET) (envelope-from olli) Date: Thu, 19 Mar 2009 15:12:39 +0100 (CET) Message-Id: <200903191412.n2JECdbl011120@lurza.secnetix.de> From: Oliver Fromme To: freebsd-standards@FreeBSD.ORG In-Reply-To: <200903181213.n2ICDPpT042350@lurza.secnetix.de> X-Newsgroups: list.freebsd-standards User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.1.2 (lurza.secnetix.de [127.0.0.1]); Thu, 19 Mar 2009 15:13:02 +0100 (CET) Cc: Subject: Re: Suspecting bug in /bin/sh's IFS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Mar 2009 14:13:04 -0000 Further debugging reveals that this is not a generic problem with field splitting, but it affects the read command only. I tried to use "set" instead of "read": ORIG_IFS="$IFS" while read line; do IFS="$IFS=" set -- $line IFS="$ORIG_IFS" key="$1" shift val="$*" echo "'$key' -- '$val'" done < config Now i get correct output for both " \t\n=" and "= \t\n" as the IFS value. So the bug is in the "read" builtin. The following patch fixes the problem: --- bin/sh/miscbltin.c.orig 2006-02-04 15:37:50.000000000 +0100 +++ bin/sh/miscbltin.c 2009-03-19 15:01:43.000000000 +0100 @@ -188,7 +188,7 @@ } if (c == '\n') break; - if (startword && *ifs == ' ' && strchr(ifs, c)) { + if (startword && strchr(ifs, c)) { continue; } startword = 0; The bug seems to exist for at least 15 years; the bogus comparison was already present in the initial import in the FreeBSD CVS repository in 1994. Any opinions on this? Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "Whatever happened to the days when hacking started at the cerebral cortex, and not at the keyboard?" -- Sid on userfriendly.org by Illiad, 2007-06-20 From owner-freebsd-standards@FreeBSD.ORG Sat Mar 21 22:04:00 2009 Return-Path: Delivered-To: freebsd-standards@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8EDD1065673 for ; Sat, 21 Mar 2009 22:04:00 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mx1.stack.nl (meestal-mk5.stack.nl [131.155.140.149]) by mx1.freebsd.org (Postfix) with ESMTP id A8E448FC12 for ; Sat, 21 Mar 2009 22:04:00 +0000 (UTC) (envelope-from jilles@stack.nl) Received: by mx1.stack.nl (Postfix, from userid 65534) id CAAF33F846; Sat, 21 Mar 2009 22:47:00 +0100 (CET) X-Spam-DCC: : X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on meestal-mk5.stack.nl X-Spam-Level: X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,J_CHICKENPOX_63, NO_RELAYS autolearn=no version=3.2.3 X-Spam-Relay-Country: Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id B37A03FB85; Sat, 21 Mar 2009 22:46:47 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 0931B22892; Sat, 21 Mar 2009 22:46:34 +0100 (CET) Date: Sat, 21 Mar 2009 22:46:34 +0100 From: Jilles Tjoelker To: Oliver Fromme Message-ID: <20090321214634.GA36395@stack.nl> References: <200903181213.n2ICDPpT042350@lurza.secnetix.de> <200903191412.n2JECdbl011120@lurza.secnetix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200903191412.n2JECdbl011120@lurza.secnetix.de> X-Operating-System: FreeBSD 7.1-PRERELEASE i386 User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-standards@FreeBSD.ORG Subject: Re: Suspecting bug in /bin/sh's IFS X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Mar 2009 22:04:01 -0000 On Thu, Mar 19, 2009 at 03:12:39PM +0100, Oliver Fromme wrote: > Further debugging reveals that this is not a generic > problem with field splitting, but it affects the read > command only. > I tried to use "set" instead of "read": > ORIG_IFS="$IFS" > while read line; do > IFS="$IFS=" > set -- $line > IFS="$ORIG_IFS" > key="$1" > shift > val="$*" > echo "'$key' -- '$val'" > done < config > Now i get correct output for both " \t\n=" and "= \t\n" > as the IFS value. So the bug is in the "read" builtin. > The following patch fixes the problem: > --- bin/sh/miscbltin.c.orig 2006-02-04 15:37:50.000000000 +0100 > +++ bin/sh/miscbltin.c 2009-03-19 15:01:43.000000000 +0100 > @@ -188,7 +188,7 @@ > } > if (c == '\n') > break; > - if (startword && *ifs == ' ' && strchr(ifs, c)) { > + if (startword && strchr(ifs, c)) { > continue; > } > startword = 0; > The bug seems to exist for at least 15 years; the bogus > comparison was already present in the initial import in > the FreeBSD CVS repository in 1994. > Any opinions on this? The code is wrong, but your patched code is also wrong. The read builtin should use the same logic as normal field splitting, with additional rules if there are more fields in the input than variables. I have noticed that NetBSD has already fixed this. I have ported these fixes over: http://www.stack.nl/~jilles/unix/sh-read-split.patch The patch is against RELENG_7, I hope it applies to -CURRENT as well. The NetBSD commit message also refers to http://www.research.att.com/~gsf/public/ifs.sh Just like their /bin/sh, our /bin/sh with the patch now passes the 'read' tests from there (there are still many other failures though). By the way, to avoid all processing by read, one must IFS= read -r VAR. Without the IFS specification, leading and trailing whitespace will still be stripped. -- Jilles Tjoelker