From owner-freebsd-standards@FreeBSD.ORG Mon Jul 2 11:07:22 2012 Return-Path: Delivered-To: freebsd-standards@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 877051065673 for ; Mon, 2 Jul 2012 11:07:22 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 4055A8FC15 for ; Mon, 2 Jul 2012 11:07:22 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q62B7MGk012753 for ; Mon, 2 Jul 2012 11:07:22 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q62B7LwH012751 for freebsd-standards@FreeBSD.org; Mon, 2 Jul 2012 11:07:21 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 2 Jul 2012 11:07:21 GMT Message-Id: <201207021107.q62B7LwH012751@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-standards@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-standards@FreeBSD.org X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 11:07:22 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o stand/166349 standards Support the assignment-allocation character for fscanf o stand/165236 standards The NONE Wi-Fi regulatory restricts use of channels 12 o stand/164787 standards dirfd() function not available when _POSIX_C_SOURCE is o kern/164674 standards [patch] [libc] vfprintf/vfwprintf return error (EOF) o o stand/162434 standards getaddrinfo: addrinfo.ai_family is an address family, o stand/154842 standards invalid request authenticator in the second and subseq o stand/150093 standards C++ std::locale support is broken o stand/130067 standards Wrong numeric limits in system headers? o stand/124860 standards flockfile(3) doesn't work when the memory has been exh o stand/121921 standards [patch] Add leap second support to at(1), atrun(8) o stand/116477 standards rm(1): rm behaves unexpectedly when using -r and relat o bin/116413 standards incorrect getconf(1) handling of unsigned constants gi o stand/116081 standards make does not work with the directive sinclude p stand/107561 standards [libc] [patch] [request] Missing SUS function tcgetsid o stand/100017 standards [Patch] Add fuser(1) functionality to fstat(1) a stand/86484 standards [patch] mkfifo(1) uses wrong permissions o stand/82654 standards C99 long double math functions are missing o stand/81287 standards [patch] fingerd(8) might send a line not ending in CRL a stand/80293 standards sysconf() does not support well-defined unistd values o stand/79056 standards [feature request] [atch] regex(3) regression tests o stand/70813 standards [patch] ls(1) not Posix compliant o stand/66357 standards make POSIX conformance problem ('sh -e' & '+' command- s kern/64875 standards [libc] [patch] [request] add a system call: fdatasync( o stand/56476 standards [patch] cd9660 unicode support simple hack o stand/54410 standards one-true-awk not POSIX compliant (no extended REs) o stand/46119 standards Priority problems for SCHED_OTHER using pthreads o stand/44365 standards [headers] [patch] [request] introduce ulong and unchar a stand/41576 standards ln(1): replacing old dir-symlinks a docs/26003 standards getgroups(2) lists NGROUPS_MAX but not syslimits.h s stand/24590 standards timezone function not compatible witn Single Unix Spec o stand/21519 standards sys/dir.h should be deprecated some more s bin/14925 standards getsubopt isn't poisonous enough 32 problems total. From owner-freebsd-standards@FreeBSD.ORG Mon Jul 2 20:28:45 2012 Return-Path: Delivered-To: freebsd-standards@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id BC3431065673 for ; Mon, 2 Jul 2012 20:28:45 +0000 (UTC) (envelope-from bjk@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A6E3C8FC14 for ; Mon, 2 Jul 2012 20:28:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q62KSjZQ096430 for ; Mon, 2 Jul 2012 20:28:45 GMT (envelope-from bjk@freebsd.org) Received: from localhost (bjk@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) with ESMTP id q62KSj36096427 for ; Mon, 2 Jul 2012 20:28:45 GMT (envelope-from bjk@freebsd.org) X-Authentication-Warning: freefall.freebsd.org: bjk owned process doing -bs Date: Mon, 2 Jul 2012 20:28:45 +0000 (UTC) From: Benjamin Kaduk To: freebsd-standards@freebsd.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: FD_SETSIZE: signed or unsigned? X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Jul 2012 20:28:45 -0000 Hi all, We've had an unsigned FD_SETSIZE since 2002 (r90135, by markm): %%%%%%%%%%%%%%%%%%%%%%%% Revision 90135 - (show annotations) Sun Feb 3 11:36:59 2002 UTC (10 years, 5 months ago) by markm File MIME type: text/plain File size: 7279 byte(s) Zero functional difference; make some integer constants unsigned, as they are used in unsigned context. This shuts lint(1) up in a few significant ways with "signed/unsigned" arithmetic warnings. %%%%%%%%%%%%%%%%%%%%%%%% Yet NetBSD, Linux, and OpenBSD all have signed (plain int) FD_SETSIZEs. This has led to various pieces of software introducing casts and workarounds for the FreeBSD behavior, which (apparently) was itself made in order to attempt to reduce the occurence of warnings. Before I go and introduce such workarounds into our codebase for $work, I wanted to check with the standards gurus to see whether there is any chance that FreeBSD is what needs fixing, first. Bruce touched on some related issues on freebsd-security a few months later in 2002 ("[provos@citi.umich.edu: OpenBSD Security Advisory: Select Boundary Condition]"), but I can't really derive anything decisive from it. In any case, it is clear that a negative value for FD_SETSIZE is nonsensical. Doing a quick survey, it seems to mostly be used as the first argument to select(2) (which is an int type argument), and as a bound check on fd numbers (before using them in select). Given the former, it seems dubious to consider ever using a value between INT_MAX and UINT_MAX, which does leave open what the actual type should be. POSIX is delightfully vague: %%%%%%%%%%%%%%%%%%%%%% The following shall be defined as a macro: FD_SETSIZE Maximum number of file descriptors in an fd_set structure. %%%%%%%%%%%%%%%%%%%%%% Further thoughts on the matter are welcome. -Ben From owner-freebsd-standards@FreeBSD.ORG Tue Jul 3 17:17:52 2012 Return-Path: Delivered-To: freebsd-standards@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 71BEB1065670; Tue, 3 Jul 2012 17:17:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail06.syd.optusnet.com.au (mail06.syd.optusnet.com.au [211.29.132.187]) by mx1.freebsd.org (Postfix) with ESMTP id 0799D8FC18; Tue, 3 Jul 2012 17:17:48 +0000 (UTC) Received: from c122-106-171-232.carlnfd1.nsw.optusnet.com.au (c122-106-171-232.carlnfd1.nsw.optusnet.com.au [122.106.171.232]) by mail06.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id q63HHdT7003863 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 4 Jul 2012 03:17:41 +1000 Date: Wed, 4 Jul 2012 03:17:39 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Benjamin Kaduk In-Reply-To: Message-ID: <20120704022929.L2234@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-standards@FreeBSD.org Subject: Re: FD_SETSIZE: signed or unsigned? X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Jul 2012 17:17:52 -0000 On Mon, 2 Jul 2012, Benjamin Kaduk wrote: > We've had an unsigned FD_SETSIZE since 2002 (r90135, by markm): > %%%%%%%%%%%%%%%%%%%%%%%% > Revision 90135 - (show annotations) > Sun Feb 3 11:36:59 2002 UTC (10 years, 5 months ago) by markm > File MIME type: text/plain > File size: 7279 byte(s) > > Zero functional difference; make some integer constants unsigned, as > they are used in unsigned context. This shuts lint(1) up in a few > significant ways with "signed/unsigned" arithmetic warnings. > %%%%%%%%%%%%%%%%%%%%%%%% > > Yet NetBSD, Linux, and OpenBSD all have signed (plain int) FD_SETSIZEs. > > This has led to various pieces of software introducing casts and workarounds > for the FreeBSD behavior, which (apparently) was itself made in order to > attempt to reduce the occurence of warnings. I didn't like it at the time, and got the similar changes to NBBY and howmany() backed out. Only the unsigned constant in howmany() was obviously wrong. It caused promotion of nearby int args to unsigned, so for example howmany(-1, 1) was UINT_MAX with type unsigned instead of -1 with type int. OTOH, howmany(-1L, 1) was -1L on arches with sizeof(u_int) < sizeof(long) (assume no padding bits throughout), but was (long)UINT_MAX on arches with sizeof(u_int) == sizeof(long). (long)UINT_MAX is implementation-defined and is normally also -1L. Maybe negative howmany()'s aren't useful, but the unsigned constant also makes the type of howmany() unsigned when it should be int, giving the same problem as for FD_SETSIZE. You don't want to know about these complications, and should let the default binary promotions work normally by not having any unusual types hidden in the implementation of howmany(). Especially when previous and other implementations dont't have them. If you really want unsigned behaviour in howmany(), then you can pass it unsigned args (howmany((x), 0U + (y)) would give the same behaviour as the hidden 1U). > Before I go and introduce such workarounds into our codebase for $work, I > wanted to check with the standards gurus to see whether there is any chance > that FreeBSD is what needs fixing, first. I would prefer to change it back to plain int. > Bruce touched on some related issues on freebsd-security a few months later > in 2002 ("[provos@citi.umich.edu: OpenBSD Security Advisory: Select Boundary > Condition]"), but I can't really derive anything decisive from it. I had forgotten about that (but complained about magic casts to `unsigned int' in file descriptor code recently :-)). > In any case, it is clear that a negative value for FD_SETSIZE is nonsensical. But comparing "int fd" with FD_SETSIZE makes sense, and having FD_SIZE a different type than int mainly gives sign extension problems. The problems are mostly only potential, but the compiler doesn't know that and gives actual problems by warning about the potential ones; -Werror then makes these fatal. IMO, unsigned types should never be used just because the values in them are expected to be always nonnegative. Use of unsigned types mainly gives (un)sign extension bugs when unsigned types are mixed with signed types in expressions. > Doing a quick survey, it seems to mostly be used as the first argument to > select(2) (which is an int type argument), and as a bound check on fd numbers > (before using them in select). Given the former, it seems dubious to > consider ever using a value between INT_MAX and UINT_MAX, which does leave > open what the actual type should be. The value of (INT_MAX + 1U) actually makes sense: if you have fds numbered 0 to INT_MAX, then the number of fds is (INT_MAX + 1U). The POSIX history reminded me of this -- old versions had an off-by-1 error for nfds, with nfds >= FD_SETSIZE an error, but it is nfds > FD_SETSIZE that is the error. Anyway, nfds is int, so it cannot be (INT_MAX + 1U), and FD_SETSIZE being (INT_MAX + 1U) is not useful since apart from being more than large enough for anyone (at least with 32 bit ints), nfds can never equal it. > POSIX is delightfully vague: > %%%%%%%%%%%%%%%%%%%%%% > The following shall be defined as a macro: > > FD_SETSIZE > > Maximum number of file descriptors in an fd_set structure. > %%%%%%%%%%%%%%%%%%%%%% The 2001 and 2007 (draft) versions are equally vague. Bruce From owner-freebsd-standards@FreeBSD.ORG Sat Jul 7 15:00:24 2012 Return-Path: Delivered-To: freebsd-standards@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 030C21065672 for ; Sat, 7 Jul 2012 15:00:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id AB9618FC0C for ; Sat, 7 Jul 2012 15:00:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q67F0Nj8004776 for ; Sat, 7 Jul 2012 15:00:23 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q67F0Nnh004775; Sat, 7 Jul 2012 15:00:23 GMT (envelope-from gnats) Resent-Date: Sat, 7 Jul 2012 15:00:23 GMT Resent-Message-Id: <201207071500.q67F0Nnh004775@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-standards@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Volodymyr Kostyrko Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 42C271065672 for ; Sat, 7 Jul 2012 14:50:34 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id 2E35D8FC16 for ; Sat, 7 Jul 2012 14:50:34 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q67EoXYW038200 for ; Sat, 7 Jul 2012 14:50:33 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id q67EoX2H038199; Sat, 7 Jul 2012 14:50:33 GMT (envelope-from nobody) Message-Id: <201207071450.q67EoX2H038199@red.freebsd.org> Date: Sat, 7 Jul 2012 14:50:33 GMT From: Volodymyr Kostyrko To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: standards/169697: syslogd(8) is not BOM aware X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Jul 2012 15:00:24 -0000 >Number: 169697 >Category: standards >Synopsis: syslogd(8) is not BOM aware >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-standards >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Jul 07 15:00:23 UTC 2012 >Closed-Date: >Last-Modified: >Originator: Volodymyr Kostyrko >Release: RELENG_9 >Organization: None >Environment: # uname -a FreeBSD limbo.xim.bz 9.0-STABLE FreeBSD 9.0-STABLE #2 r238059M: Tue Jul 3 15:27:12 EEST 2012 arcade@limbo.xim.bz:/usr/obj/usr/src/sys/MINIMALx32 i386 >Description: Sending a UTF-8 formatted string starting with BOM to syslogd via /dev/log almost works: #!/usr/bin/evn python from __future__ import unicode_literals import logging, logging.handlers logger = logging.getLogger('test') handler = logging.handlers.SysLogHandler('/dev/log') handler.setFormatter(logging.Formatter('%(name)s[%(process)s]: %(message)s')) logger.addHandler(handler) logger.critical('test') This results in following line in log: Jul 7 17:31:19 limbo test[9154]: test But hexdump of it shows: 00000000 4a 75 6c 20 20 37 20 31 37 3a 32 37 3a 32 37 20 |Jul 7 17:27:27 | 00000010 6c 69 6d 62 6f 20 ef bb bf 74 65 73 74 5b 35 36 |limbo ...test[56| 00000020 37 33 5d 3a 20 74 65 73 74 0a |73]: test.| 0000002a Note the BOM before logger name. It account as part of name, and such messages can't be rerouted via /etc/syslogd.conf as they just doesn't match. Including BOM field into the logs is also not a good thing. >How-To-Repeat: >Fix: I think BOM can be safely dropped if: - line starts with BOM; - '-8' switch is given. >Release-Note: >Audit-Trail: >Unformatted: