From owner-freebsd-standards@freebsd.org Sun Feb 25 01:00:03 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54D7BF31D4B for ; Sun, 25 Feb 2018 01:00:03 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yb0-x231.google.com (mail-yb0-x231.google.com [IPv6:2607:f8b0:4002:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1A2F8080A for ; Sun, 25 Feb 2018 01:00:02 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yb0-x231.google.com with SMTP id t18-v6so1017337ybt.0 for ; Sat, 24 Feb 2018 17:00:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Kd2dMGLPbr3pBn0Vyur84L0mMaQgOERiDPJKr1Ih2cQ=; b=OQIFbIZVCbgQmovPLajQScpy2RauAqwAhVpzCaZtWnPDvrmFADPTYGiSn8Lh181MVU R71uoSrotAtmNxnqAxT7cW3Jo8uKVzb2JovxG4M9X+P0TqFaHYgugsTRSNMIzvNVtdJu wvGH+VkpMJcVfxkZmssvmtgi02Ps2biL2tLuY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Kd2dMGLPbr3pBn0Vyur84L0mMaQgOERiDPJKr1Ih2cQ=; b=c5V1WHyryKrjUHHkW32p+Mk6yB8WIM5dQd0QhXOC/zVPffS2EJKJ3Z6oYs6YnWM/6H Hx5TlkWT0dPk3ipWEYuMkJd7WJRUWZnTMVoHxgSlzZMpUxtxsLDN1tjmguvx2LJh+TXO gUJDzYVTEM2dQ+XmJjg+hgKl2epNmbYpuVB91XR0j0hcB0bB4fRxNkSMRnGHM+/KQuB3 VqeYTnK1tdilwbwA8RJQvBPCFe3E2j3414E7t/Xyien6COPGM5LIILANv2DU/k3l0T9/ YgriYQJyCFAUQLHfXEuSzhr+k+tA4zQXHpdi8SVj9HfjAsd8v5JRW7j+F+3ech/xehzL ykWg== X-Gm-Message-State: APf1xPAnS2EZKTf96sw2snKetYfwXOQ1iO9BUmDq6tdnHz/BFMOhdCRG Rs7U71n6e69TJJN+gsnrIIzPupL6iD20ZVMyfhTbkg== X-Google-Smtp-Source: AG47ELtn0oF8c6fhskzkw64qQLxtk1SW2i7rWQAiVl7iqboIaZ6M2Q0TLsQgR9BdqLiSezaT5wN9oG4VlNtcWpRNYIE= X-Received: by 2002:a25:6006:: with SMTP id u6-v6mr4191129ybb.460.1519520402106; Sat, 24 Feb 2018 17:00:02 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a25:dfcb:0:0:0:0:0 with HTTP; Sat, 24 Feb 2018 16:59:31 -0800 (PST) In-Reply-To: References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> From: Eitan Adler Date: Sat, 24 Feb 2018 16:59:31 -0800 Message-ID: Subject: Re: Marking select(2) as restrict To: cem@freebsd.org Cc: FreeBSD Hackers , FreeBSD Standards Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 01:00:03 -0000 On 24 February 2018 at 10:55, Conrad Meyer wrote: > On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler wrote: >> After this entire thread here is the summary. If I've misrepresented >> you here please let me know. >> ... >> >> kib@ - no benefit; concerned fallout could be hard to observe >> cem@ - concerned about warnings > > Consider me a +1 to kib@. I did not voice those concerns explicitly > in earlier email because kib did already and I didn't anticipate you > would ignore him. I am not ignoring him. As I stated above I do not believe fallout is likely since most other major libc implementations have already done this: glibc: already done - https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 openbsd: already done https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 dragonflyBSD: alredy done: https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 netbsd: already done: https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 As a further check I went through the search results on github for select() and did not see any failures in the top few pages. -- Eitan Adler From owner-freebsd-standards@freebsd.org Sun Feb 25 01:28:50 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E726DF33D2A; Sun, 25 Feb 2018 01:28:49 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 3435B81D4B; Sun, 25 Feb 2018 01:28:48 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id D610A1A47E3; Sun, 25 Feb 2018 12:28:38 +1100 (AEDT) Date: Sun, 25 Feb 2018 12:28:37 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Warner Losh cc: "Conrad E. Meyer" , FreeBSD Standards , FreeBSD Hackers Subject: Re: Marking select(2) as restrict In-Reply-To: Message-ID: <20180225112627.B976@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=uyavkMrdAAAA:8 a=5jWAyVIvSIEuDVK_DRoA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 a=j2_G595jqNHTxQgNwHU2:22 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 01:28:50 -0000 On Sat, 24 Feb 2018, Warner Losh wrote: > On Sat, Feb 24, 2018 at 11:55 AM, Conrad Meyer wrote: > >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >> wrote: >>> After this entire thread here is the summary. If I've misrepresented >>> you here please let me know. >>> ... >>> >>> kib@ - no benefit; concerned fallout could be hard to observe >>> cem@ - concerned about warnings >> >> Consider me a +1 to kib@. I did not voice those concerns explicitly >> in earlier email because kib did already and I didn't anticipate you >> would ignore him. > > So there's no benefit to the change (we won't optimize better). It's hard > to observe breakage. No answer about how we'd even know if something broke > because a exp run sure as hell isn't going to tell us. > > All that militates against the change rather strongly. Your exp run will > change no minds because it is useless. Why not remove restrict from other APIs to be consistent with select()? If might break their callers just as much. Start with pselect(). sigaction() is interesting too. Without restrict for sigaction(), there is nothing (in old FreeBSD man pages) to prevent callers passing the same pointer for 'act' and 'oact'. This might be a good hack. 'act' is const, but will be overwritten on copyout if it is the same as 'oact'. The kernel could reasonably copyout to 'oact' before reading 'act'. This clobbers the input arg. I grepped all man pages in libc/sys for APIs taking 2 pointer args and not having restrict: _umtx_op() abort2() adjtime() aio_suspend() aio_waitcomplete() clock_nanosleep(), nanosleep() (most interesting. One pointer arg is const. It is unclear if that prevents aliasing, but the API is similar to that of sigaction() and POSIX added restrict for the latter only) execve() extattr_*() (at least 8 functions in man page with too many functions) fexecve() fhopen() fhstat() fhstatfs() fstatat() futimensat() getdirentries() getfh() getresgid() getresuid() gettimeofday() kenv() kevent() lgetfh() link() linkat() lio_listio() mincore() mount() mq_receive() mq_timedreceive() mq_timedsend() ppoll() (this obfuscates its first pointer arg using [] instead of *, and is missing restrict for that arg only. ppoll() is similar to pselect() except for these bugs) quotactl() rctl_*() (all 5 functions in another unsplit man page) rename() renameat() sctp_generic_recvmsg() sctp_generic_sendmsg() sctp_generic_sendmsg_iov() select() settimeofday() setitimer() (like select(). Fixed in POSIX in 2001, but not in FreeBSD) sendfile() sendto() statfs() symlink() symlinkat() utimensat() These are mostly BSD APIs. POSIX fixed almost all of the POSIX APIs in 2001. In a few cases like readlink(), there is no problem since all of the pointer args are const. Almost all of the POSIX APIs in the above list are of this type. In a few cases like statfs(), there is probably no problem because the pointer arg types are different and none of them is void *. In statfs(), one of them is also const. Most other cases are broken. E.g., POSIX added restrict to recvmsg() and sendmsg() in 2001, but the related newer sctp APIs haven't caught up with thus yet. Most FreeBSD timer APIs are missing restrict even when they are also in POSIX and not missing restrict there. sendmmsg() only has 1 pointer arg, but this is input-output which causes similar problems. It is declared as restrict. I don't know if that fixes the problems (the constraint on the implementation's ordering and restrict in the API doesn't give that AFAIK). Bruce From owner-freebsd-standards@freebsd.org Sun Feb 25 04:16:40 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1039DF008D9 for ; Sun, 25 Feb 2018 04:16:40 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic317-28.consmr.mail.bf2.yahoo.com (sonic317-28.consmr.mail.bf2.yahoo.com [74.6.129.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A880068569 for ; Sun, 25 Feb 2018 04:16:39 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: VZE0io0VM1lDfcyX3dFNBerczClz.AtmOANfB773jEMDVcP3hrA.eAttLKhUYdW Aed26_8a4uWndigq82wV7LjXl8jrgp4dP.E7ZrsB82t2XybQnGfR8hsDcoTxzXZgLbNmAgQ9vdst xKKwOgFA6TgXhDdbO6wPmvwwaomYkXC2GWfIlJwQi9vdXXNnO07tXgG.B.XzymCMw5xcTsjoNCOW 1D5Xggk7IFEBCUQT9Mz0pPX_SHB5uUQf.s8vlMZEtqV0Sj._MuiEKmxqtPTLTT1qcSO_34ft1a8M mkYzCgAkowhq3ZVofb8tjWjbJbhJroyjQaCxYhCY98CP7W3FD6lPr9BQCRhld5PX54lIUrueh0qg eeB8Z9eO2qiMvXgsOPWxUPftE13sGi3bb7Fb9MmRWyHfkFEOM9dhioazmWDZYP5atiaqQQFoysOP .rPmuafWYKtrwR7AG8IcT4oKA1WhX3BIZnEQqSTRyKZNQqsEAdsWCAp.y4R5jnxfn Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.bf2.yahoo.com with HTTP; Sun, 25 Feb 2018 04:16:39 +0000 Received: from smtp103.rhel.mail.bf1.yahoo.com (EHLO [192.168.1.25]) ([98.139.230.213]) by smtp411.mail.bf1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID 2506173844fd19f52721971e29959e28; Sun, 25 Feb 2018 03:56:22 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: Date: Sat, 24 Feb 2018 19:56:20 -0800 Cc: FreeBSD Hackers , FreeBSD Standards Content-Transfer-Encoding: quoted-printable Message-Id: <9438AC5E-56B8-46E4-AECE-5C3A194F4D1E@yahoo.com> References: To: Eitan Adler X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 04:16:40 -0000 On 2018-Feb-15, at 12:10 AM, Eitan Adler wrote: > Hi all, >=20 > POSIX requires that the fd_set arguments in select(2) be marked as > restrict. This patch attempts to implement that. >=20 > (a) Am I missing anything? > (b) Anything in particular to watch out for? > (c) Assuming an exp-run passes any reason not to commit? >=20 >=20 > Index: lib/libc/sys/select.2 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- lib/libc/sys/select.2 (revision 329296) > +++ lib/libc/sys/select.2 (working copy) > @@ -39,7 +39,7 @@ > .Sh SYNOPSIS > .In sys/select.h > .Ft int > -.Fn select "int nfds" "fd_set *readfds" "fd_set *writefds" "fd_set > *exceptfds" "struct timeval *timeout" > +.Fn select "int nfds" "fd_set * restrict readfds" "fd_set * restrict > writefds" "fd_set * restrict exceptfds" "struct timeval *timeout" > .Fn FD_SET fd &fdset > .Fn FD_CLR fd &fdset > .Fn FD_ISSET fd &fdset > Index: lib/libc/sys/select.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- lib/libc/sys/select.c (revision 329296) > +++ lib/libc/sys/select.c (working copy) > @@ -41,7 +41,7 @@ __weak_reference(__sys_select, __select); >=20 > #pragma weak select > int > -select(int n, fd_set *rs, fd_set *ws, fd_set *es, struct timeval *t) > +select(int n, fd_set * restrict rs, fd_set * restrict ws, fd_set * > restrict es, struct timeval *t) > { >=20 > return (((int (*)(int, fd_set *, fd_set *, fd_set *, struct timeval = *)) > Index: sys/sys/select.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/sys/select.h (revision 329296) > +++ sys/sys/select.h (working copy) > @@ -101,8 +101,7 @@ int pselect(int, fd_set *__restrict, fd_set *__res > const struct timespec *__restrict, const sigset_t *__restrict); > #ifndef _SELECT_DECLARED > #define _SELECT_DECLARED > -/* XXX missing restrict type-qualifier */ > -int select(int, fd_set *, fd_set *, fd_set *, struct timeval *); > +int select(int, fd_set *__restrict, fd_set *__restrict, fd_set > *__restrict, struct timeval *); > #endif > __END_DECLS > #endif /* !_KERNEL */ Going in a different direction: C++ . . . =46rom FreeBSD's cdefs.h : #if !(__GNUC__ =3D=3D 2 && __GNUC_MINOR__ =3D=3D 95) #if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901 #define __restrict #else #define __restrict restrict #endif #endif It looks to me like C++ use of cdefs.h and then, say, select.h, could easily lead to __restrict being translated to no-text. C++11 does add __STDC_VERSION__ to the "implementation-defined value, if present" category. (Quoted material is from en.cppreference.com .) This would lead C++ to not give errors/warnings for violating the constraints involved in calling a newly Linux-like implementation of select (with C99-like restrict involved). It also means that if some C++ compilers have a __restrict (-like) extension that it is not being put to use for either code generation or for reporting violations of C99-like constraints. Of course if a C++11 or later targeting defines __STDC_VERSION__ with it being >=3D 199901 then the C++ compiler would see "restrict" (no quotes) after the substitution, likely giving a syntax error. (It is not a keyword in C++.) But these types of points also apply to existing uses of __restrict after cdefs.h use (unless I missed a level of conditionality that is relevant). But at least there is some history as evidence for these. Overall result: C++ apparently only gets run-time behavior as evidence for the use of the new content of select's implementation if this is changed: no reports of abusive calls as stands. =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-standards@freebsd.org Sun Feb 25 05:52:57 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47C54F0B25F; Sun, 25 Feb 2018 05:52:57 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D37316D54B; Sun, 25 Feb 2018 05:52:56 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-qt0-x22d.google.com with SMTP id g14so15298089qti.2; Sat, 24 Feb 2018 21:52:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Acp96vYJWPn0oJeci604nQ1lm91Y0lj15KMyk5+miPQ=; b=jmdo/N0LQe84sEQGdbCPsup6c9uWx6NJ9Ed8GyS3tBG9wLt+zLizodqhrcWQ3aNPfr l6s0+qaEacccwbmBc5tQP02BUQe9jOj7xH45DDDT/v+KddBl4DLBH4FRvlkU2UUXIUGc CSezeUKr+J1ghmzqU2WuPukRJN25tq5BDQqgX1P5tl4Zqh2pyLgrlxP4kdRrt/rM6d5R u9TvRDM5ErkskIIFyooQjCo+/oz3aew/UdErFAiY8zeeXk4lXI7ZvSty1xAIaakPk0O2 jiXtM2ayskWaoT45D9c4KeAZ9ws5GMaO77uspkh2Dh9fU7kXMk0a8w1pS2T9arQTLId2 uvJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Acp96vYJWPn0oJeci604nQ1lm91Y0lj15KMyk5+miPQ=; b=fbGKkpWM9wcrfNBJ4BunA7EhqQxFlJSTPAhgpkoaUfJvn3KwvyNYrY8jYh0E8xISO5 eSt4/3OSZXFklx5MZ+2N49lR5OSX7zRXEqDWdE6KXLQXZ6L3ejKu2KnwxTumntfZYS5O +5mqz77dvIpLKQAxQNAS1fV7kp3bE8EHH6b0mR8NShM7xsImjPrBZWXSAC2R6B6bZOcy YhhwdGQ2iBA9YvIxm9KqFdOh3xwDYZ3iAW3wUW1UFcW/gw42sDRoWLw1h018u3lT8jS1 gJIo7sYonZ00hsKy3E2vITrs8Wf8nKpWR6qtI7UlYq4KkCmEpT9l4EYUPUADyDaiCPAU DNoA== X-Gm-Message-State: APf1xPDrxL9TOHXXRKmcczkL6MbZGJsgLO2aIwecJ62U8eDwT6oNfbJ5 FFIEUp6krUTZjF3PymCo33Fhtknifl43P/s2kOs= X-Google-Smtp-Source: AG47ELunZqxlnmSxLtspqbrKy4Dna02GHHOUnjdE0K8sstxylYhph7JmG/7KO4wL92dxgRitsyRN4USIS2qJgO+eP4I= X-Received: by 10.200.35.141 with SMTP id q13mr10690571qtq.73.1519537976439; Sat, 24 Feb 2018 21:52:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.32.74 with HTTP; Sat, 24 Feb 2018 21:52:55 -0800 (PST) In-Reply-To: References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> From: Stefan Blachmann Date: Sun, 25 Feb 2018 06:52:55 +0100 Message-ID: Subject: Re: Marking select(2) as restrict To: Eitan Adler Cc: cem@freebsd.org, FreeBSD Hackers , FreeBSD Standards Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 05:52:57 -0000 The Linux manual pages do not mention restrict for select(). glibc select() itself returns just ENOSYS(), if there is no alias for select(). So I guess what actually gets called is this: https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 Which in turn appears to call __sys_select: https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L346 See also: https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.html If I understand correctly, the *only* place that defines the optimizations actually being done is the static functions itself: https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L487 So maybe the actual prototypes being used for the functions for which the interpose array is used are irrelevant: https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L642 I not yet found out which functions are actually weakly aliased in, but I could imagine that adding the restrict keyword to the prototypes of the functions listed there, is possibly only of cosmetical importance, without any actual effect. If this is correct, one could be "Posix compliant" without causing any disruptive "optimization" :) Have a nice Sunday! Stefan P.S.: Maybe it would be better to avoid adding the restrict keyword in the __thr_select() function itself mentioned above, as Linux' select function seems to have no restrict: https://github.com/torvalds/linux/blob/master/fs/select.c#L1262 https://github.com/torvalds/linux/blob/master/fs/select.c#L599 On 2/25/18, Eitan Adler wrote: > On 24 February 2018 at 10:55, Conrad Meyer wrote: >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >> wrote: >>> After this entire thread here is the summary. If I've misrepresented >>> you here please let me know. >>> ... >>> >>> kib@ - no benefit; concerned fallout could be hard to observe >>> cem@ - concerned about warnings >> >> Consider me a +1 to kib@. I did not voice those concerns explicitly >> in earlier email because kib did already and I didn't anticipate you >> would ignore him. > > I am not ignoring him. As I stated above I do not believe fallout is > likely since most other major libc implementations have already done > this: > > glibc: already done - > https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 > openbsd: already done > https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 > dragonflyBSD: alredy done: > https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 > netbsd: already done: > https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 > > As a further check I went through the search results on github for > select() and did not see any failures in the top few pages. > > -- > Eitan Adler > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-standards@freebsd.org Sun Feb 25 13:51:30 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE697F2F645; Sun, 25 Feb 2018 13:51:29 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DC417DD7E; Sun, 25 Feb 2018 13:51:29 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w1PDpCuk031025 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 25 Feb 2018 15:51:15 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w1PDpCuk031025 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w1PDpBjT031024; Sun, 25 Feb 2018 15:51:11 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Sun, 25 Feb 2018 15:51:11 +0200 From: Konstantin Belousov To: Stefan Blachmann Cc: Eitan Adler , FreeBSD Hackers , FreeBSD Standards , cem@freebsd.org Subject: Re: Marking select(2) as restrict Message-ID: <20180225135111.GO94212@kib.kiev.ua> References: <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.3 (2018-01-21) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 13:51:30 -0000 On Sun, Feb 25, 2018 at 06:52:55AM +0100, Stefan Blachmann wrote: > The Linux manual pages do not mention restrict for select(). > glibc select() itself returns just ENOSYS(), if there is no alias for select(). > > So I guess what actually gets called is this: > https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 > Which in turn appears to call __sys_select: > https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L346 > See also: > https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.html > > If I understand correctly, the *only* place that defines the > optimizations actually being done is the static functions itself: > https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L487 No, you do not understand this correctly. Select(2) implementation is in kernel and cannot be affected by the userspace prototype change. It is the caller of select(2) which might be optimized unexpectedly when select claims that its arguments cannot be aliased. The use of restrict in the glibc prototype would be a good argument if clang on glibc were not a rare combination. > > So maybe the actual prototypes being used for the > functions for which the interpose array is used are irrelevant: > https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L642 > > I not yet found out which functions are actually weakly aliased in, > but I could imagine that adding the restrict keyword to the prototypes > of the functions listed there, is possibly only > of cosmetical importance, without any actual effect. > If this is correct, one could be "Posix compliant" without causing any > disruptive "optimization" :) > > Have a nice Sunday! > Stefan > > > P.S.: Maybe it would be better to avoid adding the restrict keyword in the > __thr_select() function itself mentioned above, as Linux' select > function seems to have no restrict: > https://github.com/torvalds/linux/blob/master/fs/select.c#L1262 > https://github.com/torvalds/linux/blob/master/fs/select.c#L599 > > > On 2/25/18, Eitan Adler wrote: > > On 24 February 2018 at 10:55, Conrad Meyer wrote: > >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler > >> wrote: > >>> After this entire thread here is the summary. If I've misrepresented > >>> you here please let me know. > >>> ... > >>> > >>> kib@ - no benefit; concerned fallout could be hard to observe > >>> cem@ - concerned about warnings > >> > >> Consider me a +1 to kib@. I did not voice those concerns explicitly > >> in earlier email because kib did already and I didn't anticipate you > >> would ignore him. > > > > I am not ignoring him. As I stated above I do not believe fallout is > > likely since most other major libc implementations have already done > > this: > > > > glibc: already done - > > https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 > > openbsd: already done > > https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 > > dragonflyBSD: alredy done: > > https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 > > netbsd: already done: > > https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 > > > > As a further check I went through the search results on github for > > select() and did not see any failures in the top few pages. > > > > -- > > Eitan Adler > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-standards@freebsd.org Sun Feb 25 14:47:03 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76B6EF31583 for ; Sun, 25 Feb 2018 14:47:03 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic307-9.consmr.mail.gq1.yahoo.com (sonic307-9.consmr.mail.gq1.yahoo.com [98.137.64.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 16AD97F67B for ; Sun, 25 Feb 2018 14:47:02 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: aeiaDq4VM1n0FoQTDuAiG9hKIOA3Oc1AkBzmbJ8nYGjSD0RhiAH.iWCxMjX43SL zqrX3ha8eDwnfekZAaQGGGGk9r53tg_8tP4E2n1HxkRL6IY_4M09N337lpwBwCPvNjw4YzvP.PUE KbmrFccV6Rot4slpok2VgkbjUCrvvC4FeyY3SL4TvkNJ6me05tiuXIiyofCR4sCeJnJ8jPQICMdn Z4EZYlKBtZTHPaX449G0SxlKAMj5GRMUXdRlwyEkZAxT4z0xngY0hLCY6zudmB7h.fMviVT9R1lP koXGSMjti0VKD4p4W7NoQgeycoVSCXM_zuDzukwmUIhmVnjkJ7towfyczyyAC3.iEcFh8nRB8CA2 MejPULuFkwh.ezAneYp8xQ2abFG0_pXaQtForqoX_FRJkuACRp0iRSrCB9z.xDg8q1tVvxgMK7y0 MvPtKIGfZLmcA7GAUjo_6.0XMxgqFRrtOy9scTLJZeNaa_4vLGb_cEwV7MfL4QI4- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Sun, 25 Feb 2018 14:46:56 +0000 Received: from smtp101.rhel.mail.gq1.yahoo.com (EHLO [192.168.1.25]) ([68.180.227.10]) by smtp413.mail.gq1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID e97702b39406288e9e6deaaf59d87a7d; Sun, 25 Feb 2018 14:46:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225112627.B976@besplex.bde.org> Date: Sun, 25 Feb 2018 06:46:52 -0800 Cc: Warner Losh , FreeBSD Hackers , FreeBSD Standards , "Conrad E. Meyer" Content-Transfer-Encoding: 7bit Message-Id: <5957DA7A-80CC-48D8-AC4B-8EC403658AC2@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> <20180225112627.B976@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 14:47:03 -0000 On 2018-Feb-24, at 5:28 PM, Bruce Evans wrote: > On Sat, 24 Feb 2018, Warner Losh wrote: > >> On Sat, Feb 24, 2018 at 11:55 AM, Conrad Meyer wrote: >> >>> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >>> wrote: >>>> After this entire thread here is the summary. If I've misrepresented >>>> you here please let me know. >>>> ... >>>> >>>> kib@ - no benefit; concerned fallout could be hard to observe >>>> cem@ - concerned about warnings >>> >>> Consider me a +1 to kib@. I did not voice those concerns explicitly >>> in earlier email because kib did already and I didn't anticipate you >>> would ignore him. >> >> So there's no benefit to the change (we won't optimize better). It's hard >> to observe breakage. No answer about how we'd even know if something broke >> because a exp run sure as hell isn't going to tell us. >> >> All that militates against the change rather strongly. Your exp run will >> change no minds because it is useless. > > Why not remove restrict from other APIs to be consistent with select()? If > might break their callers just as much. This appears to be about the handling of non-conforming programs. Quoting from C99's 6.7.3 "Type qualifiers" paragraph 7 about conforming programs: "The intended use of the restrict qualifier (like the register storage class) is to promote optimization, and deleting all instances of the qualifier from all preprocessing units composing a conforming program does not change its meaning (i.e. observable behavior)." So: If removing restrict results in a conforming program's observable behavior changing, then that would be evidence that the environment is incorrectly implemented relative to C99. Of course, removing restrict can make it harder to detect non-conforming programs if the compiler(s) involved are helpful when restrict is present. Going the other way: 6.7.3.1 "Formal definition of restrict" also says in its paragraph 6: "A translator is free to ignore any or all aliasing implications of uses of restrict." === Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-standards@freebsd.org Sun Feb 25 17:42:18 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64C01F03A71 for ; Sun, 25 Feb 2018 17:42:18 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic308-9.consmr.mail.gq1.yahoo.com (sonic308-9.consmr.mail.gq1.yahoo.com [98.137.68.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DDCD468CC5 for ; Sun, 25 Feb 2018 17:42:17 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: j.1WblMVM1k6TvUCt_SeQvg09_WKkQkhu6IaP98pLJwinQ.ag62IcfE3f.Zk8v7 lKYzmsWBT7q873AK8AQOWv9GbCVPq0CTssphe2l4O2n8UyY5QIY_1kCrt2BDLasnO.LqUzA2Nulj edXDub5AFq4FMH3I9oyiHJeMCAL_Y6p.kUzeSPZQ0KHzLMpoI3s8awgperhRVoYtUTPqwsN49D0v x7lGHEbesAN3I4_vZwinKgf4A2_PSuj1qD1oqau5HKI7Uaib38qPZGRHnfk2DU0qdyqHv8qtv9QQ 6Hiv4PRiZ.ou6FbuTDlMf8p.cCvW4HM4wBe9RYjOTsV4uIhQM8kunjUPmc0XbqTN61kfTevRHdpB .eJLsTtZOOLCg6nxyu.pChyIk8OUpprKRKO.IH9Q04GjzQa3.TakMXfb5eW7KlXhUUhid9dCajeG wdomriT0bUAHDayQAQS2B2SHO8wIJvfVMOWDjOqWMWpmtD6db_7YxGO50uZrkS8lBQ.OAfBQ- Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sun, 25 Feb 2018 17:42:16 +0000 Received: from smtp105.rhel.mail.gq1.yahoo.com (EHLO [192.168.1.25]) ([68.180.227.8]) by smtp413.mail.gq1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID 151b169aca54fce9868b612577f67de5; Sun, 25 Feb 2018 17:32:04 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225135111.GO94212@kib.kiev.ua> Date: Sun, 25 Feb 2018 09:32:03 -0800 Cc: Stefan Blachmann , Eitan Adler , "Conrad E. Meyer" , FreeBSD Standards , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <0C8DEA21-0F27-4D8D-8E30-42378B166FBF@yahoo.com> References: <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> <20180225135111.GO94212@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 17:42:18 -0000 On 2018-Feb-25, at 5:51 AM, Konstantin Belousov = wrote: > On Sun, Feb 25, 2018 at 06:52:55AM +0100, Stefan Blachmann wrote: >> The Linux manual pages do not mention restrict for select(). >> glibc select() itself returns just ENOSYS(), if there is no alias for = select(). >>=20 >> So I guess what actually gets called is this: >> https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 >> Which in turn appears to call __sys_select: >> = https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L34= 6 >> See also: >> = https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.h= tml >>=20 >> If I understand correctly, the *only* place that defines the >> optimizations actually being done is the static functions itself: >> = https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_sysca= lls.c#L487 > No, you do not understand this correctly. Select(2) implementation is > in kernel and cannot be affected by the userspace prototype change. It > is the caller of select(2) which might be optimized unexpectedly when > select claims that its arguments cannot be aliased. >=20 > The use of restrict in the glibc prototype would be a good argument if > clang on glibc were not a rare combination. I'm only noting where some evidence might be available . . . My understanding is that there is (at least) one desktop Linux distribution that is clang based: OpenMadriva Lx. It may be a source of information. Checking a little into its status . . . =46rom https://en.wikipedia.org/wiki/OpenMandriva_Lx : "A beta release of OpenMandriva Lx 3.0 was released in June 2016.[19] This new release came with significant changes to the core system - among other things, it was the first desktop Linux distribution that was built completely with the Clang compiler instead of GCC." https://wiki.openmandriva.org/en/3.03/Release_Notes#LLVM.2Fclang reports: "OpenMandriva provides LLVM/clang 5.0.0 as the default compiler, GCC is also available. Over 90% of packages in our main repository are now built with LLVM/clang." = https://www.openmandriva.org/en/news/article/openmandriva-lx-3-03-get-it-w= hile-it-s-hot reports: "Everything with this release, including the new Firefox Quantum 57, is compiled with LLVM/clang 5.0.0" Looking quickly, there is glibc source involved: https://github.com/OpenMandrivaAssociation/glibc/tree/master (There are other branches, such a 3.0 branch.) I have not analyzed the patches that they use. For reference for V3.03: ( = https://www.openmandriva.org/en/news/article/openmandriva-lx-3-03-get-it-w= hile-it-s-hot ) "At the hardware level there is an up-to-date kernel at 4.13.12 release and systemd 234 and for your graphics stack mesa 17.2.3 with a S3TC support enabled and xorg 1.19.5." "Our main desktop environment KDE Plasma is updated to 5.10.5 and Frameworks are at 5.39." =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-standards@freebsd.org Sun Feb 25 20:48:23 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38588F279BC; Sun, 25 Feb 2018 20:48:23 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay101.isp.belgacom.be (mailrelay101.isp.belgacom.be [195.238.20.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 596DA6FE0E; Sun, 25 Feb 2018 20:48:22 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3ABxWDYRKkFurs+5XLudmcpTZWNBhigK39O0sv0rFi?= =?us-ascii?q?tYgRLfrxwZ3uMQTl6Ol3ixeRBMOHs6kC07KempujcFRI2YyGvnEGfc4EfD4+ou?= =?us-ascii?q?JSoTYdBtWYA1bwNv/gYn9yNs1DUFh44yPzahANS47xaFLIv3K98yMZFAnhOgpp?= =?us-ascii?q?POT1HZPZg9iq2+yo9JDffwtFiCChbb9uMR67sRjfus4KjIV4N60/0AHJonxGe+?= =?us-ascii?q?RXwWNnO1eelAvi68mz4ZBu7T1et+ou+MBcX6r6eb84TaFDAzQ9L281/szrugLd?= =?us-ascii?q?QgaJ+3ART38ZkhtMAwjC8RH6QpL8uTb0u+ZhxCWXO9D9QKsqUjq+8ahkVB7oiD?= =?us-ascii?q?8GNzEn9mHXltdwh79frB64uhBz35LYbISTOfFjfK3SYMkaSHJPUMhRSSJPH4Cy?= =?us-ascii?q?YIkBD+UOIelWoJLwp0cMoBeiGQWgGP/jxiFOi3Tr3aM6yeMhEQTe0QI+HtIOsn?= =?us-ascii?q?DUp8jrOacVVuC117fHzTDZYPNQwjf29Y/FcgwgofGOWbJ9asrfyVMxGAzbk1ie?= =?us-ascii?q?tILrMymS1uQXvGiW9uxtXv+shW4/swx8oSWjyt0yhoTGh48Z0E3I+Ct3zYovON?= =?us-ascii?q?G1RkF2bNi5G5VKrS6aLZF5QsY6TmFtvyY116MJtIagfCgP1JQn3xnfa+Gbc4SQ?= =?us-ascii?q?4hLsSuKRITBgiXJmYr2/gxey8U2+xe3mUcm4ykpKritHktnIrHwCyxvT6s+cSv?= =?us-ascii?q?Rj+0euwzCP1xvJ5uFDO0A0mrLXK58nwrEuipoeqUfOEjLslEnog6Kbd18o9vWm?= =?us-ascii?q?5unpeLnqu5GROoBshgH7KKsum8i/AeoiMggJWmiW4fi81Lzh/U39W7hKgOc2nb?= =?us-ascii?q?fHv5/BPsQUu7S1AwhP0oYs8xq/FSup0MwEnXkbK1JIYBGHj4/yO1HSIfD4Duyw?= =?us-ascii?q?jEqokDpwyPDGO6fuApTJLnTZjLjherN9uAZgz18QytZE+5tSFrAHaNj+Xkjsr9?= =?us-ascii?q?vGRks6NBeowuXtBdFV2YYXWGbJCaicZvD8q1iNs94uIe3ET4gSozv4Iv4+r6ry?= =?us-ascii?q?jH09sXEHcKSD5rdRb2q3SKc1a36FaGbh149SWVwBuRAzGamz0AWP?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2B9AQANIJNa/4aF9lFcGgEBAQEBAgEBA?= =?us-ascii?q?QEIAQEBAYNPVoEAKI12dI0GAQGCATIBY4dtjhiCFoU9AoI3VRgBAgEBAQEBAQI?= =?us-ascii?q?BaiiCOCQBgkcBBTocIxALDgoJJQ8SGB4GE4R4Axmtd4cjDYEyghQBAQEBAQEEA?= =?us-ascii?q?QEBASSOLIJqiCkFoS8yCY81hF6BBJFpjRmKCx44gVFNMAiCfYJDHIF8PzeMbAE?= =?us-ascii?q?BAQ?= X-IPAS-Result: =?us-ascii?q?A2B9AQANIJNa/4aF9lFcGgEBAQEBAgEBAQEIAQEBAYNPVoE?= =?us-ascii?q?AKI12dI0GAQGCATIBY4dtjhiCFoU9AoI3VRgBAgEBAQEBAQIBaiiCOCQBgkcBB?= =?us-ascii?q?TocIxALDgoJJQ8SGB4GE4R4Axmtd4cjDYEyghQBAQEBAQEEAQEBASSOLIJqiCk?= =?us-ascii?q?FoS8yCY81hF6BBJFpjRmKCx44gVFNMAiCfYJDHIF8PzeMbAEBAQ?= Received: from 134.133-246-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.246.133.134]) by relay.skynet.be with ESMTP; 25 Feb 2018 21:48:15 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w1PKmDJt045352; Sun, 25 Feb 2018 21:48:14 +0100 (CET) (envelope-from tijl@FreeBSD.org) Date: Sun, 25 Feb 2018 21:48:13 +0100 From: Tijl Coosemans To: Konstantin Belousov Cc: Eitan Adler , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180225214813.776a9f58@kalimero.tijl.coosemans.org> In-Reply-To: <20180222105608.GE94212@kib.kiev.ua> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 20:48:23 -0000 On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: > Consider the recently changed devd code: > select(n + 1, &fd, &fd, &fd); > There, compiler can see that restrict is applied to arguments which are > given same values. Since this leads to the self-contradicting statement > fd != fd > which cannot be true, compliler in its optimizing wisdom can assume that > the code is never executing and remove it. I do not know whether clang > actually makes such transformation, but it does not sound unfeasible > looking at its other advances. There's an example in the C99 standard that indicates such a call is not necessarily undefined so compilers cannot optimise it away: EXAMPLE 3 The function parameter declarations void h(int n, int * restrict p, int * restrict q, int * restrict r) { int i; for (i = 0; i < n; i++) p[i] = q[i] + r[i]; } illustrate how an unmodified object can be aliased through two restricted pointers. In particular, if a and b are disjoint arrays, a call of the form h(100, a, b, b) has defined behavior, because array b is not modified within function h. From owner-freebsd-standards@freebsd.org Sun Feb 25 21:01:06 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 92C67F28783 for ; Sun, 25 Feb 2018 21:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 55F1570785 for ; Sun, 25 Feb 2018 21:01:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A1CC723473 for ; Sun, 25 Feb 2018 21:01:05 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w1PL15mo006641 for ; Sun, 25 Feb 2018 21:01:05 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w1PL1501006629 for freebsd-standards@FreeBSD.org; Sun, 25 Feb 2018 21:01:05 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201802252101.w1PL1501006629@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: freebsd-standards@FreeBSD.org Subject: Problem reports for freebsd-standards@FreeBSD.org that need special attention Date: Sun, 25 Feb 2018 21:01:05 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 21:01:06 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 193594 | stddef.h should define max_align_t Open | 191586 | FreeBSD doesn't validate negative edgecases in bi 2 problems total for which you should take action. From owner-freebsd-standards@freebsd.org Sun Feb 25 23:32:08 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39C80F32C0A for ; Sun, 25 Feb 2018 23:32:08 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic301-32.consmr.mail.ne1.yahoo.com (sonic301-32.consmr.mail.ne1.yahoo.com [66.163.184.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9E9476BD0 for ; Sun, 25 Feb 2018 23:32:07 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: z5KulQYVM1lgjp2MoWXlPy8ZJGIVip8A6.tbhkTTNOVBQ9h0e8ZzxdPAupJb0_x f6Kv1iETNsHJNOb.AcTPMWNcdT34qCpcUrXGi5gDEQM34dzCeF_DoEoj4zv_ZVNWazPNANcCvOot YAvHr80ubUhkqwA2tSW2ANLQLX9ietS6hYQHRjUFBKn_MAhAn8GDJl632fqahHVyaVjkh3FH2ufH YkCP4lSzTzzjwww7kk5qSP_hW_sUNqUJWbHb.uoHVFIIKBlzFDroGCzOUxbuZigJVDHTVH03EtnI pcdZB9GbZKK84IS.P.5e.tH7vDsHOJUzDoZKk7gyuNf6Wzx6p7zhiy1D6Q.AlalqxHDtqVD_.ORW i7omnaLqtCZFvDKzZMKxT6BwmnjNPDEgtOlIn4uSDpAIjgAeUR3lBmlu0_L3KP4qUkohzAVxUW44 bb5ggR6Xv9AgNcmoswSrAG6.5Cry7ie6Cq.JbFdjx4zss0rfYMu6zBh4vLE4gLMzHMS8v Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.ne1.yahoo.com with HTTP; Sun, 25 Feb 2018 23:32:01 +0000 Received: from smtp226.mail.ne1.yahoo.com (EHLO [192.168.1.25]) ([10.218.253.215]) by smtp410.mail.ne1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID ca9ce8c4b86d225e30f2b4ca91ae9c8d; Sun, 25 Feb 2018 23:31:57 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225214813.776a9f58@kalimero.tijl.coosemans.org> Date: Sun, 25 Feb 2018 15:31:56 -0800 Cc: Konstantin Belousov , Eitan Adler , Kevin Lo , FreeBSD Standards , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> To: Tijl Coosemans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 23:32:08 -0000 On 2018-Feb-25, at 12:48 PM, Tijl Coosemans wrote: > On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >> Consider the recently changed devd code: >> select(n + 1, &fd, &fd, &fd); >> There, compiler can see that restrict is applied to arguments which = are >> given same values. Since this leads to the self-contradicting = statement >> fd !=3D fd >> which cannot be true, compliler in its optimizing wisdom can assume = that >> the code is never executing and remove it. I do not know whether = clang >> actually makes such transformation, but it does not sound unfeasible >> looking at its other advances. >=20 > There's an example in the C99 standard that indicates such a call is = not > necessarily undefined so compilers cannot optimise it away: >=20 > EXAMPLE 3 > The function parameter declarations > void h(int n, int * restrict p, int * restrict q, int * restrict r) > { > int i; > for (i =3D 0; i < n; i++) > p[i] =3D q[i] + r[i]; > } > illustrate how an unmodified object can be aliased through two = restricted > pointers. In particular, if a and b are disjoint arrays, a call of = the > form h(100, a, b, b) has defined behavior, because array b is not = modified > within function h. Good point. In essence the restrictions on the caller can not be known independently of how the parameters are used in the called code --something that prototype does not specify. This does constrain what the compiler can do about potential aliasing that it might detect. A prototype that would make h's restrictions clearer is one that reports that q and r are not used to modify memory: void h(int n, int * restrict p, int const * restrict q, int const * = restrict r); With such a prototype, it is easier to known that q's "objects" and r's "objects" both simply must not overlap p's "objects". (See g from example 2 for its d+50 valid vs. d+1 invalid status.) Section 7.1 of the C Rationale says: "The restrict keyword allows the prototype to express what was previously expressed by words." But it turns out that not all the words can be eliminated if one is to know the actual criteria for what the caller is allowed to do while staying within defined behavior. There is a lot of wording for which the "example 3" h(100, a, b, b) example usage being well defined is not obvious. Such wording is without any specific such context to reference and tends to assume that all restricted pointers are used to modify the matching objects (when const does not prevent such). This need not be the case. =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-standards@freebsd.org Mon Feb 26 03:45:52 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C3DBF25917; Mon, 26 Feb 2018 03:45:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id E45EE7FD3A; Mon, 26 Feb 2018 03:45:51 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 549A8D69011; Mon, 26 Feb 2018 14:45:43 +1100 (AEDT) Date: Mon, 26 Feb 2018 14:45:42 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict In-Reply-To: <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> Message-ID: <20180226135457.B1203@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=pGLkceISAAAA:8 a=xwlSsFIUvvU5qVp6TTwA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 03:45:52 -0000 On Sun, 25 Feb 2018, Mark Millard via freebsd-standards wrote: > On 2018-Feb-25, at 12:48 PM, Tijl Coosemans wrote: > >> On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >>> Consider the recently changed devd code: >>> select(n + 1, &fd, &fd, &fd); >>> There, compiler can see that restrict is applied to arguments which are >>> given same values. Since this leads to the self-contradicting statement >>> fd != fd >>> which cannot be true, compliler in its optimizing wisdom can assume that >>> the code is never executing and remove it. I do not know whether clang >>> actually makes such transformation, but it does not sound unfeasible >>> looking at its other advances. >> >> There's an example in the C99 standard that indicates such a call is not >> necessarily undefined so compilers cannot optimise it away: >> >> EXAMPLE 3 >> The function parameter declarations >> void h(int n, int * restrict p, int * restrict q, int * restrict r) >> { >> int i; >> for (i = 0; i < n; i++) >> p[i] = q[i] + r[i]; >> } >> illustrate how an unmodified object can be aliased through two restricted >> pointers. In particular, if a and b are disjoint arrays, a call of the >> form h(100, a, b, b) has defined behavior, because array b is not modified >> within function h. > > Good point. In essence the restrictions on the caller can > not be known independently of how the parameters are used > in the called code --something that prototype does not specify. > This does constrain what the compiler can do about potential > aliasing that it might detect. > > A prototype that would make h's restrictions clearer is > one that reports that q and r are not used to modify > memory: > > void h(int n, int * restrict p, int const * restrict q, int const * restrict r); > > With such a prototype, it is easier to known that q's "objects" > and r's "objects" both simply must not overlap p's "objects". > (See g from example 2 for its d+50 valid vs. d+1 invalid status.) I think the example intentionally leaves out 'const'. I think const already prevents aliasing. 'restrict' prevents it even more. Using the combination in the exaple would make it less clear what the 'restrict' part does. In draft C99 (n869.txt), Example 3 is much more complicated and seems to be broken. It is intentionally of how const can be used in conjunction with restrict. p has qualifiers const and restrict, and q and r only have qualifier const, and it is claimed that p being restrict- qualified implies that an object accessed through p is never accessed through q or r, and that this is the precise assertion required to optimize the loop. const for p seems to be just a bug. Removing it gives an example of how restrict is not needed for the input-only args since const suffices. However, C99 TC3 (n1256.pdf) changes this signficantly by removing all the consts and adding restricts. This might be related to POSIX's inconsistencies for sigaction() and nanosleep(). For sigaction(), the input arg is const restrict and the output arg is restrict, but for nanosleep() the the input arg is just const and the output arg is unqualified. const on 1 arg seems to be enough to prevent all aliasing when there are only 2 args that could be aliased without it, but Example 3 no longer gives any hints about how restrict interacts with const. restrict gives stronger (but different) restrictions than const on aliases with globals, but so should all POSIX syscall-like functions. An application function miight access any application global state, but library functions should only access documented global state. For syscall-like functions, the global state outside of the kernel should be precisely errno. POSIX should restrict aliasing to error in a general way, but for sigaction() and nanosleep() it is clear that the pointer args cannot be aliased to errno, since they don't point to ints. Bruce From owner-freebsd-standards@freebsd.org Mon Feb 26 04:40:30 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 273E8F2969F for ; Mon, 26 Feb 2018 04:40:30 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic306-22.consmr.mail.ne1.yahoo.com (sonic306-22.consmr.mail.ne1.yahoo.com [66.163.189.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0AF881FA5 for ; Mon, 26 Feb 2018 04:40:29 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: 4LNx12kVM1mHP9pCKQ8TyhCiJXtYHjpWwm4BdDaFW3lqULKF8SBu.xSDcJSD0P9 uGYHUJV.OuMjKaXl4.cE3ArQGLB0KDlSf924AZwRLWW_81rF6YUPpkQJ0JdcjZzW0K2iEAlIdL8i eUHzr8jtgaYURUBamCRRrDmwX98URrBYgcKIBNa3Bf_xKBBKTBASHZ57fb1gNDekAZRfMrLqTGky Tyby23MlI2ZumuFjv9XyNdzSCKzmH4T3YzD4ObyoR3QnDL86mmdDCQaVzOR48eT6WxVWs2.Oyx5d 5_w3SUyDMHMkyGKxcNuOT8Ip7xnARI53drsjGpNqRhiMCw4Vf9mFuE1gUPDl.XHEI1Rze6iHpWET kkIY9ghssbT2DkxReTqKvzDh9lZ4HS8JFbe4qJF6lPcmVzTtUI8haL0zEnk0SUb82hzYnYPlZmV7 05GgA6WXgZjtIBmFfiVgiFDL2jh8VoyxEIh3Xr7._ftDJydx4FusHnAl5CorgYnTWj2iUf4nX Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.ne1.yahoo.com with HTTP; Mon, 26 Feb 2018 04:40:23 +0000 Received: from smtp230.mail.ne1.yahoo.com (EHLO [192.168.1.25]) ([10.218.253.211]) by smtp404.mail.ne1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID c920c595c0bc7ede23f066c3b2f36b8d; Mon, 26 Feb 2018 04:40:18 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180226135457.B1203@besplex.bde.org> Date: Sun, 25 Feb 2018 20:40:16 -0800 Cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Content-Transfer-Encoding: quoted-printable Message-Id: <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 04:40:30 -0000 On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: > On Sun, 25 Feb 2018, Mark Millard via freebsd-standards wrote: >=20 >> On 2018-Feb-25, at 12:48 PM, Tijl Coosemans = wrote: >>=20 >>> On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >>>> Consider the recently changed devd code: >>>> select(n + 1, &fd, &fd, &fd); >>>> There, compiler can see that restrict is applied to arguments which = are >>>> given same values. Since this leads to the self-contradicting = statement >>>> fd !=3D fd >>>> which cannot be true, compliler in its optimizing wisdom can assume = that >>>> the code is never executing and remove it. I do not know whether = clang >>>> actually makes such transformation, but it does not sound = unfeasible >>>> looking at its other advances. >>>=20 >>> There's an example in the C99 standard that indicates such a call is = not >>> necessarily undefined so compilers cannot optimise it away: >>>=20 >>> EXAMPLE 3 >>> The function parameter declarations >>> void h(int n, int * restrict p, int * restrict q, int * restrict r) >>> { >>> int i; >>> for (i =3D 0; i < n; i++) >>> p[i] =3D q[i] + r[i]; >>> } >>> illustrate how an unmodified object can be aliased through two = restricted >>> pointers. In particular, if a and b are disjoint arrays, a call of = the >>> form h(100, a, b, b) has defined behavior, because array b is not = modified >>> within function h. >>=20 >> Good point. In essence the restrictions on the caller can >> not be known independently of how the parameters are used >> in the called code --something that prototype does not specify. >> This does constrain what the compiler can do about potential >> aliasing that it might detect. >>=20 >> A prototype that would make h's restrictions clearer is >> one that reports that q and r are not used to modify >> memory: >>=20 >> void h(int n, int * restrict p, int const * restrict q, int const * = restrict r); >>=20 >> With such a prototype, it is easier to known that q's "objects" >> and r's "objects" both simply must not overlap p's "objects". >> (See g from example 2 for its d+50 valid vs. d+1 invalid status.) >=20 > I think the example intentionally leaves out 'const'. I think const > already prevents aliasing. 'restrict' prevents it even more. Using > the combination in the exaple would make it less clear what the > 'restrict' part does. Agreed: the combination would make the example less clear about restrict. So, agreed: likely deliberate for specifying just restrict's semantics. I would not suggest my alternate h prototype for that example in the standard: different purposes served. > . . . restrict is not needed for the input-only > args since const suffices.=20 Here you lost me. With q and r having both the const and the restrict, updates to p's "objects" can not change the "object(s)" q and r validly can be used to access. That can be important. Without the "restrict" for q and r (but still having the const for each) it is valid for updates to p's "objects" to change what q and r then can validly access. The 2 restrict's in question do not seem redundant to me as far as the prototype goes. They may enable optimizations in some architectures: more order of operation independence so more reordering possible. A more general context could still require words describing requirements, words beyond the restrict and const use. I would not claim that adding const where it fits would make all contexts well described. This one just happened to be nicer for the person reading the prototype. (But it would makes for a worse specification of restrict of itself to have the const's in place in example 3.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-standards@freebsd.org Mon Feb 26 10:00:27 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE805F39F68; Mon, 26 Feb 2018 10:00:26 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 108AF6D39C; Mon, 26 Feb 2018 10:00:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id DD374107332; Mon, 26 Feb 2018 21:00:22 +1100 (AEDT) Date: Mon, 26 Feb 2018 21:00:21 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict In-Reply-To: <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> Message-ID: <20180226200951.V2634@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=FNpr/6gs c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=irPJxnBdqyEU6-vdNS0A:9 a=CjuIK1q_8ugA:10 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 10:00:27 -0000 On Sun, 25 Feb 2018, Mark Millard wrote: > On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: >> . . . restrict is not needed for the input-only >> args since const suffices. > > Here you lost me. > > With q and r having both the const and the restrict, > updates to p's "objects" can not change the > "object(s)" q and r validly can be used to access. > That can be important. > > Without the "restrict" for q and r (but still > having the const for each) it is valid for updates > to p's "objects" to change what q and r then can > validly access. Yes, I forgot that all const on a pointer arg does is prevent modification through that pointer. Modifications can still occur through other args. memmove() is a canonical example. Its source arg is const, by the source is _always_ modified in the overlapping case that is the reason for existence of memmove(). memcpy() is another canonical example. Both of its args are declared with restrict, except in its man page. Its source arg is still declared as const. Without that, either arg could be modified. With that and restrict on the other arg, I think it follows that the source arg cannot be modified. It is unclear if restrict on the source arg is needed too. It is now clear(er) that POSIX's restricts for sigaction() are correct and not having them for nanosleep() is a bug. nanosleep()'s args are const struct timespec * and struct timespec *. Nothing prevents these being aliased, just like for memmove(), and unlike for memcpy(), the behaviour is not undefined when aliasing occurs. Aliasing can only occur if the pointers are equal, since unlike for mem*() they don't point to arrays (but the prototype doesn't give this information.) So nanosleep(&ts, &ts) must work, and working involves clobbering the input arg. The implementation must be careful to not write the output before reading the input (if the pointers are not equal), and callers using the same timespec for input and output must not depend on the source being const. This is just like for the non-restrict select() except it is easier to avoid problems (e.g., by copying) with a single small object than with a potentially large array for the input arg. My argument applies better to pointer args of different types. Then const prevents modifications through some pointer args and the different types prevent aliasing of the non-const args to the const args. For select() with restrict, I think the compiler cannot assert that the args don't overlap since (even without the detailed specification and Example 3), the compiler cannot know if select() modifies its args. For all that the compiler knows, select() might be a stub that never modifies or even reads anything. I think doing no accesses satifies the constraints of restrict. It might be valid for the compiler to assert that the (values pointed to by) the fdset args don't change for select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access through an fdset type would give undefined behaviour on the other fdset args; tv can still change since it has a different type). But this is precisely what is wanted for the original example where we only care about the return value -- then fdset is indeterminate after the call so we shouldn't use it; the compiler is unlikely to optimize the non-use of it and the worst that it can do is add a runtime assertion that the arg didn't change. Bruce From owner-freebsd-standards@freebsd.org Mon Feb 26 12:46:42 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1D3FF21333; Mon, 26 Feb 2018 12:46:41 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay118.isp.belgacom.be (mailrelay118.isp.belgacom.be [195.238.20.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15F877380F; Mon, 26 Feb 2018 12:46:40 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3ArAR86hFIKLnGdYmUQPj+jp1GYnF86YWxBRYc798d?= =?us-ascii?q?s5kLTJ7yp8WwAkXT6L1XgUPTWs2DsrQY07GQ6/iocFdDyK7JiGoFfp1IWk1Nou?= =?us-ascii?q?QttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXdrXKo8DEdBAj0OxZr?= =?us-ascii?q?KeTpAI7SiNm82/yv95HJbAhEmDSwbaluIBmqsA7cqtQYjYx+J6gr1xDHuGFIe+?= =?us-ascii?q?NYxWNpIVKcgRPx7dqu8ZBg7ipdpesv+9ZPXqvmcas4S6dYDCk9PGAu+MLrrxjD?= =?us-ascii?q?QhCR6XYaT24bjwBHAwnB7BH9Q5fxri73vfdz1SWGIcH7S60/VC+85Kl3VhDnlC?= =?us-ascii?q?YHNyY48G7JjMxwkLlbqw+lqxBm3oLYfJ2ZOP94c6jAf90VWHBBU95fWSJBHI2y?= =?us-ascii?q?cogBAOgPPelXs4bzqEADrQe8CAWwGO/j1iNEimHw0KYn0+ohCwbG3Ak4EtwQsX?= =?us-ascii?q?TUrtH1P7oMXOCyy6nI1ivMb/ZM1jf784jDbxcsoe2NXbJydcrc0kkhFxnbgVqO?= =?us-ascii?q?tIHrIj2b2v4Ks2iB4OptTOSigHMkpQFpujWixdoghpPXio8ay13I7zh1zYg7KN?= =?us-ascii?q?GiVUJ2b9GpHZ1NvC+ALYR2WNktQ2RwtSY/zb0JpIC0cTARyJQi2x7fc/uHc5WU?= =?us-ascii?q?4h77VOaePzN4hHV9dbK8nRmy9UmgyujiWcmu11ZGtDZFktjOtnAJzRDc9s+HSv?= =?us-ascii?q?xm/ki/3DaAzQbT6vpeLUAzj6rbJIYtwr82lpUNrUTOBiz7lFjsgKOIeUgp+/Kk?= =?us-ascii?q?5/npb7jovJOQKoF5hw7mPqQrgMO/AOA4MgYUX2ic/OSxzKHj/Uz7QLVOlfA2nL?= =?us-ascii?q?PZv47EKssAva62HhVZ0oE56xawFzumysgXnWEbLFJZfxKKl4vpNE/QIPD8Cvey?= =?us-ascii?q?mFqskC11yP/YJbLhGYjCImLEkLf7crZ381RcxxYrzdBD+5JUDakMIPzpWkDvqt?= =?us-ascii?q?PXFQQ5PBGtz+b8FNVyzIUeVn+VDa+DLazSqkSF5uw1I+aSeoAaoy39JOU/6/7p?= =?us-ascii?q?l385lkcXfbO10psPdHC4AvNmLl2XYXr2nNgOD3wFvhEjQ+DziF2NSyJcZ3WsUK?= =?us-ascii?q?Im/TE2E4ymDZ3dSY+zm7OBxzq0EodRZmBcBVDfWUvvIq+eRvwBIA+MK8l62mgO?= =?us-ascii?q?T7SsY4g5yQy1sgLmjbFgK6zd53tLm4jk0Y1J5u/X3To18id5Cs2byCnZU2B2mk?= =?us-ascii?q?smXTI79ptT50tnxQHQguBDn/VEGIkLtLtyWQAgOMuZlrQiBg=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2AmBgCOAJRa/4aF9lFcGgEBAQEBAgEBA?= =?us-ascii?q?QEIAQEBAYNPVoEAKI5qjQcBAYIBMgFjlgWCFoU9AoI/VhcBAgEBAQEBAQIBaih?= =?us-ascii?q?CDAGBaSQBgkcBBTocIxALDgoJJQ8qHgYThRStNohkghQBAQEBAQEEAQEBASSOL?= =?us-ascii?q?IsTBaFhCZQTgQSRaZckHwE2gVFNMAiCfYRbPzeIYoN1AQEB?= X-IPAS-Result: =?us-ascii?q?A2AmBgCOAJRa/4aF9lFcGgEBAQEBAgEBAQEIAQEBAYNPVoE?= =?us-ascii?q?AKI5qjQcBAYIBMgFjlgWCFoU9AoI/VhcBAgEBAQEBAQIBaihCDAGBaSQBgkcBB?= =?us-ascii?q?TocIxALDgoJJQ8qHgYThRStNohkghQBAQEBAQEEAQEBASSOLIsTBaFhCZQTgQS?= =?us-ascii?q?RaZckHwE2gVFNMAiCfYRbPzeIYoN1AQEB?= Received: from 134.133-246-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.246.133.134]) by relay.skynet.be with ESMTP; 26 Feb 2018 13:45:29 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w1QCjR7J051426; Mon, 26 Feb 2018 13:45:29 +0100 (CET) (envelope-from tijl@FreeBSD.org) Date: Mon, 26 Feb 2018 13:45:26 +0100 From: Tijl Coosemans To: Bruce Evans Cc: Mark Millard , FreeBSD Hackers , FreeBSD Standards , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180226134526.425aa33c@kalimero.tijl.coosemans.org> In-Reply-To: <20180226200951.V2634@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> <20180226200951.V2634@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 12:46:42 -0000 On Mon, 26 Feb 2018 21:00:21 +1100 (EST) Bruce Evans wrote: > For select() with restrict, I think the compiler cannot assert that > the args don't overlap since (even without the detailed specification > and Example 3), the compiler cannot know if select() modifies its args. > For all that the compiler knows, select() might be a stub that never > modifies or even reads anything. I think doing no accesses satifies > the constraints of restrict. It might be valid for the compiler to > assert that the (values pointed to by) the fdset args don't change for > select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access > through an fdset type would give undefined behaviour on the other fdset > args; tv can still change since it has a different type). But this is > precisely what is wanted for the original example where we only care > about the return value -- then fdset is indeterminate after the call > so we shouldn't use it; the compiler is unlikely to optimize the non-use > of it and the worst that it can do is add a runtime assertion that the > arg didn't change. I don't think the compiler can assert that fdset is unmodified. It's valid for select to modify fdset through one pointer if it doesn't dereference the other two. From owner-freebsd-standards@freebsd.org Tue Feb 27 21:01:13 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 672DCF2879A; Tue, 27 Feb 2018 21:01:13 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailout.stack.nl (mailout05.stack.nl [IPv6:2001:610:1108:5010::202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.stack.nl", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E768F6FADC; Tue, 27 Feb 2018 21:01:12 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mailout.stack.nl (Postfix) with ESMTP id CE9613B; Tue, 27 Feb 2018 22:01:10 +0100 (CET) Received: by toad2.stack.nl (Postfix, from userid 1677) id C6503892CC; Tue, 27 Feb 2018 22:01:10 +0100 (CET) Date: Tue, 27 Feb 2018 22:01:10 +0100 From: Jilles Tjoelker To: Bruce Evans Cc: Mark Millard , FreeBSD Hackers , Tijl Coosemans , FreeBSD Standards , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180227210110.GA76452@stack.nl> References: <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> <20180226200951.V2634@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180226200951.V2634@besplex.bde.org> User-Agent: Mutt/1.8.0 (2017-02-23) X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 21:01:13 -0000 On Mon, Feb 26, 2018 at 09:00:21PM +1100, Bruce Evans wrote: > On Sun, 25 Feb 2018, Mark Millard wrote: > > > On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: > >> . . . restrict is not needed for the input-only > >> args since const suffices. > > Here you lost me. > > With q and r having both the const and the restrict, > > updates to p's "objects" can not change the > > "object(s)" q and r validly can be used to access. > > That can be important. > > Without the "restrict" for q and r (but still > > having the const for each) it is valid for updates > > to p's "objects" to change what q and r then can > > validly access. > Yes, I forgot that all const on a pointer arg does is prevent modification > through that pointer. Modifications can still occur through other args. > memmove() is a canonical example. Its source arg is const, by the source > is _always_ modified in the overlapping case that is the reason for > existence of memmove(). In fact, it is even weaker. The C standard permits casting away const and using the resulting pointer for reading. Unless the underlying object is defined as const, is a string constant or the pointer is based on a restrict pointer to const, the pointer can also be used for writing. In FreeBSD code __DECONST should be used instead of a regular cast, but the same things can be done with the resulting pointer. > memcpy() is another canonical example. Both of its args are declared with > restrict, except in its man page. Its source arg is still declared as > const. Without that, either arg could be modified. With that and restrict > on the other arg, I think it follows that the source arg cannot be modified. > It is unclear if restrict on the source arg is needed too. The restrict on the destination alone prevents it aliasing the source. However, the restrict on the source prevents it aliasing globals and thread-locals. This is not needed for memcpy() which does not write any, but is often needed for more complicated functions. > It is now clear(er) that POSIX's restricts for sigaction() are correct > and not having them for nanosleep() is a bug. nanosleep()'s args are > const struct timespec * and struct timespec *. Nothing prevents these > being aliased, just like for memmove(), and unlike for memcpy(), the > behaviour is not undefined when aliasing occurs. Aliasing can only > occur if the pointers are equal, since unlike for mem*() they don't point > to arrays (but the prototype doesn't give this information.) So > nanosleep(&ts, &ts) must work, and working involves clobbering the input > arg. The implementation must be careful to not write the output before > reading the input (if the pointers are not equal), and callers using the > same timespec for input and output must not depend on the source being > const. This is just like for the non-restrict select() except it is > easier to avoid problems (e.g., by copying) with a single small object > than with a potentially large array for the input arg. Not having restrict on nanosleep()'s args is not much of a burden since its input must necessarily be read before the output can be written (unlike sigaction()). Therefore, I don't think this is a bug. A call to nanosleep() with two equal pointers also has the useful property of being restartable after signals without needing fixups. > My argument applies better to pointer args of different types. Then const > prevents modifications through some pointer args and the different types > prevent aliasing of the non-const args to the const args. Assuming there are no pointers to character type. > For select() with restrict, I think the compiler cannot assert that > the args don't overlap since (even without the detailed specification > and Example 3), the compiler cannot know if select() modifies its args. > For all that the compiler knows, select() might be a stub that never > modifies or even reads anything. I think doing no accesses satifies > the constraints of restrict. It might be valid for the compiler to > assert that the (values pointed to by) the fdset args don't change for > select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access > through an fdset type would give undefined behaviour on the other fdset > args; tv can still change since it has a different type). But this is > precisely what is wanted for the original example where we only care > about the return value -- then fdset is indeterminate after the call > so we shouldn't use it; the compiler is unlikely to optimize the non-use > of it and the worst that it can do is add a runtime assertion that the > arg didn't change. Agreed, with the addition that the arg may even change per Tijl Cooseman's reply. -- Jilles Tjoelker From owner-freebsd-standards@freebsd.org Thu Mar 1 22:44:16 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBF21F30F94 for ; Thu, 1 Mar 2018 22:44:16 +0000 (UTC) (envelope-from christophebeauval@hotmail.com) Received: from EUR01-VE1-obe.outbound.protection.outlook.com (mail-ve1eur01olkn0817.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1f::817]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 320567E556 for ; Thu, 1 Mar 2018 22:44:16 +0000 (UTC) (envelope-from christophebeauval@hotmail.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=SDw4/naYr5roi5frr81nur7HhgIj9Tp8M8FJZjrfBB4=; b=Gmkejk1ExEwfHfWK9aJL/T/o/Rt0P4ECpe5bNO7foTqLZ/h251aqEg6yKb5N1EafSi8tX/VYbxKqavpjHOAi4pPykpP4NJ3C/aA8vdxH3icDmgOZHzqK70YAjILjlenKUx7o6Qr7fX5lxqyhIICigdc6tOo1s2IP3B6okKE4/ComILo7VNQ3fqAy2SnAm3sy1RQGT2aFVL3XrhlahmLrvMlork0dv+H+sXaP8+h/bqh8LtI2IIaLU2RT8pL25BUB2ih5Hj8XzFyRoLPSk36uRN/n/bj6/S/yyq0guydYFauwvkCuodtcj51ukD6sJ6LLBGezVfnQuwBgbn5igH1uNg== Received: from VE1EUR01FT046.eop-EUR01.prod.protection.outlook.com (10.152.2.54) by VE1EUR01HT227.eop-EUR01.prod.protection.outlook.com (10.152.3.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.485.12; Thu, 1 Mar 2018 22:44:13 +0000 Received: from VI1PR0402MB3711.eurprd04.prod.outlook.com (10.152.2.52) by VE1EUR01FT046.mail.protection.outlook.com (10.152.3.92) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.485.12 via Frontend Transport; Thu, 1 Mar 2018 22:44:14 +0000 Received: from VI1PR0402MB3711.eurprd04.prod.outlook.com ([fe80::f055:d81e:6871:3516]) by VI1PR0402MB3711.eurprd04.prod.outlook.com ([fe80::f055:d81e:6871:3516%13]) with mapi id 15.20.0548.013; Thu, 1 Mar 2018 22:44:13 +0000 From: Christophe Beauval To: "freebsd-standards@freebsd.org" Subject: Libc getnameinfo length requirement Thread-Topic: Libc getnameinfo length requirement Thread-Index: AQHTsa7SLJirV9IYX0GJ48JXzy9Gxw== Date: Thu, 1 Mar 2018 22:44:13 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:61F61F31EF8A04CED9B1713AE09C51A6A54CFB9AB7F30A3FEC719D437984A4DE; UpperCasedChecksum:46E9C24F9E4356C05F6DFDF08BE1358FEB384838E6FB5B1DCDE0DC620584917D; SizeAsReceived:6951; Count:44 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [gzBawb+t8wbjHkoRsidnPud2tNV10Nu5] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; VE1EUR01HT227; 6:4R14j04liTX3CGiqo+qCD1bouxl3F5DlW5I1DU4BjFhTys2XxaF/xCf00x6rlatOIF8WTo2a0fAkwNfzIuF6FpWCtHlhoZYPthpvrX1HVQIl0deGOPKReCUOuqg00cw/GE5jAAVPFw9fB/0kVnZwo//vHQ3MhNHrXLv0UQZHXP4sM6GSPX1KYPj6YskinOx0pz7PwVAOnx4KtsxYpaCOuOBfGkcfNAyvgmzOGrOWpz/kM0qdFpukY3y77KdPD/5Ip68+I0EpiKQgnSLBRqmj8rlDUtlSTBHgE/Erk/jW+T9M6JFC8cds85/t++gkIU84tvq/Qtml0wpQvmktqQvOHRae55QTw01N7HTXIMSptGc=; 5:J2KNTKTfrcK6SxHH/6lU9Oc/N1XDRWzpJ9Be+Z/Ndf8e5FodlplGL8HHn6vMFqIcMN/k0/ZSjm5np9XSXYfGW4Le7qN5gZsIgGr0f6HGsqxYuG3crdIBKZDnILy+qMtDjWv5QlcZ2ajSId7w5scLjsQ7MJZqTUnwaotV1NvVnYo=; 24:AXXVGcNZUWFfDk4D5+riNv9eVfo2PH29LOV6gVr8k9AOOibomAVNm91Lb7nRYa41akQiPPyxdvJR+5ieWWL7QFqeJPFDaRf485O1e0AttWs=; 7:bng8Dupr6JAd+nBO3caaH+W3ncq1Ml/3YTTLsczMZ7xaDJet1dg8qKIPql4fiMORmGrO1MYgp8NnYdVG+f3ygXNe+amvH92QYVPVo8Mh72XomFc4jJ8H4mr1ODwjErBSq+TusPE+LChtMCGT0We8kOswFZXdMhJETYD5fjsgBHifAekAER/q4KyazGJGYI9wb2KEfFeUAyLf4po0BNGoqHDcYuUBC5t0EiUXqrEvBPPH0zB0UE+PnaASAkObLSCV x-incomingheadercount: 44 x-eopattributedmessage: 0 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1603101448)(1601125374)(1701031045); SRVR:VE1EUR01HT227; x-ms-traffictypediagnostic: VE1EUR01HT227: x-ms-office365-filtering-correlation-id: 7f7a4320-6ada-4ff2-d0ef-08d57fc5f4c2 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031); SRVR:VE1EUR01HT227; BCL:0; PCL:0; RULEID:; SRVR:VE1EUR01HT227; x-forefront-prvs: 05986C03E0 x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:VE1EUR01HT227; H:VI1PR0402MB3711.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: Content-Transfer-Encoding: base64 MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7f7a4320-6ada-4ff2-d0ef-08d57fc5f4c2 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Mar 2018 22:44:13.9173 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1EUR01HT227 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Mar 2018 22:44:17 -0000 SGVsbG8gYWxsDQoNClRoaXMgbWVzc2FnZSBhZnRlciBoYXZpbmcgcHJlc2VudGVkIHRoZSBwcm9i bGVtIGJlbG93IGluIHRoZSBmcmVlbm9kZSANCiNmcmVlYnNkIGFuZCBFRk5ldCAjYnNkY29kZSBj aGFubmVscyBhbmQgYmVpbmcgYWR2aXNlZCAoYW1vbmdzdCBvdGhlcnMpIA0KdG8gdXNlIHRoaXMg bWFpbGluZyBsaXN0Lg0KDQpXaGVuIEkgZ290IHN0cmFuZ2UgZXJyb3JzIChFQUlfRkFJTCkgZnJv bSBnZXRuYW1laW5mby1jYWxscyBpbiBhIHByaXZhdGUgDQpwcm9qZWN0LCBJIGludmVzdGlnYXRl ZCB0aGUgZnJlZWJzZC9saWJjL25ldC9nZXRuYW1laW5mby5jIHNvdXJjZSBjb2RlIA0KYW5kIHN0 dW1ibGVkIHVwb24gdGhlIHJlcXVpcmVtZW50IChpbiB0aGUgc3dpdGNoLXN0YXRlbWVudCBhdCBs aW5lIDEyNykgDQp0aGF0IHRoZSAic29ja2xlbl90IHNhbGVuIiBhcmd1bWVudCBuZWVkcyB0byBi ZSB0aGUgZXhhY3Qgc2FtZSBudW1iZXIgYXMgDQoic2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9pbiki IGZvciB0aGUgUEZfSU5FVCBwcm90b2NvbCBmYW1pbHkgYW5kIA0KInNpemVvZihzdHJ1Y3Qgc29j a2FkZHJfaW42KSIgZm9yIHRoZSBQRl9JTkVUNiBwcm90b2NvbCBmYW1pbHkuIFRoaXMgDQpmYWls cyB3aGVuIHVzaW5nIGEgInNvY2thZGRyX3N0b3JhZ2UiIHN0cnVjdCBhcyBmaXJzdCBhcmd1bWVu dCBhbmQgdGllZCANCnRvIHRoaXMgYSAic2l6ZW9mKHN0cnVjdCBzb2NrYWRkcl9zdG9yYWdlKSIg YXMgc2Vjb25kIGFyZ3VtZW50LCBhcyBjYW4gDQpiZSBmb3VuZCBhcyBleGFtcGxlIGluIFJGQyA0 MDM4IG9uIHBhZ2UgMjEgcGFyYWdyYXBoIDYuMi4zLiBGb3IgdGhlIA0KUEZfTE9DQUwgcHJvdG9j b2wgZmFtaWx5IGEgbGFyZ2VyICJzYWxlbiIgdGhhbiBpdCdzIHVzZWQgc3RydWN0IGlzIGFsc28g DQpub3QgYWxsb3dlZCBpbiB0aGF0IHN3aXRjaC4NCg0KQSBsZXNzIHN0cmljdCByZXF1aXJlbWVu dCBlaXRoZXIgYWxsb3dpbmcgYW55IGxhcmdlciBzaXplIChsaWtlIGluIA0KZ2xpYmMncyBpbXBs ZW1lbnRhdGlvbikgb3IgYXQgbGVhc3QgdGhlIHNpemUgb2YgInN0cnVjdCANCnNvY2thZGRyX3N0 b3JhZ2UiIChhcyBzdWdnZXN0ZWQgYnkgaHJzKSB3b3VsZCBzb2x2ZSB0aGlzIHByb2JsZW0uDQoN CktpbmQgcmVnYXJkcw0KDQpDaHJpc3RvcGhlIEJlYXV2YWwNCg0K From owner-freebsd-standards@freebsd.org Fri Mar 2 02:37:07 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20194F3FC95 for ; Fri, 2 Mar 2018 02:37:07 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail.allbsd.org (gatekeeper.allbsd.org [IPv6:2001:2f0:104:e001::32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gatekeeper.allbsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DC5B8651A for ; Fri, 2 Mar 2018 02:37:06 +0000 (UTC) (envelope-from hrs@FreeBSD.org) Received: from mail-d.allbsd.org ([IPv6:2409:11:a740:c00:58:65ff:fe00:b0b]) (authenticated bits=56) by mail.allbsd.org (8.15.2/8.15.2) with ESMTPSA id w222amni023762 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) (Client CN "/OU=GT07882699/OU=See+20www.rapidssl.com/resources/cps+20+28c+2915/OU=Domain+20Control+20Validated+20-+20RapidSSL+28R+29/CN=*.allbsd.org", Issuer "/C=US/O=GeoTrust+20Inc./CN=RapidSSL+20SHA256+20CA+20-+20G3"); Fri, 2 Mar 2018 11:37:00 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from alph.d.allbsd.org ([IPv6:2409:11:a740:c00:16:ceff:fe34:2700]) by mail-d.allbsd.org (8.15.2/8.15.2) with ESMTPS id w222ah7e021512 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 2 Mar 2018 11:36:43 +0900 (JST) (envelope-from hrs@FreeBSD.org) Received: from localhost (localhost [IPv6:0:0:0:0:0:0:0:1]) (authenticated bits=0) by alph.d.allbsd.org (8.15.2/8.15.2) with ESMTPA id w222afo7021509; Fri, 2 Mar 2018 11:36:43 +0900 (JST) (envelope-from hrs@FreeBSD.org) Date: Fri, 02 Mar 2018 11:18:04 +0900 (JST) Message-Id: <20180302.111804.1733124916347516747.hrs@allbsd.org> To: christophebeauval@hotmail.com Cc: freebsd-standards@freebsd.org Subject: Re: Libc getnameinfo length requirement From: Hiroki Sato In-Reply-To: References: X-PGPkey-fingerprint: BDB3 443F A5DD B3D0 A530 FFD7 4F2C D3D8 2793 CF2D X-Mailer: Mew version 6.7 on Emacs 25.3 / Mule 6.0 (HANACHIRUSATO) Mime-Version: 1.0 Content-Type: Multipart/Signed; protocol="application/pgp-signature"; micalg=pgp-sha1; boundary="--Security_Multipart0(Fri_Mar__2_11_18_04_2018_403)--" Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.99.3 at gatekeeper.allbsd.org X-Virus-Status: Clean X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.allbsd.org [IPv6:2001:2f0:104:e001:0:0:0:32]); Fri, 02 Mar 2018 11:37:02 +0900 (JST) X-Spam-Status: No, score=-95.5 required=13.0 tests=CONTENT_TYPE_PRESENT, QENCPTR1,RCVD_IN_AHBL,RCVD_IN_AHBL_PROXY,RCVD_IN_AHBL_SPAM,RCVD_IN_CHINA, RCVD_IN_CHINA_KR,RCVD_IN_TAIWAN,RDNS_NONE,URIBL_SC2_SURBL,URIBL_XS_SURBL, USER_IN_WHITELIST autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on gatekeeper.allbsd.org X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 02:37:07 -0000 ----Security_Multipart0(Fri_Mar__2_11_18_04_2018_403)-- Content-Type: Multipart/Mixed; boundary="--Next_Part(Fri_Mar__2_11_18_04_2018_371)--" Content-Transfer-Encoding: 7bit ----Next_Part(Fri_Mar__2_11_18_04_2018_371)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi, Christophe Beauval wrote in : ch> Hello all ch> ch> This message after having presented the problem below in the freenode ch> #freebsd and EFNet #bsdcode channels and being advised (amongst others) ch> to use this mailing list. ch> ch> When I got strange errors (EAI_FAIL) from getnameinfo-calls in a private ch> project, I investigated the freebsd/libc/net/getnameinfo.c source code ch> and stumbled upon the requirement (in the switch-statement at line 127) ch> that the "socklen_t salen" argument needs to be the exact same number as ch> "sizeof(struct sockaddr_in)" for the PF_INET protocol family and ch> "sizeof(struct sockaddr_in6)" for the PF_INET6 protocol family. This ch> fails when using a "sockaddr_storage" struct as first argument and tied ch> to this a "sizeof(struct sockaddr_storage)" as second argument, as can ch> be found as example in RFC 4038 on page 21 paragraph 6.2.3. For the ch> PF_LOCAL protocol family a larger "salen" than it's used struct is also ch> not allowed in that switch. ch> ch> A less strict requirement either allowing any larger size (like in ch> glibc's implementation) or at least the size of "struct ch> sockaddr_storage" (as suggested by hrs) would solve this problem. I have a patch to change getnameinfo() to accept a longer salen (attached). I do not think there is a bad side-effect or impact for backward compatibility. You can try test_getnameinfo.c to test the difference. While I'm here, I changed the return value on error because SUSv4 says EAI_FAMILY should be returned when the address length was invalid. -- Hiroki ----Next_Part(Fri_Mar__2_11_18_04_2018_371)-- Content-Type: Text/X-Patch; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="getnameinfo.c.20180302-1.diff" Index: lib/libc/net/getnameinfo.c =================================================================== --- lib/libc/net/getnameinfo.c (revision 329060) +++ lib/libc/net/getnameinfo.c (working copy) @@ -120,6 +120,16 @@ if (sa == NULL) return (EAI_FAIL); + /* + * getnameinfo() accepts an salen longer than sa->sa_len so that + * sizeof(struct sockaddr_storage) can be specified to salen as + * shown in RFC 4018 Sec.6.2.3. Note that sa->sa_len must be + * the correct length depending on sa->sa_family. + */ + if (salen < sa->sa_len) + return (EAI_FAMILY); + if (salen > sa->sa_len) + salen = sa->sa_len; afd = find_afd(sa->sa_family); if (afd == NULL) @@ -134,16 +144,16 @@ if (salen > afd->a_socklen || salen <= afd->a_socklen - sizeofmember(struct sockaddr_un, sun_path)) - return (EAI_FAIL); + return (EAI_FAMILY); break; case PF_LINK: if (salen <= afd->a_socklen - sizeofmember(struct sockaddr_dl, sdl_data)) - return (EAI_FAIL); + return (EAI_FAMILY); break; default: if (salen != afd->a_socklen) - return (EAI_FAIL); + return (EAI_FAMILY); break; } ----Next_Part(Fri_Mar__2_11_18_04_2018_371)-- Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="test_getnameinfo.c" #include #include #include #include #include #include #include int main(void) { struct sockaddr_storage ss; struct sockaddr *sa = (struct sockaddr *)&ss; struct sockaddr_in *sin = (struct sockaddr_in *)&ss; char hbuf[NI_MAXHOST]; int error; memset(&ss, 0, sizeof(ss)); sin->sin_family = AF_INET; sin->sin_len = sizeof(*sin); /* salen == sa->sa_len */ error = getnameinfo(sa, sa->sa_len, hbuf, sizeof(hbuf), NULL, 0, NI_NUMERICHOST); if (error) printf("salen == sa->sa_len: error: %s\n", gai_strerror(error)); else printf("salen == sa->sa_len: hbuf = %s\n", hbuf); /* salen > sa->sa_len */ error = getnameinfo(sa, sizeof(ss), hbuf, sizeof(hbuf), NULL, 0, NI_NUMERICHOST); if (error) printf("salen > sa->sa_len: error: %s\n", gai_strerror(error)); else printf("salen > sa->sa_len: hbuf = %s\n", hbuf); return (EXIT_SUCCESS); } ----Next_Part(Fri_Mar__2_11_18_04_2018_371)---- ----Security_Multipart0(Fri_Mar__2_11_18_04_2018_403)-- Content-Type: application/pgp-signature Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iEYEABECAAYFAlqYtFwACgkQTyzT2CeTzy3aFACfY61mz4B75vO350wdLfLvfQ+K qGMAoNEYD+jjilYMR6uwITOHfQPfKPU1 =FwBz -----END PGP SIGNATURE----- ----Security_Multipart0(Fri_Mar__2_11_18_04_2018_403)---- From owner-freebsd-standards@freebsd.org Fri Mar 2 12:32:44 2018 Return-Path: Delivered-To: freebsd-standards@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6421FF41955 for ; Fri, 2 Mar 2018 12:32:44 +0000 (UTC) (envelope-from christophebeauval@hotmail.com) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-oln040092064104.outbound.protection.outlook.com [40.92.64.104]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B5167E3E7; Fri, 2 Mar 2018 12:32:43 +0000 (UTC) (envelope-from christophebeauval@hotmail.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=+vy4guwrm/AWlGAL8dAHRApRZrJ8VXbQZ0RbbUzTjNg=; b=Qdefrd0HsV9p9opRHkU+ck81zxAd8eEgOAGrvNkJl+a26LPbpAnbQriMS2ipEA/FDmi85rxhbwABQ0H2Aj0r8277ehFdx3vADdjAaOjoBSeM2qEuX413sT72d0HmGHI9EBBSdAuDdqUF2eOEEprUkuGfsk1QAPiIYGl9Dp+bQPWW5NnT8XMSQW/fPiiA9BRBlzoO0JHhQ4ZRMqVPNiheg4TFQbcglnw9buq7eUgR589knHrjoZN3FmP1eqbyBJzvQfDRKvwnS4HRA0eDwSoq/Bn8dius/dgxB/ZTOz/npVNjEj96THnItbrG7w/BP46B46AXvuER7oMAxSbOdwOqbQ== Received: from HE1EUR01FT051.eop-EUR01.prod.protection.outlook.com (10.152.0.58) by HE1EUR01HT209.eop-EUR01.prod.protection.outlook.com (10.152.1.137) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.527.15; Fri, 2 Mar 2018 12:32:41 +0000 Received: from VI1PR0402MB3711.eurprd04.prod.outlook.com (10.152.0.51) by HE1EUR01FT051.mail.protection.outlook.com (10.152.1.71) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.527.22 via Frontend Transport; Fri, 2 Mar 2018 12:32:41 +0000 Received: from VI1PR0402MB3711.eurprd04.prod.outlook.com ([fe80::f055:d81e:6871:3516]) by VI1PR0402MB3711.eurprd04.prod.outlook.com ([fe80::f055:d81e:6871:3516%13]) with mapi id 15.20.0548.013; Fri, 2 Mar 2018 12:32:41 +0000 From: Christophe Beauval To: Hiroki Sato CC: "freebsd-standards@freebsd.org" Subject: Re: Libc getnameinfo length requirement Thread-Topic: Libc getnameinfo length requirement Thread-Index: AQHTsa7SLJirV9IYX0GJ48JXzy9Gx6O8NgYAgACrrYA= Date: Fri, 2 Mar 2018 12:32:40 +0000 Message-ID: References: <20180302.111804.1733124916347516747.hrs@allbsd.org> In-Reply-To: <20180302.111804.1733124916347516747.hrs@allbsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:039FEAE6A156C775AC2F026C46B6A4ED17F573B60A85B3A2836202A93CB5C7F6; UpperCasedChecksum:51F56403A855D6AFC9D2A4E64A1766A4345F7AE56CA5B2351642425421BDBCE0; SizeAsReceived:7221; Count:47 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [pNdLJIipd6K+kdgjBqL+4mLqqeUg8NWI] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1EUR01HT209; 6:qL9RZPFZA9MNVxnS7DMDbXCgRAmeZHfWtp0h7fP68zfwzJvYgXjMjtOcxLV3/Nxy+pBQQwIV5h5iOwCWny8nyOUera7N1X8lytzZPmm4mNkMJANY8DcoflohaoNNSAmaFu3OgrnOATP7ZIRSjV9TmYDRHILUmq2d6gSo13+Lp/sSUC68r5FNQosXDXEEYZUN8xF4y/6nzMYrNl/vwGco2kavadKEplq2C7GndMVxsNg2K6qXA3WFdgFgBX+9JIWbvhaJr/atWwL2b5p/EV7wlJHIHwYHVIgBbyB/CO7n1DefwjvCi2t4lqv5A6w8TqfJwZEkhdfytYWddmiFqL4Wl6G4bxVe0vE2LLWVzgn2vo4=; 5:obTpjfY20bb7S1mHkX0n0gRcwUGShOn0GJrXQvQyySi7QQDiX2CN4YVZpfxdpWzLXTQlWdNn57kpmejUwtA/ib+ntIFKZFOlL5AT8y9vRcy+JyTcMttjm9veWiEo1LNBd4JApYXu9pDGy7OC1NMAjQhBQsYRofxXEkaXnfDHgZE=; 24:Rh3XoVuDzlNknhpKDFkKfvpMwXE5NIwVgIFDoljIZp6ldbicfynNj/kXQUS2TtZCTj2Opv3fMe5EpUgeFAnHA83HDHpbha0cnmOQUQ+BkTE=; 7:7IfpJHbM1yXySH50HweLddUW9jVVMbcPpOU1oS9YWac8/2/156uRWLx/oi9PFi6VtglabduZ8hncdrHfj/uHtDON7FpbBIJnn5uS0lISF9YHCj0a0xejXynPA/ElOvwKfXaZGf3KejoGtz/cZanvVikkvigi2n7IJnUQXHERzoOjrwkauRsnt+F06s4BBv60yeyId+rj7Ekg36ppuC8tzDD0JNVXWj3bmSKQSlagvYS264Ma5sqn79BfdTVJvukW x-incomingheadercount: 47 x-eopattributedmessage: 0 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(201702061078)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125374)(1603101448)(1701031045); SRVR:HE1EUR01HT209; x-ms-traffictypediagnostic: HE1EUR01HT209: x-ms-office365-filtering-correlation-id: bde814fb-e3bf-4751-9d2a-08d58039ae7f x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031); SRVR:HE1EUR01HT209; BCL:0; PCL:0; RULEID:; SRVR:HE1EUR01HT209; x-forefront-prvs: 05991796DF x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:HE1EUR01HT209; H:VI1PR0402MB3711.eurprd04.prod.outlook.com; FPR:; SPF:None; LANG:; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="Windows-1252" Content-ID: Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-Network-Message-Id: bde814fb-e3bf-4751-9d2a-08d58039ae7f X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Mar 2018 12:32:41.1258 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1EUR01HT209 X-BeenThere: freebsd-standards@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Standards compliance List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 12:32:44 -0000 Hello There is a typo in the comment: RFC 4018 is mentioned, this should be 4038. There is also a problem with relying on the sa->sa_len presence and sort=20 of undoing two earlier patches from 17th April 2005, where according to=20 the commit comment, sa->sa_len is not required by POSIX:=20 https://github.com/freebsd/freebsd/commit/90a7ec34ec2cf5e36aeb76ff35c656f88= 0659274#diff-25ee506a989a23ae5d8904e113aa66d7=20 and=20 https://github.com/freebsd/freebsd/commit/ba35b6aa7602967dbfeb53dc1e32ba186= 8bd3e98#diff-25ee506a989a23ae5d8904e113aa66d7=20 . Kind regards Christophe Beauval Op 2/03/2018 om 3:18 schreef Hiroki Sato: > Hi, > > Christophe Beauval wrote > in : > > ch> Hello all > ch> > ch> This message after having presented the problem below in the freenode > ch> #freebsd and EFNet #bsdcode channels and being advised (amongst other= s) > ch> to use this mailing list. > ch> > ch> When I got strange errors (EAI_FAIL) from getnameinfo-calls in a priv= ate > ch> project, I investigated the freebsd/libc/net/getnameinfo.c source cod= e > ch> and stumbled upon the requirement (in the switch-statement at line 12= 7) > ch> that the "socklen_t salen" argument needs to be the exact same number= as > ch> "sizeof(struct sockaddr_in)" for the PF_INET protocol family and > ch> "sizeof(struct sockaddr_in6)" for the PF_INET6 protocol family. This > ch> fails when using a "sockaddr_storage" struct as first argument and ti= ed > ch> to this a "sizeof(struct sockaddr_storage)" as second argument, as ca= n > ch> be found as example in RFC 4038 on page 21 paragraph 6.2.3. For the > ch> PF_LOCAL protocol family a larger "salen" than it's used struct is al= so > ch> not allowed in that switch. > ch> > ch> A less strict requirement either allowing any larger size (like in > ch> glibc's implementation) or at least the size of "struct > ch> sockaddr_storage" (as suggested by hrs) would solve this problem. > > I have a patch to change getnameinfo() to accept a longer salen > (attached). I do not think there is a bad side-effect or impact for > backward compatibility. You can try test_getnameinfo.c to test the > difference. > > While I'm here, I changed the return value on error because SUSv4 > says EAI_FAMILY should be returned when the address length was > invalid. > > -- Hiroki