From owner-freebsd-hackers@freebsd.org Sun Feb 25 01:00:03 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54B0FF31D4A for ; Sun, 25 Feb 2018 01:00:03 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: from mail-yb0-x22e.google.com (mail-yb0-x22e.google.com [IPv6:2607:f8b0:4002:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1ACD8080B for ; Sun, 25 Feb 2018 01:00:02 +0000 (UTC) (envelope-from lists@eitanadler.com) Received: by mail-yb0-x22e.google.com with SMTP id b12-v6so4190529ybn.8 for ; Sat, 24 Feb 2018 17:00:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=eitanadler.com; s=0xdeadbeef; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Kd2dMGLPbr3pBn0Vyur84L0mMaQgOERiDPJKr1Ih2cQ=; b=OQIFbIZVCbgQmovPLajQScpy2RauAqwAhVpzCaZtWnPDvrmFADPTYGiSn8Lh181MVU R71uoSrotAtmNxnqAxT7cW3Jo8uKVzb2JovxG4M9X+P0TqFaHYgugsTRSNMIzvNVtdJu wvGH+VkpMJcVfxkZmssvmtgi02Ps2biL2tLuY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Kd2dMGLPbr3pBn0Vyur84L0mMaQgOERiDPJKr1Ih2cQ=; b=kGInYqO4AHpq1hJGSV+nOJlBwb1ZEyykHVb3Zt8mcd3zDVvnME+tjiyvBxOnaSksLk DmBz9X68k4K6HJrsTAC7MVasxxQ0sPFuMVw7HeFS/BgWEkJtRQGnYSwPcVtcOp4Pi41u /Xf2wfWocVkRFu4UxdNOJr5uByixyDktQ4gyOLgesa/uzxtGDFQokUZgy4Pq9v9b+x0f 9CQEY5W6AyUsMNYmaKTxKvT0OlIVOxz0caWlyqxdngN0t2uO2hGj2N/i1XPxD06YrNGB OZ56SmFkdEsYJFDKeZ6UBw4yo8p5cDYg1DZ4Tt/AhCYe1OHiURNPje8JRz8oRr2svwhA aVNg== X-Gm-Message-State: APf1xPB9XK1ZBBuC4GQWKNkp5sU9/PL12JET3wP4xxNNk5S9ZClDGTbn uE5OPfYHWu8dtIxICX+0UYWs7zZp7vnPMNpx45cRQQ== X-Google-Smtp-Source: AG47ELtn0oF8c6fhskzkw64qQLxtk1SW2i7rWQAiVl7iqboIaZ6M2Q0TLsQgR9BdqLiSezaT5wN9oG4VlNtcWpRNYIE= X-Received: by 2002:a25:6006:: with SMTP id u6-v6mr4191129ybb.460.1519520402106; Sat, 24 Feb 2018 17:00:02 -0800 (PST) MIME-Version: 1.0 Received: by 2002:a25:dfcb:0:0:0:0:0 with HTTP; Sat, 24 Feb 2018 16:59:31 -0800 (PST) In-Reply-To: References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> From: Eitan Adler Date: Sat, 24 Feb 2018 16:59:31 -0800 Message-ID: Subject: Re: Marking select(2) as restrict To: cem@freebsd.org Cc: FreeBSD Hackers , FreeBSD Standards Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 01:00:03 -0000 On 24 February 2018 at 10:55, Conrad Meyer wrote: > On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler wrote: >> After this entire thread here is the summary. If I've misrepresented >> you here please let me know. >> ... >> >> kib@ - no benefit; concerned fallout could be hard to observe >> cem@ - concerned about warnings > > Consider me a +1 to kib@. I did not voice those concerns explicitly > in earlier email because kib did already and I didn't anticipate you > would ignore him. I am not ignoring him. As I stated above I do not believe fallout is likely since most other major libc implementations have already done this: glibc: already done - https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 openbsd: already done https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 dragonflyBSD: alredy done: https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 netbsd: already done: https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 As a further check I went through the search results on github for select() and did not see any failures in the top few pages. -- Eitan Adler From owner-freebsd-hackers@freebsd.org Sun Feb 25 01:28:50 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E726DF33D2A; Sun, 25 Feb 2018 01:28:49 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail108.syd.optusnet.com.au (mail108.syd.optusnet.com.au [211.29.132.59]) by mx1.freebsd.org (Postfix) with ESMTP id 3435B81D4B; Sun, 25 Feb 2018 01:28:48 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail108.syd.optusnet.com.au (Postfix) with ESMTPS id D610A1A47E3; Sun, 25 Feb 2018 12:28:38 +1100 (AEDT) Date: Sun, 25 Feb 2018 12:28:37 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Warner Losh cc: "Conrad E. Meyer" , FreeBSD Standards , FreeBSD Hackers Subject: Re: Marking select(2) as restrict In-Reply-To: Message-ID: <20180225112627.B976@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=uyavkMrdAAAA:8 a=5jWAyVIvSIEuDVK_DRoA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 a=j2_G595jqNHTxQgNwHU2:22 X-Mailman-Approved-At: Sun, 25 Feb 2018 03:58:28 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 01:28:50 -0000 On Sat, 24 Feb 2018, Warner Losh wrote: > On Sat, Feb 24, 2018 at 11:55 AM, Conrad Meyer wrote: > >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >> wrote: >>> After this entire thread here is the summary. If I've misrepresented >>> you here please let me know. >>> ... >>> >>> kib@ - no benefit; concerned fallout could be hard to observe >>> cem@ - concerned about warnings >> >> Consider me a +1 to kib@. I did not voice those concerns explicitly >> in earlier email because kib did already and I didn't anticipate you >> would ignore him. > > So there's no benefit to the change (we won't optimize better). It's hard > to observe breakage. No answer about how we'd even know if something broke > because a exp run sure as hell isn't going to tell us. > > All that militates against the change rather strongly. Your exp run will > change no minds because it is useless. Why not remove restrict from other APIs to be consistent with select()? If might break their callers just as much. Start with pselect(). sigaction() is interesting too. Without restrict for sigaction(), there is nothing (in old FreeBSD man pages) to prevent callers passing the same pointer for 'act' and 'oact'. This might be a good hack. 'act' is const, but will be overwritten on copyout if it is the same as 'oact'. The kernel could reasonably copyout to 'oact' before reading 'act'. This clobbers the input arg. I grepped all man pages in libc/sys for APIs taking 2 pointer args and not having restrict: _umtx_op() abort2() adjtime() aio_suspend() aio_waitcomplete() clock_nanosleep(), nanosleep() (most interesting. One pointer arg is const. It is unclear if that prevents aliasing, but the API is similar to that of sigaction() and POSIX added restrict for the latter only) execve() extattr_*() (at least 8 functions in man page with too many functions) fexecve() fhopen() fhstat() fhstatfs() fstatat() futimensat() getdirentries() getfh() getresgid() getresuid() gettimeofday() kenv() kevent() lgetfh() link() linkat() lio_listio() mincore() mount() mq_receive() mq_timedreceive() mq_timedsend() ppoll() (this obfuscates its first pointer arg using [] instead of *, and is missing restrict for that arg only. ppoll() is similar to pselect() except for these bugs) quotactl() rctl_*() (all 5 functions in another unsplit man page) rename() renameat() sctp_generic_recvmsg() sctp_generic_sendmsg() sctp_generic_sendmsg_iov() select() settimeofday() setitimer() (like select(). Fixed in POSIX in 2001, but not in FreeBSD) sendfile() sendto() statfs() symlink() symlinkat() utimensat() These are mostly BSD APIs. POSIX fixed almost all of the POSIX APIs in 2001. In a few cases like readlink(), there is no problem since all of the pointer args are const. Almost all of the POSIX APIs in the above list are of this type. In a few cases like statfs(), there is probably no problem because the pointer arg types are different and none of them is void *. In statfs(), one of them is also const. Most other cases are broken. E.g., POSIX added restrict to recvmsg() and sendmsg() in 2001, but the related newer sctp APIs haven't caught up with thus yet. Most FreeBSD timer APIs are missing restrict even when they are also in POSIX and not missing restrict there. sendmmsg() only has 1 pointer arg, but this is input-output which causes similar problems. It is declared as restrict. I don't know if that fixes the problems (the constraint on the implementation's ordering and restrict in the API doesn't give that AFAIK). Bruce From owner-freebsd-hackers@freebsd.org Sun Feb 25 04:06:40 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F367CF00114 for ; Sun, 25 Feb 2018 04:06:39 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic317-28.consmr.mail.bf2.yahoo.com (sonic317-28.consmr.mail.bf2.yahoo.com [74.6.129.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95AE087F5A for ; Sun, 25 Feb 2018 04:06:39 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: VZE0io0VM1lDfcyX3dFNBerczClz.AtmOANfB773jEMDVcP3hrA.eAttLKhUYdW Aed26_8a4uWndigq82wV7LjXl8jrgp4dP.E7ZrsB82t2XybQnGfR8hsDcoTxzXZgLbNmAgQ9vdst xKKwOgFA6TgXhDdbO6wPmvwwaomYkXC2GWfIlJwQi9vdXXNnO07tXgG.B.XzymCMw5xcTsjoNCOW 1D5Xggk7IFEBCUQT9Mz0pPX_SHB5uUQf.s8vlMZEtqV0Sj._MuiEKmxqtPTLTT1qcSO_34ft1a8M mkYzCgAkowhq3ZVofb8tjWjbJbhJroyjQaCxYhCY98CP7W3FD6lPr9BQCRhld5PX54lIUrueh0qg eeB8Z9eO2qiMvXgsOPWxUPftE13sGi3bb7Fb9MmRWyHfkFEOM9dhioazmWDZYP5atiaqQQFoysOP .rPmuafWYKtrwR7AG8IcT4oKA1WhX3BIZnEQqSTRyKZNQqsEAdsWCAp.y4R5jnxfn Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.bf2.yahoo.com with HTTP; Sun, 25 Feb 2018 04:06:33 +0000 Received: from smtp103.rhel.mail.bf1.yahoo.com (EHLO [192.168.1.25]) ([98.139.230.213]) by smtp411.mail.bf1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID 2506173844fd19f52721971e29959e28; Sun, 25 Feb 2018 03:56:22 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: Date: Sat, 24 Feb 2018 19:56:20 -0800 Cc: FreeBSD Hackers , FreeBSD Standards Content-Transfer-Encoding: quoted-printable Message-Id: <9438AC5E-56B8-46E4-AECE-5C3A194F4D1E@yahoo.com> References: To: Eitan Adler X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 04:06:40 -0000 On 2018-Feb-15, at 12:10 AM, Eitan Adler wrote: > Hi all, >=20 > POSIX requires that the fd_set arguments in select(2) be marked as > restrict. This patch attempts to implement that. >=20 > (a) Am I missing anything? > (b) Anything in particular to watch out for? > (c) Assuming an exp-run passes any reason not to commit? >=20 >=20 > Index: lib/libc/sys/select.2 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- lib/libc/sys/select.2 (revision 329296) > +++ lib/libc/sys/select.2 (working copy) > @@ -39,7 +39,7 @@ > .Sh SYNOPSIS > .In sys/select.h > .Ft int > -.Fn select "int nfds" "fd_set *readfds" "fd_set *writefds" "fd_set > *exceptfds" "struct timeval *timeout" > +.Fn select "int nfds" "fd_set * restrict readfds" "fd_set * restrict > writefds" "fd_set * restrict exceptfds" "struct timeval *timeout" > .Fn FD_SET fd &fdset > .Fn FD_CLR fd &fdset > .Fn FD_ISSET fd &fdset > Index: lib/libc/sys/select.c > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- lib/libc/sys/select.c (revision 329296) > +++ lib/libc/sys/select.c (working copy) > @@ -41,7 +41,7 @@ __weak_reference(__sys_select, __select); >=20 > #pragma weak select > int > -select(int n, fd_set *rs, fd_set *ws, fd_set *es, struct timeval *t) > +select(int n, fd_set * restrict rs, fd_set * restrict ws, fd_set * > restrict es, struct timeval *t) > { >=20 > return (((int (*)(int, fd_set *, fd_set *, fd_set *, struct timeval = *)) > Index: sys/sys/select.h > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > --- sys/sys/select.h (revision 329296) > +++ sys/sys/select.h (working copy) > @@ -101,8 +101,7 @@ int pselect(int, fd_set *__restrict, fd_set *__res > const struct timespec *__restrict, const sigset_t *__restrict); > #ifndef _SELECT_DECLARED > #define _SELECT_DECLARED > -/* XXX missing restrict type-qualifier */ > -int select(int, fd_set *, fd_set *, fd_set *, struct timeval *); > +int select(int, fd_set *__restrict, fd_set *__restrict, fd_set > *__restrict, struct timeval *); > #endif > __END_DECLS > #endif /* !_KERNEL */ Going in a different direction: C++ . . . =46rom FreeBSD's cdefs.h : #if !(__GNUC__ =3D=3D 2 && __GNUC_MINOR__ =3D=3D 95) #if !defined(__STDC_VERSION__) || __STDC_VERSION__ < 199901 #define __restrict #else #define __restrict restrict #endif #endif It looks to me like C++ use of cdefs.h and then, say, select.h, could easily lead to __restrict being translated to no-text. C++11 does add __STDC_VERSION__ to the "implementation-defined value, if present" category. (Quoted material is from en.cppreference.com .) This would lead C++ to not give errors/warnings for violating the constraints involved in calling a newly Linux-like implementation of select (with C99-like restrict involved). It also means that if some C++ compilers have a __restrict (-like) extension that it is not being put to use for either code generation or for reporting violations of C99-like constraints. Of course if a C++11 or later targeting defines __STDC_VERSION__ with it being >=3D 199901 then the C++ compiler would see "restrict" (no quotes) after the substitution, likely giving a syntax error. (It is not a keyword in C++.) But these types of points also apply to existing uses of __restrict after cdefs.h use (unless I missed a level of conditionality that is relevant). But at least there is some history as evidence for these. Overall result: C++ apparently only gets run-time behavior as evidence for the use of the new content of select's implementation if this is changed: no reports of abusive calls as stands. =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-hackers@freebsd.org Sun Feb 25 05:52:57 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47C54F0B25F; Sun, 25 Feb 2018 05:52:57 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D37316D54B; Sun, 25 Feb 2018 05:52:56 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-qt0-x22d.google.com with SMTP id g14so15298089qti.2; Sat, 24 Feb 2018 21:52:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Acp96vYJWPn0oJeci604nQ1lm91Y0lj15KMyk5+miPQ=; b=jmdo/N0LQe84sEQGdbCPsup6c9uWx6NJ9Ed8GyS3tBG9wLt+zLizodqhrcWQ3aNPfr l6s0+qaEacccwbmBc5tQP02BUQe9jOj7xH45DDDT/v+KddBl4DLBH4FRvlkU2UUXIUGc CSezeUKr+J1ghmzqU2WuPukRJN25tq5BDQqgX1P5tl4Zqh2pyLgrlxP4kdRrt/rM6d5R u9TvRDM5ErkskIIFyooQjCo+/oz3aew/UdErFAiY8zeeXk4lXI7ZvSty1xAIaakPk0O2 jiXtM2ayskWaoT45D9c4KeAZ9ws5GMaO77uspkh2Dh9fU7kXMk0a8w1pS2T9arQTLId2 uvJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Acp96vYJWPn0oJeci604nQ1lm91Y0lj15KMyk5+miPQ=; b=fbGKkpWM9wcrfNBJ4BunA7EhqQxFlJSTPAhgpkoaUfJvn3KwvyNYrY8jYh0E8xISO5 eSt4/3OSZXFklx5MZ+2N49lR5OSX7zRXEqDWdE6KXLQXZ6L3ejKu2KnwxTumntfZYS5O +5mqz77dvIpLKQAxQNAS1fV7kp3bE8EHH6b0mR8NShM7xsImjPrBZWXSAC2R6B6bZOcy YhhwdGQ2iBA9YvIxm9KqFdOh3xwDYZ3iAW3wUW1UFcW/gw42sDRoWLw1h018u3lT8jS1 gJIo7sYonZ00hsKy3E2vITrs8Wf8nKpWR6qtI7UlYq4KkCmEpT9l4EYUPUADyDaiCPAU DNoA== X-Gm-Message-State: APf1xPDrxL9TOHXXRKmcczkL6MbZGJsgLO2aIwecJ62U8eDwT6oNfbJ5 FFIEUp6krUTZjF3PymCo33Fhtknifl43P/s2kOs= X-Google-Smtp-Source: AG47ELunZqxlnmSxLtspqbrKy4Dna02GHHOUnjdE0K8sstxylYhph7JmG/7KO4wL92dxgRitsyRN4USIS2qJgO+eP4I= X-Received: by 10.200.35.141 with SMTP id q13mr10690571qtq.73.1519537976439; Sat, 24 Feb 2018 21:52:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.32.74 with HTTP; Sat, 24 Feb 2018 21:52:55 -0800 (PST) In-Reply-To: References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> From: Stefan Blachmann Date: Sun, 25 Feb 2018 06:52:55 +0100 Message-ID: Subject: Re: Marking select(2) as restrict To: Eitan Adler Cc: cem@freebsd.org, FreeBSD Hackers , FreeBSD Standards Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 05:52:57 -0000 The Linux manual pages do not mention restrict for select(). glibc select() itself returns just ENOSYS(), if there is no alias for select(). So I guess what actually gets called is this: https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 Which in turn appears to call __sys_select: https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L346 See also: https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.html If I understand correctly, the *only* place that defines the optimizations actually being done is the static functions itself: https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L487 So maybe the actual prototypes being used for the functions for which the interpose array is used are irrelevant: https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L642 I not yet found out which functions are actually weakly aliased in, but I could imagine that adding the restrict keyword to the prototypes of the functions listed there, is possibly only of cosmetical importance, without any actual effect. If this is correct, one could be "Posix compliant" without causing any disruptive "optimization" :) Have a nice Sunday! Stefan P.S.: Maybe it would be better to avoid adding the restrict keyword in the __thr_select() function itself mentioned above, as Linux' select function seems to have no restrict: https://github.com/torvalds/linux/blob/master/fs/select.c#L1262 https://github.com/torvalds/linux/blob/master/fs/select.c#L599 On 2/25/18, Eitan Adler wrote: > On 24 February 2018 at 10:55, Conrad Meyer wrote: >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >> wrote: >>> After this entire thread here is the summary. If I've misrepresented >>> you here please let me know. >>> ... >>> >>> kib@ - no benefit; concerned fallout could be hard to observe >>> cem@ - concerned about warnings >> >> Consider me a +1 to kib@. I did not voice those concerns explicitly >> in earlier email because kib did already and I didn't anticipate you >> would ignore him. > > I am not ignoring him. As I stated above I do not believe fallout is > likely since most other major libc implementations have already done > this: > > glibc: already done - > https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 > openbsd: already done > https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 > dragonflyBSD: alredy done: > https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 > netbsd: already done: > https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 > > As a further check I went through the search results on github for > select() and did not see any failures in the top few pages. > > -- > Eitan Adler > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > From owner-freebsd-hackers@freebsd.org Sun Feb 25 14:49:40 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A91B8F319EF for ; Sun, 25 Feb 2018 14:49:40 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: from mail-ot0-x22f.google.com (mail-ot0-x22f.google.com [IPv6:2607:f8b0:4003:c0f::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 30DD27F8D1 for ; Sun, 25 Feb 2018 14:49:40 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: by mail-ot0-x22f.google.com with SMTP id w38so11236760ota.8 for ; Sun, 25 Feb 2018 06:49:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=gnDCmiqQjcMTyYi4lL3owpQrWqocMHiwTMIJjqlY9ZE=; b=rRfdWNTcPFt2LQB1QkKK2bbR0aF0VOCXS4BKCQmyhtfL55pm+OqBg6VoXVN9Fu1jzS a9lyftjfDki4OcAJhItcV+yY4NNUtqreu+M3QO/lVgjaiXiNre8UMQhW01a46OYOe2lY C1iXrtRLe7wVSgTNjGQE2Mx5A/hmlBN75xkURf+H8MZqtg+q+x8H3WwXYrOOQU3yxoJR 59Q4oUjraUwOXi3n5GepZumuHaSsbxHQE4Zi1t9jlOoB6tLMkxdlkT9nB+FS4su3hb8N W3ztmqowfnWkDS3k8zGgb3krZdBn+FAuSFt42lHowhqCsx2y8Xj5q0xt5wSksM9Bdd4r IWFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=gnDCmiqQjcMTyYi4lL3owpQrWqocMHiwTMIJjqlY9ZE=; b=IbbbXXwcMEGOJwmreQUW55kp2VFAeV1CxpzSGgkNbqZOblTYlMBpGn3ZezOg66s8l8 meqzdaTxwKSq9dWaGWrKs2ccqbYS8Zj28S6cMswwGYeNQazN8BQ2ZwlVa9XR+YvdrOfj th7iDiKIzwoYw5ai9QjSILkMMtTMjFcKCJKqd8N04jBvXK8SWoXYAyTOAF5Y0OoDxH8c wmfT1cNZhN/fWIPJ0gJ9BYzgCzd4da/lpToXUsrEVKOCEKg9Zv8NA6FTxR1OjAMP5ZCb 0iKUgPgTiV1EM/gs3ur5vksSNghPSlkqDf9pryr+M2g5BvF6mqDUVTjwo8JQ8Jr0lZ7D S6Zw== X-Gm-Message-State: APf1xPDJw0nUNU+mNBo7x7HIZafIuQEVDpXZ6jf9MhWvy+u2JoQHR+qo 5LqIk0uFFUI3+TSf1CUhGg1QuevhZuboAnjNAYAhIA== X-Google-Smtp-Source: AG47ELt6VCeNVNijiEF/CH9z8yfTZ2BQ9XZvLbPXhWi0CqSqpO2Gy4BTqPnx6F+ZdzH0UrtGRkxGjujSZmSpE11lAL0= X-Received: by 10.157.32.228 with SMTP id x91mr5728259ota.176.1519570179300; Sun, 25 Feb 2018 06:49:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.45.194 with HTTP; Sun, 25 Feb 2018 06:48:58 -0800 (PST) From: Lee D Date: Sun, 25 Feb 2018 09:48:58 -0500 Message-ID: Subject: Help, please, with getting a custom I2C real time clock module to load To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 14:49:41 -0000 Hi Everyone, I have written a new I2C driver (for the Xilinx Zynq) and a new real time clock chip driver (for the ST M41T82) to use with hardware on my custom board. This is for 11.0.1. The I2C driver works fine, but I can't seem to get my RTC driver to load. The m41t82_probe() function is never even called. Both are loaded with kldload at the moment. I think the problem is something in DRIVER_MODULE macro that is preventing the kernel from even trying to let it probe. One clue is that if I change "iicbus" to "simplebus" in the DRIVER_MODULE macro of the RTC, it will then at least call m41t82_probe(). It's like the kernel thinks that I have no iicbus driver and thus won't even try to load the RTC module. I have turned on "device iic" and "device iicbus" in my kernel config file. No messages are given when I try to load the m41t82 module. It just silently loads and does nothing. Here is a code snippet from my I2C driver: ------------------------------------------ static driver_t i2c_driver = { "i2c", i2c_methods, sizeof(struct i2c_softc), }; static devclass_t i2c_devclass; DRIVER_MODULE(iicbus, i2c, iicbus_driver, iicbus_devclass, 0, 0); DRIVER_MODULE(i2c, simplebus, i2c_driver, i2c_devclass, 0, 0); And here is a code snippet from my RTC driver: ---------------------------------------------- static device_method_t m41t82_methods[] = { DEVMETHOD(device_probe, m41t82_probe), DEVMETHOD(device_attach, m41t82_attach), DEVMETHOD(device_detach, m41t82_detach), DEVMETHOD(clock_gettime, m41t82_gettime), DEVMETHOD(clock_settime, m41t82_settime), DEVMETHOD_END }; static driver_t m41t82_driver = { "m41t82", m41t82_methods, sizeof(struct m41t82_softc), }; static devclass_t m41t82_devclass; DRIVER_MODULE(m41t82, iicbus, m41t82_driver, m41t82_devclass, NULL, NULL); MODULE_VERSION(m41t82, 1); MODULE_DEPEND(m41t82, iicbus, 1, 1, 1); This is the relevant portion of my DTS file: -------------------------------------------- ps7io@e0000000 { device_type = "soc"; compatible = "simple-bus"; #address-cells = <0x1>; #size-cells = <0x1>; ranges = <0x0 0xe0000000 0x300000>; ... Other hardware ... i2c@4000 { compatible = "xlnx,zy7_i2c"; status = "okay"; reg = <0x4000 0x1000>; #address-cells = <0x1>; #size-cells = <0x0>; rtc@d0 { status = "okay"; compatible = "st,m41t82"; reg = <0xd0>; }; }; }; Any clues about how to go about debugging this problem would be very helpful. Thanks, Lee From owner-freebsd-hackers@freebsd.org Sun Feb 25 14:47:03 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 82933F31585 for ; Sun, 25 Feb 2018 14:47:03 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic307-9.consmr.mail.gq1.yahoo.com (sonic307-9.consmr.mail.gq1.yahoo.com [98.137.64.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F39737F678 for ; Sun, 25 Feb 2018 14:47:02 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: aeiaDq4VM1n0FoQTDuAiG9hKIOA3Oc1AkBzmbJ8nYGjSD0RhiAH.iWCxMjX43SL zqrX3ha8eDwnfekZAaQGGGGk9r53tg_8tP4E2n1HxkRL6IY_4M09N337lpwBwCPvNjw4YzvP.PUE KbmrFccV6Rot4slpok2VgkbjUCrvvC4FeyY3SL4TvkNJ6me05tiuXIiyofCR4sCeJnJ8jPQICMdn Z4EZYlKBtZTHPaX449G0SxlKAMj5GRMUXdRlwyEkZAxT4z0xngY0hLCY6zudmB7h.fMviVT9R1lP koXGSMjti0VKD4p4W7NoQgeycoVSCXM_zuDzukwmUIhmVnjkJ7towfyczyyAC3.iEcFh8nRB8CA2 MejPULuFkwh.ezAneYp8xQ2abFG0_pXaQtForqoX_FRJkuACRp0iRSrCB9z.xDg8q1tVvxgMK7y0 MvPtKIGfZLmcA7GAUjo_6.0XMxgqFRrtOy9scTLJZeNaa_4vLGb_cEwV7MfL4QI4- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Sun, 25 Feb 2018 14:46:56 +0000 Received: from smtp101.rhel.mail.gq1.yahoo.com (EHLO [192.168.1.25]) ([68.180.227.10]) by smtp413.mail.gq1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID e97702b39406288e9e6deaaf59d87a7d; Sun, 25 Feb 2018 14:46:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225112627.B976@besplex.bde.org> Date: Sun, 25 Feb 2018 06:46:52 -0800 Cc: Warner Losh , FreeBSD Hackers , FreeBSD Standards , "Conrad E. Meyer" Content-Transfer-Encoding: 7bit Message-Id: <5957DA7A-80CC-48D8-AC4B-8EC403658AC2@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <23181.46427.671514.319710@khavrinen.csail.mit.edu> <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> <20180225112627.B976@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 14:47:03 -0000 On 2018-Feb-24, at 5:28 PM, Bruce Evans wrote: > On Sat, 24 Feb 2018, Warner Losh wrote: > >> On Sat, Feb 24, 2018 at 11:55 AM, Conrad Meyer wrote: >> >>> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler >>> wrote: >>>> After this entire thread here is the summary. If I've misrepresented >>>> you here please let me know. >>>> ... >>>> >>>> kib@ - no benefit; concerned fallout could be hard to observe >>>> cem@ - concerned about warnings >>> >>> Consider me a +1 to kib@. I did not voice those concerns explicitly >>> in earlier email because kib did already and I didn't anticipate you >>> would ignore him. >> >> So there's no benefit to the change (we won't optimize better). It's hard >> to observe breakage. No answer about how we'd even know if something broke >> because a exp run sure as hell isn't going to tell us. >> >> All that militates against the change rather strongly. Your exp run will >> change no minds because it is useless. > > Why not remove restrict from other APIs to be consistent with select()? If > might break their callers just as much. This appears to be about the handling of non-conforming programs. Quoting from C99's 6.7.3 "Type qualifiers" paragraph 7 about conforming programs: "The intended use of the restrict qualifier (like the register storage class) is to promote optimization, and deleting all instances of the qualifier from all preprocessing units composing a conforming program does not change its meaning (i.e. observable behavior)." So: If removing restrict results in a conforming program's observable behavior changing, then that would be evidence that the environment is incorrectly implemented relative to C99. Of course, removing restrict can make it harder to detect non-conforming programs if the compiler(s) involved are helpful when restrict is present. Going the other way: 6.7.3.1 "Formal definition of restrict" also says in its paragraph 6: "A translator is free to ignore any or all aliasing implications of uses of restrict." === Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-hackers@freebsd.org Sun Feb 25 13:51:30 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE697F2F645; Sun, 25 Feb 2018 13:51:29 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DC417DD7E; Sun, 25 Feb 2018 13:51:29 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w1PDpCuk031025 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 25 Feb 2018 15:51:15 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w1PDpCuk031025 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w1PDpBjT031024; Sun, 25 Feb 2018 15:51:11 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Sun, 25 Feb 2018 15:51:11 +0200 From: Konstantin Belousov To: Stefan Blachmann Cc: Eitan Adler , FreeBSD Hackers , FreeBSD Standards , cem@freebsd.org Subject: Re: Marking select(2) as restrict Message-ID: <20180225135111.GO94212@kib.kiev.ua> References: <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.3 (2018-01-21) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 13:51:30 -0000 On Sun, Feb 25, 2018 at 06:52:55AM +0100, Stefan Blachmann wrote: > The Linux manual pages do not mention restrict for select(). > glibc select() itself returns just ENOSYS(), if there is no alias for select(). > > So I guess what actually gets called is this: > https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 > Which in turn appears to call __sys_select: > https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L346 > See also: > https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.html > > If I understand correctly, the *only* place that defines the > optimizations actually being done is the static functions itself: > https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L487 No, you do not understand this correctly. Select(2) implementation is in kernel and cannot be affected by the userspace prototype change. It is the caller of select(2) which might be optimized unexpectedly when select claims that its arguments cannot be aliased. The use of restrict in the glibc prototype would be a good argument if clang on glibc were not a rare combination. > > So maybe the actual prototypes being used for the > functions for which the interpose array is used are irrelevant: > https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_syscalls.c#L642 > > I not yet found out which functions are actually weakly aliased in, > but I could imagine that adding the restrict keyword to the prototypes > of the functions listed there, is possibly only > of cosmetical importance, without any actual effect. > If this is correct, one could be "Posix compliant" without causing any > disruptive "optimization" :) > > Have a nice Sunday! > Stefan > > > P.S.: Maybe it would be better to avoid adding the restrict keyword in the > __thr_select() function itself mentioned above, as Linux' select > function seems to have no restrict: > https://github.com/torvalds/linux/blob/master/fs/select.c#L1262 > https://github.com/torvalds/linux/blob/master/fs/select.c#L599 > > > On 2/25/18, Eitan Adler wrote: > > On 24 February 2018 at 10:55, Conrad Meyer wrote: > >> On Sat, Feb 24, 2018 at 10:35 AM, Eitan Adler > >> wrote: > >>> After this entire thread here is the summary. If I've misrepresented > >>> you here please let me know. > >>> ... > >>> > >>> kib@ - no benefit; concerned fallout could be hard to observe > >>> cem@ - concerned about warnings > >> > >> Consider me a +1 to kib@. I did not voice those concerns explicitly > >> in earlier email because kib did already and I didn't anticipate you > >> would ignore him. > > > > I am not ignoring him. As I stated above I do not believe fallout is > > likely since most other major libc implementations have already done > > this: > > > > glibc: already done - > > https://github.com/bminor/glibc/blob/master/misc/sys/select.h#L101 > > openbsd: already done > > https://github.com/openbsd/src/blob/master/sys/sys/select.h#L128 > > dragonflyBSD: alredy done: > > https://github.com/dragonflybsd/dragonflybsd/blob/master/sys/sys/select.h#L50 > > netbsd: already done: > > https://github.com/NetBSD/src/blob/trunk/sys/sys/select.h#L69 > > > > As a further check I went through the search results on github for > > select() and did not see any failures in the top few pages. > > > > -- > > Eitan Adler > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-hackers@freebsd.org Sun Feb 25 18:00:21 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFF45F066B6 for ; Sun, 25 Feb 2018 18:00:21 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CB4769893 for ; Sun, 25 Feb 2018 18:00:20 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: b1379751-1a55-11e8-bb8e-b35b57339d60 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id b1379751-1a55-11e8-bb8e-b35b57339d60; Sun, 25 Feb 2018 17:59:58 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1PI0I1B072911; Sun, 25 Feb 2018 11:00:18 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1519581618.91697.261.camel@freebsd.org> Subject: Re: Help, please, with getting a custom I2C real time clock module to load From: Ian Lepore To: Lee D , freebsd-hackers@freebsd.org Date: Sun, 25 Feb 2018 11:00:18 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 18:00:21 -0000 On Sun, 2018-02-25 at 09:48 -0500, Lee D wrote: > Hi Everyone, > > I have written a new I2C driver (for the Xilinx Zynq) and a new real > time clock chip driver (for the ST M41T82) to use with hardware on my > custom board.  This is for 11.0.1. > [...] I just remembered... another piece of non-obvious magic that is required to be an i2c host controller whose bus is managed by ofw_iicbus is that your driver has to implement the ofw_bus_get_node method.  The implementation is trivial, just grab the one out of arm/freescale/imx/imx_i2c.c. -- Ian From owner-freebsd-hackers@freebsd.org Sun Feb 25 14:54:22 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5290AF31E59 for ; Sun, 25 Feb 2018 14:54:22 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: from mail-oi0-x229.google.com (mail-oi0-x229.google.com [IPv6:2607:f8b0:4003:c06::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D26627FDCE for ; Sun, 25 Feb 2018 14:54:21 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: by mail-oi0-x229.google.com with SMTP id c83so8952764oib.1 for ; Sun, 25 Feb 2018 06:54:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ZfHCHegVXtZ9YXVi9pBkSAQmLjEEuRHNcoaHN9limug=; b=UCINj03rrkqJTNF7MmaTmd3h7euSdlPT44sTIftWmff7SQPyk8WRPgUp+P5qbpahY+ EbvsuRLrYU+CYr46M2vcxgrrlLalt8nhs6zHsKlPzv7pvAs4yRnhPBiUlV+2qYtSLDYn uQdNLDMrZp8H6IH/LpkaiEZBKN0TpB3udQ5CAR1uSSe1f9toTZ16cJbVUwj2bICEjjSL Fzl5QiTO+4veH/7vLz4Ypt9xGEnHfft6gzWuOD4Lb7fuDmHNUHCqpnx/siiK1xndloND cCeB3y7+i1lFYZ3uL5oo6GfgAAnUvr0D0kcrye8fJLjXUxd2vspKZzTQJWfZ/UI+I3f7 uLCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ZfHCHegVXtZ9YXVi9pBkSAQmLjEEuRHNcoaHN9limug=; b=D1sse3d21R9kK+IAT9TZgGH2BvnglFUap5iNctj13MXuv8cMBZDjaZdT+Ya8ZThM6d 01PgLfqxnsSbAgUsm2Z9aK3s+JihR4Jg63kGOdAY9CNHpSVQOxHW2meVoG9Si5/3DOp9 ADPMv/dMIm6j24P1fxWw+O1SjPZPpWf2A/7VWAe9/JwZih6YJjROuOsjr9M7QWuTjLr+ GPMScOL1SfcnMEfwA/9gDMgA8nzMtj/JH+r+8kLnO/zdTVRzdfCfQTKNU9+TuU4jcGf3 M/8DuMnXV7hs/Zz/kbRo+X6ehZChWjRyYqhuQFKqegZuLg28RyRcOQHiSC/MrrtQzxwZ h13g== X-Gm-Message-State: APf1xPAI/ZxZxN/XktJcc48Gqvd6/w7YK3/pgWNHivwMBARG+tg+cPUe TetO2kt4f9oltA3iKVYZHbh0EoKRJNsWMaCFoRtCFw== X-Google-Smtp-Source: AG47ELuBWCAf6lHfYQHOFsfpZKhh6GkRx5G3ZqM0DT1x/KwL3+5EFE9n2im9sn8iazoa6DYT+AKSMagfOXjmCpEvYLk= X-Received: by 10.202.78.145 with SMTP id c139mr5036491oib.47.1519570461348; Sun, 25 Feb 2018 06:54:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.45.194 with HTTP; Sun, 25 Feb 2018 06:53:41 -0800 (PST) In-Reply-To: References: From: Lee D Date: Sun, 25 Feb 2018 09:53:41 -0500 Message-ID: Subject: Re: How to set various locales with setlocale()? To: Lee Brown Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 14:54:22 -0000 On Fri, Feb 16, 2018 at 2:29 PM, Lee Brown wrote: > On Fri, Feb 16, 2018 at 3:08 AM, Lee D wrote: > >> Do you folks have any suggestions on how to accomplish this goal, or >> how to make setlocale accept the locales listed on the system? >> >> > Different architecture, it works as advertised. Maybe ask the arm folks? > Thanks for the example. The problem ended up being my own fault. Months ago I had disabled installation of locales in the src.conf file in attempt to make my installation image smaller. Putting the locales back into the installation made it work. From owner-freebsd-hackers@freebsd.org Sun Feb 25 17:42:18 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96BACF03A72 for ; Sun, 25 Feb 2018 17:42:18 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic307-9.consmr.mail.gq1.yahoo.com (sonic307-9.consmr.mail.gq1.yahoo.com [98.137.64.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F54368CC6 for ; Sun, 25 Feb 2018 17:42:17 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: j.1WblMVM1k6TvUCt_SeQvg09_WKkQkhu6IaP98pLJwinQ.ag62IcfE3f.Zk8v7 lKYzmsWBT7q873AK8AQOWv9GbCVPq0CTssphe2l4O2n8UyY5QIY_1kCrt2BDLasnO.LqUzA2Nulj edXDub5AFq4FMH3I9oyiHJeMCAL_Y6p.kUzeSPZQ0KHzLMpoI3s8awgperhRVoYtUTPqwsN49D0v x7lGHEbesAN3I4_vZwinKgf4A2_PSuj1qD1oqau5HKI7Uaib38qPZGRHnfk2DU0qdyqHv8qtv9QQ 6Hiv4PRiZ.ou6FbuTDlMf8p.cCvW4HM4wBe9RYjOTsV4uIhQM8kunjUPmc0XbqTN61kfTevRHdpB .eJLsTtZOOLCg6nxyu.pChyIk8OUpprKRKO.IH9Q04GjzQa3.TakMXfb5eW7KlXhUUhid9dCajeG wdomriT0bUAHDayQAQS2B2SHO8wIJvfVMOWDjOqWMWpmtD6db_7YxGO50uZrkS8lBQ.OAfBQ- Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Sun, 25 Feb 2018 17:42:16 +0000 Received: from smtp105.rhel.mail.gq1.yahoo.com (EHLO [192.168.1.25]) ([68.180.227.8]) by smtp413.mail.gq1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID 151b169aca54fce9868b612577f67de5; Sun, 25 Feb 2018 17:32:04 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225135111.GO94212@kib.kiev.ua> Date: Sun, 25 Feb 2018 09:32:03 -0800 Cc: Stefan Blachmann , Eitan Adler , "Conrad E. Meyer" , FreeBSD Standards , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <0C8DEA21-0F27-4D8D-8E30-42378B166FBF@yahoo.com> References: <20180221185920.GA94212@kib.kiev.ua> <23181.50488.186767.579361@khavrinen.csail.mit.edu> <20180221201002.GC94212@kib.kiev.ua> <23181.54825.511195.393054@khavrinen.csail.mit.edu> <20180222212746.GB58772@stack.nl> <20180225135111.GO94212@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 17:42:18 -0000 On 2018-Feb-25, at 5:51 AM, Konstantin Belousov = wrote: > On Sun, Feb 25, 2018 at 06:52:55AM +0100, Stefan Blachmann wrote: >> The Linux manual pages do not mention restrict for select(). >> glibc select() itself returns just ENOSYS(), if there is no alias for = select(). >>=20 >> So I guess what actually gets called is this: >> https://github.com/udp/freebsd-libc/blob/master/sys/select.c#L48 >> Which in turn appears to call __sys_select: >> = https://github.com/udp/freebsd-libc/blob/master/include/libc_private.h#L34= 6 >> See also: >> = https://lists.freebsd.org/pipermail/freebsd-questions/2007-August/154906.h= tml >>=20 >> If I understand correctly, the *only* place that defines the >> optimizations actually being done is the static functions itself: >> = https://github.com/freebsd/freebsd/blob/master/lib/libthr/thread/thr_sysca= lls.c#L487 > No, you do not understand this correctly. Select(2) implementation is > in kernel and cannot be affected by the userspace prototype change. It > is the caller of select(2) which might be optimized unexpectedly when > select claims that its arguments cannot be aliased. >=20 > The use of restrict in the glibc prototype would be a good argument if > clang on glibc were not a rare combination. I'm only noting where some evidence might be available . . . My understanding is that there is (at least) one desktop Linux distribution that is clang based: OpenMadriva Lx. It may be a source of information. Checking a little into its status . . . =46rom https://en.wikipedia.org/wiki/OpenMandriva_Lx : "A beta release of OpenMandriva Lx 3.0 was released in June 2016.[19] This new release came with significant changes to the core system - among other things, it was the first desktop Linux distribution that was built completely with the Clang compiler instead of GCC." https://wiki.openmandriva.org/en/3.03/Release_Notes#LLVM.2Fclang reports: "OpenMandriva provides LLVM/clang 5.0.0 as the default compiler, GCC is also available. Over 90% of packages in our main repository are now built with LLVM/clang." = https://www.openmandriva.org/en/news/article/openmandriva-lx-3-03-get-it-w= hile-it-s-hot reports: "Everything with this release, including the new Firefox Quantum 57, is compiled with LLVM/clang 5.0.0" Looking quickly, there is glibc source involved: https://github.com/OpenMandrivaAssociation/glibc/tree/master (There are other branches, such a 3.0 branch.) I have not analyzed the patches that they use. For reference for V3.03: ( = https://www.openmandriva.org/en/news/article/openmandriva-lx-3-03-get-it-w= hile-it-s-hot ) "At the hardware level there is an up-to-date kernel at 4.13.12 release and systemd 234 and for your graphics stack mesa 17.2.3 with a S3TC support enabled and xorg 1.19.5." "Our main desktop environment KDE Plasma is updated to 5.10.5 and Frameworks are at 5.39." =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-hackers@freebsd.org Sun Feb 25 17:23:17 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C672FF02425 for ; Sun, 25 Feb 2018 17:23:16 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 53A08680CD for ; Sun, 25 Feb 2018 17:23:16 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 7fc9e6f3-1a50-11e8-bb8e-b35b57339d60 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id 7fc9e6f3-1a50-11e8-bb8e-b35b57339d60; Sun, 25 Feb 2018 17:22:48 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1PHN78a072827; Sun, 25 Feb 2018 10:23:07 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1519579387.91697.252.camel@freebsd.org> Subject: Re: Help, please, with getting a custom I2C real time clock module to load From: Ian Lepore To: Lee D , freebsd-hackers@freebsd.org Date: Sun, 25 Feb 2018 10:23:07 -0700 In-Reply-To: References: Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 17:23:17 -0000 On Sun, 2018-02-25 at 09:48 -0500, Lee D wrote: > Hi Everyone, > > I have written a new I2C driver (for the Xilinx Zynq) and a new real > time clock chip driver (for the ST M41T82) to use with hardware on my > custom board.  This is for 11.0.1. > > The I2C driver works fine, but I can't seem to get my RTC driver to > load.  The m41t82_probe() function is never even called. > > Both are loaded with kldload at the moment. > > I think the problem is something in DRIVER_MODULE macro that is > preventing the kernel from even trying to let it probe.  One clue is > that if I change "iicbus" to "simplebus" in the DRIVER_MODULE macro > of the RTC, it will then at least call m41t82_probe(). > > It's like the kernel thinks that I have no iicbus driver and thus > won't even try to load the RTC module. > > I have turned on "device iic" and "device iicbus" in my kernel config > file. > > No messages are given when I try to load the m41t82 module.  It just > silently loads and does nothing. > > Here is a code snippet from my I2C driver: > ------------------------------------------ > > static driver_t i2c_driver = { >   "i2c", >   i2c_methods, >   sizeof(struct i2c_softc), > }; > static devclass_t  i2c_devclass; > > DRIVER_MODULE(iicbus, i2c, iicbus_driver, iicbus_devclass, 0, 0); > DRIVER_MODULE(i2c, simplebus, i2c_driver, i2c_devclass, 0, 0); > Right here is where the disconnect is happening.  It's the ofw_iicbus driver that needs to declared in the first DRIVER_MODULE() instead of iicbus, because ofw_iicbus is the one that knows to look in the fdt data for slave devices and add them as children of the bus.  But, the extern declarations needed to do that didn't exist until I added them last week in r329526. What i2c drivers have been doing in the past, and the way to work around it in the 11.x code you're dealing with, is to leverage one of the DRIVER_MODULE() declarations that already exists in ofw_iicbus, by naming your driver "iichb" instead of "i2c".  Like this: static driver_t i2c_driver = {   "iichb",   i2c_methods,   sizeof(struct i2c_softc), }; Or you could import r329526 into the kernel source you're using and rebuild the kernel.  I do intend to MFC that change to 11-stable (in fact, I should probably do that today). When you've got these new drivers working, please consider putting them up for review at https://reviews.freebsd.org and we'll get them committed to freebsd. -- Ian From owner-freebsd-hackers@freebsd.org Sun Feb 25 17:42:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CBDDF03A7C for ; Sun, 25 Feb 2018 17:42:24 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wm0-x244.google.com (mail-wm0-x244.google.com [IPv6:2a00:1450:400c:c09::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0747868CDB for ; Sun, 25 Feb 2018 17:42:24 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wm0-x244.google.com with SMTP id q83so13349998wme.5 for ; Sun, 25 Feb 2018 09:42:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=y1W4hS0dorqzfNjCmr0OfQ/osKUIU5dgEOHHU00H4Uc=; b=BXGdaJzDhJQg8Jv9vwxE4JoZjATLaq8VbnqLL/+CWK8XnrA4jVc7zIVyAnE1FuGWOU m43OqIv546hF8pWDa7i3d+s2Sl50cFwHysHdYlOcM0tRIqZzvs/BiJAc2o5S9udmNGCD l7gwv2sNfXW3Su15qXWO6rDkwliZcCsa/6IDzWB9ajPDc/9l+c9I7TqAmd0mqc4ke6TI glluvj1OmWRF/FSozFS3GvCorpC7lPzdyovFDAZTe8fqaQY1o5dm29eWsDw1t2hMfi1U 3CVmmaMdrJrckNtKy/Lz3noYxGTq6bv5nJ2BYLdTHeYQ5uZnA+5luIaP2jZiYNyfPlnn W73Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=y1W4hS0dorqzfNjCmr0OfQ/osKUIU5dgEOHHU00H4Uc=; b=hEaK9AMMtRgFCiJM027pze5IGjsOZPhw45NfnRx2YEL/bQZ+UUIlrZm/jN1yKPuzTs jAXPYx5OlEcQxEEIcD20ZKfscIen/wvXGyyaITcKH4nWzuj8Eds3B8ouNd/GIEp6yfoE 437Nn8iXlWHU+tl2jxuvFkrPSWUILvIAgx7Z7l1Zq9cnIVTVJ7/TQUpu+iGJCWRIsOu1 QjM4xvJ8Ag6GbwyAbospZwNWZF3hFld4bQBNsvX6zbV+pdhSI4R7UNFfjKlxFHr4jfaz S8BJJsai3ZZkZRzGZSc4Hd89MOo4xyevFezoggFZL2ZuNstEYS6SsATTcLpOJLkUQe5K lmUg== X-Gm-Message-State: APf1xPA/SL9RHbNmYXjIO3mnrqQ88lM11mIWOl4dk49aPF6QdvPGd6Nu buFVmA17QHNNBkSo8IUAtBU= X-Google-Smtp-Source: AG47ELuN2X6MaIruJsggkML+skjg4Ei4c4TaikaIwfdeagofw+CpW8pI+B89TRL39CPLWRQt/nfXHA== X-Received: by 10.28.167.208 with SMTP id q199mr6360997wme.29.1519580543072; Sun, 25 Feb 2018 09:42:23 -0800 (PST) Received: from ernst.home (pD9E23F28.dip0.t-ipconnect.de. [217.226.63.40]) by smtp.gmail.com with ESMTPSA id 56sm8199114wrt.23.2018.02.25.09.42.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 25 Feb 2018 09:42:22 -0800 (PST) Date: Sun, 25 Feb 2018 18:42:20 +0100 From: Gary Jennejohn To: Lee D Cc: freebsd-hackers@freebsd.org Subject: Re: Help, please, with getting a custom I2C real time clock module to load Message-ID: <20180225184220.748e9d59@ernst.home> In-Reply-To: References: Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 17:42:24 -0000 On Sun, 25 Feb 2018 09:48:58 -0500 Lee D wrote: > Hi Everyone, > > I have written a new I2C driver (for the Xilinx Zynq) and a new real > time clock chip driver (for the ST M41T82) to use with hardware on my > custom board. This is for 11.0.1. > > The I2C driver works fine, but I can't seem to get my RTC driver to > load. The m41t82_probe() function is never even called. > > Both are loaded with kldload at the moment. > > I think the problem is something in DRIVER_MODULE macro that is > preventing the kernel from even trying to let it probe. One clue is > that if I change "iicbus" to "simplebus" in the DRIVER_MODULE macro > of the RTC, it will then at least call m41t82_probe(). > > It's like the kernel thinks that I have no iicbus driver and thus > won't even try to load the RTC module. > > I have turned on "device iic" and "device iicbus" in my kernel config > file. > > No messages are given when I try to load the m41t82 module. It just > silently loads and does nothing. > > Here is a code snippet from my I2C driver: > ------------------------------------------ > > static driver_t i2c_driver = { > "i2c", > i2c_methods, > sizeof(struct i2c_softc), > }; > static devclass_t i2c_devclass; > > DRIVER_MODULE(iicbus, i2c, iicbus_driver, iicbus_devclass, 0, 0); > DRIVER_MODULE(i2c, simplebus, i2c_driver, i2c_devclass, 0, 0); > This should give you a clue: grep DRIVER_MODULE /sys/dev/iicbus/iic.c DRIVER_MODULE(iic, iicbus, iic_driver, iic_devclass, 0, 0); grep DRIVER_MODULE /sys/dev/iicbus/rtc8583.c DRIVER_MODULE(rtc8583, iicbus, rtc8583_driver, rtc8583_devclass, NULL, NULL); [snip] -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Sun Feb 25 17:57:19 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7ACF7F0655B for ; Sun, 25 Feb 2018 17:57:19 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 039396971D for ; Sun, 25 Feb 2018 17:57:18 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: 38329ce8-1a55-11e8-b951-f99fef315fd9 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id 38329ce8-1a55-11e8-b951-f99fef315fd9; Sun, 25 Feb 2018 17:56:35 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1PHvA62072887; Sun, 25 Feb 2018 10:57:10 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1519581430.91697.258.camel@freebsd.org> Subject: Re: Help, please, with getting a custom I2C real time clock module to load From: Ian Lepore To: gljennjohn@gmail.com, Lee D Cc: freebsd-hackers@freebsd.org Date: Sun, 25 Feb 2018 10:57:10 -0700 In-Reply-To: <20180225184220.748e9d59@ernst.home> References: <20180225184220.748e9d59@ernst.home> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 17:57:19 -0000 On Sun, 2018-02-25 at 18:42 +0100, Gary Jennejohn wrote: > On Sun, 25 Feb 2018 09:48:58 -0500 > Lee D wrote: > > > > > Hi Everyone, > > > > I have written a new I2C driver (for the Xilinx Zynq) and a new real > > time clock chip driver (for the ST M41T82) to use with hardware on my > > custom board.  This is for 11.0.1. > > > > The I2C driver works fine, but I can't seem to get my RTC driver to > > load.  The m41t82_probe() function is never even called. > > > > Both are loaded with kldload at the moment. > > > > I think the problem is something in DRIVER_MODULE macro that is > > preventing the kernel from even trying to let it probe.  One clue is > > that if I change "iicbus" to "simplebus" in the DRIVER_MODULE macro > > of the RTC, it will then at least call m41t82_probe(). > > > > It's like the kernel thinks that I have no iicbus driver and thus > > won't even try to load the RTC module. > > > > I have turned on "device iic" and "device iicbus" in my kernel config > > file. > > > > No messages are given when I try to load the m41t82 module.  It just > > silently loads and does nothing. > > > > Here is a code snippet from my I2C driver: > > ------------------------------------------ > > > > static driver_t i2c_driver = { > >   "i2c", > >   i2c_methods, > >   sizeof(struct i2c_softc), > > }; > > static devclass_t  i2c_devclass; > > > > DRIVER_MODULE(iicbus, i2c, iicbus_driver, iicbus_devclass, 0, 0); > > DRIVER_MODULE(i2c, simplebus, i2c_driver, i2c_devclass, 0, 0); > > > This should give you a clue: > > grep DRIVER_MODULE /sys/dev/iicbus/iic.c > DRIVER_MODULE(iic, iicbus, iic_driver, iic_devclass, 0, 0); > Nope, not applicable.  Lee's problem was due to not getting ofw_iicbus connected to his new host controller driver so that the children of iicbus would get enumerated.  iic.c is not an i2c host controller whose child is an iicbus, it is itself a child of iicbus. The real problem is that until last week it was impossible for an FDT- based host controller driver to have the proper DRIVER_MODULE() statement because some extern declarations were missing, and the way existing drivers had been working around that was to set their name to "iichb". -- Ian > grep DRIVER_MODULE /sys/dev/iicbus/rtc8583.c > DRIVER_MODULE(rtc8583, iicbus, rtc8583_driver, rtc8583_devclass, NULL, NULL); > > [snip] > From owner-freebsd-hackers@freebsd.org Sun Feb 25 20:48:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38588F279BC; Sun, 25 Feb 2018 20:48:23 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay101.isp.belgacom.be (mailrelay101.isp.belgacom.be [195.238.20.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 596DA6FE0E; Sun, 25 Feb 2018 20:48:22 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3ABxWDYRKkFurs+5XLudmcpTZWNBhigK39O0sv0rFi?= =?us-ascii?q?tYgRLfrxwZ3uMQTl6Ol3ixeRBMOHs6kC07KempujcFRI2YyGvnEGfc4EfD4+ou?= =?us-ascii?q?JSoTYdBtWYA1bwNv/gYn9yNs1DUFh44yPzahANS47xaFLIv3K98yMZFAnhOgpp?= =?us-ascii?q?POT1HZPZg9iq2+yo9JDffwtFiCChbb9uMR67sRjfus4KjIV4N60/0AHJonxGe+?= =?us-ascii?q?RXwWNnO1eelAvi68mz4ZBu7T1et+ou+MBcX6r6eb84TaFDAzQ9L281/szrugLd?= =?us-ascii?q?QgaJ+3ART38ZkhtMAwjC8RH6QpL8uTb0u+ZhxCWXO9D9QKsqUjq+8ahkVB7oiD?= =?us-ascii?q?8GNzEn9mHXltdwh79frB64uhBz35LYbISTOfFjfK3SYMkaSHJPUMhRSSJPH4Cy?= =?us-ascii?q?YIkBD+UOIelWoJLwp0cMoBeiGQWgGP/jxiFOi3Tr3aM6yeMhEQTe0QI+HtIOsn?= =?us-ascii?q?DUp8jrOacVVuC117fHzTDZYPNQwjf29Y/FcgwgofGOWbJ9asrfyVMxGAzbk1ie?= =?us-ascii?q?tILrMymS1uQXvGiW9uxtXv+shW4/swx8oSWjyt0yhoTGh48Z0E3I+Ct3zYovON?= =?us-ascii?q?G1RkF2bNi5G5VKrS6aLZF5QsY6TmFtvyY116MJtIagfCgP1JQn3xnfa+Gbc4SQ?= =?us-ascii?q?4hLsSuKRITBgiXJmYr2/gxey8U2+xe3mUcm4ykpKritHktnIrHwCyxvT6s+cSv?= =?us-ascii?q?Rj+0euwzCP1xvJ5uFDO0A0mrLXK58nwrEuipoeqUfOEjLslEnog6Kbd18o9vWm?= =?us-ascii?q?5unpeLnqu5GROoBshgH7KKsum8i/AeoiMggJWmiW4fi81Lzh/U39W7hKgOc2nb?= =?us-ascii?q?fHv5/BPsQUu7S1AwhP0oYs8xq/FSup0MwEnXkbK1JIYBGHj4/yO1HSIfD4Duyw?= =?us-ascii?q?jEqokDpwyPDGO6fuApTJLnTZjLjherN9uAZgz18QytZE+5tSFrAHaNj+Xkjsr9?= =?us-ascii?q?vGRks6NBeowuXtBdFV2YYXWGbJCaicZvD8q1iNs94uIe3ET4gSozv4Iv4+r6ry?= =?us-ascii?q?jH09sXEHcKSD5rdRb2q3SKc1a36FaGbh149SWVwBuRAzGamz0AWP?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2B9AQANIJNa/4aF9lFcGgEBAQEBAgEBA?= =?us-ascii?q?QEIAQEBAYNPVoEAKI12dI0GAQGCATIBY4dtjhiCFoU9AoI3VRgBAgEBAQEBAQI?= =?us-ascii?q?BaiiCOCQBgkcBBTocIxALDgoJJQ8SGB4GE4R4Axmtd4cjDYEyghQBAQEBAQEEA?= =?us-ascii?q?QEBASSOLIJqiCkFoS8yCY81hF6BBJFpjRmKCx44gVFNMAiCfYJDHIF8PzeMbAE?= =?us-ascii?q?BAQ?= X-IPAS-Result: =?us-ascii?q?A2B9AQANIJNa/4aF9lFcGgEBAQEBAgEBAQEIAQEBAYNPVoE?= =?us-ascii?q?AKI12dI0GAQGCATIBY4dtjhiCFoU9AoI3VRgBAgEBAQEBAQIBaiiCOCQBgkcBB?= =?us-ascii?q?TocIxALDgoJJQ8SGB4GE4R4Axmtd4cjDYEyghQBAQEBAQEEAQEBASSOLIJqiCk?= =?us-ascii?q?FoS8yCY81hF6BBJFpjRmKCx44gVFNMAiCfYJDHIF8PzeMbAEBAQ?= Received: from 134.133-246-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.246.133.134]) by relay.skynet.be with ESMTP; 25 Feb 2018 21:48:15 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w1PKmDJt045352; Sun, 25 Feb 2018 21:48:14 +0100 (CET) (envelope-from tijl@FreeBSD.org) Date: Sun, 25 Feb 2018 21:48:13 +0100 From: Tijl Coosemans To: Konstantin Belousov Cc: Eitan Adler , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180225214813.776a9f58@kalimero.tijl.coosemans.org> In-Reply-To: <20180222105608.GE94212@kib.kiev.ua> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 20:48:23 -0000 On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: > Consider the recently changed devd code: > select(n + 1, &fd, &fd, &fd); > There, compiler can see that restrict is applied to arguments which are > given same values. Since this leads to the self-contradicting statement > fd != fd > which cannot be true, compliler in its optimizing wisdom can assume that > the code is never executing and remove it. I do not know whether clang > actually makes such transformation, but it does not sound unfeasible > looking at its other advances. There's an example in the C99 standard that indicates such a call is not necessarily undefined so compilers cannot optimise it away: EXAMPLE 3 The function parameter declarations void h(int n, int * restrict p, int * restrict q, int * restrict r) { int i; for (i = 0; i < n; i++) p[i] = q[i] + r[i]; } illustrate how an unmodified object can be aliased through two restricted pointers. In particular, if a and b are disjoint arrays, a call of the form h(100, a, b, b) has defined behavior, because array b is not modified within function h. From owner-freebsd-hackers@freebsd.org Sun Feb 25 22:21:37 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87284F2E461 for ; Sun, 25 Feb 2018 22:21:37 +0000 (UTC) (envelope-from leeb@ratnaling.org) Received: from mail-io0-x22a.google.com (mail-io0-x22a.google.com [IPv6:2607:f8b0:4001:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F66873F40 for ; Sun, 25 Feb 2018 22:21:36 +0000 (UTC) (envelope-from leeb@ratnaling.org) Received: by mail-io0-x22a.google.com with SMTP id d71so6091386iog.4 for ; Sun, 25 Feb 2018 14:21:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ratnaling-org.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=S3DBGBmSQ2h0GXZYyG/qovObhd8go2ij+jw8SSLHKAE=; b=zTQFCou6UfmDXhkhZk+Smuvzv+r9xL+CboVJCItKAElWGUiX3lFi8lh+MGJ512cH/H EQaWEu05L+bGtWlcko5WwL+QNUO+CVTVm0VJkGXX6CHKVkPyTk1ppU7PfJyWAhDmw6Sl M4Mh/T7J8doI5EmbhXHax/5dv2uq192ayhIPaAtFRdXtRNJvTaOQNXd/vdNiEsAX6AaS oWq7ck5Dq53NCX3a9KfxE32CSSvHCLye8b0twO/x+6uCrWiRvLO+/kL/yCo4ysgoq9gw g9VsbxSU5NcMqmun4+3NZp+0lU9zgkdL+UdfGYDGcxVamTvM1LK2TtuYDZtytQxdPE8m 1KsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=S3DBGBmSQ2h0GXZYyG/qovObhd8go2ij+jw8SSLHKAE=; b=KwDm+L9YopNwxu5KtTxT+NgX5dTey6yABaFMw1ctQJeXRutFK/qCTb/+VO/8qisSer 6MRr1Ttq0uWN6X3I6PJ7CxJuMGQaFqWKaBOZghVE1iDoRDM+i8PqGLhItLF3pN7Ny4Jo iDKBKmfIa/fhyiOSxofZCSQ8vi2Eggxw4I9lWkdKGnFmvIJxXQdLFUoDgMIRB/RAlxWJ x5w4p49EKAVTr60vlUgXyB/cLvR9A/8kAEw9hX0oGCCCRLAnHfElVswLqM1xJYQj7jno MDBillu8CNVwlLROi/UFZuLIUjX+bcTnQRsMyKywqr1BEMXG08EpG2swKurX0AJbA0Ee Ch5w== X-Gm-Message-State: APf1xPAuOa2pKvLNvOKSPXl1XuZahOs40ncWz3b7z3Q1JR1yhx5Z83I7 y8pjbDj2qN+GhZoRVdrrwjdX54zQS8jaCtucRuQO6Q== X-Google-Smtp-Source: AG47ELut00VhNKal898I2SMOps9uKTzsch+C+mVP9HiIeJdJ47U6V9lw5S0GUbmthieC1lMeXhDzNLCMc2pexL+byt8= X-Received: by 10.107.10.155 with SMTP id 27mr10211212iok.259.1519597296049; Sun, 25 Feb 2018 14:21:36 -0800 (PST) MIME-Version: 1.0 Received: by 10.79.225.66 with HTTP; Sun, 25 Feb 2018 14:21:35 -0800 (PST) In-Reply-To: References: From: Lee Brown Date: Sun, 25 Feb 2018 14:21:35 -0800 Message-ID: Subject: Re: How to set various locales with setlocale()? To: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 22:21:37 -0000 On Sun, Feb 25, 2018 at 6:53 AM, Lee D wrote: > On Fri, Feb 16, 2018 at 2:29 PM, Lee Brown wrote: > > On Fri, Feb 16, 2018 at 3:08 AM, Lee D wrote: > > > >> Do you folks have any suggestions on how to accomplish this goal, or > >> how to make setlocale accept the locales listed on the system? > >> > >> > > Different architecture, it works as advertised. Maybe ask the arm folks? > > > > Thanks for the example. > > The problem ended up being my own fault. Months ago I had disabled > installation of locales in the src.conf file in attempt to make my > installation image smaller. > > Putting the locales back into the installation made it work. > Don't ya hate it when you shoot yourself in the foot. I'm surprised I can stand at this point. From owner-freebsd-hackers@freebsd.org Sun Feb 25 23:32:08 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46A1FF32C0B for ; Sun, 25 Feb 2018 23:32:08 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic301-32.consmr.mail.ne1.yahoo.com (sonic301-32.consmr.mail.ne1.yahoo.com [66.163.184.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C9E7876BCF for ; Sun, 25 Feb 2018 23:32:07 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: z5KulQYVM1lgjp2MoWXlPy8ZJGIVip8A6.tbhkTTNOVBQ9h0e8ZzxdPAupJb0_x f6Kv1iETNsHJNOb.AcTPMWNcdT34qCpcUrXGi5gDEQM34dzCeF_DoEoj4zv_ZVNWazPNANcCvOot YAvHr80ubUhkqwA2tSW2ANLQLX9ietS6hYQHRjUFBKn_MAhAn8GDJl632fqahHVyaVjkh3FH2ufH YkCP4lSzTzzjwww7kk5qSP_hW_sUNqUJWbHb.uoHVFIIKBlzFDroGCzOUxbuZigJVDHTVH03EtnI pcdZB9GbZKK84IS.P.5e.tH7vDsHOJUzDoZKk7gyuNf6Wzx6p7zhiy1D6Q.AlalqxHDtqVD_.ORW i7omnaLqtCZFvDKzZMKxT6BwmnjNPDEgtOlIn4uSDpAIjgAeUR3lBmlu0_L3KP4qUkohzAVxUW44 bb5ggR6Xv9AgNcmoswSrAG6.5Cry7ie6Cq.JbFdjx4zss0rfYMu6zBh4vLE4gLMzHMS8v Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.ne1.yahoo.com with HTTP; Sun, 25 Feb 2018 23:32:01 +0000 Received: from smtp226.mail.ne1.yahoo.com (EHLO [192.168.1.25]) ([10.218.253.215]) by smtp410.mail.ne1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID ca9ce8c4b86d225e30f2b4ca91ae9c8d; Sun, 25 Feb 2018 23:31:57 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180225214813.776a9f58@kalimero.tijl.coosemans.org> Date: Sun, 25 Feb 2018 15:31:56 -0800 Cc: Konstantin Belousov , Eitan Adler , Kevin Lo , FreeBSD Standards , FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> To: Tijl Coosemans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Feb 2018 23:32:08 -0000 On 2018-Feb-25, at 12:48 PM, Tijl Coosemans wrote: > On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >> Consider the recently changed devd code: >> select(n + 1, &fd, &fd, &fd); >> There, compiler can see that restrict is applied to arguments which = are >> given same values. Since this leads to the self-contradicting = statement >> fd !=3D fd >> which cannot be true, compliler in its optimizing wisdom can assume = that >> the code is never executing and remove it. I do not know whether = clang >> actually makes such transformation, but it does not sound unfeasible >> looking at its other advances. >=20 > There's an example in the C99 standard that indicates such a call is = not > necessarily undefined so compilers cannot optimise it away: >=20 > EXAMPLE 3 > The function parameter declarations > void h(int n, int * restrict p, int * restrict q, int * restrict r) > { > int i; > for (i =3D 0; i < n; i++) > p[i] =3D q[i] + r[i]; > } > illustrate how an unmodified object can be aliased through two = restricted > pointers. In particular, if a and b are disjoint arrays, a call of = the > form h(100, a, b, b) has defined behavior, because array b is not = modified > within function h. Good point. In essence the restrictions on the caller can not be known independently of how the parameters are used in the called code --something that prototype does not specify. This does constrain what the compiler can do about potential aliasing that it might detect. A prototype that would make h's restrictions clearer is one that reports that q and r are not used to modify memory: void h(int n, int * restrict p, int const * restrict q, int const * = restrict r); With such a prototype, it is easier to known that q's "objects" and r's "objects" both simply must not overlap p's "objects". (See g from example 2 for its d+50 valid vs. d+1 invalid status.) Section 7.1 of the C Rationale says: "The restrict keyword allows the prototype to express what was previously expressed by words." But it turns out that not all the words can be eliminated if one is to know the actual criteria for what the caller is allowed to do while staying within defined behavior. There is a lot of wording for which the "example 3" h(100, a, b, b) example usage being well defined is not obvious. Such wording is without any specific such context to reference and tends to assume that all restricted pointers are used to modify the matching objects (when const does not prevent such). This need not be the case. =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-hackers@freebsd.org Mon Feb 26 04:40:30 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34798F296A0 for ; Mon, 26 Feb 2018 04:40:30 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) Received: from sonic306-22.consmr.mail.ne1.yahoo.com (sonic306-22.consmr.mail.ne1.yahoo.com [66.163.189.84]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0A6681FA3 for ; Mon, 26 Feb 2018 04:40:29 +0000 (UTC) (envelope-from marklmi26-fbsd@yahoo.com) X-YMail-OSG: 4LNx12kVM1mHP9pCKQ8TyhCiJXtYHjpWwm4BdDaFW3lqULKF8SBu.xSDcJSD0P9 uGYHUJV.OuMjKaXl4.cE3ArQGLB0KDlSf924AZwRLWW_81rF6YUPpkQJ0JdcjZzW0K2iEAlIdL8i eUHzr8jtgaYURUBamCRRrDmwX98URrBYgcKIBNa3Bf_xKBBKTBASHZ57fb1gNDekAZRfMrLqTGky Tyby23MlI2ZumuFjv9XyNdzSCKzmH4T3YzD4ObyoR3QnDL86mmdDCQaVzOR48eT6WxVWs2.Oyx5d 5_w3SUyDMHMkyGKxcNuOT8Ip7xnARI53drsjGpNqRhiMCw4Vf9mFuE1gUPDl.XHEI1Rze6iHpWET kkIY9ghssbT2DkxReTqKvzDh9lZ4HS8JFbe4qJF6lPcmVzTtUI8haL0zEnk0SUb82hzYnYPlZmV7 05GgA6WXgZjtIBmFfiVgiFDL2jh8VoyxEIh3Xr7._ftDJydx4FusHnAl5CorgYnTWj2iUf4nX Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.ne1.yahoo.com with HTTP; Mon, 26 Feb 2018 04:40:23 +0000 Received: from smtp230.mail.ne1.yahoo.com (EHLO [192.168.1.25]) ([10.218.253.211]) by smtp404.mail.ne1.yahoo.com (JAMES SMTP Server ) with ESMTPA ID c920c595c0bc7ede23f066c3b2f36b8d; Mon, 26 Feb 2018 04:40:18 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: Marking select(2) as restrict From: Mark Millard In-Reply-To: <20180226135457.B1203@besplex.bde.org> Date: Sun, 25 Feb 2018 20:40:16 -0800 Cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Content-Transfer-Encoding: quoted-printable Message-Id: <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 04:40:30 -0000 On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: > On Sun, 25 Feb 2018, Mark Millard via freebsd-standards wrote: >=20 >> On 2018-Feb-25, at 12:48 PM, Tijl Coosemans = wrote: >>=20 >>> On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >>>> Consider the recently changed devd code: >>>> select(n + 1, &fd, &fd, &fd); >>>> There, compiler can see that restrict is applied to arguments which = are >>>> given same values. Since this leads to the self-contradicting = statement >>>> fd !=3D fd >>>> which cannot be true, compliler in its optimizing wisdom can assume = that >>>> the code is never executing and remove it. I do not know whether = clang >>>> actually makes such transformation, but it does not sound = unfeasible >>>> looking at its other advances. >>>=20 >>> There's an example in the C99 standard that indicates such a call is = not >>> necessarily undefined so compilers cannot optimise it away: >>>=20 >>> EXAMPLE 3 >>> The function parameter declarations >>> void h(int n, int * restrict p, int * restrict q, int * restrict r) >>> { >>> int i; >>> for (i =3D 0; i < n; i++) >>> p[i] =3D q[i] + r[i]; >>> } >>> illustrate how an unmodified object can be aliased through two = restricted >>> pointers. In particular, if a and b are disjoint arrays, a call of = the >>> form h(100, a, b, b) has defined behavior, because array b is not = modified >>> within function h. >>=20 >> Good point. In essence the restrictions on the caller can >> not be known independently of how the parameters are used >> in the called code --something that prototype does not specify. >> This does constrain what the compiler can do about potential >> aliasing that it might detect. >>=20 >> A prototype that would make h's restrictions clearer is >> one that reports that q and r are not used to modify >> memory: >>=20 >> void h(int n, int * restrict p, int const * restrict q, int const * = restrict r); >>=20 >> With such a prototype, it is easier to known that q's "objects" >> and r's "objects" both simply must not overlap p's "objects". >> (See g from example 2 for its d+50 valid vs. d+1 invalid status.) >=20 > I think the example intentionally leaves out 'const'. I think const > already prevents aliasing. 'restrict' prevents it even more. Using > the combination in the exaple would make it less clear what the > 'restrict' part does. Agreed: the combination would make the example less clear about restrict. So, agreed: likely deliberate for specifying just restrict's semantics. I would not suggest my alternate h prototype for that example in the standard: different purposes served. > . . . restrict is not needed for the input-only > args since const suffices.=20 Here you lost me. With q and r having both the const and the restrict, updates to p's "objects" can not change the "object(s)" q and r validly can be used to access. That can be important. Without the "restrict" for q and r (but still having the const for each) it is valid for updates to p's "objects" to change what q and r then can validly access. The 2 restrict's in question do not seem redundant to me as far as the prototype goes. They may enable optimizations in some architectures: more order of operation independence so more reordering possible. A more general context could still require words describing requirements, words beyond the restrict and const use. I would not claim that adding const where it fits would make all contexts well described. This one just happened to be nicer for the person reading the prototype. (But it would makes for a worse specification of restrict of itself to have the const's in place in example 3.) =3D=3D=3D Mark Millard marklmi at yahoo.com ( markmi at dsl-only.net is going away in 2018-Feb, late) From owner-freebsd-hackers@freebsd.org Mon Feb 26 12:46:42 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1D3FF21333; Mon, 26 Feb 2018 12:46:41 +0000 (UTC) (envelope-from tijl@freebsd.org) Received: from mailrelay118.isp.belgacom.be (mailrelay118.isp.belgacom.be [195.238.20.145]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "relay.skynet.be", Issuer "GlobalSign Organization Validation CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 15F877380F; Mon, 26 Feb 2018 12:46:40 +0000 (UTC) (envelope-from tijl@freebsd.org) X-Belgacom-Dynamic: yes IronPort-PHdr: =?us-ascii?q?9a23=3ArAR86hFIKLnGdYmUQPj+jp1GYnF86YWxBRYc798d?= =?us-ascii?q?s5kLTJ7yp8WwAkXT6L1XgUPTWs2DsrQY07GQ6/iocFdDyK7JiGoFfp1IWk1Nou?= =?us-ascii?q?QttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBXdrXKo8DEdBAj0OxZr?= =?us-ascii?q?KeTpAI7SiNm82/yv95HJbAhEmDSwbaluIBmqsA7cqtQYjYx+J6gr1xDHuGFIe+?= =?us-ascii?q?NYxWNpIVKcgRPx7dqu8ZBg7ipdpesv+9ZPXqvmcas4S6dYDCk9PGAu+MLrrxjD?= =?us-ascii?q?QhCR6XYaT24bjwBHAwnB7BH9Q5fxri73vfdz1SWGIcH7S60/VC+85Kl3VhDnlC?= =?us-ascii?q?YHNyY48G7JjMxwkLlbqw+lqxBm3oLYfJ2ZOP94c6jAf90VWHBBU95fWSJBHI2y?= =?us-ascii?q?cogBAOgPPelXs4bzqEADrQe8CAWwGO/j1iNEimHw0KYn0+ohCwbG3Ak4EtwQsX?= =?us-ascii?q?TUrtH1P7oMXOCyy6nI1ivMb/ZM1jf784jDbxcsoe2NXbJydcrc0kkhFxnbgVqO?= =?us-ascii?q?tIHrIj2b2v4Ks2iB4OptTOSigHMkpQFpujWixdoghpPXio8ay13I7zh1zYg7KN?= =?us-ascii?q?GiVUJ2b9GpHZ1NvC+ALYR2WNktQ2RwtSY/zb0JpIC0cTARyJQi2x7fc/uHc5WU?= =?us-ascii?q?4h77VOaePzN4hHV9dbK8nRmy9UmgyujiWcmu11ZGtDZFktjOtnAJzRDc9s+HSv?= =?us-ascii?q?xm/ki/3DaAzQbT6vpeLUAzj6rbJIYtwr82lpUNrUTOBiz7lFjsgKOIeUgp+/Kk?= =?us-ascii?q?5/npb7jovJOQKoF5hw7mPqQrgMO/AOA4MgYUX2ic/OSxzKHj/Uz7QLVOlfA2nL?= =?us-ascii?q?PZv47EKssAva62HhVZ0oE56xawFzumysgXnWEbLFJZfxKKl4vpNE/QIPD8Cvey?= =?us-ascii?q?mFqskC11yP/YJbLhGYjCImLEkLf7crZ381RcxxYrzdBD+5JUDakMIPzpWkDvqt?= =?us-ascii?q?PXFQQ5PBGtz+b8FNVyzIUeVn+VDa+DLazSqkSF5uw1I+aSeoAaoy39JOU/6/7p?= =?us-ascii?q?l385lkcXfbO10psPdHC4AvNmLl2XYXr2nNgOD3wFvhEjQ+DziF2NSyJcZ3WsUK?= =?us-ascii?q?Im/TE2E4ymDZ3dSY+zm7OBxzq0EodRZmBcBVDfWUvvIq+eRvwBIA+MK8l62mgO?= =?us-ascii?q?T7SsY4g5yQy1sgLmjbFgK6zd53tLm4jk0Y1J5u/X3To18id5Cs2byCnZU2B2mk?= =?us-ascii?q?smXTI79ptT50tnxQHQguBDn/VEGIkLtLtyWQAgOMuZlrQiBg=3D=3D?= X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: =?us-ascii?q?A2AmBgCOAJRa/4aF9lFcGgEBAQEBAgEBA?= =?us-ascii?q?QEIAQEBAYNPVoEAKI5qjQcBAYIBMgFjlgWCFoU9AoI/VhcBAgEBAQEBAQIBaih?= =?us-ascii?q?CDAGBaSQBgkcBBTocIxALDgoJJQ8qHgYThRStNohkghQBAQEBAQEEAQEBASSOL?= =?us-ascii?q?IsTBaFhCZQTgQSRaZckHwE2gVFNMAiCfYRbPzeIYoN1AQEB?= X-IPAS-Result: =?us-ascii?q?A2AmBgCOAJRa/4aF9lFcGgEBAQEBAgEBAQEIAQEBAYNPVoE?= =?us-ascii?q?AKI5qjQcBAYIBMgFjlgWCFoU9AoI/VhcBAgEBAQEBAQIBaihCDAGBaSQBgkcBB?= =?us-ascii?q?TocIxALDgoJJQ8qHgYThRStNohkghQBAQEBAQEEAQEBASSOLIsTBaFhCZQTgQS?= =?us-ascii?q?RaZckHwE2gVFNMAiCfYRbPzeIYoN1AQEB?= Received: from 134.133-246-81.adsl-dyn.isp.belgacom.be (HELO kalimero.tijl.coosemans.org) ([81.246.133.134]) by relay.skynet.be with ESMTP; 26 Feb 2018 13:45:29 +0100 Received: from kalimero.tijl.coosemans.org (kalimero.tijl.coosemans.org [127.0.0.1]) by kalimero.tijl.coosemans.org (8.15.2/8.15.2) with ESMTP id w1QCjR7J051426; Mon, 26 Feb 2018 13:45:29 +0100 (CET) (envelope-from tijl@FreeBSD.org) Date: Mon, 26 Feb 2018 13:45:26 +0100 From: Tijl Coosemans To: Bruce Evans Cc: Mark Millard , FreeBSD Hackers , FreeBSD Standards , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180226134526.425aa33c@kalimero.tijl.coosemans.org> In-Reply-To: <20180226200951.V2634@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> <20180226200951.V2634@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 12:46:42 -0000 On Mon, 26 Feb 2018 21:00:21 +1100 (EST) Bruce Evans wrote: > For select() with restrict, I think the compiler cannot assert that > the args don't overlap since (even without the detailed specification > and Example 3), the compiler cannot know if select() modifies its args. > For all that the compiler knows, select() might be a stub that never > modifies or even reads anything. I think doing no accesses satifies > the constraints of restrict. It might be valid for the compiler to > assert that the (values pointed to by) the fdset args don't change for > select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access > through an fdset type would give undefined behaviour on the other fdset > args; tv can still change since it has a different type). But this is > precisely what is wanted for the original example where we only care > about the return value -- then fdset is indeterminate after the call > so we shouldn't use it; the compiler is unlikely to optimize the non-use > of it and the worst that it can do is add a runtime assertion that the > arg didn't change. I don't think the compiler can assert that fdset is unmodified. It's valid for select to modify fdset through one pointer if it doesn't dereference the other two. From owner-freebsd-hackers@freebsd.org Mon Feb 26 03:45:52 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9C3DBF25917; Mon, 26 Feb 2018 03:45:52 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail109.syd.optusnet.com.au (mail109.syd.optusnet.com.au [211.29.132.80]) by mx1.freebsd.org (Postfix) with ESMTP id E45EE7FD3A; Mon, 26 Feb 2018 03:45:51 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail109.syd.optusnet.com.au (Postfix) with ESMTPS id 549A8D69011; Mon, 26 Feb 2018 14:45:43 +1100 (AEDT) Date: Mon, 26 Feb 2018 14:45:42 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict In-Reply-To: <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> Message-ID: <20180226135457.B1203@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=pGLkceISAAAA:8 a=xwlSsFIUvvU5qVp6TTwA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 X-Mailman-Approved-At: Mon, 26 Feb 2018 15:11:23 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 03:45:52 -0000 On Sun, 25 Feb 2018, Mark Millard via freebsd-standards wrote: > On 2018-Feb-25, at 12:48 PM, Tijl Coosemans wrote: > >> On Thu, 22 Feb 2018 12:56:08 +0200 Konstantin Belousov wrote: >>> Consider the recently changed devd code: >>> select(n + 1, &fd, &fd, &fd); >>> There, compiler can see that restrict is applied to arguments which are >>> given same values. Since this leads to the self-contradicting statement >>> fd != fd >>> which cannot be true, compliler in its optimizing wisdom can assume that >>> the code is never executing and remove it. I do not know whether clang >>> actually makes such transformation, but it does not sound unfeasible >>> looking at its other advances. >> >> There's an example in the C99 standard that indicates such a call is not >> necessarily undefined so compilers cannot optimise it away: >> >> EXAMPLE 3 >> The function parameter declarations >> void h(int n, int * restrict p, int * restrict q, int * restrict r) >> { >> int i; >> for (i = 0; i < n; i++) >> p[i] = q[i] + r[i]; >> } >> illustrate how an unmodified object can be aliased through two restricted >> pointers. In particular, if a and b are disjoint arrays, a call of the >> form h(100, a, b, b) has defined behavior, because array b is not modified >> within function h. > > Good point. In essence the restrictions on the caller can > not be known independently of how the parameters are used > in the called code --something that prototype does not specify. > This does constrain what the compiler can do about potential > aliasing that it might detect. > > A prototype that would make h's restrictions clearer is > one that reports that q and r are not used to modify > memory: > > void h(int n, int * restrict p, int const * restrict q, int const * restrict r); > > With such a prototype, it is easier to known that q's "objects" > and r's "objects" both simply must not overlap p's "objects". > (See g from example 2 for its d+50 valid vs. d+1 invalid status.) I think the example intentionally leaves out 'const'. I think const already prevents aliasing. 'restrict' prevents it even more. Using the combination in the exaple would make it less clear what the 'restrict' part does. In draft C99 (n869.txt), Example 3 is much more complicated and seems to be broken. It is intentionally of how const can be used in conjunction with restrict. p has qualifiers const and restrict, and q and r only have qualifier const, and it is claimed that p being restrict- qualified implies that an object accessed through p is never accessed through q or r, and that this is the precise assertion required to optimize the loop. const for p seems to be just a bug. Removing it gives an example of how restrict is not needed for the input-only args since const suffices. However, C99 TC3 (n1256.pdf) changes this signficantly by removing all the consts and adding restricts. This might be related to POSIX's inconsistencies for sigaction() and nanosleep(). For sigaction(), the input arg is const restrict and the output arg is restrict, but for nanosleep() the the input arg is just const and the output arg is unqualified. const on 1 arg seems to be enough to prevent all aliasing when there are only 2 args that could be aliased without it, but Example 3 no longer gives any hints about how restrict interacts with const. restrict gives stronger (but different) restrictions than const on aliases with globals, but so should all POSIX syscall-like functions. An application function miight access any application global state, but library functions should only access documented global state. For syscall-like functions, the global state outside of the kernel should be precisely errno. POSIX should restrict aliasing to error in a general way, but for sigaction() and nanosleep() it is clear that the pointer args cannot be aliased to errno, since they don't point to ints. Bruce From owner-freebsd-hackers@freebsd.org Mon Feb 26 16:27:40 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1635EF31185 for ; Mon, 26 Feb 2018 16:27:40 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: from mail-ot0-x22f.google.com (mail-ot0-x22f.google.com [IPv6:2607:f8b0:4003:c0f::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 950D27CCF8; Mon, 26 Feb 2018 16:27:39 +0000 (UTC) (envelope-from embaudarm@gmail.com) Received: by mail-ot0-x22f.google.com with SMTP id m22so456890otf.10; Mon, 26 Feb 2018 08:27:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=fpW0/oEAQAnx/HWMwBtZURUgAovfivmdvDKvszry5lM=; b=aziSCyfV1q6Ml/dSIhhvK16Y5xQ4FF/4LD0nOLmRclKSCzHygy1511xpIWjOsZ37kS r5Pp891EgvpZgf8qXWFZVFpuSMr7qwd3k2xQZM4KLYrdx6WeujpXWz6f/SGeX7u7HL4s gqeVVJtbk9sXiiKjLACUlSeXyJ/S8DztRxQuetCA3i76j9DSJE51gFMpnwGL6WcLae0T 8ZoMjJUXaBCDJk4UK4nlcvEh2UYM5Xe5/AOWBHm2Czxrb1rT787TcqpgW6+U5wA1pTTu YVifarRoZl3uYjbxp0CeGIlPCprwvKn+zCm3jidkqRIPQsF6cLIZl6v5+l5TN9WtvYCp k5jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=fpW0/oEAQAnx/HWMwBtZURUgAovfivmdvDKvszry5lM=; b=eZMuRdWu+NbKVActtIpyiF3u2ekOgBwqXoIw0FuuL4DrMUhu1v7DtfMbiqxgn54h+r chRwGzuM2f/piXEIndBahQvjE43cnyaz/mzW8G6Wrr4AAsKdGFiu5VMGTx23MRZIykHo EpDtFoqe7O7XBYDGFVdS07nzsWiTUYtTpznBKsqF3EnDHs8E1uPPm9oLgLBsijUM7VbR YSOrBwmDi5JK8t7sIr0G+GwHc1dyElvvUbLsrZgYreNjhKAEmii5UkOJOhwpRR8JAzoK G6HO5krfATmbvqpI0ZA+mlc0wIRR30Aah10QFDXwDL8gNoBFRYKA8ArFA+h/tW5stJ5Y 6ktg== X-Gm-Message-State: APf1xPA9xIk5D3cosUGnqJSJ9ZqkTjzuh/Hkh96RN+Ke1q25sp5K39+T KIAQLeWGUphbJKM4qjdOGheK/Xmp/heXiuiAMtM= X-Google-Smtp-Source: AH8x227UqhTl73n0yiv21Y4rs5CsLqDvzui0Ll+DCx46+C15sCkr+4Xhz9kS5rUKVRr89fwJ5CeMYvI6KKdnxZyD52Q= X-Received: by 10.157.53.10 with SMTP id o10mr7674935otc.283.1519662457682; Mon, 26 Feb 2018 08:27:37 -0800 (PST) MIME-Version: 1.0 Received: by 10.157.45.194 with HTTP; Mon, 26 Feb 2018 08:26:57 -0800 (PST) In-Reply-To: <1519579387.91697.252.camel@freebsd.org> References: <1519579387.91697.252.camel@freebsd.org> From: Lee D Date: Mon, 26 Feb 2018 11:26:57 -0500 Message-ID: Subject: Re: Help, please, with getting a custom I2C real time clock module to load To: Ian Lepore Cc: FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 16:27:40 -0000 On Sun, Feb 25, 2018 at 12:23 PM, Ian Lepore wrote: > On Sun, 2018-02-25 at 09:48 -0500, Lee D wrote: >> Hi Everyone, >> >> I have written a new I2C driver (for the Xilinx Zynq) and a new real >> time clock chip driver (for the ST M41T82) to use with hardware on my >> custom board. This is for 11.0.1. > > Right here is where the disconnect is happening. It's the ofw_iicbus > driver that needs to declared in the first DRIVER_MODULE() instead of > iicbus, because ofw_iicbus is the one that knows to look in the fdt > data for slave devices and add them as children of the bus. But, the > extern declarations needed to do that didn't exist until I added them > last week in r329526. > > What i2c drivers have been doing in the past, and the way to work > around it in the 11.x code you're dealing with, is to leverage one of > the DRIVER_MODULE() declarations that already exists in ofw_iicbus, by > naming your driver "iichb" instead of "i2c". Like this: > > static driver_t i2c_driver = { > "iichb", > i2c_methods, > sizeof(struct i2c_softc), > }; > > Or you could import r329526 into the kernel source you're using and > rebuild the kernel. I do intend to MFC that change to 11-stable (in > fact, I should probably do that today). > > When you've got these new drivers working, please consider putting them > up for review at https://reviews.freebsd.org and we'll get them > committed to freebsd. > > -- Ian > Thanks so much for the advice, I was able to get the RTC driver to load using the iichb trick. I would be happy to contribute the code assuming I can get it to work. I will probably move to 11.1 in the near future. Lee From owner-freebsd-hackers@freebsd.org Tue Feb 27 06:31:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B886BF28D7B for ; Tue, 27 Feb 2018 06:31:02 +0000 (UTC) (envelope-from siddharth.muralee@gmail.com) Received: from mail-it0-x236.google.com (mail-it0-x236.google.com [IPv6:2607:f8b0:4001:c0b::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D14484CD3 for ; Tue, 27 Feb 2018 06:31:02 +0000 (UTC) (envelope-from siddharth.muralee@gmail.com) Received: by mail-it0-x236.google.com with SMTP id u5so14025980itc.1 for ; Mon, 26 Feb 2018 22:31:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Gfwf0ioDjgbDErE/BYG0pSrwJy85ocWKgdd0SWgQ1To=; b=SMZD/U+E6N2oaSCGIF0hOIvxCm1bdLzV6FQSNsP6QgdO1YjOmRa32DIExvcWxrtSRk Otvr+X5E1ZQq7NcLOeVkLwUhXqIrcuJDpKtRT0giwtehSqXCFX/jbJLmpNrW2bx988XA DVpM8yNT1gECh2h9/do0jYsXR2RBjO68THsNL2qS8+bhHrxbkK0v9448jy51D13ec1SF +KsnfspzzEcC9Zk3rtEZA9f1a9JrZCuSPX+L/A+VjD5MFEMsIR/HB02ALc8Qqm0cWgOX NyOMuununpnB8TRLpO5B8LdTRHzEjYQiZHa5j9xkDaTW/MhcfsuzV2kU1p0hTug5MLLk 42Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Gfwf0ioDjgbDErE/BYG0pSrwJy85ocWKgdd0SWgQ1To=; b=XnpTbgK/PgDkGUf3lh8fiKR5lw+OMpTV70Xnw+NewEUwZ2m77430kX1xZi8IaBwuUC qTMKnovNhw0l1wEgf+ycVd2grUcZZxWZod+wzQ299psS6hMcdU3a3Gz825hj6En2Tf94 6uOBgsDZeh343htH14qMGgHfFecEsqepvrGejwnXCybzHlI3ci9LGuLE4Di36V3hJvdz EdC+euZ/HyZ3SDPTQgXCkwwrS9RJ97BAdca40qW+obWHPNFmCEtTk3o3eLHNxJTzWqTe 1KcdqR6pAgrDr0V0lfi5J6mGJFGqynKkQqliEwDsLQJ6oJJlqEwWZtlf5ExHhBjTvbEJ duWQ== X-Gm-Message-State: APf1xPA1M/9kwrpVaKjSEgZNG6FJ8UZgkUhr9V2IBt7x+xU4RL1qjr+m mcJPvE/0HQVeaZvcgoi9rq4yWXaMQG/Q/MglBYo= X-Google-Smtp-Source: AH8x225uRNIfUWz9x8topWvLF1JXLuTYPIH0yHrl/LG98iK4+wGP5HARu0J8Q9HMa+/5n2QWHA8Navypr0CRKLKmXWQ= X-Received: by 10.36.50.85 with SMTP id j82mr16456871ita.82.1519713061393; Mon, 26 Feb 2018 22:31:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.2.127.23 with HTTP; Mon, 26 Feb 2018 22:30:40 -0800 (PST) From: Siddharth Muralee Date: Tue, 27 Feb 2018 12:00:40 +0530 Message-ID: Subject: [GSoC] Kernel Fuzzing suite To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 06:31:03 -0000 Hi there, I am an undergraduate student at Amrita University and I would like to work on the Kernel Fuzzing Suite project that has been suggested in the projects page. I am a security enthusiast and I am quite familiar with both user-land and kernel-land exploitation. I am also familiar with using automated tools like PIN, AFL, Angr for automated binary analysis and vulnerability detection. I have been taking a look at the current kernel fuzzers already present for FreeBSD like Syzkaller, and TriforceAFL for OpenBSD. I have also been comparing these with other Kernel fuzzers like Trinity and the relatively new DiFuzz. I would like to know how to start working on this project. Since no mentors have been assigned to this project I also don't know who to contact regarding queries. --=20 Regards, Siddharth M Second Year B.Tech (CSE) Student, Amrita School of Engineering, Kollam * Blog * *---------------------------------------* *=E2=80=9CMost people get ahead during the time that others waste.**"* From owner-freebsd-hackers@freebsd.org Mon Feb 26 10:00:27 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE805F39F68; Mon, 26 Feb 2018 10:00:26 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 108AF6D39C; Mon, 26 Feb 2018 10:00:25 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id DD374107332; Mon, 26 Feb 2018 21:00:22 +1100 (AEDT) Date: Mon, 26 Feb 2018 21:00:21 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Mark Millard cc: Tijl Coosemans , FreeBSD Standards , FreeBSD Hackers , Kevin Lo Subject: Re: Marking select(2) as restrict In-Reply-To: <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> Message-ID: <20180226200951.V2634@besplex.bde.org> References: <20180221032247.GA81670@ns.kevlo.org> <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=FNpr/6gs c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=kj9zAlcOel0A:10 a=PO7r1zJSAAAA:8 a=irPJxnBdqyEU6-vdNS0A:9 a=CjuIK1q_8ugA:10 X-Mailman-Approved-At: Tue, 27 Feb 2018 18:10:21 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Feb 2018 10:00:27 -0000 On Sun, 25 Feb 2018, Mark Millard wrote: > On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: >> . . . restrict is not needed for the input-only >> args since const suffices. > > Here you lost me. > > With q and r having both the const and the restrict, > updates to p's "objects" can not change the > "object(s)" q and r validly can be used to access. > That can be important. > > Without the "restrict" for q and r (but still > having the const for each) it is valid for updates > to p's "objects" to change what q and r then can > validly access. Yes, I forgot that all const on a pointer arg does is prevent modification through that pointer. Modifications can still occur through other args. memmove() is a canonical example. Its source arg is const, by the source is _always_ modified in the overlapping case that is the reason for existence of memmove(). memcpy() is another canonical example. Both of its args are declared with restrict, except in its man page. Its source arg is still declared as const. Without that, either arg could be modified. With that and restrict on the other arg, I think it follows that the source arg cannot be modified. It is unclear if restrict on the source arg is needed too. It is now clear(er) that POSIX's restricts for sigaction() are correct and not having them for nanosleep() is a bug. nanosleep()'s args are const struct timespec * and struct timespec *. Nothing prevents these being aliased, just like for memmove(), and unlike for memcpy(), the behaviour is not undefined when aliasing occurs. Aliasing can only occur if the pointers are equal, since unlike for mem*() they don't point to arrays (but the prototype doesn't give this information.) So nanosleep(&ts, &ts) must work, and working involves clobbering the input arg. The implementation must be careful to not write the output before reading the input (if the pointers are not equal), and callers using the same timespec for input and output must not depend on the source being const. This is just like for the non-restrict select() except it is easier to avoid problems (e.g., by copying) with a single small object than with a potentially large array for the input arg. My argument applies better to pointer args of different types. Then const prevents modifications through some pointer args and the different types prevent aliasing of the non-const args to the const args. For select() with restrict, I think the compiler cannot assert that the args don't overlap since (even without the detailed specification and Example 3), the compiler cannot know if select() modifies its args. For all that the compiler knows, select() might be a stub that never modifies or even reads anything. I think doing no accesses satifies the constraints of restrict. It might be valid for the compiler to assert that the (values pointed to by) the fdset args don't change for select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access through an fdset type would give undefined behaviour on the other fdset args; tv can still change since it has a different type). But this is precisely what is wanted for the original example where we only care about the return value -- then fdset is indeterminate after the call so we shouldn't use it; the compiler is unlikely to optimize the non-use of it and the worst that it can do is add a runtime assertion that the arg didn't change. Bruce From owner-freebsd-hackers@freebsd.org Tue Feb 27 21:01:13 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 672DCF2879A; Tue, 27 Feb 2018 21:01:13 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from mailout.stack.nl (mailout05.stack.nl [IPv6:2001:610:1108:5010::202]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.stack.nl", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E768F6FADC; Tue, 27 Feb 2018 21:01:12 +0000 (UTC) (envelope-from jilles@stack.nl) Received: from toad2.stack.nl (toad2.stack.nl [IPv6:2001:610:1108:5010::161]) by mailout.stack.nl (Postfix) with ESMTP id CE9613B; Tue, 27 Feb 2018 22:01:10 +0100 (CET) Received: by toad2.stack.nl (Postfix, from userid 1677) id C6503892CC; Tue, 27 Feb 2018 22:01:10 +0100 (CET) Date: Tue, 27 Feb 2018 22:01:10 +0100 From: Jilles Tjoelker To: Bruce Evans Cc: Mark Millard , FreeBSD Hackers , Tijl Coosemans , FreeBSD Standards , Kevin Lo Subject: Re: Marking select(2) as restrict Message-ID: <20180227210110.GA76452@stack.nl> References: <20180221104400.GU94212@kib.kiev.ua> <20180222112752.10da7e51@kalimero.tijl.coosemans.org> <20180222105608.GE94212@kib.kiev.ua> <20180225214813.776a9f58@kalimero.tijl.coosemans.org> <2909E983-953A-4463-959C-F3C386BC6C9A@yahoo.com> <20180226135457.B1203@besplex.bde.org> <1A2830F4-A00B-4C56-8D28-C46715DC7C9E@yahoo.com> <20180226200951.V2634@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180226200951.V2634@besplex.bde.org> User-Agent: Mutt/1.8.0 (2017-02-23) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 21:01:13 -0000 On Mon, Feb 26, 2018 at 09:00:21PM +1100, Bruce Evans wrote: > On Sun, 25 Feb 2018, Mark Millard wrote: > > > On 2018-Feb-25, at 7:45 PM, Bruce Evans wrote: > >> . . . restrict is not needed for the input-only > >> args since const suffices. > > Here you lost me. > > With q and r having both the const and the restrict, > > updates to p's "objects" can not change the > > "object(s)" q and r validly can be used to access. > > That can be important. > > Without the "restrict" for q and r (but still > > having the const for each) it is valid for updates > > to p's "objects" to change what q and r then can > > validly access. > Yes, I forgot that all const on a pointer arg does is prevent modification > through that pointer. Modifications can still occur through other args. > memmove() is a canonical example. Its source arg is const, by the source > is _always_ modified in the overlapping case that is the reason for > existence of memmove(). In fact, it is even weaker. The C standard permits casting away const and using the resulting pointer for reading. Unless the underlying object is defined as const, is a string constant or the pointer is based on a restrict pointer to const, the pointer can also be used for writing. In FreeBSD code __DECONST should be used instead of a regular cast, but the same things can be done with the resulting pointer. > memcpy() is another canonical example. Both of its args are declared with > restrict, except in its man page. Its source arg is still declared as > const. Without that, either arg could be modified. With that and restrict > on the other arg, I think it follows that the source arg cannot be modified. > It is unclear if restrict on the source arg is needed too. The restrict on the destination alone prevents it aliasing the source. However, the restrict on the source prevents it aliasing globals and thread-locals. This is not needed for memcpy() which does not write any, but is often needed for more complicated functions. > It is now clear(er) that POSIX's restricts for sigaction() are correct > and not having them for nanosleep() is a bug. nanosleep()'s args are > const struct timespec * and struct timespec *. Nothing prevents these > being aliased, just like for memmove(), and unlike for memcpy(), the > behaviour is not undefined when aliasing occurs. Aliasing can only > occur if the pointers are equal, since unlike for mem*() they don't point > to arrays (but the prototype doesn't give this information.) So > nanosleep(&ts, &ts) must work, and working involves clobbering the input > arg. The implementation must be careful to not write the output before > reading the input (if the pointers are not equal), and callers using the > same timespec for input and output must not depend on the source being > const. This is just like for the non-restrict select() except it is > easier to avoid problems (e.g., by copying) with a single small object > than with a potentially large array for the input arg. Not having restrict on nanosleep()'s args is not much of a burden since its input must necessarily be read before the output can be written (unlike sigaction()). Therefore, I don't think this is a bug. A call to nanosleep() with two equal pointers also has the useful property of being restartable after signals without needing fixups. > My argument applies better to pointer args of different types. Then const > prevents modifications through some pointer args and the different types > prevent aliasing of the non-const args to the const args. Assuming there are no pointers to character type. > For select() with restrict, I think the compiler cannot assert that > the args don't overlap since (even without the detailed specification > and Example 3), the compiler cannot know if select() modifies its args. > For all that the compiler knows, select() might be a stub that never > modifies or even reads anything. I think doing no accesses satifies > the constraints of restrict. It might be valid for the compiler to > assert that the (values pointed to by) the fdset args don't change for > select(nfd, &fdset, &fdset, &fdset, &tv) (because any write access > through an fdset type would give undefined behaviour on the other fdset > args; tv can still change since it has a different type). But this is > precisely what is wanted for the original example where we only care > about the return value -- then fdset is indeterminate after the call > so we shouldn't use it; the compiler is unlikely to optimize the non-use > of it and the worst that it can do is add a runtime assertion that the > arg didn't change. Agreed, with the addition that the arg may even change per Tijl Cooseman's reply. -- Jilles Tjoelker From owner-freebsd-hackers@freebsd.org Tue Feb 27 22:30:55 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FBFFF2FA86 for ; Tue, 27 Feb 2018 22:30:55 +0000 (UTC) (envelope-from torek@elf.torek.net) Received: from elf.torek.net (mail.torek.net [96.90.199.121]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "elf.torek.net", Issuer "elf.torek.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D083674520 for ; Tue, 27 Feb 2018 22:30:54 +0000 (UTC) (envelope-from torek@elf.torek.net) Received: from elf.torek.net (localhost [127.0.0.1]) by elf.torek.net (8.15.2/8.15.2) with ESMTPS id w1RMUO2Q079463 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 27 Feb 2018 14:30:24 -0800 (PST) (envelope-from torek@elf.torek.net) Received: (from torek@localhost) by elf.torek.net (8.15.2/8.15.2/Submit) id w1RMUOmL079462 for freebsd-hackers@freebsd.org; Tue, 27 Feb 2018 14:30:24 -0800 (PST) (envelope-from torek) Date: Tue, 27 Feb 2018 14:30:24 -0800 (PST) From: Chris Torek Message-Id: <201802272230.w1RMUOmL079462@elf.torek.net> To: freebsd-hackers@freebsd.org Subject: Re: Marking select(2) as restrict In-Reply-To: <20180227210110.GA76452@stack.nl> X-Greylist: inspected by milter-greylist-4.6.2 (elf.torek.net [127.0.0.1]); Tue, 27 Feb 2018 14:30:24 -0800 (PST) for IP:'127.0.0.1' DOMAIN:'localhost' HELO:'elf.torek.net' FROM:'torek@elf.torek.net' RCPT:'' X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (elf.torek.net [127.0.0.1]); Tue, 27 Feb 2018 14:30:24 -0800 (PST) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 22:30:55 -0000 If we can back up a bit, there are (at least) two mostly-separate issues: * What types and qualfiiers to use for various parameters: These are dictated by standards, and we should just conform to whatever is in the standard unless there's some major reason to do otherwise. POSIX says that we *should* use "restrict" here: http://pubs.opengroup.org/onlinepubs/009696699/functions/pselect.html so if no one has a major reason to do otherwise (such as "no conforming program can tell that we didn't and all it will achieve is nothing" :-) ) I'd say add the restrict. (As kib and others have noted, it achieves a lot of nothing, so the priority for this particular conformance seems low.) * Select itself: it makes no sense *in the caller* to pass the same &fd_set argument to more than one parameter unless they are all input-only. The kernel is going to write on them, and there is no guarantee that the kernel will write them in any particular order. Hence if you write: ret = select(nfds, &a, &a, &a, timeout); you are nearly guaranteed to have a bug: this call can only be correct if the only value you intend to inspect is "ret". (That, in fact, is what devd was doing: if ret==0, daemonize, and in any case throw away the returned results in the by-pointer arguments. Hence not a bug -- unless ... well, read on.) The remaining question is: can even that be correct? In particular, should the kernel be required to use copy-in/copy-out semantics, or can it do the equivalent (minus error checking and a lot of other important details) of: int select(int nfds, fd_set *in, fd_set *out, fd_set *ex, struct timeval *tvp) { some_complicated_type v_in, v_out, v_ex; v1 = op(*in); *in = function(v1); v2 = op(*out); ... } ? If the latter is *permitted*, then the call passing "&a" three times is *always* wrong, since *in might have been clobbered before *out is ever loaded. If a program breaks because it was doing this, well, it was already broken, we were just lucky that it worked anyway. (Was this good luck, or bad luck? :-) ) Currently, the kernel does in fact copy them all in, then futz with them, then copy them all out (for good reasons). So the devd usage always works. But will the kernel *always* do this, or is that an accident of the implementation? If the kernel *is* always going to do this, the documentation should say so, at which point the POSIX requirement for the restrict on the parameter is itself a bug (as far as BSD is concerned). That would call for feature test macros around the declaration. So, if we declare that select(nfds, &a, &a, &a, timeout) *is* legal and meaningful in BSD, *don't* just put in the restrict: use feature test macros as appropriate to handle any conformance issues. But please also update select(2) to call out when this is OK (e.g., in devd's particular use case). Chris From owner-freebsd-hackers@freebsd.org Wed Feb 28 00:13:23 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38E6EF3736F for ; Wed, 28 Feb 2018 00:13:23 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DCAFC78963 for ; Wed, 28 Feb 2018 00:13:22 +0000 (UTC) (envelope-from brooks@spindle.one-eyed-alien.net) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id 831395A9F27; Wed, 28 Feb 2018 00:13:16 +0000 (UTC) Date: Wed, 28 Feb 2018 00:13:16 +0000 From: Brooks Davis To: Siddharth Muralee Cc: freebsd-hackers@freebsd.org Subject: Re: [GSoC] Kernel Fuzzing suite Message-ID: <20180228001316.GA21774@spindle.one-eyed-alien.net> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Nq2Wo0NMKNjxTN9z" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.1 (2017-09-22) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 00:13:23 -0000 --Nq2Wo0NMKNjxTN9z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 27, 2018 at 12:00:40PM +0530, Siddharth Muralee wrote: > Hi there, > I am an undergraduate student at Amrita University and I would like to > work on the Kernel Fuzzing Suite project that has been suggested in the > projects page. I am a security enthusiast and I am quite familiar with both > user-land and kernel-land exploitation. I am also familiar with using > automated tools like PIN, AFL, Angr for automated binary analysis and > vulnerability detection. > I have been taking a look at the current kernel fuzzers already > present for FreeBSD like Syzkaller, and TriforceAFL for OpenBSD. I have > also been comparing these with other Kernel fuzzers like Trinity and the > relatively new DiFuzz. I would like to know how to start working on this > project. Since no mentors have been assigned to this project I also don't > know who to contact regarding queries. I'd suggest suggest looking for one of the existing frameworks that does work at least minimally, but has incomplete coverage it proposing a project to enhance things to support FreeBSD. For example, my understanding of the status of Syzkaller is that is supports syscalls that are identical to those on Linux. That presumably means that there are many syscalls including quite important ones that aren't covered. -- Brooks --Nq2Wo0NMKNjxTN9z Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJalfQbAAoJEKzQXbSebgfAZJ8IAJq4ur3tP1NJRs9vbD3I+JGB zMcaQQAV9/FGmWffNtFnApa1GrokFitC0Y4XJxaiYclNSi67KSsYLfDLkCL+keB0 fsoFCdlReo1Dsoi+d06uSWFhX2FZCFuwHVdhiIiZ0uwmxpqAX4V2gycvgixfLUVz y3Zm0YFKWsQdnVO4nR78dKFCUEi2VdxSVX3FCRQII+DjM2HINsvEwFLggpPZDh/l jjNLZ7s1IsTCYgGPiOH0WSlcMw+KWzrN7AbmVQSlSM6uUKQxui31oyqzrlGii+QO LAB0YD4T45OaA7AIyp5mzWqYICE4mSFiiZcxOU+2/fbjypRLkzvP6/oLBiYU6hE= =s9DR -----END PGP SIGNATURE----- --Nq2Wo0NMKNjxTN9z-- From owner-freebsd-hackers@freebsd.org Wed Feb 28 02:39:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEE33F414EF for ; Wed, 28 Feb 2018 02:39:02 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 403E67E838 for ; Wed, 28 Feb 2018 02:39:02 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-io0-x230.google.com with SMTP id g21so1539522ioj.5 for ; Tue, 27 Feb 2018 18:39:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=tA3Ew35D0CVn2W3jIV+veXByZTEczAEfglDUDnWAe3s=; b=bSBEzJdf3paJh2vLlswPgZgcGNUgHYQRgKDDocw2xoRrsC3MH7Wl8UHv0nW6xdHf5P HWSF9xYMMQzndWqfOHJ+xeaBrLm6PX9LCesSbh4YM/UQFndITryRQVmYChKweEZT3J++ WfRHW0xcyqtbWV5tHpOwWDi8OHjL5tYCzbawRhlNxtUamwyasVSZvmurLJJFzYUsDfVg RauseCoWCPevvzSD95kQMIF5p2O5dhmTyexu+4rNoMcUN/FVkGvy8knmT/o9SIwInSPt DkkLCKJS+mkiXi7zd/vogIfe4iMejPFUxVGUtGYDkhzj27xsRKyPQQgjx8mgXmm5gPv6 aIAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=tA3Ew35D0CVn2W3jIV+veXByZTEczAEfglDUDnWAe3s=; b=cyu78RbEE2f5X+UjU4JV00/SemBofkWHVpin/N055u0TFFQ8RE6ZEqADE48zrLABj0 JGdQq7iDQyv2zfxeIJc8jOAIo+kQunqgyal+ZwT8UqDRAzLhw5ihjU4kj0MI7jFJOQVo any6Eb2uPic1tzdQxbEW85u6KOiMggKkjD/sonFuCee81HWVg+vGYxZuJc/N+zvAOG0/ 2Cavm454hlVj+zQMtIO8r3AE1COywiYPRtyxx0ATfvdctcAQrZYOJMiqpH6PhMkfVcLj tDKHtv8QH6iTIRL2kGB6I0fupDdY7L5EQZPte9P0sG4P5ETt4s9AR0/dPxWX6EmNlU8/ 9nIg== X-Gm-Message-State: APf1xPA2zAF7UpZWW+z7gtTXw+Iuh22ePqIuwdohw2SLO0r8dNarYaOD DISCQZ1Ddta/sbRPmtu71+vAnJOlAsx+wUDD+yANSw== X-Google-Smtp-Source: AG47ELtxgC45wi1QA770s1BCpWkrGwPjXKxu7aMXccUryauysTtSEQgw5zCprjuDum6li+sz1G+ThbRSq7HYYvSnZHY= X-Received: by 10.107.134.95 with SMTP id i92mr18081954iod.210.1519785541674; Tue, 27 Feb 2018 18:39:01 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.163.13 with HTTP; Tue, 27 Feb 2018 18:38:41 -0800 (PST) In-Reply-To: References: From: Ed Maste Date: Tue, 27 Feb 2018 21:38:41 -0500 X-Google-Sender-Auth: UbTJk8MnNGnHErJa-KhBTh6Wnm8 Message-ID: Subject: Re: syzkaller for freebsd again To: Dmitry Vyukov Cc: FreeBSD Hackers , syzkaller Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 02:39:02 -0000 On 21 December 2017 at 04:26, Dmitry Vyukov wrote: > > I wanted to point out that freebsd support in syzkaller is still far > from being complete. We still need better descriptions of system calls > and kernel code coverage, report parsing need improvements as well. > For linux we are now finding 100+ bugs per months in a completely Hi Dmitry, Yes, I had one of my co-op students work on automation for setting up and running Syzkaller (in this case, on Packet.net's infrastructure). It's certainly still quite early for us; we hadn't yet done work on Syzkaller itself for FreeBSD. I think the most important change for us to make effective use of Syzkaller is going to be having kernel coverage support. I have two new Waterloo co-op students for this Jan-Apr work term and one of them is getting close to having a working kcov implementation; once this is ready we'll pick up the execution again. > We could setup a similar thing for freebsd, but for that we need > support for building freebsd kernel and GCE-compatible images. For > linux that code lives here: > https://github.com/google/syzkaller/blob/master/pkg/kernel/kernel.go > https://github.com/google/syzkaller/blob/master/pkg/kernel/generated.go Thanks, we'll take a look at this too. The FreeBSD release engineering team produces GCE images so much of the infrastructure exists already. Right now it's only straightforward to build FreeBSD from FreeBSD, so it might take some work to integrate this with the setup you describe here. From owner-freebsd-hackers@freebsd.org Wed Feb 28 09:40:50 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49AA0F38683 for ; Wed, 28 Feb 2018 09:40:50 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: from mail-wr0-x234.google.com (mail-wr0-x234.google.com [IPv6:2a00:1450:400c:c0c::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B8C746E3F5 for ; Wed, 28 Feb 2018 09:40:49 +0000 (UTC) (envelope-from gljennjohn@gmail.com) Received: by mail-wr0-x234.google.com with SMTP id v111so1660755wrb.3 for ; Wed, 28 Feb 2018 01:40:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references:reply-to :mime-version:content-transfer-encoding; bh=28TZZRSRce2QwWaKn0mQqerpESIB9a2ydoyhywymPYs=; b=QMQ5H81kq8AgslJ57ySW1+TWuvSzTcEjHIcDGIc/qrDes1EJBcYGdbF1mXMw7J7rkO heYFMuRUeob7HAoJQe6SPa2CZCtm7NxNsbA8UebfS6XIRoXZ/V1IuAKQ+ErTsWAWve6G L1NWn7eRLrSRsO7tF9pApC9Hw6jNIZaHrBNl7VAqhAYFvpWiRYz5Cm2iZ2PBdmM4zAq2 /BzrBkNpAFgIO5ghpfTGCMJq2U2nIfYf2/X8TWmF2eDBC+cbqKSIYhlB3hiXpUIAX3Nv haIXKbltZGzRbdYnx0j883qPPOsxsPGlFKro4tjIGoFdvLuAaL/vyf//855NwbHkXp+f ZG/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:reply-to:mime-version:content-transfer-encoding; bh=28TZZRSRce2QwWaKn0mQqerpESIB9a2ydoyhywymPYs=; b=l3VA6Vi+bHwtxIqN/MxT2bTOYAOjLoPhIimRWVVz5iVX3Xwe24jKAg5DwYiu/SwdcX FVHU1q50kadRmpFbYM0pSDjHckVUxAvLzYdhhWu599LZkaYTr7POZT+oBoSo0BJmloBm /sSj2EqDZyDRggDEXIl/0/LfeuX1iZXZ1XGG238oUINrkIbE1dnLcIxLgyg3I5hwU3jE sv9iSDGpp4KU89swoaNGkqzk/lgG79PvYnnpqKWhsf+5Lb47B77QtpLE41iaAJSYCFc3 EYb8hew4Pb/FN+ha+CzhrWnEGN7CiU6SyciYy3khK+++83glpqSa0vYpgXEoY6KDYr9w 95aA== X-Gm-Message-State: APf1xPB4m+6tPeNTdQkUzoQp2NllH/fiGl7lyErd71bHjiHnpkPyrvaV OuTQ2B6ES0r007gdnkhwqncrOA== X-Google-Smtp-Source: AH8x224H9wGN2VFM0cjamePvmv5sNoz4gsmLfojPk3OLjYg4Xx19fiSQiMDJxZ9fic+GtWPwp2hjRA== X-Received: by 10.223.191.10 with SMTP id p10mr16459113wrh.160.1519810848407; Wed, 28 Feb 2018 01:40:48 -0800 (PST) Received: from ernst.home (p5B02321F.dip0.t-ipconnect.de. [91.2.50.31]) by smtp.gmail.com with ESMTPSA id b136sm1052938wme.34.2018.02.28.01.40.46 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 28 Feb 2018 01:40:47 -0800 (PST) Date: Wed, 28 Feb 2018 10:40:39 +0100 From: Gary Jennejohn To: Chris Torek Cc: freebsd-hackers@freebsd.org Subject: Re: Marking select(2) as restrict Message-ID: <20180228104039.1680ca80@ernst.home> In-Reply-To: <201802272230.w1RMUOmL079462@elf.torek.net> References: <20180227210110.GA76452@stack.nl> <201802272230.w1RMUOmL079462@elf.torek.net> Reply-To: gljennjohn@gmail.com X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.31; amd64-portbld-freebsd12.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 09:40:50 -0000 On Tue, 27 Feb 2018 14:30:24 -0800 (PST) Chris Torek wrote: > If we can back up a bit, there are (at least) two mostly-separate > issues: > > * What types and qualfiiers to use for various parameters: > These are dictated by standards, and we should just conform > to whatever is in the standard unless there's some major > reason to do otherwise. > > POSIX says that we *should* use "restrict" here: > http://pubs.opengroup.org/onlinepubs/009696699/functions/pselect.html > > so if no one has a major reason to do otherwise (such as > "no conforming program can tell that we didn't and all it will > achieve is nothing" :-) ) I'd say add the restrict. > > (As kib and others have noted, it achieves a lot of nothing, so > the priority for this particular conformance seems low.) > > * Select itself: it makes no sense *in the caller* to pass > the same &fd_set argument to more than one parameter unless > they are all input-only. The kernel is going to write on them, > and there is no guarantee that the kernel will write them in any > particular order. Hence if you write: > > ret = select(nfds, &a, &a, &a, timeout); > > you are nearly guaranteed to have a bug: this call can only > be correct if the only value you intend to inspect is "ret". > > (That, in fact, is what devd was doing: if ret==0, daemonize, > and in any case throw away the returned results in the by-pointer > arguments. Hence not a bug -- unless ... well, read on.) > > The remaining question is: can even that be correct? In > particular, should the kernel be required to use copy-in/copy-out > semantics, or can it do the equivalent (minus error checking > and a lot of other important details) of: > > int select(int nfds, fd_set *in, fd_set *out, fd_set *ex, > struct timeval *tvp) { > some_complicated_type v_in, v_out, v_ex; > > v1 = op(*in); > *in = function(v1); > v2 = op(*out); > ... > } > > ? If the latter is *permitted*, then the call passing "&a" > three times is *always* wrong, since *in might have been > clobbered before *out is ever loaded. If a program breaks > because it was doing this, well, it was already broken, we were > just lucky that it worked anyway. (Was this good luck, or bad > luck? :-) ) > > Currently, the kernel does in fact copy them all in, > then futz with them, then copy them all out (for good reasons). > So the devd usage always works. But will the kernel *always* > do this, or is that an accident of the implementation? > Linux also does a copyin (copy_from_user) of each FD_SET. Linux also does not use restrict in select. It would be a very bad design decision to not copy each FD_SET separately into the kernel. > If the kernel *is* always going to do this, the documentation > should say so, at which point the POSIX requirement for the > restrict on the parameter is itself a bug (as far as BSD is > concerned). That would call for feature test macros around > the declaration. > > So, if we declare that select(nfds, &a, &a, &a, timeout) *is* > legal and meaningful in BSD, *don't* just put in the restrict: > use feature test macros as appropriate to handle any conformance > issues. But please also update select(2) to call out when this > is OK (e.g., in devd's particular use case). > It would be good to explicitly state that it is only safe to use pointers to a single FD_SET when only the error return is of interest and the results in the FD_SET will be ignored. -- Gary Jennejohn From owner-freebsd-hackers@freebsd.org Wed Feb 28 12:31:42 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 186FDF217A9 for ; Wed, 28 Feb 2018 12:31:42 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8F6374F04 for ; Wed, 28 Feb 2018 12:31:41 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from coleburn.home.andric.com (coleburn.home.andric.com [192.168.0.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id BCEC4EF11; Wed, 28 Feb 2018 13:31:34 +0100 (CET) From: Dimitry Andric Message-Id: <584CE3E6-98F4-4D19-8371-979654F38A3C@FreeBSD.org> Content-Type: multipart/signed; boundary="Apple-Mail=_226320BA-6BA4-4D12-8831-3EF9CEF758A9"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Marking select(2) as restrict Date: Wed, 28 Feb 2018 13:31:26 +0100 In-Reply-To: <20180228104039.1680ca80@ernst.home> Cc: Chris Torek , freebsd-hackers@freebsd.org To: Gary Jennejohn References: <20180227210110.GA76452@stack.nl> <201802272230.w1RMUOmL079462@elf.torek.net> <20180228104039.1680ca80@ernst.home> X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 12:31:42 -0000 --Apple-Mail=_226320BA-6BA4-4D12-8831-3EF9CEF758A9 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii On 28 Feb 2018, at 10:40, Gary Jennejohn wrote: > > On Tue, 27 Feb 2018 14:30:24 -0800 (PST) > Chris Torek wrote: > >> If we can back up a bit, there are (at least) two mostly-separate >> issues: >> >> * What types and qualfiiers to use for various parameters: >> These are dictated by standards, and we should just conform >> to whatever is in the standard unless there's some major >> reason to do otherwise. >> >> POSIX says that we *should* use "restrict" here: >> http://pubs.opengroup.org/onlinepubs/009696699/functions/pselect.html >> >> so if no one has a major reason to do otherwise (such as >> "no conforming program can tell that we didn't and all it will >> achieve is nothing" :-) ) I'd say add the restrict. >> >> (As kib and others have noted, it achieves a lot of nothing, so >> the priority for this particular conformance seems low.) >> >> * Select itself: it makes no sense *in the caller* to pass >> the same &fd_set argument to more than one parameter unless >> they are all input-only. The kernel is going to write on them, >> and there is no guarantee that the kernel will write them in any >> particular order. Hence if you write: >> >> ret = select(nfds, &a, &a, &a, timeout); >> >> you are nearly guaranteed to have a bug: this call can only >> be correct if the only value you intend to inspect is "ret". >> >> (That, in fact, is what devd was doing: if ret==0, daemonize, >> and in any case throw away the returned results in the by-pointer >> arguments. Hence not a bug -- unless ... well, read on.) >> >> The remaining question is: can even that be correct? In >> particular, should the kernel be required to use copy-in/copy-out >> semantics, or can it do the equivalent (minus error checking >> and a lot of other important details) of: >> >> int select(int nfds, fd_set *in, fd_set *out, fd_set *ex, >> struct timeval *tvp) { >> some_complicated_type v_in, v_out, v_ex; >> >> v1 = op(*in); >> *in = function(v1); >> v2 = op(*out); >> ... >> } >> >> ? If the latter is *permitted*, then the call passing "&a" >> three times is *always* wrong, since *in might have been >> clobbered before *out is ever loaded. If a program breaks >> because it was doing this, well, it was already broken, we were >> just lucky that it worked anyway. (Was this good luck, or bad >> luck? :-) ) >> >> Currently, the kernel does in fact copy them all in, >> then futz with them, then copy them all out (for good reasons). >> So the devd usage always works. But will the kernel *always* >> do this, or is that an accident of the implementation? >> > > Linux also does a copyin (copy_from_user) of each FD_SET. Linux > also does not use restrict in select. Not in the kernel, where it is effectively declared as: asmlinkage long sys_select(int n, fd_set __user *inp, fd_set __user *outp, fd_set __user *exp, struct timeval __user *tvp); but definitely in userland, where glibc has: extern int __select (int __nfds, fd_set *__restrict __readfds, fd_set *__restrict __writefds, fd_set *__restrict __exceptfds, struct timeval *__restrict __timeout); So for any normal program the entry point to select() is certainly using restrict. > It would be a very bad > design decision to not copy each FD_SET separately into the > kernel. Indeed, Linux's sys_select allocates 6 separate bitmaps, 3 for the incoming fd_sets, and 3 for the outgoing ones. If the outgoing ones would all use the same pointer, it is just going to end up calling copy_to_user() 3 times on the same area, overwriting the previous results. That would still be unsafe for some situations. > It would be good to explicitly state that it is only safe to > use pointers to a single FD_SET when only the error return is of > interest and the results in the FD_SET will be ignored. Agreed. -Dimitry --Apple-Mail=_226320BA-6BA4-4D12-8831-3EF9CEF758A9 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCWpahHgAKCRCwXqMKLiCW o4BjAJ9lJeyAjTRLq0q9p5/snjjbVvpo9QCgoyRbGrTdgQnsuRsaTgn1GT+Lc/Q= =1awd -----END PGP SIGNATURE----- --Apple-Mail=_226320BA-6BA4-4D12-8831-3EF9CEF758A9-- From owner-freebsd-hackers@freebsd.org Wed Feb 28 18:43:22 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A844CF3C414 for ; Wed, 28 Feb 2018 18:43:22 +0000 (UTC) (envelope-from dvyukov@google.com) Received: from mail-pf0-x22d.google.com (mail-pf0-x22d.google.com [IPv6:2607:f8b0:400e:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3168687497 for ; Wed, 28 Feb 2018 18:43:22 +0000 (UTC) (envelope-from dvyukov@google.com) Received: by mail-pf0-x22d.google.com with SMTP id y186so1363858pfb.2 for ; Wed, 28 Feb 2018 10:43:22 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=t3gyz0jt+BdRixwh96i8WbuzrV+wIvmhiS9bz5mQkX8=; b=MA79FuknzhYWla2asooXhXnNCTuoX8fEYDGNJu3VPWw2JRJLdGvtojJ+4g0h2XplQg yPLzDknQEHMeoH8r50VHaONpFHBV3hfYGO+nGyP+y8w3MpMRUZjc4Yhvehy1UaemL55G fYEj2nT3PsrtYIaQ5dNTRclDQfkVsSU59nB8pr9RXV1ia6gb0z8hEFEykfFuJUz7Nfu6 xrqZpuowwp7mGJ5GAjH0oGpa8caKERe4A1aPR59aHv/tHyIbzDnEDMZP9UkVaGcwY2Wr 0vw2t7e3xRZ7IdsRezf8p8Lwf3wblgL9LF2tBQqJd4rmmSIIS4fjCHEK8odzdpoYGgST sRRw== X-Gm-Message-State: APf1xPDWk8kqKb5kKiHgl0HyQwtawmsweWiPN3yvtSTE3JUJnDj1/LEe yOVHgnZkZuKVNsjDIqI1OODP2uPtY/UO3gzk0/ryPg== X-Google-Smtp-Source: AH8x2254MMJDTy8vTUmprljFtSUEYDJhwf+0HQ1HjHX1P7+59SPGtr9sHehGWq5QH0plLavcKGlDh02ptVBzQivWoUs= X-Received: by 10.98.110.71 with SMTP id j68mr16837069pfc.93.1519843401073; Wed, 28 Feb 2018 10:43:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.140.151 with HTTP; Wed, 28 Feb 2018 10:43:00 -0800 (PST) In-Reply-To: References: From: Dmitry Vyukov Date: Wed, 28 Feb 2018 19:43:00 +0100 Message-ID: Subject: Re: syzkaller for freebsd again To: Ed Maste Cc: FreeBSD Hackers , syzkaller Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Wed, 28 Feb 2018 20:07:56 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2018 18:43:22 -0000 On Wed, Feb 28, 2018 at 3:38 AM, Ed Maste wrote: > On 21 December 2017 at 04:26, Dmitry Vyukov wrote: >> >> I wanted to point out that freebsd support in syzkaller is still far >> from being complete. We still need better descriptions of system calls >> and kernel code coverage, report parsing need improvements as well. >> For linux we are now finding 100+ bugs per months in a completely > > Hi Dmitry, > > Yes, I had one of my co-op students work on automation for setting up > and running Syzkaller (in this case, on Packet.net's infrastructure). > It's certainly still quite early for us; we hadn't yet done work on > Syzkaller itself for FreeBSD. > > I think the most important change for us to make effective use of > Syzkaller is going to be having kernel coverage support. I have two > new Waterloo co-op students for this Jan-Apr work term and one of them > is getting close to having a working kcov implementation; once this is > ready we'll pick up the execution again. Hi Ed, Yes, coverage would be great. Assuming that the kernel interface is not radically different from linux, changes on syzkaller side should be trivial. Ready to merge that when you are ready. >> We could setup a similar thing for freebsd, but for that we need >> support for building freebsd kernel and GCE-compatible images. For >> linux that code lives here: >> https://github.com/google/syzkaller/blob/master/pkg/kernel/kernel.go >> https://github.com/google/syzkaller/blob/master/pkg/kernel/generated.go > > Thanks, we'll take a look at this too. The FreeBSD release engineering > team produces GCE images so much of the infrastructure exists already. > Right now it's only straightforward to build FreeBSD from FreeBSD, so > it might take some work to integrate this with the setup you describe > here. We could create another master VM with freebsd. Should not be a problem. Since all code is Go porting should be almost zero effort too. The syz-ci thing (which continuously builds kernels and images) can also run locally (using, say, qemu VMs for actual testing). So you could make it work locally first (which will be a useful thing in itself), and once that works, we can start looking at setting up real continuous testing. From owner-freebsd-hackers@freebsd.org Thu Mar 1 02:01:48 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A211F37918 for ; Thu, 1 Mar 2018 02:01:48 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A81AC71E8F; Thu, 1 Mar 2018 02:01:47 +0000 (UTC) (envelope-from carpeddiem@gmail.com) Received: by mail-it0-x232.google.com with SMTP id n7so5829879ita.5; Wed, 28 Feb 2018 18:01:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Z7PgiypVaTYk2xEhcLz4PBaVGsSZZV7IpzU/YGFt1ak=; b=Ndtk0pzSQMPllVQY8O50R0ecsdrnFQPKLQrdWX4ubJfOfR9E8W4zY4IuSMiofZeSL5 9o4eNWlAzCWvZNC2Hl+lDZZj9fw9L3fThzntafatrW/q8HsTKdK476NhFcxYAcobGplt 00oIg/YmWTd6T5Kzti9X8SlcOo6ob2wsfOO1tYrO6/6gM+y93fD1Y1V/fl0wQN/P1gx7 XapW0iHyYxCNAtITtIIDwIbKVRdEmX27tNGNuRTagHCC6Ga3tPglrHN5cVlZign9X0ZD NqEXptpo0W3HYDgQr5kTh1Hv1ewx27DFj2EO2PuV5k/PkdEYGzBx97xdQhEij4kfZeIy HjfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Z7PgiypVaTYk2xEhcLz4PBaVGsSZZV7IpzU/YGFt1ak=; b=avCj70KdtfIZbT/gcDUAOgP7fAZi4114F9fUzP7sSpNpBOPxBt/WnpHOvehuZiHETz WykOE/TR2gnbpjB0OAx0w3IAAK/1izuYTjK3Om1csEc5A6YJ1d38vsicBXwplZUYq9qG bN3Od9pir2WS/IzOYHrDUv8N2Vjn42aFlvrDXGEk84isziCLkpnKEXxLA2VlpBatLVk0 TNZon2Cb+c2xj7kBwq7WzBmSWquBXnXq91OPJsHggz1YfUhRlk3ExzPXo9bclQjXuzEw p82n1LJ7v8YxywBr47buKN5AZGSubS7siwWCVhJcdjQCiaFkQwz3GDHxpLXau0nWESfa QUYQ== X-Gm-Message-State: APf1xPDZJwuYMPoaqkbHoShuTrfMVe50TyfaHUXbni+TGcW7oaRbKk0w 0jK0rI74MBcjKwPIjII0dv3/nUNu36bLAcPSNZ5mOD0Q X-Google-Smtp-Source: AG47ELtVXk+KcepDFAS5vra7qeNvIZMQA8ghSksjZgFoaKwXPBUAe1Ey2DlRvtTfIAuVWqmZgbvKJvVqAO/8ONNFvfk= X-Received: by 10.36.47.135 with SMTP id j129mr788153itj.78.1519869706771; Wed, 28 Feb 2018 18:01:46 -0800 (PST) MIME-Version: 1.0 Sender: carpeddiem@gmail.com Received: by 10.107.163.13 with HTTP; Wed, 28 Feb 2018 18:01:26 -0800 (PST) In-Reply-To: <20180228001316.GA21774@spindle.one-eyed-alien.net> References: <20180228001316.GA21774@spindle.one-eyed-alien.net> From: Ed Maste Date: Wed, 28 Feb 2018 21:01:26 -0500 X-Google-Sender-Auth: 3b7GPG0FG2tn4mj0SyWpM_Ed-_g Message-ID: Subject: Re: [GSoC] Kernel Fuzzing suite To: Brooks Davis Cc: Siddharth Muralee , FreeBSD Hackers Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Mar 2018 02:01:48 -0000 On 27 February 2018 at 19:13, Brooks Davis wrote: > > I'd suggest suggest looking for one of the existing frameworks that does > work at least minimally, but has incomplete coverage it proposing a > project to enhance things to support FreeBSD. For example, my > understanding of the status of Syzkaller is that is supports syscalls > that are identical to those on Linux. That presumably means that there > are many syscalls including quite important ones that aren't covered. There's a good amount of work to be done on Syzkaller for FreeBSD still - there's a list at the bottom of https://github.com/google/syzkaller/blob/master/docs/freebsd.md. One of my Waterloo co-op students from last term worked on automation for Syzkaller/FreeBSD, and Mitchell, one of this term's students, is working on the first item from the above list: kernel coverage support. I'd say the next most important item is second on that list, extending the set of syscalls supported by Syzkaller. From owner-freebsd-hackers@freebsd.org Thu Mar 1 19:40:45 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0E724F45F97 for ; Thu, 1 Mar 2018 19:40:45 +0000 (UTC) (envelope-from ss.16u10049@btech.nitdgp.ac.in) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A36E9766FA for ; Thu, 1 Mar 2018 19:40:43 +0000 (UTC) (envelope-from ss.16u10049@btech.nitdgp.ac.in) Received: by mail-oi0-x231.google.com with SMTP id u73so5394303oie.3 for ; Thu, 01 Mar 2018 11:40:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=btech-nitdgp-ac-in.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=E/SKnNdhvMy5h4aG69+xZfiEelLGbDUumn387DKi3NM=; b=eCEARHC/U4UJD7ypsnPqW1W16e9Zg8tRydrpoQ2LRfLpHg06zCl/HIexUSEUMtlJb0 lizjoRF7Zv5sntYwJoWXTolgTv3buJKoZ2EtmBO/5vjf/F3xx7KzNdinlOW0adgLNk3Y pUH5Db2sBBqbxnSOl8msWnPRHtx+aIqhcxyL7ixn7hU+RsjE0KoOjMzSnsYY+aIexHX1 wqJ5axPyywrFWXlGnWhqi73PF76sXmA2R83tQgQfpbaI1+42aObdsM4wxS3fp3eIAPJF +zy7BP547Ck8tSaCbJReoldPH3Imlla0P+0dlLGGOBdGyps0aoJRJfOHT2QH4/Panlt8 aQiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=E/SKnNdhvMy5h4aG69+xZfiEelLGbDUumn387DKi3NM=; b=nw/k6G5Q8Awz+dvK3dYodfOaWqL8OoI1+oHt2YFYJDTAMKnAXUJrugBnH25P31a56q s6iH0xQ8Z8y1dfaWUmQFobUGf5CGGcdyrzLUGh6dn3GvWPiRJrUv1uhaPYejnyTN89Nv KRi07ziq7VfuOLD4MKGG0LAIlB4ojnjihgwsQOr7Pb6ZLWM3PUaran0BY54FthNYFwo8 OqlqmoEjJsGKsJsvVf4BCePVK+SzZTnqNoXwONRXunJJapbjXGS8cM2NxaK20ZkVBeEH 6YeDkg6t76DRhGQcc73USzNMhFqhA8VmvjVuBV2FnTls5ON4jm2imMkj0+tKYSk1JCq7 5Jkg== X-Gm-Message-State: AElRT7FQLVAoCQN4PKUDNeWzxbJBshL3LxP1RiJDNZSDL6e94VMkNJQF xUrK0rty5b/qk/tZbps7kNBGKSlmx6VZhxDGVi5GF2UJ X-Google-Smtp-Source: AG47ELsHEkNn78Tzzzj9Z+HGwqbwaxZ4E8SgDmLVOOxgL/63j4Cm42Tbf3lfczmWGaRjKMLWLtsDoot8NIOIo7CVFxo= X-Received: by 10.202.179.66 with SMTP id c63mr2082653oif.115.1519933242829; Thu, 01 Mar 2018 11:40:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.202.206.211 with HTTP; Thu, 1 Mar 2018 11:40:42 -0800 (PST) From: "SUDHANSHU SAURAV ." Date: Fri, 2 Mar 2018 01:10:42 +0530 Message-ID: Subject: query regarding Gsoc2018 To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Mar 2018 19:40:45 -0000 Sir, Thanks for providing such a guidelines for Gsoc 2018.Can you tell me the github repository link of this bhyve emulator containing instructions and also suggest me how can I use Intel XED tool to create a test control for instructions emulation code. From owner-freebsd-hackers@freebsd.org Fri Mar 2 01:02:26 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C0D1F39EFE for ; Fri, 2 Mar 2018 01:02:26 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA74D8360A for ; Fri, 2 Mar 2018 01:02:24 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: by mail-qt0-x22f.google.com with SMTP id j4so10058067qth.8 for ; Thu, 01 Mar 2018 17:02:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:from:date:message-id:subject:to; bh=tnj+F1L7p/a3Xsj7XVfd+aUK/YX2CFrSPCdv+ioS/QI=; b=pDsPYJzUtQNepUUqp2r2ar1OvK49L4rSw6Kx12Pqs3dHt454ExbBCtLeznnx99VohL G/++h0lnEPeI07DeIojt9lwIbpsJW+vg10W3Gm+QHHGC/1t/RmZXbTi+alZnem6dlUAk d1YUg/mbQkrP2Q91h+sPc3j7/oQGeSSVEC8jgw8UiaVetWF6JxeR0vNSiGX+yxTyejwH CuBACgrN35wKSLMryHmpX+0jdATihCXCFLPj+tZme2hkAHfh9DS3gN9Cua3ng7AM1WgS Eso9rrDpL7/kH5BcNblWgWpY7RRWqgIGI+uP2uYUc1goM1TNv24XkcllHFsCv+/xe83Z CsXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=tnj+F1L7p/a3Xsj7XVfd+aUK/YX2CFrSPCdv+ioS/QI=; b=P0ikiaKWRIno0thICZHgIY90N/rQq8t2RNmAQKQja4x7ntvRGr0bfhfs09SoKXd2b1 rVbyWb4OAjx05RApCWjwPN43MzMQRmVtBj9CINuTGqTDX3qBB66V4W7GAUuZ494nrMz1 SyY190Xov9FgNoMfhzPm66FtYpQcJ1482uQYs4chbHJAN1NeWOVoC924EbFU+Cv5+ITs B0vN+QNXr4Ubc1OkQvEkUu6N8thFcSa5rhx0VWl0McHRKvNc5KW41io09alBKz0nw6+S 5/iji7tMzWW4rd5B8Fviy2ECJ0mPRcDmTV9ulj1CTzOxBBeq+5Ba5Vi2bfoQEWEdTWhT jOWg== X-Gm-Message-State: AElRT7E7TGyXrqe2cJ5YGYA0ZrjfLwnIyvclxR5MJSpN7BaZWDE1TZIL 5j1iRcsgRD7VqT8iuNQD+g1p6vDCzADFLIu17Tr8XZTn X-Google-Smtp-Source: AG47ELs1OiebKX+pJ5P36VQK1BcOgB1GV5QROXYRNq94/RRm+Xso5jMtew03gFb0DXWm2i0irvNmq1smwsCfSWlMvvc= X-Received: by 10.200.34.1 with SMTP id o1mr6212942qto.103.1519952543482; Thu, 01 Mar 2018 17:02:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.237.63.125 with HTTP; Thu, 1 Mar 2018 17:02:23 -0800 (PST) From: Joe Maloney Date: Thu, 1 Mar 2018 20:02:23 -0500 Message-ID: Subject: OpenRC 0.35 for FreeBSD To: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 01:02:26 -0000 Hello hackers, I have been working on a single diff version of OpenRC for FreeBSD: https://github.com/pkgdemon/freebsd/commit/b6885cd533c848a1b4f3582f48e40c883669b35c Why OpenRC? The licensing is right, and it's a way of adding modern features to service management without reinventing the wheel. That's my sales pitch. This newest itteration is a result of a year, and a half of feedback from the import into TrueOS. Eventually it will be upstreamed into TrueOS to further reduce the diff again between TrueOS, and FreeBSD. The single commit in this new fork allows coexistence with rc.d. It also uses dhcient again although there isn't a service to start it directly yet but it does get started with netif. I have just recently replaced OpenRC's bootmisc service with a dummy service that launches cleartmp, cleanvar. Once I replace the remaining 7 or so scripts with FreeBSD converted scripts I will rework the dependancies to remove the uncessary dummy scripts during a second audit. I have been working on a conversion tool which eases some of the conversion of the remaining 89, or so scripts. As is this implemtnation only touches top level Makefiles, mtree, adds new files, and adds a modified init which changes pathnames.h to call /etc/openrc. This is work in progress so I am not quite done, and I have been making tickets in the fork to keep track. I was thinking for example of making a kenv option which would allow a single init to switch pathnames settings perhaps. At the same time I was trying to avoid being intrusive by touching service, or init. So therefore there is service, and rc-service. There is init, and openrc-init. There is rc, and openrc. There is rc.shutdown, and openrc.shutdown. You just change init_path for loader, and that is that. I have full details in the wiki at the fork, and tickets for the items I plan to tackle next: https://github.com/pkgdemon/freebsd/wiki https://github.com/pkgdemon/freebsd/issues I have had a lot of fun working on this learning how base is tied together, how bsdinstall is started, learning the internals of rc.d scripts, etc. I would appreciate any feedback. Thanks. Joe Maloney From owner-freebsd-hackers@freebsd.org Fri Mar 2 03:19:32 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1879F42CE8 for ; Fri, 2 Mar 2018 03:19:32 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-qk0-x232.google.com (mail-qk0-x232.google.com [IPv6:2607:f8b0:400d:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5D25368432; Fri, 2 Mar 2018 03:19:32 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-qk0-x232.google.com with SMTP id w142so10383935qkb.8; Thu, 01 Mar 2018 19:19:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=uyVFVlc94GtoVy5IkyLc10DmiMlzGMqmoVZPQ/UCXUU=; b=AHLQZGPTfkcTE5kmxHTI3qCgA9/L1lWU5DQZaLZxtUJNLUQ/447tJdDozkh+R8ddOe Aay8ruw6cGpR4BgwhlvFjA6xFINWoCFE0k1WMOv7F76A+3eMpoO5izpVF53OikFeBCAf j+Tv32kuBr9aEvnokDZmcdIKsxe9ux39Ao9Kf6gRa06yCZBpjfe9diKKWuTalgqluYA0 WriU8xflY/Ih4bSr+AtvLuCt0nPZoMxxDcjzK2ah6HQuHsOcQ/0fwHS3GC2/xNMR7hJw sQEucTZywgnJtoHN/kFkc/P1NC1rr8a+gcFQlnkAsC/ahZ+7Mcn292MlItE7Xsiq6Su3 NbVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=uyVFVlc94GtoVy5IkyLc10DmiMlzGMqmoVZPQ/UCXUU=; b=DVLuTmsMVVxuLq05a2eV0uPwquSfqnEbmMVSXPwiLUILKZ/tZqMW4ExCVJpF4VSrV6 NUPuWLwqTLhPp9BhkbFRqbDvkRCKhcLfhfNcgJmR+79icDL5yze8pcog2ZbKzuGLAQTB kzjRNA2x183j2B7SkMRDUxqlQ2IFjYv29tY72pXKtvrqDDByfu8uvNAcLdMDXgUPtSTx BMZVhv/ZoSFsynAEfel3H5IYNY9+UbwJP5ZlnFc5x473oHRY8JBPR7eVitOheGkZwx2j MTrvvieBgWWB7mk65fzFAAIgx5E+HoaLqO8MdcK/f9Jb1mihA6dF9lHDupJagRlcwWYU FYDg== X-Gm-Message-State: AElRT7GJf3NnuW9R76C2L64F9a8EHKtjMhtiRrh+Nlp23MJ3aQZGY/yk 90FlC2z4brtqjxUhsx88QdmiXggFNXULry1dViI= X-Google-Smtp-Source: AG47ELuI1LyUmObELMw77kHg03zKIaOlxX8wneq0dC/diCfsHMadSGpGov2+46BwRldJ3lMypguDg/yY+G86YScHHk0= X-Received: by 10.55.75.77 with SMTP id y74mr6448733qka.44.1519960771871; Thu, 01 Mar 2018 19:19:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.32.74 with HTTP; Thu, 1 Mar 2018 19:19:31 -0800 (PST) In-Reply-To: References: <4db65b52-5a82-15ff-1629-0371a619f89b@freebsd.org> From: Stefan Blachmann Date: Fri, 2 Mar 2018 04:19:31 +0100 Message-ID: Subject: Re: New microcode updating tool for FreeBSD To: Sean Bruno Cc: freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 03:19:33 -0000 Hi guys, Intel has released a bunch of new microcodes the last days. As these are not released to the public, you might want to look again into https://forums.freebsd.org/threads/introducing-cpupdate-a-microcode-tool-for-freebsd.64588/ There you find how to obtain the microcodes in case your computer manufacturer hasn't supplied BIOS updates. I have updated the cpupdate tool, too. It does the automatic activation of the new CPU features for meltdown/spectre mitigation now, too. So you do no longer need devcpu-data/cpucontrol. If you tried devcpu-data/cpucontrol and wondered why it does not work, here is background: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192487#c0 Have fun testing the new microcodes and FreeBSD's first mitigation patches! Stefan From owner-freebsd-hackers@freebsd.org Fri Mar 2 05:11:46 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D936CF24148 for ; Fri, 2 Mar 2018 05:11:46 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5A1B26C8D5; Fri, 2 Mar 2018 05:11:45 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id w225Bc3v086938 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 2 Mar 2018 06:11:38 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: sblachmann@gmail.com Received: from [10.58.0.4] (dadv@[10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id w225BW4d076099 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 2 Mar 2018 12:11:32 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: New microcode updating tool for FreeBSD To: Stefan Blachmann , Sean Bruno References: <4db65b52-5a82-15ff-1629-0371a619f89b@freebsd.org> Cc: freebsd-hackers@freebsd.org From: Eugene Grosbein Message-ID: <5A98DD05.60606@grosbein.net> Date: Fri, 2 Mar 2018 12:11:33 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 05:11:47 -0000 02.03.2018 10:19, Stefan Blachmann: > Intel has released a bunch of new microcodes the last days. > As these are not released to the public, you might want to look again into > https://forums.freebsd.org/threads/introducing-cpupdate-a-microcode-tool-for-freebsd.64588/ > > There you find how to obtain the microcodes in case your computer > manufacturer hasn't supplied BIOS updates. > > I have updated the cpupdate tool, too. > It does the automatic activation of the new CPU features for > meltdown/spectre mitigation now, too. So you do no longer need > devcpu-data/cpucontrol. > > If you tried devcpu-data/cpucontrol and wondered why it does not work, > here is background: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192487#c0 > > Have fun testing the new microcodes and FreeBSD's first mitigation patches! > Stefan I've just committed new port sysutils/cpupdate for your work. Please notify if you make significant changes so the port does not stale. From owner-freebsd-hackers@freebsd.org Fri Mar 2 05:44:54 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83E8AF269FF for ; Fri, 2 Mar 2018 05:44:54 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2868C6E005; Fri, 2 Mar 2018 05:44:54 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-qk0-x231.google.com with SMTP id o25so10650647qkl.7; Thu, 01 Mar 2018 21:44:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DLL8vl8tXRe34ftadxotVjYTIWoKMt6Nf3dI6yVPR78=; b=SlJxhVrxduIi7DPhvn6Ffm94SAQzpiyLhlMrGNrHZFUuQZ82CUy3aJbSvlISS+VWFy rlNrdmG2/FZogcr6PYUZXqjBbPiE6XtzShNlD5Q90ZDsiPPwYB4Cvwxr5RgrMA7Ztb6D lS/nP+oZEMzcV3LRuxpg8nqwEmma1UhVCadkvEqIk1cErwudGElqdJFMeEDhwCFnOxIQ RPyCLdRgWlLrV8UtyW5FOZoH1xaW108cg9d90aMLkWIbdnjw5MyAy7sDIEz9+dGT+SYV 7r2SVRVxIq866YapIT9rRgEvwX+gbvqJgLDpyAJpw4fZ8Wjr6uxHddG6cTwe6D84a1ho TCQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DLL8vl8tXRe34ftadxotVjYTIWoKMt6Nf3dI6yVPR78=; b=JcjvgFWQZPKx1Di2kOedRXcRkXFNw9/IAscxdBkAHvWbukSPfkoM/g+Hs1Va1Ffw2D 5wqJSR7NjZG6VERgwD3E0PrfFpuHqHpTZVlP6gD9TA8GpDwsEPbATyFnt9Cs7gW+Eg4N V0UUGVmVqovsgCyV06D9LlBRgLedLQPaUNuH66F5+055pjjjGb0H2B+IffLlFls8PpB3 V7e30jmkDXJE0fu54L8lxW5TCK/LdIxJixWy8/Nvvrp5WF1aqsfwYL3mTSg2NNP64Tev aLf549z0uqYSkTIkrf+h6NhvHJnyQZKoQZX9bCQEJPebAG5rvIBw80Zn4h9B0DAq7gWF GsPQ== X-Gm-Message-State: AElRT7Hl+o5y0fPpBBoKlVE2mA0XrflaonuB9qH4buy82AjwzaBwInsp IK85jG89IP/W4S358kye05+vbjjuxzLmgoBvOr8= X-Google-Smtp-Source: AG47ELvtS0sSVqgZ8VCTpDsnr7UhGGtsTt+xmURY/t8CbHilP7mRy4zdYn0rIAfVCjfc6vm/GTKkzeGY8s/uifRUNjg= X-Received: by 10.55.48.144 with SMTP id w138mr5980707qkw.199.1519969493597; Thu, 01 Mar 2018 21:44:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.32.74 with HTTP; Thu, 1 Mar 2018 21:44:53 -0800 (PST) In-Reply-To: <5A98DD05.60606@grosbein.net> References: <4db65b52-5a82-15ff-1629-0371a619f89b@freebsd.org> <5A98DD05.60606@grosbein.net> From: Stefan Blachmann Date: Fri, 2 Mar 2018 06:44:53 +0100 Message-ID: Subject: Re: New microcode updating tool for FreeBSD To: Eugene Grosbein Cc: Sean Bruno , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 05:44:54 -0000 Thank you Eugene! I'll do some reworking the next days. (E.g. formatting to FreeBSD style, breaking down the big functions a bit, cleaning up messy things, write manpage, etc.). I'll will notify you when I got feedback from a few testers that reworking didn't break anything obvious so you can update the port. On 3/2/18, Eugene Grosbein wrote: > 02.03.2018 10:19, Stefan Blachmann: > >> Intel has released a bunch of new microcodes the last days. >> As these are not released to the public, you might want to look again >> into >> https://forums.freebsd.org/threads/introducing-cpupdate-a-microcode-tool-for-freebsd.64588/ >> >> There you find how to obtain the microcodes in case your computer >> manufacturer hasn't supplied BIOS updates. >> >> I have updated the cpupdate tool, too. >> It does the automatic activation of the new CPU features for >> meltdown/spectre mitigation now, too. So you do no longer need >> devcpu-data/cpucontrol. >> >> If you tried devcpu-data/cpucontrol and wondered why it does not work, >> here is background: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=192487#c0 >> >> Have fun testing the new microcodes and FreeBSD's first mitigation >> patches! >> Stefan > > I've just committed new port sysutils/cpupdate for your work. > Please notify if you make significant changes so the port does not stale. > > From owner-freebsd-hackers@freebsd.org Fri Mar 2 06:23:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F06D2F2812C for ; Fri, 2 Mar 2018 06:23:30 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from goliath.siemens.de (goliath.siemens.de [192.35.17.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "goliath.siemens.de", Issuer "Siemens Issuing CA Internet Server 2017" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7287A6F5F6 for ; Fri, 2 Mar 2018 06:23:30 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w226Ir57002891 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 2 Mar 2018 07:18:53 +0100 Received: from curry.mchp.siemens.de (curry.mchp.siemens.de [139.25.40.130]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w226IrDZ016543 for ; Fri, 2 Mar 2018 07:18:53 +0100 Received: (from user@localhost) by curry.mchp.siemens.de (8.15.2/8.15.2) id w226IrbW098668; Date: Fri, 2 Mar 2018 07:18:52 +0100 From: Andre Albsmeier To: freebsd-hackers@FreeBSD.org Cc: Andre.Albsmeier@siemens.com Subject: Adding support for MosChip 9912 PCIe (serial/parallel) cards Message-ID: <20180302061852.GA7887@bali> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 06:23:31 -0000 I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial ports) sitting here which does not get detected on 11.1. I tried to simply add it to the uart and ppc drivers with --- sys/dev/uart/uart_bus_pci.c.ORI 2018-02-12 06:17:57.000000000 +0100 +++ sys/dev/uart/uart_bus_pci.c 2018-03-01 18:27:05.212040000 +0100 @@ -158,6 +158,8 @@ "MosChip MCS9904 PCIe to Peripheral Controller", 0x10 }, { 0x9710, 0x9922, 0xa000, 0x1000, "MosChip MCS9922 PCIe to Peripheral Controller", 0x10 }, +{ 0x9710, 0x9912, 0xa000, 0x1000, + "MosChip MCS9912 PCIe to Peripheral Controller", 0x10 }, { 0xdeaf, 0x9051, 0xffff, 0, "Middle Digital PC Weasel Serial Port", 0x10 }, { 0xffff, 0, 0xffff, 0, NULL, 0, 0} }; and --- sys/dev/ppc/ppc_pci.c.ORI 2017-09-03 17:55:53.000000000 +0200 +++ sys/dev/ppc/ppc_pci.c 2018-03-01 18:27:23.000000000 +0100 @@ -93,6 +93,7 @@ { 0x98659710, "MosChip MCS9865 1284 Printer port", 0x10 }, { 0x99009710, "MosChip MCS9900 PCIe to Peripheral Controller", 0x10 }, { 0x99019710, "MosChip MCS9901 PCIe to Peripheral Controller", 0x10 }, + { 0x99129710, "MosChip MCS9912 PCIe to Peripheral Controller", 0x10 }, { 0xffff } }; Now, when loading ppc.ko and uart.ko it gives me in dmesg: ppc1: port 0xd010-0xd017,0xd000-0xd007 mem 0x89201000-0x89201fff,0x89200000-0x89200fff irq 20 at device 0.2 on pci9 ppc1: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode ppbus0: on ppc1 lpt0: on ppbus0 lpt0: Interrupt-driven port driver bug: Unable to set devclass (class: ppc devname: (unknown)) ppc0: parallel port not found. ppc0: parallel port not found. uart2: port 0xd030-0xd037 mem 0x89205000-0x89205fff,0x89204000-0x89204fff irq 22 at device 0.0 on pci9 uart3: port 0xd020-0xd027 mem 0x89203000-0x89203fff,0x89202000-0x89202fff irq 23 at device 0.1 on pci9 driver bug: Unable to set devclass (class: uart devname: (unknown)) driver bug: Unable to set devclass (class: uart devname: (unknown)) driver bug: Unable to set devclass (class: ppc devname: (unknown)) uart4: <16550 or compatible> port 0x3e8-0x3ef irq 7 on acpi0 What am I missing here? From owner-freebsd-hackers@freebsd.org Fri Mar 2 08:44:43 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF225F30728 for ; Fri, 2 Mar 2018 08:44:43 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from mail.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "0x20.net", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5072274DDA for ; Fri, 2 Mar 2018 08:44:43 +0000 (UTC) (envelope-from lars@e.0x20.net) Received: from e.0x20.net (mail.0x20.net [46.251.251.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.0x20.net (Postfix) with ESMTPS id 9B4ABAF877; Fri, 2 Mar 2018 09:37:57 +0100 (CET) Received: (from lars@localhost) by e.0x20.net (8.15.2/8.15.2/Submit) id w228buOJ094962; Fri, 2 Mar 2018 09:37:56 +0100 (CET) (envelope-from lars) Date: Fri, 2 Mar 2018 09:37:56 +0100 From: Lars Engels To: Joe Maloney Cc: freebsd-hackers@freebsd.org Subject: Re: OpenRC 0.35 for FreeBSD Message-ID: <20180302083756.GH34685@e.0x20.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Editor: VIM - Vi IMproved 8.0 User-Agent: Mutt/1.9.3 (2018-01-21) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 08:44:43 -0000 On Thu, Mar 01, 2018 at 08:02:23PM -0500, Joe Maloney wrote: > Hello hackers, > I have been working on a single diff version of OpenRC for FreeBSD: > > https://github.com/pkgdemon/freebsd/commit/b6885cd533c848a1b4f3582f48e40c883669b35c Thanks for your work! > > Why OpenRC? The licensing is right, and it's a way of adding modern > features to service management without reinventing the wheel. That's > my sales pitch. Hm, that does not convince me. FreeBSD's rc is also BSD licensed and did not reinvent any wheel. Could you maybe give some comparison between OpenRC and our rc? What does OpenRC better? Thanks! Lars From owner-freebsd-hackers@freebsd.org Fri Mar 2 13:47:48 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9998EF24F18 for ; Fri, 2 Mar 2018 13:47:48 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL SHA256 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AD7E8196D for ; Fri, 2 Mar 2018 13:47:47 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.15.1/8.15.1/NETPLEX) with ESMTP id w22DaeR8027483; Fri, 2 Mar 2018 08:36:40 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Fri, 02 Mar 2018 08:36:40 -0500 (EST) Date: Fri, 2 Mar 2018 08:36:40 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Andre Albsmeier cc: freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards In-Reply-To: <20180302061852.GA7887@bali> Message-ID: References: <20180302061852.GA7887@bali> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 13:47:48 -0000 On Fri, 2 Mar 2018, Andre Albsmeier wrote: > I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial > ports) sitting here which does not get detected on 11.1. I tried > to simply add it to the uart and ppc drivers with > [ ... ] Do you try adding similar support to puc_pci_devices[] in sys/dev/puc/pucdata.c? -- DE From owner-freebsd-hackers@freebsd.org Fri Mar 2 15:28:03 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27EFCF2CB32 for ; Fri, 2 Mar 2018 15:28:03 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D12A86557; Fri, 2 Mar 2018 15:28:02 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x232.google.com with SMTP id f75so13886829lfg.6; Fri, 02 Mar 2018 07:28:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=1sCMo6gFM4KFa0C31gLgvvroeYS7NasPulLDo2Nvoe8=; b=aIg7JcLun0vO360nTSkK8uRw5JmhDhZsEFUpbkgs8aStWWCLiJ5GkN81kN2PVlAmdb DWEOQwiWsl2BpI9QUpMMhZ4v/nif0KjA2kiDlPUXVrDEJi4AVGkh1uA2V4nxG7eIVT3q d8QUkGyibTUhcedM2SPyUCHL4qs58/mGkJ5KMA2tkgdkEbO5thSGZdpto8iGR6KMCnVu FNPdEKsZrhz907v+5Itfsgcn1F0WfgMNg5g0W7neG8GxSQ9YBUy+jjJNFlJ4JIhDQnL3 FTdJy1BMtP8pjrbJsO0Jb06dQvfo+3gmsL4YZwYqwQ7Cw/4muuagV9a5aZntGL53ER+R K83A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=1sCMo6gFM4KFa0C31gLgvvroeYS7NasPulLDo2Nvoe8=; b=ErHuDy4YTRKPAktxaZbD1eKyauR+9O2SjBdxSuIhGaBwdilpLZuDpM0veBL1d+cdMx BmeiTLFf4r+iPqfBBGY1vMvwFyg6IXi3RGXljcQfvrb9bN0UhTBLIrpIZhKJnOoo3J3J ECdn2uO5aSjk97K3kAejMT2s0HkD1tm2TGIs2ZO2i9tofhQhwT9FbfhKFYWlpEhJW9Sr zEfKGQBmd69QJIJeZRPd9n2fpbKiU/4pTTjErjqO9RsD45vDdj+/gQOo6ywRAwElq4rG yKf6jBalWNQLR3jOXQXJC0MXXQ6N01aBHp5UuoO4T7nhQa461LRB7CfuL9UCZTqJVmvd aA3A== X-Gm-Message-State: APf1xPCZ2eVCNaFFRrEdiqJg8z0f0AwWIsxY7AM9JtEAOAkPytbwCPuu K23Ou2r3bNidnPis9YzpbW6kGrkxrmHiRu+HnsE= X-Google-Smtp-Source: AG47ELuih2CLwqbn/lCNrpdKaZn/0Emmv9IYYHPxj5bFs++AY9/qfuootDk7V0+RKcHB2uAbCMLq4aDM9JLYKfRCcdk= X-Received: by 10.46.27.211 with SMTP id c80mr4355791ljf.46.1520004480699; Fri, 02 Mar 2018 07:28:00 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.2.195 with HTTP; Fri, 2 Mar 2018 07:28:00 -0800 (PST) In-Reply-To: <20180302083756.GH34685@e.0x20.net> References: <20180302083756.GH34685@e.0x20.net> From: Alan Somers Date: Fri, 2 Mar 2018 08:28:00 -0700 X-Google-Sender-Auth: cCn8S6HAQIm1CM9GUqJ4882ij2E Message-ID: Subject: Re: OpenRC 0.35 for FreeBSD To: Lars Engels Cc: Joe Maloney , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 15:28:03 -0000 On Fri, Mar 2, 2018 at 1:37 AM, Lars Engels wrote: > On Thu, Mar 01, 2018 at 08:02:23PM -0500, Joe Maloney wrote: > > Hello hackers, > > I have been working on a single diff version of OpenRC for FreeBSD: > > > > https://github.com/pkgdemon/freebsd/commit/ > b6885cd533c848a1b4f3582f48e40c883669b35c > > Thanks for your work! > > > > > Why OpenRC? The licensing is right, and it's a way of adding modern > > features to service management without reinventing the wheel. That's > > my sales pitch. > > Hm, that does not convince me. FreeBSD's rc is also BSD licensed and did > not reinvent any wheel. > Could you maybe give some comparison between OpenRC and our rc? What > does OpenRC better? > > Thanks! > > Lars > Parallel startup, mainly. From owner-freebsd-hackers@freebsd.org Fri Mar 2 15:39:33 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDAE2F2DA35 for ; Fri, 2 Mar 2018 15:39:33 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: from mail-pf0-x22d.google.com (mail-pf0-x22d.google.com [IPv6:2607:f8b0:400e:c00::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 555908707A for ; Fri, 2 Mar 2018 15:39:33 +0000 (UTC) (envelope-from 6yearold@gmail.com) Received: by mail-pf0-x22d.google.com with SMTP id y186so4151960pfb.2 for ; Fri, 02 Mar 2018 07:39:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=I0cbI2MfASeW1FfYyLgkAFhXnETgudCpI201/iSt8VU=; b=eJ/1Dt4buuSoYU+4QXujQ1M1uDUFVguG6m09CodxaCIjMyjPsExk/FQex9IIV1+2PY 6q3sih5eTK2NiehfGzgAog4/i3yMxNk9RW5201Izl61+yCdN1uN+4NfjlK0iWpToPyAt JRAxfMCF1JNX5uGnTuEZTWlgKnQXXDSOllcOa/GVYBxe04CtMb4ADAV7Aqbq/QBHj0We rbXXyaFZZ2tXwp1iHATeI79DRr0cuCNA8RvktIqmwW1qrwxL1kJtJ9IR7X9vTlX34CNF 7evfD7+zPSIyLk8p3f9AUMCZNhnHJyD62jqeAl0oXuyVYcgHRFundf/L9/yD1g2PQEDg c1Jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=I0cbI2MfASeW1FfYyLgkAFhXnETgudCpI201/iSt8VU=; b=f2ziqFavO1HhBzb8YOnLNvp2CVBFA8CDUATDtt57/7T1M8x1mJLlyiBrQelUmCaA0K 0ZRVFykA38G4JOE4AaYlYahAljJhfkuQpj7zxCXE9t7cjTHcWuZdUhOxOJ3DR/zOtKsQ QqcXKuva0uAVn+Rn3Wc0llSNdfH8mvAtkYpeKb9ZT1UnedhOv0WpNQYbSOU/eRLr5AIL cmgssMxOfiNkB+GSt1iKQP2ivDNn8GyuVjxaTpX73hVfrjr6ZF4b8fIQMs5L6QIuPR8l +PCCmWzNOQKmQnFDWuW+q3HFkNGgLVind94W2yqT0wVs+JUYKUZgCpRcpc0CGy8UhT9k qC7A== X-Gm-Message-State: APf1xPDP3GmR7g5ygUy5S3o28sr5Wyf7PZlrK9rp8tPc9ysqkYz9uktr S4IdkJlaWhNiTSGW4WF/BpctaZT8sJJYVkyOQUs= X-Google-Smtp-Source: AG47ELuZbzl0sUGPG+op2fHCbsir5u3GEBGTGPDWLZmtI6a6KFgLOKXg/5LlRhNC/V27wYqzCeXGi/DT+gAAWFURYfQ= X-Received: by 10.101.69.4 with SMTP id n4mr4937145pgq.184.1520005172406; Fri, 02 Mar 2018 07:39:32 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.179.134 with HTTP; Fri, 2 Mar 2018 07:39:01 -0800 (PST) In-Reply-To: References: From: Gleb Popov <6yearold@gmail.com> Date: Fri, 2 Mar 2018 18:39:01 +0300 Message-ID: Subject: Re: OpenRC 0.35 for FreeBSD To: Joe Maloney Cc: freebsd-hackers Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 15:39:34 -0000 On Fri, Mar 2, 2018 at 4:02 AM, Joe Maloney wrote: > Hello hackers, > I have been working on a single diff version of OpenRC for FreeBSD: > > https://github.com/pkgdemon/freebsd/commit/b6885cd533c848a1b4f3582f48e40c > 883669b35c > > Why OpenRC? The licensing is right, and it's a way of adding modern > features to service management without reinventing the wheel. That's > my sales pitch. This newest itteration is a result of a year, and a > half of feedback from the import into TrueOS. Eventually it will be > upstreamed into TrueOS to further reduce the diff again between > TrueOS, and FreeBSD. > > The single commit in this new fork allows coexistence with rc.d. It > also uses dhcient again although there isn't a service to start it > directly yet but it does get started with netif. I have just recently > replaced OpenRC's bootmisc service with a dummy service that launches > cleartmp, cleanvar. Once I replace the remaining 7 or so scripts with > FreeBSD converted scripts I will rework the dependancies to remove the > uncessary dummy scripts during a second audit. I have been working on > a conversion tool which eases some of the conversion of the remaining > 89, or so scripts. > > As is this implemtnation only touches top level Makefiles, mtree, adds > new files, and adds a modified init which changes pathnames.h to call > /etc/openrc. This is work in progress so I am not quite done, and I > have been making tickets in the fork to keep track. I was thinking > for example of making a kenv option which would allow a single init to > switch pathnames settings perhaps. At the same time I was trying to > avoid being intrusive by touching service, or init. So therefore > there is service, and rc-service. There is init, and openrc-init. > There is rc, and openrc. There is rc.shutdown, and openrc.shutdown. > You just change init_path for loader, and that is that. > > I have full details in the wiki at the fork, and tickets for the items > I plan to tackle next: > > https://github.com/pkgdemon/freebsd/wiki > https://github.com/pkgdemon/freebsd/issues > > I have had a lot of fun working on this learning how base is tied > together, how bsdinstall is started, learning the internals of rc.d > scripts, etc. I would appreciate any feedback. Thanks. > > Joe Maloney > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > I've been seeing nosh release announcements for a long time and was thinking that this is what is going to replace current rc. Am I wrong? From owner-freebsd-hackers@freebsd.org Fri Mar 2 15:43:38 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE01FF2E02B for ; Fri, 2 Mar 2018 15:43:38 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 643F7874F0; Fri, 2 Mar 2018 15:43:38 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from [134.153.27.124] (unknown [127.0.1.132]) by freefall.freebsd.org (Postfix) with ESMTP id 24ACCFFED; Fri, 2 Mar 2018 15:43:38 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) From: "Jonathan Anderson" To: "Lars Engels" Cc: "Joe Maloney" , freebsd-hackers@freebsd.org Subject: Re: OpenRC 0.35 for FreeBSD Date: Fri, 02 Mar 2018 12:13:37 -0330 X-Mailer: MailMate (1.10r5443) Message-ID: In-Reply-To: <20180302083756.GH34685@e.0x20.net> References: <20180302083756.GH34685@e.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 15:43:38 -0000 On 2 Mar 2018, at 5:07, Lars Engels wrote: > On Thu, Mar 01, 2018 at 08:02:23PM -0500, Joe Maloney wrote: >> [...] >> Why OpenRC? The licensing is right, and it's a way of adding modern >> features to service management without reinventing the wheel. That's >> my sales pitch. > > Hm, that does not convince me. FreeBSD's rc is also BSD licensed and > did > not reinvent any wheel. > Could you maybe give some comparison between OpenRC and our rc? What > does OpenRC better? In addition to asking that question (which is a good one), I think that we need to understand the differences among the various permissively-licensed init/inetd replacements. Personally I think that the time to replace rc is drawing near, but there are a number of options that I've heard of vying for consideration: - finit - jobd (is this still a thing?) - nosh - OpenRC - runit ... not to mention more out-there ideas like launchd. These decisions are never purely technical (as it's a matter of "technically sound" + "someone is willing to do the work"), but I for one wouldn't want to switch something as fundamental as the rc system without having a broad conversation about the merits of the various options and coming to some kind of rough consensus. Also, of course, I have an interest in choosing an init system that can play nicely with sandboxing services, but that's just one consideration among many. Jon -- Jonathan Anderson jonathan@FreeBSD.org From owner-freebsd-hackers@freebsd.org Fri Mar 2 16:11:24 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA998F309B5 for ; Fri, 2 Mar 2018 16:11:23 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "freefall.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A37DD690FA; Fri, 2 Mar 2018 16:11:23 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) Received: from [134.153.27.124] (unknown [127.0.1.132]) by freefall.freebsd.org (Postfix) with ESMTP id 6231A10DDB; Fri, 2 Mar 2018 16:11:23 +0000 (UTC) (envelope-from jonathan@FreeBSD.org) From: "Jonathan Anderson" To: "Lars Engels" Cc: "Joe Maloney" , freebsd-hackers@freebsd.org Subject: Re: OpenRC 0.35 for FreeBSD Date: Fri, 02 Mar 2018 12:41:22 -0330 X-Mailer: MailMate (1.10r5443) Message-ID: <35AC3784-D02B-4EB1-9F82-E522B7A7730E@FreeBSD.org> In-Reply-To: References: <20180302083756.GH34685@e.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 16:11:24 -0000 On 2 Mar 2018, at 12:13, Jonathan Anderson wrote: > > [...] there are a number of options that I've heard of vying for > consideration: > > - finit > - jobd (is this still a thing?) > - nosh > - OpenRC > - runit Oh, and also s6: https://skarnet.org/software/s6/why.html Jon -- Jonathan Anderson jonathan@FreeBSD.org From owner-freebsd-hackers@freebsd.org Fri Mar 2 18:35:13 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 80732F3B9B2 for ; Fri, 2 Mar 2018 18:35:13 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF62670AB2 for ; Fri, 2 Mar 2018 18:35:12 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: by mail-lf0-x229.google.com with SMTP id i80so14678971lfg.5 for ; Fri, 02 Mar 2018 10:35:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mime-version :content-disposition:user-agent; bh=1dxE3Qt8dHrgNn3we+1wUnY+iFbnSWXkgwd2433DiVQ=; b=TiA6ON/LSWOPRzkGPyBeUkNBIX8nsYRf31P3IPCiHoZyoEAI9d4G7mBih+BnX+Mmnc 241nP5la/umil7XW2BlvBNeRMyv0aU2LbHYDbCpWWPTZi27ZdfZCguydoPtKCjjQUxAw fm/SnplR98lLxFiUaETg84E0FFiKPYXiDd9/zPN89X/lEmp7B8GxkajRLfcmszV/gP6Y /kM6QSBvzKOJeWTJN51z5LSVoIz/ggO6yRBMwPgNcRhIsFFwr5rZ3w5JHpJP689ZUKhY PT/GUWw4sqZrGpRVrmWOAareD50e/Y+sH7gcG9Ohlxb8INw53ncqOqmMpB4C/qtKSJbk f8xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mime-version:content-disposition:user-agent; bh=1dxE3Qt8dHrgNn3we+1wUnY+iFbnSWXkgwd2433DiVQ=; b=FLogR9D7t3xSuzLuRTEcFI8uSPmL+y+hKcSuL1Vxoajh9gQwFwTl7aByABtm8U9njL hXmWGLFvk2N1tUR5rnpR/StRGm2YF94ptWXQmp2lIVlQHWCB0cH0Qc7QPe76t1e8hSMQ oDVwMHtGkXNsjusmyHZe/TBCVVbCmHokAPLJTW5ekU524oxtpRQOh6YohnAiJVmpU/HZ 7KV2MBe7b6knBogahqXTTL56HMvve0FvNe4DYQed1sefP2d3F9U7rJqxXnoCmVmh5eFe BvFc4Ekjz5N36dnNWSmfb6ujBW5zXWeT7RBloDPoHIK9DOHMzZdRUxfh0LgBXefosHi/ 7z4A== X-Gm-Message-State: AElRT7HB1FVeXnWibDzm12FUAdNza/lDxmpy72YYS3MIOWwClsY5UZJC IdAnTi40h4BbnILsGmo9AORQIFBT X-Google-Smtp-Source: AG47ELs74YI+BBaUpoJZXigch7/hFo8loE4wYpHrL0uQjUHsOjTIuZx0pnj5HfKF0TbKRBHPLHlIpg== X-Received: by 10.46.2.11 with SMTP id 11mr4747739ljc.0.1520015711508; Fri, 02 Mar 2018 10:35:11 -0800 (PST) Received: from x-wing (87-206-170-77.dynamic.chello.pl. [87.206.170.77]) by smtp.gmail.com with ESMTPSA id t28sm1446115lfd.71.2018.03.02.10.35.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 02 Mar 2018 10:35:09 -0800 (PST) Sender: Mariusz Zaborski Date: Fri, 2 Mar 2018 19:35:14 +0100 From: Mariusz Zaborski To: cl-capsicum-discuss@lists.cam.ac.uk Cc: freebsd-hackers@freebsd.org Subject: unlinkfd Message-ID: <20180302183514.GA99279@x-wing> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline User-Agent: Mutt/1.9.2 (2017-12-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Mar 2018 18:35:13 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hello, Today I would like to propose a new syscall called unlinkfd(2) which came up during a discussion with Ed Maste. Currently in UNIX we can=E2=80=99t remove files safely. If we will try to d= o so we always end up in a race condition. For example when we open a file, and che= ck it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the file we a= re trying to unlink could be a different one than the one we were fstating just a moment= ago. Another reason of implementing unlinkfd(2) came to us when we were trying to sandbox some applications like: uudecode/b64decode or bspatch. It occured to us that we don=E2=80=99t have a good way of removing single files. Of co= urse we can try to determine in which directory we are in, and then open this directory= and remove a single file. It looks even more bizarre if we would think about a program which operates= on multiple files. If we would analyze a situation with two totally different directories like `/tmp` and `/home/oshogbo` we would end up with pre opening a root directory or keeping as many directories as we are working on open. All of that effort only to remove two files. This make it totally impractic= al! I think that opening directories also presents some wider attack vector bec= ause we are keeping a single descriptor to a directory only to remove one file. Unfortunately this means that an attacker can remove all files in that dire= ctory. I proposed this as well on the last Capsicum call. There was a suggestion t= hat instead of doing a single syscall maybe we should have a Casper service that will allow us to remove files. Another idea was that we should perhaps rede= sign programs to create some subdirs work on the subdirs and then remove all fil= es in this subdir. I don=E2=80=99t feel that creating a Casper service is a good = idea because we still have exactly the same issue of race condition. In my opinion creat= ing subdirs is also a problem for us. First we would need to redesign some of our tools and I think we should simplyfiy capsicumizition of the process instead of making it harder. Secondly we can create a temporary subdirectory but what will remove it? We are going back to having a fd to directory in which we just created a su= bdir. Another way would be to have Casper service which would remove a directory = but with the risk of RC. In conclusion, I think we need syscall like unlinkfd(2), which turn out tah= t it is easy to implement. The only downside of this implementation is that we n= ot only need to provide a fd but also a path file. This is because inodes nor vnodes don=E2=80=99t contain filenames. We are comparing vnodes of the fd a= nd the given path, if they are exactly the same we remove a file. In the syscall we are = using a fd so there is no Ambient Authority because we are proving that we already have access to that file. Thanks to that the syscall can be safely used with Caspsicum. I have already discussed this with some people and they said `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s do some= thing with that idea! If you are intereted in patch you can find it here: https://reviews.freebsd.org/D14567 Thanks, --=20 Mariusz Zaborski oshogbo//vx | http://oshogbo.vexillium.org FreeBSD commiter | https://freebsd.org Software developer | http://wheelsystems.com If it's not broken, let's fix it till it is!!1 --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEkD1x0xkJXVVY1Gwf38KEGuLGxWQFAlqZmVkACgkQ38KEGuLG xWSNJQ/+Om5z6D8nvxGL2xCGDx8f/0zJ+z97AkzWtEMd+YSY2nUVVif/mIp3baqY GoqpESoKEOuXugk7gYL6uEN1LqMkNwxyP7bNoIOts8yLmtflDOa44z3rWkkDlzm8 VLtz9FfmL2NBOt5z+sQrUuliMlBOhCgmZzHdb7iCmja06cGo2hERtXsOUK7nOATq 2oxLzMs827tpTVU5Y62LnbHG0wdj9qbKJW77dF7xtZjh7iZZv76uebBZzyXm79s6 wiD8HV2kc5guvZSctF+f7xcnlN0vwQcpKEAXVfrzUJgYO/spySGcNehPSjAbUTE4 BDRKC192vYxGSYkiX+nDtCYXVCj+yy52T5ikdocB8Tl6CAb+68Jmp1zNXuVo8vP2 nEr0CSVwSWTd8Y8Shz6NIjYZ6KWjVYYlbyY315R1ycu/XzLwmGnjZmVD/DCa+7z0 WWZvfHQYhhQMiu8Jp5EMLfK2VWklVAQLVp3xIANEaxoWDjpJnfNM7WmXpn3m8rcO 68pogZ4O4iNtgZ9PTbgcczw7HAYkkNsv3A1tl1KJVaTGrHSA9xMaHe37sNANK5AH thb1kl0Mjw+gcKXnKAXpy1ev30nvXzjTLAlC52WTto0Y1KJyd37p2CCyTvFsFqai DOnqxbw6ZU/8XlRegnXxxSuvEdaChWFnLfYyNexVYsUW3phPVak= =LZpw -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF-- From owner-freebsd-hackers@freebsd.org Sat Mar 3 06:53:36 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 525D8F450B1 for ; Sat, 3 Mar 2018 06:53:36 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from david.siemens.de (david.siemens.de [192.35.17.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "david.siemens.de", Issuer "Siemens Issuing CA Internet Server 2017" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC801716BE; Sat, 3 Mar 2018 06:53:35 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w236i1DY023406 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 3 Mar 2018 07:44:01 +0100 Received: from curry.mchp.siemens.de (curry.mchp.siemens.de [139.25.40.130]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w236i0px013446; Sat, 3 Mar 2018 07:44:00 +0100 Received: (from user@localhost) by curry.mchp.siemens.de (8.15.2/8.15.2) id w236i0Eo005205; Date: Sat, 3 Mar 2018 07:44:00 +0100 From: Andre Albsmeier To: Daniel Eischen Cc: Andre Albsmeier , freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards Message-ID: <20180303064400.GA27337@bali> References: <20180302061852.GA7887@bali> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 06:53:36 -0000 On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: > On Fri, 2 Mar 2018, Andre Albsmeier wrote: > > > I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial > > ports) sitting here which does not get detected on 11.1. I tried > > to simply add it to the uart and ppc drivers with > > > [ ... ] > > Do you try adding similar support to puc_pci_devices[] in > sys/dev/puc/pucdata.c? Just tried that: @@ -1204,6 +1204,11 @@ PUC_PORT_1S1P, 0x10, 4, 0, }, +{ 0x9710, 0x9912, 0xa000, 0x3012, + "NetMos NM9912 Dual UART and 1284 Printer port", + DEFAULT_RCLK, + PUC_PORT_2S1P, 0x10, 4, 0, +}, { 0x9710, 0x9865, 0xa000, 0x3012, "NetMos NM9865 Dual UART and 1284 Printer port", DEFAULT_RCLK, But the results are exactly the same. It also doesn't matter if puc.ko is loaded at all. -Andre From owner-freebsd-hackers@freebsd.org Sat Mar 3 11:28:47 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE5E4F32C8F for ; Sat, 3 Mar 2018 11:28:46 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: from mail-wr0-x231.google.com (mail-wr0-x231.google.com [IPv6:2a00:1450:400c:c0c::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5422F7C687; Sat, 3 Mar 2018 11:28:46 +0000 (UTC) (envelope-from etnapierala@gmail.com) Received: by mail-wr0-x231.google.com with SMTP id k9so12548316wre.9; Sat, 03 Mar 2018 03:28:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:mail-followup-to :references:mime-version:content-disposition:in-reply-to:user-agent; bh=d85evfgHq/nldJ+7l4GsN8hG90XgeQv00l3yR5aUo3o=; b=DlXbl1F9s8M0tlSbiHePpsVV9cu+yvpGbxSioj/dPK6QxAsyrusFcMGENI9zNr+sjQ dVyQ9mRZnz5lj4Yw97gH/Bx71SJaa8+CtvLBz9FO31zXpmN0PU3jelWfmIukGnXW1dkE YjGfe33pmlXEACpLkDUhYw34sDwIHXu9/IOx9IRvBO+i/JkF1xXobc3YRn3nb4TnxIXW Q3j+dTAATBDeHREV246/CGu5GLlNOdgboe1klnmYLQtXXZ7O2zKaOd/UxZQbI1Pj2rEM WdhfQFdwwhyJ9zIZ5zauaHejj30JS/rlo2jw8Lwwh+7TopeH11TXnk6JKD9xRda//GlS HuMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=d85evfgHq/nldJ+7l4GsN8hG90XgeQv00l3yR5aUo3o=; b=uEM7yakljUkmyDHGXQjT+hVkREnEuETG4ZNhWRprku29uqpexSSIu+0JAuROUeg1Rs SM02pvULe2YCRotU3ehHw8hR9y05tzwKWqpOhdT5v66a8RY/jS2hKf4DLaVwM2ZzK+zj dz/T/M42K1uGIeNY8gwqKFFARaXARTpny2GqTnrDbGM1Z/lBD8xeWjRjsabMTIf48xSZ x0n6jX2zea/V7Jdq7BiDBgmeSgIYZaSecsQGTeD8nXIZDFocVwI1AX5T2fNaoxIampjP ITbngZnw3fiMwg5uJFqn6JuEQTgMyUPry235EndAkMI07Ut+O8q1EFm6ApZMYNwLPQ0u urzA== X-Gm-Message-State: APf1xPA/fPhyCoWfUom89IblMeZSMGDMaxsBQGXeviOjU8ZCafv04gUW YOy3YGU5wRtWo9QNBs2D1XzjZw== X-Google-Smtp-Source: AG47ELvoiWDBd3x1WoxeU/omWhySNa9aU6acUCKEw09Iizcpq8SWrZjos1w+j3OLeuZ2yNNvTbrfMA== X-Received: by 10.223.187.72 with SMTP id x8mr7103405wrg.217.1520076524186; Sat, 03 Mar 2018 03:28:44 -0800 (PST) Received: from brick (cpc92302-cmbg19-2-0-cust461.5-4.cable.virginm.net. [82.1.209.206]) by smtp.gmail.com with ESMTPSA id y6sm2582568wmy.14.2018.03.03.03.28.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 Mar 2018 03:28:43 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Date: Sat, 3 Mar 2018 11:28:41 +0000 From: Edward Tomasz =?utf-8?Q?Napiera=C5=82a?= To: Alan Somers Cc: Lars Engels , Joe Maloney , "freebsd-hackers@freebsd.org" Subject: Re: OpenRC 0.35 for FreeBSD Message-ID: <20180303112841.GA2029@brick> Mail-Followup-To: Alan Somers , Lars Engels , Joe Maloney , "freebsd-hackers@freebsd.org" References: <20180302083756.GH34685@e.0x20.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.3 (2018-01-21) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 11:28:47 -0000 On 0302T0828, Alan Somers wrote: > On Fri, Mar 2, 2018 at 1:37 AM, Lars Engels wrote: > > > On Thu, Mar 01, 2018 at 08:02:23PM -0500, Joe Maloney wrote: > > > Hello hackers, > > > I have been working on a single diff version of OpenRC for FreeBSD: > > > > > > https://github.com/pkgdemon/freebsd/commit/ > > b6885cd533c848a1b4f3582f48e40c883669b35c > > > > Thanks for your work! > > > > > > > > Why OpenRC? The licensing is right, and it's a way of adding modern > > > features to service management without reinventing the wheel. That's > > > my sales pitch. > > > > Hm, that does not convince me. FreeBSD's rc is also BSD licensed and did > > not reinvent any wheel. > > Could you maybe give some comparison between OpenRC and our rc? What > > does OpenRC better? > > > > Thanks! > > > > Lars > > > > Parallel startup, mainly. Some time ago I played with an idea of making it possible within our existing rcNG infrastructure. I've modified rcorder(8) to add a "-p" flag, which modifies its output from the current "one script per line" to "list of scripts that can be run in parallel per line", using the existing rc metadata ("PROVIDES" et al), and modified /etc/rc to make use of that. I had to give it up due to other obligations, but should anyone want to pick it up, the code is here: https://reviews.freebsd.org/D3715 There's also an unrelated review for the "supervise" functionality for rc scripts, which includes changes for sshd, cron, and syslogd: https://reviews.freebsd.org/D7474 From owner-freebsd-hackers@freebsd.org Sat Mar 3 09:53:27 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BE45F2BE72 for ; Sat, 3 Mar 2018 09:53:27 +0000 (UTC) (envelope-from justin@specialbusservice.com) Received: from mail-vk0-x229.google.com (mail-vk0-x229.google.com [IPv6:2607:f8b0:400c:c05::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 26E8377FB9 for ; Sat, 3 Mar 2018 09:53:27 +0000 (UTC) (envelope-from justin@specialbusservice.com) Received: by mail-vk0-x229.google.com with SMTP id t126so7143247vkb.11 for ; Sat, 03 Mar 2018 01:53:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=specialbusservice.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=OMHDWcJR+ZNc7+qiSY1kzOvqmZVsR68yYjd4B0GBmpE=; b=GJG8yaFaicDaMbt08IHRK1gDRaVeRDSaz1Jzwk0vDrUx+vE+dWyWnaUqFeJHrilPLT 4jiHS0Gvd+dAryGn3LDGN75UUjExpFJjg78I04wIJj3OoSZcAwzt0e8b1l7zqPDPdvNe s6cQtjnHmuxCW3cWHpiG0HDXagkhZP5XUASsU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=OMHDWcJR+ZNc7+qiSY1kzOvqmZVsR68yYjd4B0GBmpE=; b=FSgzFh4lGS0x0IVVRQvi2uy0EGaN8wrfqVaUp4+DeziIPUpOrdic9a9AgVxz328VIt nQNix24Vw3dBhMbfOpbGkjUEsXYrxgNKwO27savDxCirFYMjfP51fBgp/UQwlqIWC12x 1/E/mFINO4ipgobP71mtG1Nl5EyYT1+Ac9OAiN5rHzzVos86/ru7lhFntpgyCJsLCbME 2uPKtS90bXEA1CTo5yN2VpxUBo5kdzQJmSgS+93W43isx/URlzPdz3PD1F5K/RvNSwtI WjPpCIn5pmxhH0uSwwfSZI//L82ATcU6tl/unw8WcaG6bTWCz1wnBv/XLj/8I5x3gZq8 4t2A== X-Gm-Message-State: APf1xPD/7r5OLnVjideKMkELYXgVu0dbYXAvK18fMYE2WiB5QnzioELf AaDFZojlon968cCGurmm01VaeiRpV7D5rFlD0VvhPA== X-Google-Smtp-Source: AG47ELuKmA9wUBGB6aGqaAKkDEtrUKIU/o75hHOQX+P7/c2JBherxn893ug67syrCnCaTzIOH8Axv3Z281ZZ+ICUIvU= X-Received: by 10.31.242.75 with SMTP id q72mr5716632vkh.188.1520070806355; Sat, 03 Mar 2018 01:53:26 -0800 (PST) MIME-Version: 1.0 Received: by 10.103.2.81 with HTTP; Sat, 3 Mar 2018 01:53:25 -0800 (PST) In-Reply-To: <20180302183514.GA99279@x-wing> References: <20180302183514.GA99279@x-wing> From: Justin Cormack Date: Sat, 3 Mar 2018 09:53:25 +0000 Message-ID: Subject: Re: [capsicum] unlinkfd To: Mariusz Zaborski Cc: "" , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Sat, 03 Mar 2018 13:27:55 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 09:53:27 -0000 I think it would make sense to have an unlinkfd() that unlinks the file fro= m everywhere, so it does not need a name to be specified. This might be hard to implement. For temporary files, I really like Linux memfd_create(2) that opens an anon= ymous file without a name. This semantics is really useful. (Linux memfd also has additional options for sealing the file fo make it immutable which are very useful for safely passing files between processes.) Having a way to make unnamed temporary files solves a lot of deletion issues as the file never needs to be unlinked. On 2 March 2018 at 18:35, Mariusz Zaborski wrote: > Hello, > > Today I would like to propose a new syscall called unlinkfd(2) which came= up > during a discussion with Ed Maste. > > Currently in UNIX we can=E2=80=99t remove files safely. If we will try to= do so we > always end up in a race condition. For example when we open a file, and c= heck > it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the file we= are trying to > unlink could be a different one than the one we were fstating just a mome= nt ago. > > Another reason of implementing unlinkfd(2) came to us when we were trying > to sandbox some applications like: uudecode/b64decode or bspatch. It occu= red > to us that we don=E2=80=99t have a good way of removing single files. Of = course we can > try to determine in which directory we are in, and then open this directo= ry and > remove a single file. > > It looks even more bizarre if we would think about a program which operat= es on > multiple files. If we would analyze a situation with two totally differen= t > directories like `/tmp` and `/home/oshogbo` we would end up with pre open= ing > a root directory or keeping as many directories as we are working on open= . > All of that effort only to remove two files. This make it totally impract= ical! > > I think that opening directories also presents some wider attack vector b= ecause > we are keeping a single descriptor to a directory only to remove one file= . > Unfortunately this means that an attacker can remove all files in that di= rectory. > > I proposed this as well on the last Capsicum call. There was a suggestion= that > instead of doing a single syscall maybe we should have a Casper service t= hat > will allow us to remove files. Another idea was that we should perhaps re= design > programs to create some subdirs work on the subdirs and then remove all f= iles in > this subdir. I don=E2=80=99t feel that creating a Casper service is a goo= d idea because > we still have exactly the same issue of race condition. In my opinion cre= ating > subdirs is also a problem for us. > > First we would need to redesign some of our tools and I think we should > simplyfiy capsicumizition of the process instead of making it harder. > > Secondly we can create a temporary subdirectory but what will remove it? > We are going back to having a fd to directory in which we just created a = subdir. > Another way would be to have Casper service which would remove a director= y but > with the risk of RC. > > In conclusion, I think we need syscall like unlinkfd(2), which turn out t= aht it > is easy to implement. The only downside of this implementation is that we= not > only need to provide a fd but also a path file. This is because inodes no= r > vnodes don=E2=80=99t contain filenames. We are comparing vnodes of the fd= and the given > path, if they are exactly the same we remove a file. In the syscall we ar= e using > a fd so there is no Ambient Authority because we are proving that we alre= ady > have access to that file. Thanks to that the syscall can be safely used w= ith > Caspsicum. I have already discussed this with some people and they said > `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s do so= mething with that idea! > If you are intereted in patch you can find it here: > https://reviews.freebsd.org/D14567 > > Thanks, > -- > Mariusz Zaborski > oshogbo//vx | http://oshogbo.vexillium.org > FreeBSD commiter | https://freebsd.org > Software developer | http://wheelsystems.com > If it's not broken, let's fix it till it is!!1 From owner-freebsd-hackers@freebsd.org Sat Mar 3 10:41:14 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8B97F2F5CC for ; Sat, 3 Mar 2018 10:41:13 +0000 (UTC) (envelope-from robert.watson@cl.cam.ac.uk) Received: from cyrus.watson.org (cyrus.watson.org [204.107.128.30]) by mx1.freebsd.org (Postfix) with ESMTP id 9CA2D7A45A; Sat, 3 Mar 2018 10:41:13 +0000 (UTC) (envelope-from robert.watson@cl.cam.ac.uk) Received: from [10.0.1.23] (host109-151-50-63.range109-151.btcentralplus.com [109.151.50.63]) by cyrus.watson.org (Postfix) with ESMTPSA id 0E879A16E5; Sat, 3 Mar 2018 10:41:11 +0000 (UTC) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: [capsicum] unlinkfd From: "Robert N. M. Watson" In-Reply-To: Date: Sat, 3 Mar 2018 10:41:07 +0000 Cc: Mariusz Zaborski , "" , freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> References: <20180302183514.GA99279@x-wing> To: Justin Cormack X-Mailer: Apple Mail (2.3445.5.20) X-Mailman-Approved-At: Sat, 03 Mar 2018 13:30:35 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 10:41:14 -0000 FWIW, this is part of why we introduced anonymous POSIX shared memory = objects with Capsicum in FreeBSD -- we allow shm_open(2) to be passed a = SHM_ANON special name, which causes the creation of a swap-backed, = mappable file-like object that can have I/O, memory mapping, etc, = performed on it .. but never has any persistent state across reboots = even in the event of a crash. With Capsicum you can then refine a file descriptor to the otherwise = writable object to be read-only for the purposes of delegation. There is = not, however, a mechanism to "freeze" the state of the object causing = other outstanding writable descriptors to become read-only -- certainly = something could be added, but some care regarding VM semantics would be = required -- in particular, so that faults could not be experienced as a = result of an memory store performed before the "freeze" but issued to = VFS only later. I certainly have no objection to an unlinkat(2) system call -- it's = unfortunate that a full suite of the at(2) APIs wasn't introduced in the = first place. It would be worth checking that no one else (e.g., Solaris, = Mac OS X, Linux) hasn't already added an unlinkat(2) that we can match = API semantics for. I think I take the view that for truly anonymous = objects, shm_open(2) without a name (or the Linux equiv) is the right = thing -- and hence unlinkat(2) is for more conventional use cases where = the final pathname element is known. On directories: There, I find myself falling back on a Casper-like = service, since GC'ing a single anonymous memory object is = straightforward, but GC'ing a directory hierarchy is a more messy = business. Robert > On 3 Mar 2018, at 09:53, Justin Cormack = wrote: >=20 > I think it would make sense to have an unlinkfd() that unlinks the = file from > everywhere, so it does not need a name to be specified. This might be > hard to implement. >=20 > For temporary files, I really like Linux memfd_create(2) that opens an = anonymous > file without a name. This semantics is really useful. (Linux memfd = also has > additional options for sealing the file fo make it immutable which are = very > useful for safely passing files between processes.) Having a way to = make > unnamed temporary files solves a lot of deletion issues as the file > never needs to > be unlinked. >=20 >=20 > On 2 March 2018 at 18:35, Mariusz Zaborski = wrote: >> Hello, >>=20 >> Today I would like to propose a new syscall called unlinkfd(2) which = came up >> during a discussion with Ed Maste. >>=20 >> Currently in UNIX we can=E2=80=99t remove files safely. If we will = try to do so we >> always end up in a race condition. For example when we open a file, = and check >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the = file we are trying to >> unlink could be a different one than the one we were fstating just a = moment ago. >>=20 >> Another reason of implementing unlinkfd(2) came to us when we were = trying >> to sandbox some applications like: uudecode/b64decode or bspatch. It = occured >> to us that we don=E2=80=99t have a good way of removing single files. = Of course we can >> try to determine in which directory we are in, and then open this = directory and >> remove a single file. >>=20 >> It looks even more bizarre if we would think about a program which = operates on >> multiple files. If we would analyze a situation with two totally = different >> directories like `/tmp` and `/home/oshogbo` we would end up with pre = opening >> a root directory or keeping as many directories as we are working on = open. >> All of that effort only to remove two files. This make it totally = impractical! >>=20 >> I think that opening directories also presents some wider attack = vector because >> we are keeping a single descriptor to a directory only to remove one = file. >> Unfortunately this means that an attacker can remove all files in = that directory. >>=20 >> I proposed this as well on the last Capsicum call. There was a = suggestion that >> instead of doing a single syscall maybe we should have a Casper = service that >> will allow us to remove files. Another idea was that we should = perhaps redesign >> programs to create some subdirs work on the subdirs and then remove = all files in >> this subdir. I don=E2=80=99t feel that creating a Casper service is a = good idea because >> we still have exactly the same issue of race condition. In my opinion = creating >> subdirs is also a problem for us. >>=20 >> First we would need to redesign some of our tools and I think we = should >> simplyfiy capsicumizition of the process instead of making it harder. >>=20 >> Secondly we can create a temporary subdirectory but what will remove = it? >> We are going back to having a fd to directory in which we just = created a subdir. >> Another way would be to have Casper service which would remove a = directory but >> with the risk of RC. >>=20 >> In conclusion, I think we need syscall like unlinkfd(2), which turn = out taht it >> is easy to implement. The only downside of this implementation is = that we not >> only need to provide a fd but also a path file. This is because = inodes nor >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of = the fd and the given >> path, if they are exactly the same we remove a file. In the syscall = we are using >> a fd so there is no Ambient Authority because we are proving that we = already >> have access to that file. Thanks to that the syscall can be safely = used with >> Caspsicum. I have already discussed this with some people and they = said >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s = do something with that idea! >> If you are intereted in patch you can find it here: >> https://reviews.freebsd.org/D14567 >>=20 >> Thanks, >> -- >> Mariusz Zaborski >> oshogbo//vx | http://oshogbo.vexillium.org >> FreeBSD commiter | https://freebsd.org >> Software developer | http://wheelsystems.com >> If it's not broken, let's fix it till it is!!1 >=20 From owner-freebsd-hackers@freebsd.org Sat Mar 3 10:49:28 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 705C7F2FFFE for ; Sat, 3 Mar 2018 10:49:28 +0000 (UTC) (envelope-from david@gwynne.id.au) Received: from pb-smtp1.pobox.com (pb-smtp1.pobox.com [64.147.108.70]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1E6BC7ACAF; Sat, 3 Mar 2018 10:49:27 +0000 (UTC) (envelope-from david@gwynne.id.au) Received: from pb-smtp1.pobox.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id 159D2BBDBA; Sat, 3 Mar 2018 05:49:21 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=pobox.com; h=content-type :mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; s=sasl; bh= s3yoZS/ndR6DlUHJBOktj1LPrC8=; b=QgI6NBv5KAUphwYFmJLe/LF8u5dSjkYw HplHJbLgib8IbkbdspSHOgi/Xd8yE4F5+/dh4B9jQ3Ijpxr83m2Zx+mC9tgFmCVS pFRjBVw3dTmNr+K9/jltBDaE1nFRu1fDlsZLMNGFC7gee5tLEojZb+uA9g7yc+xw DPhKBYVB2Ww= Received: from pb-smtp1.nyi.icgroup.com (unknown [127.0.0.1]) by pb-smtp1.pobox.com (Postfix) with ESMTP id 0ED68BBDB9; Sat, 3 Mar 2018 05:49:21 -0500 (EST) Received: from [10.0.127.67] (unknown [60.241.41.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by pb-smtp1.pobox.com (Postfix) with ESMTPSA id E1C3EBBDB7; Sat, 3 Mar 2018 05:49:18 -0500 (EST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: [capsicum] unlinkfd From: David Gwynne In-Reply-To: Date: Sat, 3 Mar 2018 20:49:43 +1000 Cc: Mariusz Zaborski , "" , freebsd-hackers@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20180302183514.GA99279@x-wing> To: Justin Cormack X-Mailer: Apple Mail (2.3445.5.20) X-Pobox-Relay-ID: 86DE5CB0-1ED0-11E8-8C66-44CE1968708C-51140664!pb-smtp1.pobox.com X-Mailman-Approved-At: Sat, 03 Mar 2018 13:37:34 +0000 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 10:49:28 -0000 > On 3 Mar 2018, at 7:53 pm, Justin Cormack = wrote: >=20 > I think it would make sense to have an unlinkfd() that unlinks the = file from > everywhere, so it does not need a name to be specified. This might be > hard to implement. >=20 > For temporary files, I really like Linux memfd_create(2) that opens an = anonymous > file without a name. This semantics is really useful. (Linux memfd = also has > additional options for sealing the file fo make it immutable which are = very > useful for safely passing files between processes.) Having a way to = make > unnamed temporary files solves a lot of deletion issues as the file > never needs to > be unlinked. maybe you could get close enough to that with a new flag for = open(2)/openat(2). eg, open("/backing/mount/point/randomname", = O_CREAT|O_UNLINK); >=20 >=20 > On 2 March 2018 at 18:35, Mariusz Zaborski = wrote: >> Hello, >>=20 >> Today I would like to propose a new syscall called unlinkfd(2) which = came up >> during a discussion with Ed Maste. >>=20 >> Currently in UNIX we can=E2=80=99t remove files safely. If we will = try to do so we >> always end up in a race condition. For example when we open a file, = and check >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the = file we are trying to >> unlink could be a different one than the one we were fstating just a = moment ago. >>=20 >> Another reason of implementing unlinkfd(2) came to us when we were = trying >> to sandbox some applications like: uudecode/b64decode or bspatch. It = occured >> to us that we don=E2=80=99t have a good way of removing single files. = Of course we can >> try to determine in which directory we are in, and then open this = directory and >> remove a single file. >>=20 >> It looks even more bizarre if we would think about a program which = operates on >> multiple files. If we would analyze a situation with two totally = different >> directories like `/tmp` and `/home/oshogbo` we would end up with pre = opening >> a root directory or keeping as many directories as we are working on = open. >> All of that effort only to remove two files. This make it totally = impractical! >>=20 >> I think that opening directories also presents some wider attack = vector because >> we are keeping a single descriptor to a directory only to remove one = file. >> Unfortunately this means that an attacker can remove all files in = that directory. >>=20 >> I proposed this as well on the last Capsicum call. There was a = suggestion that >> instead of doing a single syscall maybe we should have a Casper = service that >> will allow us to remove files. Another idea was that we should = perhaps redesign >> programs to create some subdirs work on the subdirs and then remove = all files in >> this subdir. I don=E2=80=99t feel that creating a Casper service is a = good idea because >> we still have exactly the same issue of race condition. In my opinion = creating >> subdirs is also a problem for us. >>=20 >> First we would need to redesign some of our tools and I think we = should >> simplyfiy capsicumizition of the process instead of making it harder. >>=20 >> Secondly we can create a temporary subdirectory but what will remove = it? >> We are going back to having a fd to directory in which we just = created a subdir. >> Another way would be to have Casper service which would remove a = directory but >> with the risk of RC. >>=20 >> In conclusion, I think we need syscall like unlinkfd(2), which turn = out taht it >> is easy to implement. The only downside of this implementation is = that we not >> only need to provide a fd but also a path file. This is because = inodes nor >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of = the fd and the given >> path, if they are exactly the same we remove a file. In the syscall = we are using >> a fd so there is no Ambient Authority because we are proving that we = already >> have access to that file. Thanks to that the syscall can be safely = used with >> Caspsicum. I have already discussed this with some people and they = said >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s = do something with that idea! >> If you are intereted in patch you can find it here: >> https://reviews.freebsd.org/D14567 >>=20 >> Thanks, >> -- >> Mariusz Zaborski >> oshogbo//vx | http://oshogbo.vexillium.org >> FreeBSD commiter | https://freebsd.org >> Software developer | http://wheelsystems.com >> If it's not broken, let's fix it till it is!!1 >=20 From owner-freebsd-hackers@freebsd.org Sat Mar 3 11:21:31 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87145F325D8 for ; Sat, 3 Mar 2018 11:21:31 +0000 (UTC) (envelope-from benl@google.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1BDD17C26C for ; Sat, 3 Mar 2018 11:21:31 +0000 (UTC) (envelope-from benl@google.com) Received: by mail-it0-x232.google.com with SMTP id w63so4459320ita.3 for ; Sat, 03 Mar 2018 03:21:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+nw0QLwXyx2oWj8ZRbnBmccX+ahAV9qBipJELGpnd0Y=; b=sUutoUbZvamX9Z3lRAzSeciAbcWhZ6H8ARY0KbNlyzMubdTkS+WNBEmGn8eJIY6gDE i0Jn1nrnouzu0OipPHs+tUpN4MeaVpMLdKEtKaZBq9qLSIQu+d9noeE95QaWkE1/1FNo naWZc8WnKf08SQwPXYdIexhPYZQ3YhSMCyJW679P6rP/6vKnvjrIaEn/stBgn98bNe1O V/HMaSae8ATo0HnG8KfriIlzL7q7z8ieKCehoPYbW8h/6+W5W6daH1HhiEWn+p8e6MPT cZGMH+wH3BY9lrpbM038g+2q17CiKZ6TJrpDtDnhOFqp5fxg2h889j1WaPBotXdlmkIg OTvg== X-Gm-Message-State: AElRT7EStN+l3w5H585POQi/zyFDgK/9fYmuvwdIp23rjenbLExOF5SS HH+8IMon52PVYjzKaGs6nhg/9abltkfDQNNj0kDHrURP X-Google-Smtp-Source: AG47ELsFf/RAmyz8H27hKl/fj6T6sKP2k0P4gOBPzgPt3lEYiEBiifu/ifR5E/zCmBvP2k0pXQKXM3stDmHQrhEjQKQ= X-Received: by 10.36.48.135 with SMTP id q129mr6300422itq.23.1520076090129; Sat, 03 Mar 2018 03:21:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.167.204 with HTTP; Sat, 3 Mar 2018 03:21:29 -0800 (PST) In-Reply-To: <20180302183514.GA99279@x-wing> References: <20180302183514.GA99279@x-wing> From: Ben Laurie Date: Sat, 3 Mar 2018 11:21:29 +0000 Message-ID: Subject: Re: [capsicum] unlinkfd To: Mariusz Zaborski Cc: "" , freebsd-hackers@freebsd.org X-Mailman-Approved-At: Sat, 03 Mar 2018 13:42:07 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 11:21:31 -0000 On 2 March 2018 at 18:35, Mariusz Zaborski wrote: > Hello, > > Today I would like to propose a new syscall called unlinkfd(2) which came > up > during a discussion with Ed Maste. > > Currently in UNIX we can=E2=80=99t remove files safely. If we will try to= do so we > always end up in a race condition. For example when we open a file, and > check > it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the file we= are > trying to > unlink could be a different one than the one we were fstating just a > moment ago. > > Another reason of implementing unlinkfd(2) came to us when we were trying > to sandbox some applications like: uudecode/b64decode or bspatch. It > occured > to us that we don=E2=80=99t have a good way of removing single files. Of = course we > can > try to determine in which directory we are in, and then open this > directory and > remove a single file. > > It looks even more bizarre if we would think about a program which > operates on > multiple files. If we would analyze a situation with two totally differen= t > directories like `/tmp` and `/home/oshogbo` we would end up with pre > opening > a root directory or keeping as many directories as we are working on open= . > All of that effort only to remove two files. This make it totally > impractical! > > I think that opening directories also presents some wider attack vector > because > we are keeping a single descriptor to a directory only to remove one file= . > Unfortunately this means that an attacker can remove all files in that > directory. > > I proposed this as well on the last Capsicum call. There was a suggestion > that > instead of doing a single syscall maybe we should have a Casper service > that > will allow us to remove files. Another idea was that we should perhaps > redesign > programs to create some subdirs work on the subdirs and then remove all > files in > this subdir. I don=E2=80=99t feel that creating a Casper service is a goo= d idea > because > we still have exactly the same issue of race condition. In my opinion > creating > subdirs is also a problem for us. > > First we would need to redesign some of our tools and I think we should > simplyfiy capsicumizition of the process instead of making it harder. > > Secondly we can create a temporary subdirectory but what will remove it? > We are going back to having a fd to directory in which we just created a > subdir. > Another way would be to have Casper service which would remove a director= y > but > with the risk of RC. > > In conclusion, I think we need syscall like unlinkfd(2), which turn out > taht it > is easy to implement. The only downside of this implementation is that we > not > only need to provide a fd but also a path file. This is because inodes no= r > vnodes don=E2=80=99t contain filenames. We are comparing vnodes of the fd= and the > given > path, if they are exactly the same we remove a file. In the syscall we ar= e > using > a fd so there is no Ambient Authority because we are proving that we > already > have access to that file. That seems incorrect. You are proving you have access to the inode, not the directory entry. So, for example, I could create a link to a file I wanted to remove, that I don't have permission to remove, then use this call to unlink it. > Thanks to that the syscall can be safely used with > Caspsicum. I have already discussed this with some people and they said > `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s do so= mething with that > idea! > If you are intereted in patch you can find it here: > https://reviews.freebsd.org/D14567 > > Thanks, > -- > Mariusz Zaborski > oshogbo//vx | http://oshogbo.vexillium.org > FreeBSD commiter | https://freebsd.org > Software developer | http://wheelsystems.com > If it's not broken, let's fix it till it is!!1 > From owner-freebsd-hackers@freebsd.org Sat Mar 3 12:16:40 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8E04AF36B4C for ; Sat, 3 Mar 2018 12:16:40 +0000 (UTC) (envelope-from robert.watson@cl.cam.ac.uk) Received: from cyrus.watson.org (cyrus.watson.org [204.107.128.30]) by mx1.freebsd.org (Postfix) with ESMTP id 381437E732; Sat, 3 Mar 2018 12:16:40 +0000 (UTC) (envelope-from robert.watson@cl.cam.ac.uk) Received: from [10.0.1.23] (host109-151-50-63.range109-151.btcentralplus.com [109.151.50.63]) by cyrus.watson.org (Postfix) with ESMTPSA id E64C11E59F; Sat, 3 Mar 2018 12:16:38 +0000 (UTC) From: "Robert N. M. Watson" Message-Id: <1ED213FC-D0CA-46D9-B6D1-EA261F2B80F5@cl.cam.ac.uk> Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: [capsicum] unlinkfd Date: Sat, 3 Mar 2018 12:16:34 +0000 In-Reply-To: Cc: Justin Cormack , "" , freebsd-hackers@freebsd.org, Mariusz Zaborski To: Alex Richardson References: <20180302183514.GA99279@x-wing> <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> X-Mailer: Apple Mail (2.3445.5.20) X-Mailman-Approved-At: Sat, 03 Mar 2018 15:07:03 +0000 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 12:16:40 -0000 In general, in UNIX, "unlink" is a namespace operation relative to a = directory, and not an operation on a file, so I wouldn't expect to have = a system call that searches a directory looking for a matching file, but = rather always a call that specifies the specific segment to remove (as = there may well be more than one of them). It seems to me like there are a few different use cases: (1) Just want some temporary non-persistent file-like storage please. = Here, swap-backed anonymous objects are probably generally preferable, = although if they will be huge, perhaps a filesystem is a better place to = back them. (2) Want a temporary (non-persistent) hierarchal namespace full of = file-like things. This need is not well served, as you need to not only = create this within a current filesystem, but garbage collection of the = results is not reliable in the presence of crashes/etc. (3) Want capability-based access to a persistent hierarchal namespace = full of files. This is well served by the current at(2) system calls = along with filesystems, although there are API gaps (e.g., a lack of = unlinkat(2) in FreeBSD). Because of the complexity of (2), a Casper service is likely the way to = go. We should fill the API gaps on (3) through new POSIX-like at(2). For = (1), the real issue is if the current swap-backed APIs are insufficient, = in which case a Casper service might be the way to go. Robert > On 3 Mar 2018, at 11:31, Alex Richardson = wrote: >=20 > Linux has a unlinkat() system call = (https://linux.die.net/man/2/unlinkat = ) but it doesn't seem to have a = flag that lets you unlink the fd itself. > Possibly pathname =3D=3D NULL and AT_EMPTY_PATH could mean unlink the = fd but I haven't tried whether that works. > It also has a AT_REMOVEDIR flag to make it function as rmdirat(). >=20 > Alex >=20 > On 3 March 2018 at 10:41, Robert N. M. Watson = > wrote: > FWIW, this is part of why we introduced anonymous POSIX shared memory = objects with Capsicum in FreeBSD -- we allow shm_open(2) to be passed a = SHM_ANON special name, which causes the creation of a swap-backed, = mappable file-like object that can have I/O, memory mapping, etc, = performed on it .. but never has any persistent state across reboots = even in the event of a crash. >=20 > With Capsicum you can then refine a file descriptor to the otherwise = writable object to be read-only for the purposes of delegation. There is = not, however, a mechanism to "freeze" the state of the object causing = other outstanding writable descriptors to become read-only -- certainly = something could be added, but some care regarding VM semantics would be = required -- in particular, so that faults could not be experienced as a = result of an memory store performed before the "freeze" but issued to = VFS only later. >=20 > I certainly have no objection to an unlinkat(2) system call -- it's = unfortunate that a full suite of the at(2) APIs wasn't introduced in the = first place. It would be worth checking that no one else (e.g., Solaris, = Mac OS X, Linux) hasn't already added an unlinkat(2) that we can match = API semantics for. I think I take the view that for truly anonymous = objects, shm_open(2) without a name (or the Linux equiv) is the right = thing -- and hence unlinkat(2) is for more conventional use cases where = the final pathname element is known. >=20 > On directories: There, I find myself falling back on a Casper-like = service, since GC'ing a single anonymous memory object is = straightforward, but GC'ing a directory hierarchy is a more messy = business. >=20 > Robert >=20 > > On 3 Mar 2018, at 09:53, Justin Cormack = > = wrote: > > > > I think it would make sense to have an unlinkfd() that unlinks the = file from > > everywhere, so it does not need a name to be specified. This might = be > > hard to implement. > > > > For temporary files, I really like Linux memfd_create(2) that opens = an anonymous > > file without a name. This semantics is really useful. (Linux memfd = also has > > additional options for sealing the file fo make it immutable which = are very > > useful for safely passing files between processes.) Having a way to = make > > unnamed temporary files solves a lot of deletion issues as the file > > never needs to > > be unlinked. > > > > > > On 2 March 2018 at 18:35, Mariusz Zaborski > wrote: > >> Hello, > >> > >> Today I would like to propose a new syscall called unlinkfd(2) = which came up > >> during a discussion with Ed Maste. > >> > >> Currently in UNIX we can=E2=80=99t remove files safely. If we will = try to do so we > >> always end up in a race condition. For example when we open a file, = and check > >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the = file we are trying to > >> unlink could be a different one than the one we were fstating just = a moment ago. > >> > >> Another reason of implementing unlinkfd(2) came to us when we were = trying > >> to sandbox some applications like: uudecode/b64decode or bspatch. = It occured > >> to us that we don=E2=80=99t have a good way of removing single = files. Of course we can > >> try to determine in which directory we are in, and then open this = directory and > >> remove a single file. > >> > >> It looks even more bizarre if we would think about a program which = operates on > >> multiple files. If we would analyze a situation with two totally = different > >> directories like `/tmp` and `/home/oshogbo` we would end up with = pre opening > >> a root directory or keeping as many directories as we are working = on open. > >> All of that effort only to remove two files. This make it totally = impractical! > >> > >> I think that opening directories also presents some wider attack = vector because > >> we are keeping a single descriptor to a directory only to remove = one file. > >> Unfortunately this means that an attacker can remove all files in = that directory. > >> > >> I proposed this as well on the last Capsicum call. There was a = suggestion that > >> instead of doing a single syscall maybe we should have a Casper = service that > >> will allow us to remove files. Another idea was that we should = perhaps redesign > >> programs to create some subdirs work on the subdirs and then remove = all files in > >> this subdir. I don=E2=80=99t feel that creating a Casper service is = a good idea because > >> we still have exactly the same issue of race condition. In my = opinion creating > >> subdirs is also a problem for us. > >> > >> First we would need to redesign some of our tools and I think we = should > >> simplyfiy capsicumizition of the process instead of making it = harder. > >> > >> Secondly we can create a temporary subdirectory but what will = remove it? > >> We are going back to having a fd to directory in which we just = created a subdir. > >> Another way would be to have Casper service which would remove a = directory but > >> with the risk of RC. > >> > >> In conclusion, I think we need syscall like unlinkfd(2), which turn = out taht it > >> is easy to implement. The only downside of this implementation is = that we not > >> only need to provide a fd but also a path file. This is because = inodes nor > >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of = the fd and the given > >> path, if they are exactly the same we remove a file. In the syscall = we are using > >> a fd so there is no Ambient Authority because we are proving that = we already > >> have access to that file. Thanks to that the syscall can be safely = used with > >> Caspsicum. I have already discussed this with some people and they = said > >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s = do something with that idea! > >> If you are intereted in patch you can find it here: > >> https://reviews.freebsd.org/D14567 = > >> > >> Thanks, > >> -- > >> Mariusz Zaborski > >> oshogbo//vx | http://oshogbo.vexillium.org = > >> FreeBSD commiter | https://freebsd.org = > >> Software developer | http://wheelsystems.com = > >> If it's not broken, let's fix it till it is!!1 > > >=20 >=20 >=20 From owner-freebsd-hackers@freebsd.org Sat Mar 3 12:46:37 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E446F390F2 for ; Sat, 3 Mar 2018 12:46:37 +0000 (UTC) (envelope-from alr48@hermes.cam.ac.uk) Received: from ppsw-30.csi.cam.ac.uk (ppsw-30.csi.cam.ac.uk [131.111.8.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C40B87FBC0 for ; Sat, 3 Mar 2018 12:46:36 +0000 (UTC) (envelope-from alr48@hermes.cam.ac.uk) X-Cam-AntiVirus: no malware found X-Cam-ScannerInfo: http://help.uis.cam.ac.uk/email-scanner-virus Received: from mail-it0-f51.google.com ([209.85.214.51]:51321) by ppsw-30.csi.cam.ac.uk (smtp.hermes.cam.ac.uk [131.111.8.156]:587) with esmtpsa (PLAIN:alr48) (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) id 1es6Yc-0008Na-eE (Exim 4.90_1) for freebsd-hackers@freebsd.org (return-path ); Sat, 03 Mar 2018 12:46:34 +0000 Received: by mail-it0-f51.google.com with SMTP id u66so4582120ith.1 for ; Sat, 03 Mar 2018 04:46:31 -0800 (PST) X-Gm-Message-State: AElRT7EySffJbac4ikWgJTcXjQ5rWuD5JtoTROVf+Vs5/bKaoLIUJHMQ 77Bs4h45KrMVATBhEPaMNHc+Th/nEoSSDxX+Q3U= X-Received: by 10.36.3.67 with SMTP id e64mt7305684ite.46.1520081190443; Sat, 03 Mar 2018 04:46:30 -0800 (PST) MIME-Version: 1.0 Received: by 10.2.169.133 with HTTP; Sat, 3 Mar 2018 04:46:10 -0800 (PST) In-Reply-To: <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> References: <20180302183514.GA99279@x-wing> <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> From: Alexander Richardson Date: Sat, 3 Mar 2018 12:46:10 +0000 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [capsicum] unlinkfd Cc: "" , freebsd-hackers@freebsd.org Sender: "A.L. Richardson" X-Mailman-Approved-At: Sat, 03 Mar 2018 15:08:04 +0000 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 12:46:37 -0000 Linux has a unlinkat() system call (https://linux.die.net/man/2/unlinkat) but it doesn't seem to have a flag that lets you unlink the fd itself. Possibly pathname =3D=3D NULL and AT_EMPTY_PATH could mean unlink the fd bu= t I haven't tried whether that works. It also has a AT_REMOVEDIR flag to make it function as rmdirat(). On 3 March 2018 at 10:41, Robert N. M. Watson wrote: > FWIW, this is part of why we introduced anonymous POSIX shared memory > objects with Capsicum in FreeBSD -- we allow shm_open(2) to be passed a > SHM_ANON special name, which causes the creation of a swap-backed, mappab= le > file-like object that can have I/O, memory mapping, etc, performed on it = .. > but never has any persistent state across reboots even in the event of a > crash. > > With Capsicum you can then refine a file descriptor to the otherwise > writable object to be read-only for the purposes of delegation. There is > not, however, a mechanism to "freeze" the state of the object causing oth= er > outstanding writable descriptors to become read-only -- certainly somethi= ng > could be added, but some care regarding VM semantics would be required -- > in particular, so that faults could not be experienced as a result of an > memory store performed before the "freeze" but issued to VFS only later. > > I certainly have no objection to an unlinkat(2) system call -- it's > unfortunate that a full suite of the at(2) APIs wasn't introduced in the > first place. It would be worth checking that no one else (e.g., Solaris, > Mac OS X, Linux) hasn't already added an unlinkat(2) that we can match AP= I > semantics for. I think I take the view that for truly anonymous objects, > shm_open(2) without a name (or the Linux equiv) is the right thing -- and > hence unlinkat(2) is for more conventional use cases where the final > pathname element is known. > > On directories: There, I find myself falling back on a Casper-like > service, since GC'ing a single anonymous memory object is straightforward= , > but GC'ing a directory hierarchy is a more messy business. > > Robert > > > On 3 Mar 2018, at 09:53, Justin Cormack > wrote: > > > > I think it would make sense to have an unlinkfd() that unlinks the file > from > > everywhere, so it does not need a name to be specified. This might be > > hard to implement. > > > > For temporary files, I really like Linux memfd_create(2) that opens an > anonymous > > file without a name. This semantics is really useful. (Linux memfd also > has > > additional options for sealing the file fo make it immutable which are > very > > useful for safely passing files between processes.) Having a way to mak= e > > unnamed temporary files solves a lot of deletion issues as the file > > never needs to > > be unlinked. > > > > > > On 2 March 2018 at 18:35, Mariusz Zaborski wrote: > >> Hello, > >> > >> Today I would like to propose a new syscall called unlinkfd(2) which > came up > >> during a discussion with Ed Maste. > >> > >> Currently in UNIX we can=E2=80=99t remove files safely. If we will try= to do so > we > >> always end up in a race condition. For example when we open a file, an= d > check > >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the file= we are > trying to > >> unlink could be a different one than the one we were fstating just a > moment ago. > >> > >> Another reason of implementing unlinkfd(2) came to us when we were > trying > >> to sandbox some applications like: uudecode/b64decode or bspatch. It > occured > >> to us that we don=E2=80=99t have a good way of removing single files. = Of course > we can > >> try to determine in which directory we are in, and then open this > directory and > >> remove a single file. > >> > >> It looks even more bizarre if we would think about a program which > operates on > >> multiple files. If we would analyze a situation with two totally > different > >> directories like `/tmp` and `/home/oshogbo` we would end up with pre > opening > >> a root directory or keeping as many directories as we are working on > open. > >> All of that effort only to remove two files. This make it totally > impractical! > >> > >> I think that opening directories also presents some wider attack vecto= r > because > >> we are keeping a single descriptor to a directory only to remove one > file. > >> Unfortunately this means that an attacker can remove all files in that > directory. > >> > >> I proposed this as well on the last Capsicum call. There was a > suggestion that > >> instead of doing a single syscall maybe we should have a Casper servic= e > that > >> will allow us to remove files. Another idea was that we should perhaps > redesign > >> programs to create some subdirs work on the subdirs and then remove al= l > files in > >> this subdir. I don=E2=80=99t feel that creating a Casper service is a = good idea > because > >> we still have exactly the same issue of race condition. In my opinion > creating > >> subdirs is also a problem for us. > >> > >> First we would need to redesign some of our tools and I think we shoul= d > >> simplyfiy capsicumizition of the process instead of making it harder. > >> > >> Secondly we can create a temporary subdirectory but what will remove i= t? > >> We are going back to having a fd to directory in which we just created > a subdir. > >> Another way would be to have Casper service which would remove a > directory but > >> with the risk of RC. > >> > >> In conclusion, I think we need syscall like unlinkfd(2), which turn ou= t > taht it > >> is easy to implement. The only downside of this implementation is that > we not > >> only need to provide a fd but also a path file. This is because inodes > nor > >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of the= fd and > the given > >> path, if they are exactly the same we remove a file. In the syscall we > are using > >> a fd so there is no Ambient Authority because we are proving that we > already > >> have access to that file. Thanks to that the syscall can be safely use= d > with > >> Caspsicum. I have already discussed this with some people and they sai= d > >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s do= something with > that idea! > >> If you are intereted in patch you can find it here: > >> https://reviews.freebsd.org/D14567 > >> > >> Thanks, > >> -- > >> Mariusz Zaborski > >> oshogbo//vx | http://oshogbo.vexillium.org > >> FreeBSD commiter | https://freebsd.org > >> Software developer | http://wheelsystems.com > >> If it's not broken, let's fix it till it is!!1 > > > > > From owner-freebsd-hackers@freebsd.org Sat Mar 3 15:29:25 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E78BBF44ECF for ; Sat, 3 Mar 2018 15:29:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3A5D68618F for ; Sat, 3 Mar 2018 15:29:24 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x22e.google.com with SMTP id t204so17301535lff.9 for ; Sat, 03 Mar 2018 07:29:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=PxdOXDSeLQ18khZk6HDA28FolaSH8W/xwTO6B+lFEtg=; b=eMSgHZU1gJqvCQrKUco3+gpq9xRE4JsfidrbmC0XN88CwfuT5MeT86xadEcO1HZnXu xA84tavTOVoeRfGpe4MSUTz09Bm/XlWxZtF8UGnBsXcrMMyduLJR3HGe5JJmNZg4lb6l eyOOsm86NmSc5W6Vg0m7y1yNSTcNckT0Pm7wyTP21qZRRb+SH+tzwORjMUDpst4K+5fM CesojrGasEjeD9jeMN0q0j6m4hBSZgp7qnCWXJY9Mp9d8fcFGGqtDx9K6QSuEkzsvxb1 sJcHObeld6QDXpYqzYhelGg9pVY0bHtTDt40B8ft53Fno7fXnvygITa4+GHZPWAUrTYl ojYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=PxdOXDSeLQ18khZk6HDA28FolaSH8W/xwTO6B+lFEtg=; b=Onox5zE0wJ2Z/wk+3g1xEZwXp8IVSW5kGwyVxh0BO14Fg/j4A58I6ZSikMRRIwORvX p33ZQ8ny2W4/rFMUlegSKnGUHtBzy35FkwpfiDsgwmxp6iF1MwZkX9VeIaQvtUBTsIQ8 EtBKVyJ+3CWq52K+vWJB3jETwIJWwcOoanTEjcoPII7Rl+InbfOxTYqTBIxj4a8ZGWNV AVpYp5y6knoH6NFnj3JWn86iRxFhA4lf/IFvV9YxtXP3P+iKYQUixyG/LPYVhDQFHO84 x/8Ev8kZcLWnxeohvqT76D0iVP0c3dtfqC4LnYX/HMIhd3g823u/ybGIygb39e5nS/qI lV3A== X-Gm-Message-State: AElRT7H0Og0T2wy7zLMayGNwI2K3neINYb1PfHFeQMCeUiWTxKEkvSBq CG+8o0NqXl30e2TKVZRloUDBddgiv0F3H1rl9os= X-Google-Smtp-Source: AG47ELuuIet9LtmwB60ANS9ONeuBFcbszTOfd6HBXQtDmezT6Sds778se9ZDSFh5fIWTHglrHqvfXjFYwl+Zs4pGiNw= X-Received: by 10.46.27.211 with SMTP id c80mr1784872ljf.46.1520090962503; Sat, 03 Mar 2018 07:29:22 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.2.195 with HTTP; Sat, 3 Mar 2018 07:29:21 -0800 (PST) In-Reply-To: References: <20180302183514.GA99279@x-wing> <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> From: Alan Somers Date: Sat, 3 Mar 2018 08:29:21 -0700 X-Google-Sender-Auth: VE96_IHEqR2H9ZOXvAJAaqRSSoQ Message-ID: Subject: Re: [capsicum] unlinkfd To: Alexander Richardson Cc: "" , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 15:29:25 -0000 In fact, FreeBSD has that same unlinkat(2) system call. But it doesn't solve Mariusz's problem. He's concerned about race conditions. With either unlink(2) or unlinkat(2), there's no way to ensure that the directory entry you remove is for the file you think it is. Because after reading/writing a file and before unlinking it, some other processes could've unlinked it and created a new one with the same name. It's this race condition that Mariuz seeks to solve with unlinkfd. -Alan On Sat, Mar 3, 2018 at 5:46 AM, Alexander Richardson < Alexander.Richardson@cl.cam.ac.uk> wrote: > Linux has a unlinkat() system call (https://linux.die.net/man/2/unlinkat) > but it doesn't seem to have a flag that lets you unlink the fd itself. > Possibly pathname =3D=3D NULL and AT_EMPTY_PATH could mean unlink the fd = but I > haven't tried whether that works. > It also has a AT_REMOVEDIR flag to make it function as rmdirat(). > > On 3 March 2018 at 10:41, Robert N. M. Watson > wrote: > > > FWIW, this is part of why we introduced anonymous POSIX shared memory > > objects with Capsicum in FreeBSD -- we allow shm_open(2) to be passed a > > SHM_ANON special name, which causes the creation of a swap-backed, > mappable > > file-like object that can have I/O, memory mapping, etc, performed on i= t > .. > > but never has any persistent state across reboots even in the event of = a > > crash. > > > > With Capsicum you can then refine a file descriptor to the otherwise > > writable object to be read-only for the purposes of delegation. There i= s > > not, however, a mechanism to "freeze" the state of the object causing > other > > outstanding writable descriptors to become read-only -- certainly > something > > could be added, but some care regarding VM semantics would be required = -- > > in particular, so that faults could not be experienced as a result of a= n > > memory store performed before the "freeze" but issued to VFS only later= . > > > > I certainly have no objection to an unlinkat(2) system call -- it's > > unfortunate that a full suite of the at(2) APIs wasn't introduced in th= e > > first place. It would be worth checking that no one else (e.g., Solaris= , > > Mac OS X, Linux) hasn't already added an unlinkat(2) that we can match > API > > semantics for. I think I take the view that for truly anonymous objects= , > > shm_open(2) without a name (or the Linux equiv) is the right thing -- a= nd > > hence unlinkat(2) is for more conventional use cases where the final > > pathname element is known. > > > > On directories: There, I find myself falling back on a Casper-like > > service, since GC'ing a single anonymous memory object is > straightforward, > > but GC'ing a directory hierarchy is a more messy business. > > > > Robert > > > > > On 3 Mar 2018, at 09:53, Justin Cormack > > wrote: > > > > > > I think it would make sense to have an unlinkfd() that unlinks the fi= le > > from > > > everywhere, so it does not need a name to be specified. This might be > > > hard to implement. > > > > > > For temporary files, I really like Linux memfd_create(2) that opens a= n > > anonymous > > > file without a name. This semantics is really useful. (Linux memfd al= so > > has > > > additional options for sealing the file fo make it immutable which ar= e > > very > > > useful for safely passing files between processes.) Having a way to > make > > > unnamed temporary files solves a lot of deletion issues as the file > > > never needs to > > > be unlinked. > > > > > > > > > On 2 March 2018 at 18:35, Mariusz Zaborski > wrote: > > >> Hello, > > >> > > >> Today I would like to propose a new syscall called unlinkfd(2) which > > came up > > >> during a discussion with Ed Maste. > > >> > > >> Currently in UNIX we can=E2=80=99t remove files safely. If we will t= ry to do > so > > we > > >> always end up in a race condition. For example when we open a file, > and > > check > > >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the fi= le we are > > trying to > > >> unlink could be a different one than the one we were fstating just a > > moment ago. > > >> > > >> Another reason of implementing unlinkfd(2) came to us when we were > > trying > > >> to sandbox some applications like: uudecode/b64decode or bspatch. It > > occured > > >> to us that we don=E2=80=99t have a good way of removing single files= . Of > course > > we can > > >> try to determine in which directory we are in, and then open this > > directory and > > >> remove a single file. > > >> > > >> It looks even more bizarre if we would think about a program which > > operates on > > >> multiple files. If we would analyze a situation with two totally > > different > > >> directories like `/tmp` and `/home/oshogbo` we would end up with pre > > opening > > >> a root directory or keeping as many directories as we are working on > > open. > > >> All of that effort only to remove two files. This make it totally > > impractical! > > >> > > >> I think that opening directories also presents some wider attack > vector > > because > > >> we are keeping a single descriptor to a directory only to remove one > > file. > > >> Unfortunately this means that an attacker can remove all files in th= at > > directory. > > >> > > >> I proposed this as well on the last Capsicum call. There was a > > suggestion that > > >> instead of doing a single syscall maybe we should have a Casper > service > > that > > >> will allow us to remove files. Another idea was that we should perha= ps > > redesign > > >> programs to create some subdirs work on the subdirs and then remove > all > > files in > > >> this subdir. I don=E2=80=99t feel that creating a Casper service is = a good > idea > > because > > >> we still have exactly the same issue of race condition. In my opinio= n > > creating > > >> subdirs is also a problem for us. > > >> > > >> First we would need to redesign some of our tools and I think we > should > > >> simplyfiy capsicumizition of the process instead of making it harder= . > > >> > > >> Secondly we can create a temporary subdirectory but what will remove > it? > > >> We are going back to having a fd to directory in which we just creat= ed > > a subdir. > > >> Another way would be to have Casper service which would remove a > > directory but > > >> with the risk of RC. > > >> > > >> In conclusion, I think we need syscall like unlinkfd(2), which turn > out > > taht it > > >> is easy to implement. The only downside of this implementation is th= at > > we not > > >> only need to provide a fd but also a path file. This is because inod= es > > nor > > >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of t= he fd and > > the given > > >> path, if they are exactly the same we remove a file. In the syscall = we > > are using > > >> a fd so there is no Ambient Authority because we are proving that we > > already > > >> have access to that file. Thanks to that the syscall can be safely > used > > with > > >> Caspsicum. I have already discussed this with some people and they > said > > >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99s = do something with > > that idea! > > >> If you are intereted in patch you can find it here: > > >> https://reviews.freebsd.org/D14567 > > >> > > >> Thanks, > > >> -- > > >> Mariusz Zaborski > > >> oshogbo//vx | http://oshogbo.vexillium.org > > >> FreeBSD commiter | https://freebsd.org > > >> Software developer | http://wheelsystems.com > > >> If it's not broken, let's fix it till it is!!1 > > > > > > > > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org= " > From owner-freebsd-hackers@freebsd.org Sat Mar 3 15:45:39 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 961DEF4616C for ; Sat, 3 Mar 2018 15:45:39 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B68686E43; Sat, 3 Mar 2018 15:45:38 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w23FjMAa025959 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 3 Mar 2018 17:45:25 +0200 (EET) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w23FjMAa025959 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w23FjM60025956; Sat, 3 Mar 2018 17:45:22 +0200 (EET) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Sat, 3 Mar 2018 17:45:22 +0200 From: Konstantin Belousov To: "Robert N. M. Watson" Cc: Alex Richardson , "" , Mariusz Zaborski , Justin Cormack , freebsd-hackers@freebsd.org Subject: Re: [capsicum] unlinkfd Message-ID: <20180303154522.GH3194@kib.kiev.ua> References: <20180302183514.GA99279@x-wing> <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> <1ED213FC-D0CA-46D9-B6D1-EA261F2B80F5@cl.cam.ac.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1ED213FC-D0CA-46D9-B6D1-EA261F2B80F5@cl.cam.ac.uk> User-Agent: Mutt/1.9.3 (2018-01-21) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 15:45:39 -0000 On Sat, Mar 03, 2018 at 12:16:34PM +0000, Robert N. M. Watson wrote: > (3) Want capability-based access to a persistent hierarchal namespace > full of files. This is well served by the current at(2) system calls > along with filesystems, although there are API gaps (e.g., a lack of > unlinkat(2) in FreeBSD). Why do you claim this ? We have unlinkat(2). From owner-freebsd-hackers@freebsd.org Sat Mar 3 15:53:28 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BC1DF46C36 for ; Sat, 3 Mar 2018 15:53:28 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL SHA256 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE23E87566 for ; Sat, 3 Mar 2018 15:53:27 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.15.1/8.15.1/NETPLEX) with ESMTP id w23FrK71027892; Sat, 3 Mar 2018 10:53:20 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Sat, 03 Mar 2018 10:53:20 -0500 (EST) Date: Sat, 3 Mar 2018 10:53:20 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Andre Albsmeier cc: freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards In-Reply-To: <20180303064400.GA27337@bali> Message-ID: References: <20180302061852.GA7887@bali> <20180303064400.GA27337@bali> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 15:53:28 -0000 On Sat, 3 Mar 2018, Andre Albsmeier wrote: > On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: >> On Fri, 2 Mar 2018, Andre Albsmeier wrote: >> >>> I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial >>> ports) sitting here which does not get detected on 11.1. I tried >>> to simply add it to the uart and ppc drivers with >>> >> [ ... ] >> >> Do you try adding similar support to puc_pci_devices[] in >> sys/dev/puc/pucdata.c? > > Just tried that: > > @@ -1204,6 +1204,11 @@ > PUC_PORT_1S1P, 0x10, 4, 0, > }, > > +{ 0x9710, 0x9912, 0xa000, 0x3012, > + "NetMos NM9912 Dual UART and 1284 Printer port", > + DEFAULT_RCLK, > + PUC_PORT_2S1P, 0x10, 4, 0, > +}, > { 0x9710, 0x9865, 0xa000, 0x3012, > "NetMos NM9865 Dual UART and 1284 Printer port", > DEFAULT_RCLK, > > But the results are exactly the same. It also doesn't > matter if puc.ko is loaded at all. Are you sure your subvendor and subdevice are correct? I would add some device_printf()'s in puc_pci.c::puc_pci_match() and boot with/enable bootverbose. Try to see where it's failing. -- DE From owner-freebsd-hackers@freebsd.org Sat Mar 3 18:43:52 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 991B4F3113E for ; Sat, 3 Mar 2018 18:43:52 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EDAC16FDF4; Sat, 3 Mar 2018 18:43:51 +0000 (UTC) (envelope-from oshogbo.vx@gmail.com) Received: by mail-lf0-x22d.google.com with SMTP id i80so17780872lfg.5; Sat, 03 Mar 2018 10:43:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=b4+dB4wwY/NkxEuO1DCP95STQD1r8RYEUDeWqFmH6Cg=; b=To31DY3sNCbidKgLap0k0VlIrK8tFpQoSjHbwJnhTWbqbL7rxfnmTz+rl5xTIX4MT9 wT81zLDudHFIvnudIMit5ft2pb0x91j1srv9LWnv5lho1kWZb4H3v0KWslAXdotau0NU xOhcj0u5hDtQTcBpehex8hSmZ5gq7tlRmR0RCw4w25MymZl5kzRHuU6b8xFA84QUvnqk 4CM3F1/VxAwKNjAXMngUYF5S/6JLeLxiTo5QuNLYJScnnxbjaawQnSn8tM9KTHr3hHCO 4du+oh6PIYqnHpGXpJ6r/zY2CmQQCp+BUTMnvGE8heyr2bcAiQeZddYOFSRIRqWEiIFK FVzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=b4+dB4wwY/NkxEuO1DCP95STQD1r8RYEUDeWqFmH6Cg=; b=Z0k5KaiFLyqghNGePOmAjc7GySfx60PRh7/JGffYudMAbed+OxFO9pXoBYbJfXFCgi x8ulUkY9NDfq9lQYkTi7o8WT72u+KDgV2jakvUMufH17nMEBYJ8CLS1NS6LilVdMKgV/ yQE12xcwdMi9ZV/fBXh5RX5/Wc1Fg/tV0pdCiQUTo4em6S/ihmvDzfdFNJ0W78krxwmy EyFWCGXiNkUivHSKYhaCb/x4yr0WFdmW8rydz3dSyBbQbgG9K4a7uCAVj0DV4kLSpkCk PtJjO4TNZj+M09OpxXL5vYGASu+Vs+LPtxqLncMu4eZ8NqPbydVkeR7nd+IlwfqSFdR0 USpw== X-Gm-Message-State: AElRT7EyiqC7iB4S04NXjDPdtISEnFDPjm+z7Ss2MyhMefS7Tp7xnjag OY/qi7BqyojhIqUqQVgE7avQ7/v8 X-Google-Smtp-Source: AG47ELt39142nDj4scSOcuEbQ72JBtcYVApV1EuiKA6Q3rBwZVVaK7za37IP7QmsnM8sjevhYn4bxg== X-Received: by 10.25.215.197 with SMTP id q66mr6088016lfi.89.1520102630291; Sat, 03 Mar 2018 10:43:50 -0800 (PST) Received: from x-wing (87-206-170-77.dynamic.chello.pl. [87.206.170.77]) by smtp.gmail.com with ESMTPSA id n15sm1911777lfn.15.2018.03.03.10.43.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 03 Mar 2018 10:43:49 -0800 (PST) Sender: Mariusz Zaborski Date: Sat, 3 Mar 2018 19:43:54 +0100 From: Mariusz Zaborski To: "Robert N. M. Watson" Cc: Alan Somers , "" , Alexander Richardson , "freebsd-hackers@freebsd.org" Subject: Re: [capsicum] unlinkfd Message-ID: <20180303184354.GA48406@x-wing> References: <20180302183514.GA99279@x-wing> <17DE0BFF-42A2-4CD7-B09C-ABA2606C4041@cl.cam.ac.uk> <6E83EC9C-56C5-40E7-AED0-2692A15F7FD3@cl.cam.ac.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="huq684BweRXVnRxX" Content-Disposition: inline In-Reply-To: <6E83EC9C-56C5-40E7-AED0-2692A15F7FD3@cl.cam.ac.uk> User-Agent: Mutt/1.9.2 (2017-12-15) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 18:43:53 -0000 --huq684BweRXVnRxX Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I feel that there is two different things we can think about: - What we would implement in the capability system if we would build it from scratch. Here shm_open(2) and SHM_ANON can be solution to our problems. - On the other hand we have a working operating system and we can't expect = that all our programs that are already implemented will fit to those assumptio= ns nor ask developers to rewrite many existing programs. On Sat, Mar 03, 2018 at 05:16:38PM +0000, Robert N. M. Watson wrote: > New _check() variants of the unlinkat(2) and rmdirat(2) system calls migh= t do the trick -- e.g., >=20 > int unlinkat_check(dirfd, name, checkfd); > int rmdirat_check(dirfd, name, checkfd); >=20 Similar API was proposed on the review. This solves the issue with RC. Unfortunately it's not solve the problem with guessing in which directory we will work in. When I think about sandboxing for example rm(1) we would need to preopen ro= ot directory, or preopen all directories we will work in. Both solution just d= on't feel right. I'm not saying that the unlinkfd is the right and only solution - I'm just = trying to solve problem we identified while sandboxing apps. I'm glad we started t= his discussion I hope we will work some compromise between all presented challe= nges. Thanks, --=20 Mariusz Zaborski oshogbo//vx | http://oshogbo.vexillium.org FreeBSD commiter | https://freebsd.org Software developer | http://wheelsystems.com If it's not broken, let's fix it till it is!!1 > The calls would succeed only if 'name' refers to the filesystem object pa= ssed via checkfd. This would retain UNIX-style directory behaviour but allo= ws an atomic check that the object is as expected. >=20 > Of course, what you do about it if it turns out the check fails is anothe= r question... Better not to have a name at all, hence shm_open(SHM_ANON, ..= =2E) -- although just for file objects, and not directory hierarchies. >=20 > Robert >=20 > > On 3 Mar 2018, at 15:29, Alan Somers wrote: > >=20 > > In fact, FreeBSD has that same unlinkat(2) system call. But it doesn't= solve Mariusz's problem. He's concerned about race conditions. With eith= er unlink(2) or unlinkat(2), there's no way to ensure that the directory en= try you remove is for the file you think it is. Because after reading/writ= ing a file and before unlinking it, some other processes could've unlinked = it and created a new one with the same name. It's this race condition that= Mariuz seeks to solve with unlinkfd. > > -Alan > >=20 > > On Sat, Mar 3, 2018 at 5:46 AM, Alexander Richardson > wrote: > > Linux has a unlinkat() system call (https://linux.die.net/man/2/unlinka= t ) > > but it doesn't seem to have a flag that lets you unlink the fd itself. > > Possibly pathname =3D=3D NULL and AT_EMPTY_PATH could mean unlink the f= d but I > > haven't tried whether that works. > > It also has a AT_REMOVEDIR flag to make it function as rmdirat(). > >=20 > > On 3 March 2018 at 10:41, Robert N. M. Watson > > > wrote: > >=20 > > > FWIW, this is part of why we introduced anonymous POSIX shared memory > > > objects with Capsicum in FreeBSD -- we allow shm_open(2) to be passed= a > > > SHM_ANON special name, which causes the creation of a swap-backed, ma= ppable > > > file-like object that can have I/O, memory mapping, etc, performed on= it .. > > > but never has any persistent state across reboots even in the event o= f a > > > crash. > > > > > > With Capsicum you can then refine a file descriptor to the otherwise > > > writable object to be read-only for the purposes of delegation. There= is > > > not, however, a mechanism to "freeze" the state of the object causing= other > > > outstanding writable descriptors to become read-only -- certainly som= ething > > > could be added, but some care regarding VM semantics would be require= d -- > > > in particular, so that faults could not be experienced as a result of= an > > > memory store performed before the "freeze" but issued to VFS only lat= er. > > > > > > I certainly have no objection to an unlinkat(2) system call -- it's > > > unfortunate that a full suite of the at(2) APIs wasn't introduced in = the > > > first place. It would be worth checking that no one else (e.g., Solar= is, > > > Mac OS X, Linux) hasn't already added an unlinkat(2) that we can matc= h API > > > semantics for. I think I take the view that for truly anonymous objec= ts, > > > shm_open(2) without a name (or the Linux equiv) is the right thing --= and > > > hence unlinkat(2) is for more conventional use cases where the final > > > pathname element is known. > > > > > > On directories: There, I find myself falling back on a Casper-like > > > service, since GC'ing a single anonymous memory object is straightfor= ward, > > > but GC'ing a directory hierarchy is a more messy business. > > > > > > Robert > > > > > > > On 3 Mar 2018, at 09:53, Justin Cormack > > > > wrote: > > > > > > > > I think it would make sense to have an unlinkfd() that unlinks the = file > > > from > > > > everywhere, so it does not need a name to be specified. This might = be > > > > hard to implement. > > > > > > > > For temporary files, I really like Linux memfd_create(2) that opens= an > > > anonymous > > > > file without a name. This semantics is really useful. (Linux memfd = also > > > has > > > > additional options for sealing the file fo make it immutable which = are > > > very > > > > useful for safely passing files between processes.) Having a way to= make > > > > unnamed temporary files solves a lot of deletion issues as the file > > > > never needs to > > > > be unlinked. > > > > > > > > > > > > On 2 March 2018 at 18:35, Mariusz Zaborski > wrote: > > > >> Hello, > > > >> > > > >> Today I would like to propose a new syscall called unlinkfd(2) whi= ch > > > came up > > > >> during a discussion with Ed Maste. > > > >> > > > >> Currently in UNIX we can=E2=80=99t remove files safely. If we will= try to do so > > > we > > > >> always end up in a race condition. For example when we open a file= , and > > > check > > > >> it with fstat, etc. then we want to unlink(2) it=E2=80=A6 but the = file we are > > > trying to > > > >> unlink could be a different one than the one we were fstating just= a > > > moment ago. > > > >> > > > >> Another reason of implementing unlinkfd(2) came to us when we were > > > trying > > > >> to sandbox some applications like: uudecode/b64decode or bspatch. = It > > > occured > > > >> to us that we don=E2=80=99t have a good way of removing single fil= es. Of course > > > we can > > > >> try to determine in which directory we are in, and then open this > > > directory and > > > >> remove a single file. > > > >> > > > >> It looks even more bizarre if we would think about a program which > > > operates on > > > >> multiple files. If we would analyze a situation with two totally > > > different > > > >> directories like `/tmp` and `/home/oshogbo` we would end up with p= re > > > opening > > > >> a root directory or keeping as many directories as we are working = on > > > open. > > > >> All of that effort only to remove two files. This make it totally > > > impractical! > > > >> > > > >> I think that opening directories also presents some wider attack v= ector > > > because > > > >> we are keeping a single descriptor to a directory only to remove o= ne > > > file. > > > >> Unfortunately this means that an attacker can remove all files in = that > > > directory. > > > >> > > > >> I proposed this as well on the last Capsicum call. There was a > > > suggestion that > > > >> instead of doing a single syscall maybe we should have a Casper se= rvice > > > that > > > >> will allow us to remove files. Another idea was that we should per= haps > > > redesign > > > >> programs to create some subdirs work on the subdirs and then remov= e all > > > files in > > > >> this subdir. I don=E2=80=99t feel that creating a Casper service i= s a good idea > > > because > > > >> we still have exactly the same issue of race condition. In my opin= ion > > > creating > > > >> subdirs is also a problem for us. > > > >> > > > >> First we would need to redesign some of our tools and I think we s= hould > > > >> simplyfiy capsicumizition of the process instead of making it hard= er. > > > >> > > > >> Secondly we can create a temporary subdirectory but what will remo= ve it? > > > >> We are going back to having a fd to directory in which we just cre= ated > > > a subdir. > > > >> Another way would be to have Casper service which would remove a > > > directory but > > > >> with the risk of RC. > > > >> > > > >> In conclusion, I think we need syscall like unlinkfd(2), which tur= n out > > > taht it > > > >> is easy to implement. The only downside of this implementation is = that > > > we not > > > >> only need to provide a fd but also a path file. This is because in= odes > > > nor > > > >> vnodes don=E2=80=99t contain filenames. We are comparing vnodes of= the fd and > > > the given > > > >> path, if they are exactly the same we remove a file. In the syscal= l we > > > are using > > > >> a fd so there is no Ambient Authority because we are proving that = we > > > already > > > >> have access to that file. Thanks to that the syscall can be safely= used > > > with > > > >> Caspsicum. I have already discussed this with some people and they= said > > > >> `Hey I already had that idea a while ago=E2=80=A6` so let=E2=80=99= s do something with > > > that idea! > > > >> If you are intereted in patch you can find it here: > > > >> https://reviews.freebsd.org/D14567 > > > >> > > > >> Thanks, > > > >> -- > > > >> Mariusz Zaborski > > > >> oshogbo//vx | http://oshogbo.vexillium.org > > > >> FreeBSD commiter | https://freebsd.org > > > >> Software developer | http://wheelsystems.com > > > >> If it's not broken, let's fix it till it is!!1 > > > > > > > > > > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailin= g list > > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.o= rg " > >=20 >=20 --huq684BweRXVnRxX Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEkD1x0xkJXVVY1Gwf38KEGuLGxWQFAlqa7N8ACgkQ38KEGuLG xWSNRw/9FwcsJBggBQ9oCckXSdsxGN4SVwTkrD9TSr1HmQ9f204BcbbLody5btwd xL/vCfh16MMRgJhFmsKxZEmhykVaBRfd4ZNZ9nVVY4t/RfzjbU+B61C5PyQ/TV/m hyVMB1V8VIDOsiDs9MWL+v6sUY6e483vx9j3VmAqR1IAwyybB9mvuTZ1ZTpe81Iu av2FAWjmlwpWRJWzFMthxMqVkWBn1QINM2EOHKzdgpG4fl+8iwc5DUV0Hjtaofba R1kaUXvhuo2NTGMLqa5NTC1yoBttYPyQGYTmv3WUI4DdbK/0TwM9hmUJhXRJgA8+ D2iOietYwwllTlz5V4mkBTs6O9g3Dwde3QVHeGLV1eDdLgMYNk0uXHj2zzI1KU4p VUXCTmgw1VR/dTdCchPKuxJmgKqRSi5MZhIP+E47sRTQernfNoj2zp3brdLeG0wG tJR+FFTF0TC81rF2BPU6iWptuajXqiHWw5rrP5l6tr3Sk/qLiR0axc1muc7PjW7K VJxrswG82KApJBQt+GgUjFySrvSZ5DiXs+zCOcrUjNvK1Tf0foAARD9ZXaAEYtpL sFmpdfHjP3pOHP8zTG+1UrO00RiT0nO4dodzSFnMEBjmEfrBu2GNN5ygL8ATPenl FbNUB/bQ6/xYsnQrVROBuLJu69klIoEfUUMtESt3K5mNUVn3MeE= =+11b -----END PGP SIGNATURE----- --huq684BweRXVnRxX-- From owner-freebsd-hackers@freebsd.org Sat Mar 3 18:44:08 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADF4AF3119D for ; Sat, 3 Mar 2018 18:44:08 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from goliath.siemens.de (goliath.siemens.de [192.35.17.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "goliath.siemens.de", Issuer "Siemens Issuing CA Internet Server 2017" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 255B36FE12; Sat, 3 Mar 2018 18:44:07 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from mail2.siemens.de (mail2.siemens.de [139.25.208.11]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w23Ii0DR015281 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 3 Mar 2018 19:44:00 +0100 Received: from curry.mchp.siemens.de (curry.mchp.siemens.de [139.25.40.130]) by mail2.siemens.de (8.15.2/8.15.2) with ESMTP id w23Ii0eQ020706; Sat, 3 Mar 2018 19:44:00 +0100 Received: (from user@localhost) by curry.mchp.siemens.de (8.15.2/8.15.2) id w23Ii0W3006773; Date: Sat, 3 Mar 2018 19:43:59 +0100 From: Andre Albsmeier To: Daniel Eischen Cc: Andre Albsmeier , freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards Message-ID: <20180303184359.GA29745@bali> References: <20180302061852.GA7887@bali> <20180303064400.GA27337@bali> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 18:44:08 -0000 On Sat, 03-Mar-2018 at 10:53:20 -0500, Daniel Eischen wrote: > On Sat, 3 Mar 2018, Andre Albsmeier wrote: > > > On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: > >> On Fri, 2 Mar 2018, Andre Albsmeier wrote: > >> > >>> I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial > >>> ports) sitting here which does not get detected on 11.1. I tried > >>> to simply add it to the uart and ppc drivers with > >>> > >> [ ... ] > >> > >> Do you try adding similar support to puc_pci_devices[] in > >> sys/dev/puc/pucdata.c? > > > > Just tried that: > > > > @@ -1204,6 +1204,11 @@ > > PUC_PORT_1S1P, 0x10, 4, 0, > > }, > > > > +{ 0x9710, 0x9912, 0xa000, 0x3012, > > + "NetMos NM9912 Dual UART and 1284 Printer port", > > + DEFAULT_RCLK, > > + PUC_PORT_2S1P, 0x10, 4, 0, > > +}, > > { 0x9710, 0x9865, 0xa000, 0x3012, > > "NetMos NM9865 Dual UART and 1284 Printer port", > > DEFAULT_RCLK, > > > > But the results are exactly the same. It also doesn't > > matter if puc.ko is loaded at all. > > Are you sure your subvendor and subdevice are correct? I would No ;-). I have to use: 0x9710, 0x9912, 0xa000, 0x2000, Now I have the following behaviour: When I load puc.ko I get: puc0: at device 0.2 on pci9 If I load ppc.ko now, I get: ppc0: parallel port not found. But if I unload puc and ppc and load ppc again, I get: ppc0: port 0xd000-0xd007,0xd008-0xd00f mem 0x89200000-0x89200fff irq 20 at device 0.2 on pci9 ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port For all this I have to disable /boot/device.hints -- otherwise the messages "driver bug: Unable to set devclass" comes back. So I think there are two problems: First the settings of the ISA stuff in /boot/device.hints conflicted with the settings the driver probed. This would be easy to solve -- just disable them in /boot/device.hints. Second it appears that ppc only attaches if puc did some kind of initialisation first. But we have to detach puc so the ppc can attach. -Andre From owner-freebsd-hackers@freebsd.org Sat Mar 3 19:41:59 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 88753F35A18 for ; Sat, 3 Mar 2018 19:41:59 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.netplex.net", Issuer "RapidSSL SHA256 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3323172E2B for ; Sat, 3 Mar 2018 19:41:58 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.15.1/8.15.1/NETPLEX) with ESMTP id w23JfwxO038994; Sat, 3 Mar 2018 14:41:58 -0500 X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.4.3 (mail.netplex.net [204.213.176.9]); Sat, 03 Mar 2018 14:41:58 -0500 (EST) Date: Sat, 3 Mar 2018 14:41:58 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net Reply-To: Daniel Eischen To: Andre Albsmeier cc: freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards In-Reply-To: <20180303184359.GA29745@bali> Message-ID: References: <20180302061852.GA7887@bali> <20180303064400.GA27337@bali> <20180303184359.GA29745@bali> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 19:41:59 -0000 On Sat, 3 Mar 2018, Andre Albsmeier wrote: > On Sat, 03-Mar-2018 at 10:53:20 -0500, Daniel Eischen wrote: >> On Sat, 3 Mar 2018, Andre Albsmeier wrote: >> >>> On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: >>>> On Fri, 2 Mar 2018, Andre Albsmeier wrote: >>>> >>>>> I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial >>>>> ports) sitting here which does not get detected on 11.1. I tried >>>>> to simply add it to the uart and ppc drivers with >>>>> >>>> [ ... ] >>>> >>>> Do you try adding similar support to puc_pci_devices[] in >>>> sys/dev/puc/pucdata.c? >>> >>> Just tried that: >>> >>> @@ -1204,6 +1204,11 @@ >>> PUC_PORT_1S1P, 0x10, 4, 0, >>> }, >>> >>> +{ 0x9710, 0x9912, 0xa000, 0x3012, >>> + "NetMos NM9912 Dual UART and 1284 Printer port", >>> + DEFAULT_RCLK, >>> + PUC_PORT_2S1P, 0x10, 4, 0, >>> +}, >>> { 0x9710, 0x9865, 0xa000, 0x3012, >>> "NetMos NM9865 Dual UART and 1284 Printer port", >>> DEFAULT_RCLK, >>> >>> But the results are exactly the same. It also doesn't >>> matter if puc.ko is loaded at all. >> >> Are you sure your subvendor and subdevice are correct? I would > > No ;-). I have to use: 0x9710, 0x9912, 0xa000, 0x2000, > > Now I have the following behaviour: > > When I load puc.ko I get: > puc0: at device 0.2 on pci9 > > If I load ppc.ko now, I get: > ppc0: parallel port not found. > > But if I unload puc and ppc and load ppc again, I get: > > ppc0: port 0xd000-0xd007,0xd008-0xd00f mem 0x89200000-0x89200fff irq 20 at device 0.2 on pci9 > ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode > ppbus0: on ppc0 > lpt0: on ppbus0 > lpt0: Interrupt-driven port > > For all this I have to disable /boot/device.hints -- otherwise > the messages "driver bug: Unable to set devclass" comes back. > > So I think there are two problems: > > First the settings of the ISA stuff in /boot/device.hints conflicted > with the settings the driver probed. This would be easy to solve -- > just disable them in /boot/device.hints. > > Second it appears that ppc only attaches if puc did some kind of > initialisation first. But we have to detach puc so the ppc can attach. Strange. Did you try setting puc_load="YES" in /boot/loader.conf and rebooting? Or are you just loading and unloading modules for now? -- DE From owner-freebsd-hackers@freebsd.org Sat Mar 3 20:40:16 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0525F3A5DF for ; Sat, 3 Mar 2018 20:40:16 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from thoth.sbs.de (thoth.sbs.de [192.35.17.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thoth.sbs.de", Issuer "Siemens Issuing CA Internet Server 2017" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 289A67527E; Sat, 3 Mar 2018 20:40:15 +0000 (UTC) (envelope-from Andre.Albsmeier@siemens.com) Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w23KNjut028719 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 3 Mar 2018 21:23:45 +0100 Received: from curry.mchp.siemens.de (curry.mchp.siemens.de [139.25.40.130]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w23KNjwr008608; Sat, 3 Mar 2018 21:23:45 +0100 Received: (from user@localhost) by curry.mchp.siemens.de (8.15.2/8.15.2) id w23KNjo9007033; Date: Sat, 3 Mar 2018 21:23:45 +0100 From: Andre Albsmeier To: Daniel Eischen Cc: Andre Albsmeier , freebsd-hackers@FreeBSD.org Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards Message-ID: <20180303202345.GA32478@bali> References: <20180302061852.GA7887@bali> <20180303064400.GA27337@bali> <20180303184359.GA29745@bali> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.2 (2016-11-26) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 20:40:16 -0000 On Sat, 03-Mar-2018 at 14:41:58 -0500, Daniel Eischen wrote: > On Sat, 3 Mar 2018, Andre Albsmeier wrote: > > > On Sat, 03-Mar-2018 at 10:53:20 -0500, Daniel Eischen wrote: > >> On Sat, 3 Mar 2018, Andre Albsmeier wrote: > >> > >>> On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: > >>>> On Fri, 2 Mar 2018, Andre Albsmeier wrote: > >>>> > >>>>> I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial > >>>>> ports) sitting here which does not get detected on 11.1. I tried > >>>>> to simply add it to the uart and ppc drivers with > >>>>> > >>>> [ ... ] > >>>> > >>>> Do you try adding similar support to puc_pci_devices[] in > >>>> sys/dev/puc/pucdata.c? > >>> > >>> Just tried that: > >>> > >>> @@ -1204,6 +1204,11 @@ > >>> PUC_PORT_1S1P, 0x10, 4, 0, > >>> }, > >>> > >>> +{ 0x9710, 0x9912, 0xa000, 0x3012, > >>> + "NetMos NM9912 Dual UART and 1284 Printer port", > >>> + DEFAULT_RCLK, > >>> + PUC_PORT_2S1P, 0x10, 4, 0, > >>> +}, > >>> { 0x9710, 0x9865, 0xa000, 0x3012, > >>> "NetMos NM9865 Dual UART and 1284 Printer port", > >>> DEFAULT_RCLK, > >>> > >>> But the results are exactly the same. It also doesn't > >>> matter if puc.ko is loaded at all. > >> > >> Are you sure your subvendor and subdevice are correct? I would > > > > No ;-). I have to use: 0x9710, 0x9912, 0xa000, 0x2000, > > > > Now I have the following behaviour: > > > > When I load puc.ko I get: > > puc0: at device 0.2 on pci9 > > > > If I load ppc.ko now, I get: > > ppc0: parallel port not found. > > > > But if I unload puc and ppc and load ppc again, I get: > > > > ppc0: port 0xd000-0xd007,0xd008-0xd00f mem 0x89200000-0x89200fff irq 20 at device 0.2 on pci9 > > ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode > > ppbus0: on ppc0 > > lpt0: on ppbus0 > > lpt0: Interrupt-driven port > > > > For all this I have to disable /boot/device.hints -- otherwise > > the messages "driver bug: Unable to set devclass" comes back. > > > > So I think there are two problems: > > > > First the settings of the ISA stuff in /boot/device.hints conflicted > > with the settings the driver probed. This would be easy to solve -- > > just disable them in /boot/device.hints. > > > > Second it appears that ppc only attaches if puc did some kind of > > initialisation first. But we have to detach puc so the ppc can attach. > > Strange. Did you try setting puc_load="YES" in /boot/loader.conf > and rebooting? Or are you just loading and unloading modules > for now? The latter. After each successful try I reboot to get rid of that magic initialisation puc does. But maybe I am getting closer: The card has 2 serial and 1 parallel port. But these 3 ports appear in pciconf as 3 seperate devices: none7@pci0:9:0:0: class=0x070002 card=0x1000a000 chip=0x99129710 rev=0x00 hdr=0x00 vendor = 'MosChip Semiconductor Technology Ltd.' device = 'PCIe 9912 Multi-I/O Controller' class = simple comms subclass = UART none8@pci0:9:0:1: class=0x070002 card=0x1000a000 chip=0x99129710 rev=0x00 hdr=0x00 vendor = 'MosChip Semiconductor Technology Ltd.' device = 'PCIe 9912 Multi-I/O Controller' class = simple comms subclass = UART none9@pci0:9:0:2: class=0x070103 card=0x2000a000 chip=0x99129710 rev=0x00 hdr=0x00 vendor = 'MosChip Semiconductor Technology Ltd.' device = 'PCIe 9912 Multi-I/O Controller' class = simple comms subclass = parallel port Looking at the source code of puc, I find a place where it checks the number of ports a device provides and aborts if it has only one port. This would match my understanding that puc only exists in order to help uart and ppc to work with multi-port devices where all ports are handled by a single device in pciconf. Until now, I used PUC_PORT_2S1P in my puc entry. So puc assumes that the card 0x2000a000 has 2 serial and 1 parallel port but this is probably wrong as this 0x2000a000 card only has one parallel port and the serials are handled by other devices. So the correct entry would be PUC_PORT_1P but with one single port puc does not attach. If all this is correct, I assume that puc should be left alone and ppc_pci.c is the thing I have to look at. This also matches the fact that uart attaches well to these two serial ports without the need of loading and unloading puc first so there is probably some kind of device specific initialisation missing in ppc_pci.c... -Andre From owner-freebsd-hackers@freebsd.org Sat Mar 3 22:03:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC2D0F4029E for ; Sat, 3 Mar 2018 22:03:01 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-qk0-x232.google.com (mail-qk0-x232.google.com [IPv6:2607:f8b0:400d:c09::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AAB478C68; Sat, 3 Mar 2018 22:03:01 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-qk0-x232.google.com with SMTP id s188so16383134qkb.2; Sat, 03 Mar 2018 14:03:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9yOi6fFg5xVF4dDH3fKWNXX6mPrEVr34TuPmdjTCo30=; b=TcCWmq4Ua2ulcrd22mdRaM1is6Pujyy6z5VT0n2pL7MFPpghNUyl/4tr8BA3biatSe GdDaqeqnKNtKK8CRF+PNac+0nIWBLFEmJccdbLzJc9MrT/GXPNwR1ishnkQfe3yx4Dbe d5+5m9lyx9A2TfUVS6t2LBmB+eRClpAJ/F4HBAH3NPn1J/NvuScw3FV/pBbLN5b2kXlO A8+A7fdhOlD5NXmrPo+kvAP6W58rMLivok3Je3HZ4evvHPnUsSyhSpJcAmJsShqvhJX4 rjb0x02VRjFmLPpSonUUYM6fTn2/Pkgmx9+BkRM8BM9OYDY9UkOLgDiXEZyVILh2rUku Yqhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9yOi6fFg5xVF4dDH3fKWNXX6mPrEVr34TuPmdjTCo30=; b=mjhaBHqJuQpura8JNMgpW0e1ee5b3MiGzK6bzmbZUx0cnep6UHCl4D9K1wlPsmrzk4 v3YDudc32PUFlg6jB+Qvzwbg5CVvfvrSC2/HQnXLndC7NnvPPDRY/0tCaUjp2sZhwqnB czi/gXujUy0D7HT9K7sYmux/68+6Gf1AgSU/1bTA2OWVERuMZ2D6aIHWy5/D1fAbjK47 881tgdflo5IEshCtWG5B07vGFZj4sZXHtmYST1fIPr1L95RnuigTC5SD1h4ru9PZbSYo Xadg7HFSLjha7MT6LUaaX0cUdF4EHVgDGQY2Tyi3sUUB8rHGh94HjAPVSuzxYY/xB+tt +6ag== X-Gm-Message-State: AElRT7FYVpwBkGpNmNoT4F6TzN6Wikv6UxyxeHMTdaB0WH6FPu4/gNAZ hIDXHNO5/sG0Slzn/H4jKHEa/K6zDQIGI5QJ4PE= X-Google-Smtp-Source: AG47ELsTnZLIxuUPkQx5rk0Vd803sBnKaOKYpeZXHUeHzgTfGIMZxor3U3SmEwFpD3GHAZqg1Svdsz+bq0mcjecxixw= X-Received: by 10.55.148.6 with SMTP id w6mr15308706qkd.293.1520114581044; Sat, 03 Mar 2018 14:03:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.32.74 with HTTP; Sat, 3 Mar 2018 14:03:00 -0800 (PST) In-Reply-To: <20180303202345.GA32478@bali> References: <20180302061852.GA7887@bali> <20180303064400.GA27337@bali> <20180303184359.GA29745@bali> <20180303202345.GA32478@bali> From: Stefan Blachmann Date: Sat, 3 Mar 2018 23:03:00 +0100 Message-ID: Subject: Re: Adding support for MosChip 9912 PCIe (serial/parallel) cards To: Andre Albsmeier Cc: Daniel Eischen , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Mar 2018 22:03:02 -0000 In the forums there is a thread where a guy had to download the chips' datasheet to obtain the correct IDs to add to pucdata.c. You find code and links here: https://forums.freebsd.org/threads/pcie-6-port-serial-card.63489/ HTH On 3/3/18, Andre Albsmeier wrote: > On Sat, 03-Mar-2018 at 14:41:58 -0500, Daniel Eischen wrote: >> On Sat, 3 Mar 2018, Andre Albsmeier wrote: >> >> > On Sat, 03-Mar-2018 at 10:53:20 -0500, Daniel Eischen wrote: >> >> On Sat, 3 Mar 2018, Andre Albsmeier wrote: >> >> >> >>> On Fri, 02-Mar-2018 at 08:36:40 -0500, Daniel Eischen wrote: >> >>>> On Fri, 2 Mar 2018, Andre Albsmeier wrote: >> >>>> >> >>>>> I have a MosChip 9912 card (PCIe card with 1 parallel and 2 serial >> >>>>> ports) sitting here which does not get detected on 11.1. I tried >> >>>>> to simply add it to the uart and ppc drivers with >> >>>>> >> >>>> [ ... ] >> >>>> >> >>>> Do you try adding similar support to puc_pci_devices[] in >> >>>> sys/dev/puc/pucdata.c? >> >>> >> >>> Just tried that: >> >>> >> >>> @@ -1204,6 +1204,11 @@ >> >>> PUC_PORT_1S1P, 0x10, 4, 0, >> >>> }, >> >>> >> >>> +{ 0x9710, 0x9912, 0xa000, 0x3012, >> >>> + "NetMos NM9912 Dual UART and 1284 Printer port", >> >>> + DEFAULT_RCLK, >> >>> + PUC_PORT_2S1P, 0x10, 4, 0, >> >>> +}, >> >>> { 0x9710, 0x9865, 0xa000, 0x3012, >> >>> "NetMos NM9865 Dual UART and 1284 Printer port", >> >>> DEFAULT_RCLK, >> >>> >> >>> But the results are exactly the same. It also doesn't >> >>> matter if puc.ko is loaded at all. >> >> >> >> Are you sure your subvendor and subdevice are correct? I would >> > >> > No ;-). I have to use: 0x9710, 0x9912, 0xa000, 0x2000, >> > >> > Now I have the following behaviour: >> > >> > When I load puc.ko I get: >> > puc0: at device 0.2 on >> > pci9 >> > >> > If I load ppc.ko now, I get: >> > ppc0: parallel port not found. >> > >> > But if I unload puc and ppc and load ppc again, I get: >> > >> > ppc0: port >> > 0xd000-0xd007,0xd008-0xd00f mem 0x89200000-0x89200fff irq 20 at device >> > 0.2 on pci9 >> > ppc0: Generic chipset (EPP/NIBBLE) in COMPATIBLE mode >> > ppbus0: on ppc0 >> > lpt0: on ppbus0 >> > lpt0: Interrupt-driven port >> > >> > For all this I have to disable /boot/device.hints -- otherwise >> > the messages "driver bug: Unable to set devclass" comes back. >> > >> > So I think there are two problems: >> > >> > First the settings of the ISA stuff in /boot/device.hints conflicted >> > with the settings the driver probed. This would be easy to solve -- >> > just disable them in /boot/device.hints. >> > >> > Second it appears that ppc only attaches if puc did some kind of >> > initialisation first. But we have to detach puc so the ppc can attach. >> >> Strange. Did you try setting puc_load="YES" in /boot/loader.conf >> and rebooting? Or are you just loading and unloading modules >> for now? > > The latter. After each successful try I reboot to get rid of that > magic initialisation puc does. > > But maybe I am getting closer: The card has 2 serial and 1 parallel > port. But these 3 ports appear in pciconf as 3 seperate devices: > > none7@pci0:9:0:0: class=0x070002 card=0x1000a000 chip=0x99129710 > rev=0x00 hdr=0x00 > vendor = 'MosChip Semiconductor Technology Ltd.' > device = 'PCIe 9912 Multi-I/O Controller' > class = simple comms > subclass = UART > none8@pci0:9:0:1: class=0x070002 card=0x1000a000 chip=0x99129710 > rev=0x00 hdr=0x00 > vendor = 'MosChip Semiconductor Technology Ltd.' > device = 'PCIe 9912 Multi-I/O Controller' > class = simple comms > subclass = UART > none9@pci0:9:0:2: class=0x070103 card=0x2000a000 chip=0x99129710 > rev=0x00 hdr=0x00 > vendor = 'MosChip Semiconductor Technology Ltd.' > device = 'PCIe 9912 Multi-I/O Controller' > class = simple comms > subclass = parallel port > > Looking at the source code of puc, I find a place where it checks > the number of ports a device provides and aborts if it has only > one port. This would match my understanding that puc only exists > in order to help uart and ppc to work with multi-port devices > where all ports are handled by a single device in pciconf. > > Until now, I used PUC_PORT_2S1P in my puc entry. So puc assumes > that the card 0x2000a000 has 2 serial and 1 parallel port but > this is probably wrong as this 0x2000a000 card only has one > parallel port and the serials are handled by other devices. > So the correct entry would be PUC_PORT_1P but with one single > port puc does not attach. > > If all this is correct, I assume that puc should be left alone > and ppc_pci.c is the thing I have to look at. > > This also matches the fact that uart attaches well to these two > serial ports without the need of loading and unloading puc first > so there is probably some kind of device specific initialisation > missing in ppc_pci.c... > > -Andre > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >