From owner-freebsd-arch@FreeBSD.ORG Sun Feb 22 08:59:43 2015 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BAE271A2 for ; Sun, 22 Feb 2015 08:59:43 +0000 (UTC) Received: from dchagin.static.corbina.net (dchagin.static.corbina.ru [78.107.232.239]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "dchagin.static.corbina.net", Issuer "dchagin.static.corbina.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E9F8FE for ; Sun, 22 Feb 2015 08:59:41 +0000 (UTC) Received: from dchagin.static.corbina.net (localhost [127.0.0.1]) by dchagin.static.corbina.net (8.14.9/8.14.9) with ESMTP id t1M8xW2M013524 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 22 Feb 2015 11:59:32 +0300 (MSK) (envelope-from dchagin@dchagin.static.corbina.net) Received: (from dchagin@localhost) by dchagin.static.corbina.net (8.14.9/8.14.9/Submit) id t1M8xVDQ013523; Sun, 22 Feb 2015 11:59:31 +0300 (MSK) (envelope-from dchagin) Date: Sun, 22 Feb 2015 11:59:31 +0300 From: Chagin Dmitry To: Andrew Wilcox Subject: Re: RFC: Added utimensat(2) to Linuxulator (was RE: Current status of utimensat(2)) Message-ID: <20150222085931.GA13515@dchagin.static.corbina.net> References: <004f01d03d34$3caed970$b60c8c50$@Wilcox-Tech.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <004f01d03d34$3caed970$b60c8c50$@Wilcox-Tech.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: 'Jilles Tjoelker' , freebsd-arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Feb 2015 08:59:43 -0000 On Sat, Jan 31, 2015 at 02:59:34AM -0600, Andrew Wilcox wrote: > Jilles Tjoelker sent 29 January 2015 17:06: > > I committed utimensat/futimens to head. I also made a start at Linuxulator > > utimensat, see attached patch. Please complete it and test it. > > > > -- > > Jilles Tjoelker > > > I have merged some of the things Jilles' patch does into the patch I was already working on. I have tested a number of corner cases with it, and compared the results to a running Linux system. I have also run software which all use utimensat(2) on Linux, such as recent tar(1), touch(1), and Python 3.3's shutil library, inside the Linuxulator with this patch applied. I have found this patch's behaviour to be correct in these tests. Please let me know of any comments or issues. > > I will be additionally trying to merge this into dchagin's lemul branch, but that discussion will take place on Phabricator (and will involve testing on amd64-native Linuxulator, which I have not yet done). > I commited your patch to the lemul branch with small style(9) modifications. for now lemul is up-to-date, please, test. -- Have fun! chd From owner-freebsd-arch@FreeBSD.ORG Mon Feb 23 14:48:24 2015 Return-Path: Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B53A69A3; Mon, 23 Feb 2015 14:48:24 +0000 (UTC) Received: from pmta1.delivery8.ore.mailhop.org (pmta1.delivery8.ore.mailhop.org [54.191.158.99]) by mx1.freebsd.org (Postfix) with ESMTP id 9522DD51; Mon, 23 Feb 2015 14:48:21 +0000 (UTC) Received: from smtp3.ore.mailhop.org (172.31.36.112) by pmta1.delivery1.ore.mailhop.org id htcups20r84e; Mon, 23 Feb 2015 14:47:48 +0000 (envelope-from ) Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp3.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YPuIw-000553-GN; Mon, 23 Feb 2015 14:48:14 +0000 Received: from fb864.hippie.lan (fb864.hippie.lan [172.22.42.242]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t1NEmCqq002182; Mon, 23 Feb 2015 07:48:13 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX18pbTXQXku9xM8pFlbEPGAF Message-ID: <1424702892.56366.31.camel@freebsd.org> Subject: Call for review: overriding osrelease and osreldate in jails From: Ian Lepore To: "freebsd-arch@FreeBSD.org" , freebsd-jail@FreeBSD.org Date: Mon, 23 Feb 2015 07:48:12 -0700 Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.8 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2015 14:48:24 -0000 I've added the ability to specify the values returned by sysctl (and thus by uname) for kern.osrelease and kern.osreldate within a jail. The changes are available for review: https://reviews.freebsd.org/D1948 This allows things like running an 8.4 jail on a 10.1 system such that within the jail the version is reliably spoofed as 8.4. While the uname values can be overridden with env vars, the env vars can be wiped out by scripts that use env(1). Changing the values returned by sysctl is more reliable. -- Ian From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 01:20:34 2015 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2000865E for ; Tue, 24 Feb 2015 01:20:34 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D9C11E11 for ; Tue, 24 Feb 2015 01:20:33 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1O1KRUQ096569 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Feb 2015 17:20:27 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1O1KQZK096568; Mon, 23 Feb 2015 17:20:26 -0800 (PST) (envelope-from jmg) Date: Mon, 23 Feb 2015 17:20:26 -0800 From: John-Mark Gurney To: arch@FreeBSD.org Subject: locks and kernel randomness... Message-ID: <20150224012026.GY46794@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Mon, 23 Feb 2015 17:20:27 -0800 (PST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 01:20:34 -0000 I'm working on simplifying kernel randomness interfaces. I would like to get read of all weak random generators, and this means replacing read_random and random(9) w/ effectively arc4rand(9) (to be replaced by ChaCha or Keccak in the future). The issue is that random(9) is called from any number of contexts, such as the scheduler. This makes locking a bit more interesting. Currently, both arc4rand(9) and yarrow/fortuna use a default mtx lock to protect their state. This obviously isn't compatible w/ the scheduler, and possibly other calling contexts. I have a patch[1] that unifies the random interface. It converts a few of the locks from mtx default to mtx spin to deal w/ this. If/when this is accepted, my next plan is to convert away from arc4rand, to either ChaCha or Keccak. I already have another patch that converts arc4rand and friends over to ChaCha. This patch does use PCPU data and sched_pin to help eliminate locks, but this does need more study. We could either do a restartable loop (but there might be too much state to safely do) or a critical section (though running chacha a bunch of times could have impact). [1] https://reviews.freebsd.org/D1956 -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 01:57:32 2015 Return-Path: Delivered-To: arch@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E212EFA for ; Tue, 24 Feb 2015 01:57:32 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 531E824E for ; Tue, 24 Feb 2015 01:57:32 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1O1vMi9005887 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 03:57:22 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1O1vMi9005887 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1O1vMH3005886; Tue, 24 Feb 2015 03:57:22 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 24 Feb 2015 03:57:22 +0200 From: Konstantin Belousov To: John-Mark Gurney Subject: Re: locks and kernel randomness... Message-ID: <20150224015721.GT74514@kib.kiev.ua> References: <20150224012026.GY46794@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150224012026.GY46794@funkthat.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: arch@FreeBSD.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 01:57:33 -0000 On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: > I'm working on simplifying kernel randomness interfaces. I would like > to get read of all weak random generators, and this means replacing > read_random and random(9) w/ effectively arc4rand(9) (to be replaced > by ChaCha or Keccak in the future). > > The issue is that random(9) is called from any number of contexts, such > as the scheduler. This makes locking a bit more interesting. Currently, > both arc4rand(9) and yarrow/fortuna use a default mtx lock to protect > their state. This obviously isn't compatible w/ the scheduler, and > possibly other calling contexts. > > I have a patch[1] that unifies the random interface. It converts a few > of the locks from mtx default to mtx spin to deal w/ this. This is definitely an overkill. The rebalancing minor use of randomness absolutely does not require cryptographical-strenght randomness to select a moment to rebalance thread queue. Imposing the spin lock on the whole random machinery just to allow the same random gathering code to be used for balance_ticks is detriment to the system responsivness. Scheduler is fine even with congruential generators, as you could see in the cpu_search(), look for the '69069'. Please do not enforce yet another spinlock for the system. From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 02:04:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1CE624B5 for ; Tue, 24 Feb 2015 02:04:22 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C4617353 for ; Tue, 24 Feb 2015 02:04:21 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 73291E497 for ; Mon, 23 Feb 2015 21:04:14 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.872 X-Spam-Level: X-Spam-Status: No, score=-3.872 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.165, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RZh-0gDceNMv for ; Mon, 23 Feb 2015 21:04:14 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 206A1E48E for ; Mon, 23 Feb 2015 21:04:14 -0500 (EST) Message-ID: <54EBDC1C.3060007@astrodoggroup.com> Date: Mon, 23 Feb 2015 18:04:12 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> In-Reply-To: <20150224015721.GT74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 02:04:22 -0000 On 02/23/15 17:57, Konstantin Belousov wrote: > On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: >> I'm working on simplifying kernel randomness interfaces. I would >> like to get read of all weak random generators, and this means >> replacing read_random and random(9) w/ effectively arc4rand(9) >> (to be replaced by ChaCha or Keccak in the future). >> >> The issue is that random(9) is called from any number of >> contexts, such as the scheduler. This makes locking a bit more >> interesting. Currently, both arc4rand(9) and yarrow/fortuna use >> a default mtx lock to protect their state. This obviously isn't >> compatible w/ the scheduler, and possibly other calling >> contexts. >> >> I have a patch[1] that unifies the random interface. It converts >> a few of the locks from mtx default to mtx spin to deal w/ this. > This is definitely an overkill. The rebalancing minor use of > randomness absolutely does not require cryptographical-strenght > randomness to select a moment to rebalance thread queue. Imposing > the spin lock on the whole random machinery just to allow the same > random gathering code to be used for balance_ticks is detriment to > the system responsivness. Scheduler is fine even with congruential > generators, as you could see in the cpu_search(), look for the > '69069'. > > Please do not enforce yet another spinlock for the system. > _______________________________________________ The patch attached to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches sched_balance to use get_cyclecount, which is also a suitable source of entropy for this purpose. It would also be possible to make the scheduler deterministic here, using cpuid or some such thing to make sure all CPUs don't fire the balancer at the same time. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 02:09:03 2015 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC4E75EE for ; Tue, 24 Feb 2015 02:09:03 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CDD8392 for ; Tue, 24 Feb 2015 02:09:03 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1O292WQ096909 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Feb 2015 18:09:02 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1O292nB096908; Mon, 23 Feb 2015 18:09:02 -0800 (PST) (envelope-from jmg) Date: Mon, 23 Feb 2015 18:09:02 -0800 From: John-Mark Gurney To: Konstantin Belousov Subject: Re: locks and kernel randomness... Message-ID: <20150224020902.GZ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150224015721.GT74514@kib.kiev.ua> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Mon, 23 Feb 2015 18:09:02 -0800 (PST) Cc: arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 02:09:03 -0000 Konstantin Belousov wrote this message on Tue, Feb 24, 2015 at 03:57 +0200: > On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: > > I'm working on simplifying kernel randomness interfaces. I would like > > to get read of all weak random generators, and this means replacing > > read_random and random(9) w/ effectively arc4rand(9) (to be replaced > > by ChaCha or Keccak in the future). > > > > The issue is that random(9) is called from any number of contexts, such > > as the scheduler. This makes locking a bit more interesting. Currently, > > both arc4rand(9) and yarrow/fortuna use a default mtx lock to protect > > their state. This obviously isn't compatible w/ the scheduler, and > > possibly other calling contexts. > > > > I have a patch[1] that unifies the random interface. It converts a few > > of the locks from mtx default to mtx spin to deal w/ this. > This is definitely an overkill. The rebalancing minor use of randomness > absolutely does not require cryptographical-strenght randomness to > select a moment to rebalance thread queue. Imposing the spin lock on > the whole random machinery just to allow the same random gathering code > to be used for balance_ticks is detriment to the system responsivness. > Scheduler is fine even with congruential generators, as you could see in > the cpu_search(), look for the '69069'. Then why doesn't it use this then? This is exactly why I don't want random to be a congruential generator... If you're so sure that you don't need cryptographic secure and that it's a performance benefit to do so, then you're free to roll your own, but almost all of the time, code won't meet both requirements. I haven't audited all the places where random is currently being called that might require not sleeping. The scheduler happens to be the first one I ran into... > Please do not enforce yet another spinlock for the system. I sent this email asking for help for how to avoid a spin lock. I'd appreciate if you could suggest how to improve my patch. Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 02:21:39 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 34393B0D for ; Tue, 24 Feb 2015 02:21:39 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C56A179F for ; Tue, 24 Feb 2015 02:21:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 4AF59E48B for ; Mon, 23 Feb 2015 21:21:37 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.871 X-Spam-Level: X-Spam-Status: No, score=-3.871 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.164, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id UjJg-4Djubv4 for ; Mon, 23 Feb 2015 21:21:37 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id E948BE495 for ; Mon, 23 Feb 2015 21:21:36 -0500 (EST) Message-ID: <54EBE02F.5070705@astrodoggroup.com> Date: Mon, 23 Feb 2015 18:21:35 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <20150224020902.GZ46794@funkthat.com> In-Reply-To: <20150224020902.GZ46794@funkthat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 02:21:39 -0000 On 02/23/15 18:09, John-Mark Gurney wrote: > Konstantin Belousov wrote this message on Tue, Feb 24, 2015 at > 03:57 +0200: >> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney >> wrote: >>> I'm working on simplifying kernel randomness interfaces. I >>> would like to get read of all weak random generators, and this >>> means replacing read_random and random(9) w/ effectively >>> arc4rand(9) (to be replaced by ChaCha or Keccak in the >>> future). >>> >>> The issue is that random(9) is called from any number of >>> contexts, such as the scheduler. This makes locking a bit more >>> interesting. Currently, both arc4rand(9) and yarrow/fortuna >>> use a default mtx lock to protect their state. This obviously >>> isn't compatible w/ the scheduler, and possibly other calling >>> contexts. >>> >>> I have a patch[1] that unifies the random interface. It >>> converts a few of the locks from mtx default to mtx spin to >>> deal w/ this. >> This is definitely an overkill. The rebalancing minor use of >> randomness absolutely does not require cryptographical-strenght >> randomness to select a moment to rebalance thread queue. Imposing >> the spin lock on the whole random machinery just to allow the >> same random gathering code to be used for balance_ticks is >> detriment to the system responsivness. Scheduler is fine even >> with congruential generators, as you could see in the >> cpu_search(), look for the '69069'. > > Then why doesn't it use this then? This is exactly why I don't > want random to be a congruential generator... If you're so sure > that you don't need cryptographic secure and that it's a > performance benefit to do so, then you're free to roll your own, > but almost all of the time, code won't meet both requirements. > > I haven't audited all the places where random is currently being > called that might require not sleeping. The scheduler happens to > be the first one I ran into... > >> Please do not enforce yet another spinlock for the system. > > I sent this email asking for help for how to avoid a spin lock. > I'd appreciate if you could suggest how to improve my patch. > > Thanks. > It seems to me that "random()" *should* return truly cryptographic randomness, while some other cheap mechanism (say randomish() or notrandom()), that explicitly isn't of cryptographic quality should be used for cases where "Sort of random" is good enough. This way, developers can actually decide which one they're going to use and where it's worth the performance cost to get real randomness. The scheduler doesn't happen to meet the latter case, so it's probably not a great example case and should be treated differently. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 02:43:01 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D5B1DEDE for ; Tue, 24 Feb 2015 02:43:01 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5F49D98E for ; Tue, 24 Feb 2015 02:43:01 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1O2gpUV016052 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 04:42:51 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1O2gpUV016052 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1O2gohf016051; Tue, 24 Feb 2015 04:42:50 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 24 Feb 2015 04:42:50 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150224024250.GV74514@kib.kiev.ua> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54EBDC1C.3060007@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 02:43:01 -0000 On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: > > > On 02/23/15 17:57, Konstantin Belousov wrote: > > On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: > >> I'm working on simplifying kernel randomness interfaces. I would > >> like to get read of all weak random generators, and this means > >> replacing read_random and random(9) w/ effectively arc4rand(9) > >> (to be replaced by ChaCha or Keccak in the future). > >> > >> The issue is that random(9) is called from any number of > >> contexts, such as the scheduler. This makes locking a bit more > >> interesting. Currently, both arc4rand(9) and yarrow/fortuna use > >> a default mtx lock to protect their state. This obviously isn't > >> compatible w/ the scheduler, and possibly other calling > >> contexts. > >> > >> I have a patch[1] that unifies the random interface. It converts > >> a few of the locks from mtx default to mtx spin to deal w/ this. > > This is definitely an overkill. The rebalancing minor use of > > randomness absolutely does not require cryptographical-strenght > > randomness to select a moment to rebalance thread queue. Imposing > > the spin lock on the whole random machinery just to allow the same > > random gathering code to be used for balance_ticks is detriment to > > the system responsivness. Scheduler is fine even with congruential > > generators, as you could see in the cpu_search(), look for the > > '69069'. > > > > Please do not enforce yet another spinlock for the system. > > _______________________________________________ > > The patch attached to > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches > sched_balance to use get_cyclecount, which is also a suitable source > of entropy for this purpose. > > It would also be possible to make the scheduler deterministic here, > using cpuid or some such thing to make sure all CPUs don't fire the > balancer at the same time. > The patch in the PR is probably in the right direction, but might be too simple, unless somebody dispel my fallacy. I remember seeing claims that on the very low-end embedded devices the get_cyclecount() method may be non-functional, i.e. returning some constant, probably 0. I somehow associate MIPS arch with this bias. From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 04:23:51 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA729CFF for ; Tue, 24 Feb 2015 04:23:51 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 97C0B657 for ; Tue, 24 Feb 2015 04:23:51 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1O4NnBL097868 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 23 Feb 2015 20:23:49 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1O4NmJW097867; Mon, 23 Feb 2015 20:23:48 -0800 (PST) (envelope-from jmg) Date: Mon, 23 Feb 2015 20:23:48 -0800 From: John-Mark Gurney To: Konstantin Belousov Subject: Re: locks and kernel randomness... Message-ID: <20150224042348.GA46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150224024250.GV74514@kib.kiev.ua> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Mon, 23 Feb 2015 20:23:49 -0800 (PST) Cc: Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 04:23:51 -0000 Konstantin Belousov wrote this message on Tue, Feb 24, 2015 at 04:42 +0200: > On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: > > > > > > On 02/23/15 17:57, Konstantin Belousov wrote: > > > On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: > > >> I'm working on simplifying kernel randomness interfaces. I would > > >> like to get read of all weak random generators, and this means > > >> replacing read_random and random(9) w/ effectively arc4rand(9) > > >> (to be replaced by ChaCha or Keccak in the future). > > >> > > >> The issue is that random(9) is called from any number of > > >> contexts, such as the scheduler. This makes locking a bit more > > >> interesting. Currently, both arc4rand(9) and yarrow/fortuna use > > >> a default mtx lock to protect their state. This obviously isn't > > >> compatible w/ the scheduler, and possibly other calling > > >> contexts. > > >> > > >> I have a patch[1] that unifies the random interface. It converts > > >> a few of the locks from mtx default to mtx spin to deal w/ this. > > > This is definitely an overkill. The rebalancing minor use of > > > randomness absolutely does not require cryptographical-strenght > > > randomness to select a moment to rebalance thread queue. Imposing > > > the spin lock on the whole random machinery just to allow the same > > > random gathering code to be used for balance_ticks is detriment to > > > the system responsivness. Scheduler is fine even with congruential > > > generators, as you could see in the cpu_search(), look for the > > > '69069'. > > > > > > Please do not enforce yet another spinlock for the system. > > > _______________________________________________ > > > > The patch attached to > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches > > sched_balance to use get_cyclecount, which is also a suitable source > > of entropy for this purpose. > > > > It would also be possible to make the scheduler deterministic here, > > using cpuid or some such thing to make sure all CPUs don't fire the > > balancer at the same time. > > > > The patch in the PR is probably in the right direction, but might be too > simple, unless somebody dispel my fallacy. I remember seeing claims that > on the very low-end embedded devices the get_cyclecount() method may > be non-functional, i.e. returning some constant, probably 0. I somehow > associate MIPS arch with this bias. Well, the docs say: The speed and the maximum value of each counter is CPU-dependent. Some CPUs (such as the Intel 80486) do not have such a register, so get_cyclecount() on these platforms returns a (monotonic) combination of numbers represented by the structure returned by binuptime(9). The clang builtin of cycle counter is documented as returning 0 when it isn't available. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 04:36:49 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90C12FD0 for ; Tue, 24 Feb 2015 04:36:49 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 304127E5 for ; Tue, 24 Feb 2015 04:36:48 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 11D17E480; Mon, 23 Feb 2015 23:36:46 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.871 X-Spam-Level: X-Spam-Status: No, score=-3.871 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.164, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 05McCMVgZfHK; Mon, 23 Feb 2015 23:36:45 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 8C333E47F; Mon, 23 Feb 2015 23:36:45 -0500 (EST) Message-ID: <54EBFFDC.4090905@astrodoggroup.com> Date: Mon, 23 Feb 2015 20:36:44 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Konstantin Belousov Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> In-Reply-To: <20150224024250.GV74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 04:36:49 -0000 On 02/23/15 18:42, Konstantin Belousov wrote: > On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: >> >> >> On 02/23/15 17:57, Konstantin Belousov wrote: >>> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: >>>> I'm working on simplifying kernel randomness interfaces. I would >>>> like to get read of all weak random generators, and this means >>>> replacing read_random and random(9) w/ effectively arc4rand(9) >>>> (to be replaced by ChaCha or Keccak in the future). >>>> >>>> The issue is that random(9) is called from any number of >>>> contexts, such as the scheduler. This makes locking a bit more >>>> interesting. Currently, both arc4rand(9) and yarrow/fortuna use >>>> a default mtx lock to protect their state. This obviously isn't >>>> compatible w/ the scheduler, and possibly other calling >>>> contexts. >>>> >>>> I have a patch[1] that unifies the random interface. It converts >>>> a few of the locks from mtx default to mtx spin to deal w/ this. >>> This is definitely an overkill. The rebalancing minor use of >>> randomness absolutely does not require cryptographical-strenght >>> randomness to select a moment to rebalance thread queue. Imposing >>> the spin lock on the whole random machinery just to allow the same >>> random gathering code to be used for balance_ticks is detriment to >>> the system responsivness. Scheduler is fine even with congruential >>> generators, as you could see in the cpu_search(), look for the >>> '69069'. >>> >>> Please do not enforce yet another spinlock for the system. >>> _______________________________________________ >> >> The patch attached to >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches >> sched_balance to use get_cyclecount, which is also a suitable source >> of entropy for this purpose. >> >> It would also be possible to make the scheduler deterministic here, >> using cpuid or some such thing to make sure all CPUs don't fire the >> balancer at the same time. >> > > The patch in the PR is probably in the right direction, but might be too > simple, unless somebody dispel my fallacy. I remember seeing claims that > on the very low-end embedded devices the get_cyclecount() method may > be non-functional, i.e. returning some constant, probably 0. I somehow > associate MIPS arch with this bias. > Talking to some of the arm and MIPS developers, it appears get_cyclecount() may be slow on some older ARM hardware... (In particular, hardware that doesn't support SMP anyway.) However, after a quick test on some machines here, I don't think this function actually needs randomness, due to the large number of other pathways ULE uses to balance load. New patch attached to the PR that simply removes the randomness entirely. From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 06:42:04 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BA6F87BD for ; Tue, 24 Feb 2015 06:42:04 +0000 (UTC) Received: from mail107.syd.optusnet.com.au (mail107.syd.optusnet.com.au [211.29.132.53]) by mx1.freebsd.org (Postfix) with ESMTP id 0000F63D for ; Tue, 24 Feb 2015 06:42:02 +0000 (UTC) Received: from c211-30-166-197.carlnfd1.nsw.optusnet.com.au (c211-30-166-197.carlnfd1.nsw.optusnet.com.au [211.30.166.197]) by mail107.syd.optusnet.com.au (Postfix) with ESMTPS id 4F712D445DC; Tue, 24 Feb 2015 17:15:45 +1100 (AEDT) Date: Tue, 24 Feb 2015 17:15:45 +1100 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: John-Mark Gurney Subject: Re: locks and kernel randomness... In-Reply-To: <20150224042348.GA46794@funkthat.com> Message-ID: <20150224162133.J1477@besplex.bde.org> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224042348.GA46794@funkthat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.1 cv=A5NVYcmG c=1 sm=1 tr=0 a=KA6XNC2GZCFrdESI5ZmdjQ==:117 a=PO7r1zJSAAAA:8 a=kj9zAlcOel0A:10 a=JzwRw_2MAAAA:8 a=6I5d2MoRAAAA:8 a=BnExqU6ZlYZmNHGlKycA:9 a=CjuIK1q_8ugA:10 Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 06:42:04 -0000 On Mon, 23 Feb 2015, John-Mark Gurney wrote: > Konstantin Belousov wrote this message on Tue, Feb 24, 2015 at 04:42 +0200: >> On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: >>> >>> The patch attached to >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches >>> sched_balance to use get_cyclecount, which is also a suitable source >>> of entropy for this purpose. >>> >>> It would also be possible to make the scheduler deterministic here, >>> using cpuid or some such thing to make sure all CPUs don't fire the >>> balancer at the same time. >> >> The patch in the PR is probably in the right direction, but might be too >> simple, unless somebody dispel my fallacy. I remember seeing claims that >> on the very low-end embedded devices the get_cyclecount() method may >> be non-functional, i.e. returning some constant, probably 0. I somehow >> associate MIPS arch with this bias. mips seems to use only a hardware cycle counter now. This is obfuscated by spelling the function name as mips_rd_ ## count in its implementation. The only arch with a very slow get_cyclecount() now seem to be i386 without a TSC and arm (some cases). > Well, the docs say: > The speed and the maximum value of each counter is CPU-dependent. Some > CPUs (such as the Intel 80486) do not have such a register, so > get_cyclecount() on these platforms returns a (monotonic) combination of > numbers represented by the structure returned by binuptime(9). The docs are wrong of course. Almost as bad as the design of this function. arm now does: X #ifdef _KERNEL X #if __ARM_ARCH >= 6 X #include X #endif X static __inline uint64_t X get_cyclecount(void) X { X #if __ARM_ARCH >= 6 X return cp15_pmccntr_get(); X #else /* No performance counters, so use binuptime(9). This is slooooow */ X struct bintime bt; X X binuptime(&bt); X return ((uint64_t)bt.sec << 56 | bt.frac >> 8); X #endif X } X #endif Versions used to return the highly non-monotonic (bt.sec ^ bt_frac). This was best for randomness. But get_cyclecount() was often abused for timestamps. So arm now does the above. It is still non-monotonic. It wraps every 256 seconds. It discards noise in the lower 8 bits of bt.frac to keep predictable values in the bt.sec. The noise is often 0, but may contain fairly predictable values from ntpd adjustments. i386 now uses cpu_ticks(). This partly defeats one of the excuses for the existence of get_cyclecount(). Using binuptime() directly instead of get_cyclecount() micro-optimized to an inline rdtsc would have cost a whole extra 10-20 cycles on modern x86. Using cpu_ticks() directly restores some of this cost. In the best case, rdtsc is now 2 function calls away (1 indirect: call cpu_ticks(), then an indirect call to rdtsc()). The costs of these things on (old) Athlon64 are: inline rdtsc: 13 cycles counting loop overhead add 1 direct function call: +5 cycles add another function call: direct +12 cycles; indirect +10 cycles 28 cycles for a function like cpu_ticks(). IIRC, binuptime() took 58 cycles on (older) AthlonXP. Newer x86 and maybe Intel instead of AMD changes these times significantly. Intel rdtsc always seemed to be slower than AMD rdtsc. Synchronization for P-state invariance is expensive. So rdtc on newer x86 takes 40-70 cycles. The extras for timecounters are more in the noise. Something like 90-100 cycles total, with slightly more than half for hardware. When the TSC is not available, cpu_ticks() may use another cputicker, but usually there is none, since if there is a cputicker then it is a TSC by another name, and cpu_ticks uses a timecounter. It doesn't do this in the safe way using binuptime(), but returns the hardware timecount with racy adjustments for monotonicity. It has to read the hardware timecounter, and that is the slow part. So this method is only used in cases where it is too slow to use. Apart from its implementation bugs, cpu_ticks() is better than the special adjustment for arm. get_cyclecount() is abused for timestamps in de, ktr and sctp. There is no support for converting these "timestamps" to human-readable values even in the easy case where get_cyclecount() returns a monotonic counter. The network places should just use binuptime(). ktr shouldn't do that, and is probably broken in at least some cases where it does do that via get_cyclecount() using the hardware timecounter. Bruce From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 13:46:36 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6C885214 for ; Tue, 24 Feb 2015 13:46:36 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0D1CCBA1 for ; Tue, 24 Feb 2015 13:46:35 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1ODkRFr064626 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 15:46:27 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1ODkRFr064626 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1ODkRJX064625; Tue, 24 Feb 2015 15:46:27 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 24 Feb 2015 15:46:27 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150224134627.GX74514@kib.kiev.ua> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <54EBFFDC.4090905@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54EBFFDC.4090905@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 13:46:36 -0000 On Mon, Feb 23, 2015 at 08:36:44PM -0800, Harrison Grundy wrote: > > > On 02/23/15 18:42, Konstantin Belousov wrote: > > On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: > >> > >> > >> On 02/23/15 17:57, Konstantin Belousov wrote: > >>> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: > >>>> I'm working on simplifying kernel randomness interfaces. I would > >>>> like to get read of all weak random generators, and this means > >>>> replacing read_random and random(9) w/ effectively arc4rand(9) > >>>> (to be replaced by ChaCha or Keccak in the future). > >>>> > >>>> The issue is that random(9) is called from any number of > >>>> contexts, such as the scheduler. This makes locking a bit more > >>>> interesting. Currently, both arc4rand(9) and yarrow/fortuna use > >>>> a default mtx lock to protect their state. This obviously isn't > >>>> compatible w/ the scheduler, and possibly other calling > >>>> contexts. > >>>> > >>>> I have a patch[1] that unifies the random interface. It converts > >>>> a few of the locks from mtx default to mtx spin to deal w/ this. > >>> This is definitely an overkill. The rebalancing minor use of > >>> randomness absolutely does not require cryptographical-strenght > >>> randomness to select a moment to rebalance thread queue. Imposing > >>> the spin lock on the whole random machinery just to allow the same > >>> random gathering code to be used for balance_ticks is detriment to > >>> the system responsivness. Scheduler is fine even with congruential > >>> generators, as you could see in the cpu_search(), look for the > >>> '69069'. > >>> > >>> Please do not enforce yet another spinlock for the system. > >>> _______________________________________________ > >> > >> The patch attached to > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 switches > >> sched_balance to use get_cyclecount, which is also a suitable source > >> of entropy for this purpose. > >> > >> It would also be possible to make the scheduler deterministic here, > >> using cpuid or some such thing to make sure all CPUs don't fire the > >> balancer at the same time. > >> > > > > The patch in the PR is probably in the right direction, but might be too > > simple, unless somebody dispel my fallacy. I remember seeing claims that > > on the very low-end embedded devices the get_cyclecount() method may > > be non-functional, i.e. returning some constant, probably 0. I somehow > > associate MIPS arch with this bias. > > > > Talking to some of the arm and MIPS developers, it appears > get_cyclecount() may be slow on some older ARM hardware... (In > particular, hardware that doesn't support SMP anyway.) > > However, after a quick test on some machines here, I don't think this > function actually needs randomness, due to the large number of other > pathways ULE uses to balance load. Well, system does benefit from some irregularity in a scheduler. It is in fact required for locks to work. > > New patch attached to the PR that simply removes the randomness entirely. Since my groundless worries about get_cyclecount() were defeated, I think your first patch is fine and is the way to go. From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 14:56:34 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04726BFD for ; Tue, 24 Feb 2015 14:56:34 +0000 (UTC) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7C053EA for ; Tue, 24 Feb 2015 14:56:33 +0000 (UTC) Received: by pdbfl12 with SMTP id fl12so33876792pdb.2 for ; Tue, 24 Feb 2015 06:56:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=qd2rJu/hVVb/0fXWErgCPKWobS+HXNHUWrUalW7+5Vk=; b=Bifx7ZCsYpGcS75KbCWDzuIaQ+R+BZ+k0D4Sc9QW3yCEUl0yZ7vrT2FzI7DwibWGJy V9hQ3gsQQHSxEO9wAzyfLyEhdB0KVtPQlUzSeNgwv36GAfx3GeaS1Hh0l8HVAdmxGgIY mWyihjLtu+WQy7UkBQfe/ucTSHQXADDlIrVy71jCVOsGqOWiViwuE21sO7ML+0t71kAG Mv+vNZqWJzgx3p9EWq6iGQ3rYJvXD9id75WfMHwuEotk9x2gHPwoND3tIOwoEmCD0b+q g+2vkU5zhHD5TOQmMGRJ3oryHTXU3QH+s5J46qomD2HpIYCl+hYc0m5eptwgplAgqEkj y79g== X-Gm-Message-State: ALoCoQmle8iXR5nU3o+t2xts49qQYtB0F/sVKTe59AHLAHCI6LhcM5JPd812MDoI7j9v6LSV5Nyw X-Received: by 10.70.41.161 with SMTP id g1mr25374574pdl.43.1424789786845; Tue, 24 Feb 2015 06:56:26 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id fc6sm9049022pab.6.2015.02.24.06.56.25 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 06:56:26 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150224024250.GV74514@kib.kiev.ua> Date: Tue, 24 Feb 2015 07:56:24 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.2070.6) Cc: Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 14:56:34 -0000 > On Feb 23, 2015, at 7:42 PM, Konstantin Belousov = wrote: >=20 > On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: >>=20 >>=20 >> On 02/23/15 17:57, Konstantin Belousov wrote: >>> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: >>>> I'm working on simplifying kernel randomness interfaces. I would >>>> like to get read of all weak random generators, and this means >>>> replacing read_random and random(9) w/ effectively arc4rand(9) >>>> (to be replaced by ChaCha or Keccak in the future). >>>>=20 >>>> The issue is that random(9) is called from any number of >>>> contexts, such as the scheduler. This makes locking a bit more >>>> interesting. Currently, both arc4rand(9) and yarrow/fortuna use >>>> a default mtx lock to protect their state. This obviously isn't >>>> compatible w/ the scheduler, and possibly other calling >>>> contexts. >>>>=20 >>>> I have a patch[1] that unifies the random interface. It converts >>>> a few of the locks from mtx default to mtx spin to deal w/ this. >>> This is definitely an overkill. The rebalancing minor use of >>> randomness absolutely does not require cryptographical-strenght >>> randomness to select a moment to rebalance thread queue. Imposing >>> the spin lock on the whole random machinery just to allow the same >>> random gathering code to be used for balance_ticks is detriment to >>> the system responsivness. Scheduler is fine even with congruential >>> generators, as you could see in the cpu_search(), look for the >>> '69069'. >>>=20 >>> Please do not enforce yet another spinlock for the system.=20 >>> _______________________________________________ >>=20 >> The patch attached to >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197922 switches >> sched_balance to use get_cyclecount, which is also a suitable source >> of entropy for this purpose. >>=20 >> It would also be possible to make the scheduler deterministic here, >> using cpuid or some such thing to make sure all CPUs don't fire the >> balancer at the same time. >>=20 >=20 > The patch in the PR is probably in the right direction, but might be = too > simple, unless somebody dispel my fallacy. I remember seeing claims = that > on the very low-end embedded devices the get_cyclecount() method may > be non-functional, i.e. returning some constant, probably 0. I somehow > associate MIPS arch with this bias. arm v4/v5 don=E2=80=99t have get_cyclecount() in hardware. It simply = doesn=E2=80=99t exist. However, this patch is only for SMP, which also isn=E2=80=99t available = on arm v4/v5 in our tree. MIPS=E2=80=99 get cycle count, though, has been defined since R4k days = and so much software depends on it, it would surprise me if that was eliminated to = save silicon. Then again, if you want to change random(), provide a weak_random() = that=E2=80=99s the traditional non-crypto thing that=E2=80=99s fast and lockless. That = would make it easy to audit in our tree. The scheduler doesn=E2=80=99t need cryptographic = randomness, it just needs to make different choices sometimes to ensure its notion of = fairness. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 14:58:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 953FFCC0 for ; Tue, 24 Feb 2015 14:58:22 +0000 (UTC) Received: from mail-pd0-f177.google.com (mail-pd0-f177.google.com [209.85.192.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 64A545E3 for ; Tue, 24 Feb 2015 14:58:22 +0000 (UTC) Received: by pdbfl12 with SMTP id fl12so33883099pdb.4 for ; Tue, 24 Feb 2015 06:58:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=Kn8s07214KOR7UmcUTt1A6DN3YqzmFc4csgwx7+x3aM=; b=V1GJaLqEuWQRqO66q1G7km1EAMoLlF9ymkBLT1mCl66ND2qPMF1jIdJ2NGXulUh7rH hBhp7W24WEJQ0DKxRdCfDeSONMwfIt0WPcDHftGyjJ6vrClMVXWcaPa8neYwqmd/Gabb hp6O5b20npEDt4A4gOBmihNOBKPQbOsORU/ZVHPTfOJuHrJXGJBplwViV9QZBJbBUkPV ZzCNi9iX5AipptW+3/+fX1zGehDsOUarmd4OR90YBLmJZltvVq7ydDOs1VEenLmPgTTv mrm9eKIfq7Ipmhi800SfPxA23O+5+qoAKaR7i5mcL/JmiSvZVU8shs19A7U6KHrHmOxe G8Hw== X-Gm-Message-State: ALoCoQnvWh3m9V5yCKZLJXnXRN3BQHcFljDzt34KLKhEP6AePaQP6HBPlzisu37iJJHs1KFZLilr X-Received: by 10.68.69.108 with SMTP id d12mr4262793pbu.137.1424789901699; Tue, 24 Feb 2015 06:58:21 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id jw8sm38372088pbc.73.2015.02.24.06.58.20 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 06:58:21 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <54EBFFDC.4090905@astrodoggroup.com> Date: Tue, 24 Feb 2015 07:58:19 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <54EBFFDC.4090905@astrodoggroup.com> To: Harrison Grundy X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 14:58:22 -0000 > On Feb 23, 2015, at 9:36 PM, Harrison Grundy = wrote: >=20 >=20 >=20 > On 02/23/15 18:42, Konstantin Belousov wrote: >> On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy wrote: >>>=20 >>>=20 >>> On 02/23/15 17:57, Konstantin Belousov wrote: >>>> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney wrote: >>>>> I'm working on simplifying kernel randomness interfaces. I would >>>>> like to get read of all weak random generators, and this means >>>>> replacing read_random and random(9) w/ effectively arc4rand(9) >>>>> (to be replaced by ChaCha or Keccak in the future). >>>>>=20 >>>>> The issue is that random(9) is called from any number of >>>>> contexts, such as the scheduler. This makes locking a bit more >>>>> interesting. Currently, both arc4rand(9) and yarrow/fortuna use >>>>> a default mtx lock to protect their state. This obviously isn't >>>>> compatible w/ the scheduler, and possibly other calling >>>>> contexts. >>>>>=20 >>>>> I have a patch[1] that unifies the random interface. It converts >>>>> a few of the locks from mtx default to mtx spin to deal w/ this. >>>> This is definitely an overkill. The rebalancing minor use of >>>> randomness absolutely does not require cryptographical-strenght >>>> randomness to select a moment to rebalance thread queue. Imposing >>>> the spin lock on the whole random machinery just to allow the same >>>> random gathering code to be used for balance_ticks is detriment to >>>> the system responsivness. Scheduler is fine even with congruential >>>> generators, as you could see in the cpu_search(), look for the >>>> '69069'. >>>>=20 >>>> Please do not enforce yet another spinlock for the system.=20 >>>> _______________________________________________ >>>=20 >>> The patch attached to >>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D197922 switches >>> sched_balance to use get_cyclecount, which is also a suitable source >>> of entropy for this purpose. >>>=20 >>> It would also be possible to make the scheduler deterministic here, >>> using cpuid or some such thing to make sure all CPUs don't fire the >>> balancer at the same time. >>>=20 >>=20 >> The patch in the PR is probably in the right direction, but might be = too >> simple, unless somebody dispel my fallacy. I remember seeing claims = that >> on the very low-end embedded devices the get_cyclecount() method may >> be non-functional, i.e. returning some constant, probably 0. I = somehow >> associate MIPS arch with this bias. >>=20 >=20 > Talking to some of the arm and MIPS developers, it appears > get_cyclecount() may be slow on some older ARM hardware... (In > particular, hardware that doesn't support SMP anyway.) It simply doesn=E2=80=99t exist on older ARM hardware. Some SoCs have something similar to a real-time clock that you can read, but that=E2=80=99= s not reliable for this use. > However, after a quick test on some machines here, I don't think this > function actually needs randomness, due to the large number of other > pathways ULE uses to balance load. >=20 > New patch attached to the PR that simply removes the randomness = entirely. Are you sure about that? Warner From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 17:08:05 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22C817A2 for ; Tue, 24 Feb 2015 17:08:05 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B99FD844 for ; Tue, 24 Feb 2015 17:08:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 4C50AE3E5 for ; Tue, 24 Feb 2015 12:08:02 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.869 X-Spam-Level: X-Spam-Status: No, score=-3.869 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.162, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id i0qCbApSVs1G for ; Tue, 24 Feb 2015 12:08:02 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id E4795E22B for ; Tue, 24 Feb 2015 12:08:01 -0500 (EST) Message-ID: <54ECAFF2.2050903@astrodoggroup.com> Date: Tue, 24 Feb 2015 09:08:02 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <54EBFFDC.4090905@astrodoggroup.com> In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 17:08:05 -0000 On 02/24/15 06:58, Warner Losh wrote: > >> On Feb 23, 2015, at 9:36 PM, Harrison Grundy >> wrote: >> >> >> >> On 02/23/15 18:42, Konstantin Belousov wrote: >>> On Mon, Feb 23, 2015 at 06:04:12PM -0800, Harrison Grundy >>> wrote: >>>> >>>> >>>> On 02/23/15 17:57, Konstantin Belousov wrote: >>>>> On Mon, Feb 23, 2015 at 05:20:26PM -0800, John-Mark Gurney >>>>> wrote: >>>>>> I'm working on simplifying kernel randomness interfaces. >>>>>> I would like to get read of all weak random generators, >>>>>> and this means replacing read_random and random(9) w/ >>>>>> effectively arc4rand(9) (to be replaced by ChaCha or >>>>>> Keccak in the future). >>>>>> >>>>>> The issue is that random(9) is called from any number of >>>>>> contexts, such as the scheduler. This makes locking a >>>>>> bit more interesting. Currently, both arc4rand(9) and >>>>>> yarrow/fortuna use a default mtx lock to protect their >>>>>> state. This obviously isn't compatible w/ the scheduler, >>>>>> and possibly other calling contexts. >>>>>> >>>>>> I have a patch[1] that unifies the random interface. It >>>>>> converts a few of the locks from mtx default to mtx spin >>>>>> to deal w/ this. >>>>> This is definitely an overkill. The rebalancing minor use >>>>> of randomness absolutely does not require >>>>> cryptographical-strenght randomness to select a moment to >>>>> rebalance thread queue. Imposing the spin lock on the whole >>>>> random machinery just to allow the same random gathering >>>>> code to be used for balance_ticks is detriment to the >>>>> system responsivness. Scheduler is fine even with >>>>> congruential generators, as you could see in the >>>>> cpu_search(), look for the '69069'. >>>>> >>>>> Please do not enforce yet another spinlock for the system. >>>>> _______________________________________________ >>>> >>>> The patch attached to >>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 >>>> switches sched_balance to use get_cyclecount, which is also a >>>> suitable source of entropy for this purpose. >>>> >>>> It would also be possible to make the scheduler deterministic >>>> here, using cpuid or some such thing to make sure all CPUs >>>> don't fire the balancer at the same time. >>>> >>> >>> The patch in the PR is probably in the right direction, but >>> might be too simple, unless somebody dispel my fallacy. I >>> remember seeing claims that on the very low-end embedded >>> devices the get_cyclecount() method may be non-functional, i.e. >>> returning some constant, probably 0. I somehow associate MIPS >>> arch with this bias. >>> >> >> Talking to some of the arm and MIPS developers, it appears >> get_cyclecount() may be slow on some older ARM hardware... (In >> particular, hardware that doesn't support SMP anyway.) > > It simply doesn’t exist on older ARM hardware. Some SoCs have > something similar to a real-time clock that you can read, but > that’s not reliable for this use. > >> However, after a quick test on some machines here, I don't think >> this function actually needs randomness, due to the large number >> of other pathways ULE uses to balance load. >> >> New patch attached to the PR that simply removes the randomness >> entirely. > > Are you sure about that? I'm testing on an 8 core AMD Bulldozer machine without any noticable issues. You could game the scheduler a bit by running near the "beginning" of the balance interval, but the preemption, idle steal, and priority recalculation-caused migrations pretty much wipe out the effect. That being said, get_cyclecount is pretty cheap, and this code doesn't run *that* often, so if there's a rare edge case I'm not running into that benefits from it, I suspect it's worth keeping the... 'faux' randomness in there somewhere. Anyone else want to test it? --- Harrison > > Warner > > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch To > unsubscribe, send any mail to > "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 17:40:57 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 60ADE1B2 for ; Tue, 24 Feb 2015 17:40:57 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3529FBFA for ; Tue, 24 Feb 2015 17:40:57 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1OHer9W004332 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 09:40:53 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1OHerSJ004331; Tue, 24 Feb 2015 09:40:53 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 09:40:53 -0800 From: John-Mark Gurney To: Warner Losh Subject: Re: locks and kernel randomness... Message-ID: <20150224174053.GG46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 09:40:53 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 17:40:57 -0000 Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > Then again, if you want to change random(), provide a weak_random() that???s > the traditional non-crypto thing that???s fast and lockless. That would make it easy > to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > just needs to make different choices sometimes to ensure its notion of fairness. I do not support having a weak_random... If the consumer is sure enough that you don't need a secure random, then they can pick an LCG and implement it themselves and deal (or not) w/ the locking issues... It appears that the scheduler had an LCG but for some reason the authors didn't feel like using it here.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 18:01:30 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0D97C5D0 for ; Tue, 24 Feb 2015 18:01:30 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id EB45CE8B for ; Tue, 24 Feb 2015 18:01:29 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.133.26.10]) by elvis.mu.org (Postfix) with ESMTPSA id DE9D4341F910; Tue, 24 Feb 2015 10:01:28 -0800 (PST) Message-ID: <54ECBD4B.6000007@freebsd.org> Date: Tue, 24 Feb 2015 13:04:59 -0500 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: John-Mark Gurney , Warner Losh Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> In-Reply-To: <20150224174053.GG46794@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:01:30 -0000 On 2/24/15 12:40 PM, John-Mark Gurney wrote: > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: >> Then again, if you want to change random(), provide a weak_random() that???s >> the traditional non-crypto thing that???s fast and lockless. That would make it easy >> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it >> just needs to make different choices sometimes to ensure its notion of fairness. > > I do not support having a weak_random... If the consumer is sure > enough that you don't need a secure random, then they can pick an LCG > and implement it themselves and deal (or not) w/ the locking issues... > > It appears that the scheduler had an LCG but for some reason the authors > didn't feel like using it here.. > The way I read this argument is that no low quality sources of randomness shall be allowed. So we should get rid of rand(3)? When do we deprecate that? Your argument doesn't hold water. -Alfred From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 18:03:52 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB799754 for ; Tue, 24 Feb 2015 18:03:52 +0000 (UTC) Received: from mail-pa0-f53.google.com (mail-pa0-f53.google.com [209.85.220.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9A537EBB for ; Tue, 24 Feb 2015 18:03:52 +0000 (UTC) Received: by pablf10 with SMTP id lf10so37703327pab.12 for ; Tue, 24 Feb 2015 10:03:46 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=ZxHyHyJdYRPvWo4m/bEcshGVwA7uobtfE+IsOzyoWbk=; b=bPN2+QpA2QBmft7N+gO32bML70pdN9Z6RSbMfkpSxJUREpg2g6BZrh12tqpVZsK5RJ b4DKZa8ToXO4o6Zk7/7dvmLgBryUruLMC5mqa/4hED4yUk5kpUPXr76PCCS2S+LXAFeb nQiuGLvwxEkftdVcYfyht+ZFgT/2CDH5LO+zK9TnupGhJ2PifW4h8AKpsxxpcFg56SIM JDsjNeBKus3jUkOUxxZYLez90NvXj6BlnC9dWeBopQtF4NF4EwwWfEVrqaKPNFPlnajE SINHmYse2N05rVW5NCCRRQhK81n4duNeoTacPnaKD+HjFiiBVXRkNC3N/uvWI3lBAfWu VGeQ== X-Gm-Message-State: ALoCoQmtJ2iTFrzRf/2Smk+8FIVRc8ap37aYHFjDgRX1Y0aUCQSdbHrNA2EN4Dhg5hQqi0JmnDpA X-Received: by 10.66.165.105 with SMTP id yx9mr30423678pab.145.1424801025921; Tue, 24 Feb 2015 10:03:45 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id rr9sm38801902pbc.39.2015.02.24.10.03.44 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 10:03:45 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150224174053.GG46794@funkthat.com> Date: Tue, 24 Feb 2015 11:03:42 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:03:52 -0000 > On Feb 24, 2015, at 10:40 AM, John-Mark Gurney = wrote: >=20 > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: >> Then again, if you want to change random(), provide a weak_random() = that???s >> the traditional non-crypto thing that???s fast and lockless. That = would make it easy >> to audit in our tree. The scheduler doesn???t need cryptographic = randomness, it >> just needs to make different choices sometimes to ensure its notion = of fairness. >=20 > I do not support having a weak_random... If the consumer is sure > enough that you don't need a secure random, then they can pick an LCG > and implement it themselves and deal (or not) w/ the locking issues... >=20 > It appears that the scheduler had an LCG but for some reason the = authors > didn't feel like using it here.. Why don=E2=80=99t you support having a common random routine that=E2=80=99= s to mix the pot, but not cryptographically secure? Lots of algorithms use them, and = having a common one would keep us from reinventing the wheel. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 18:25:11 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7316BEE9; Tue, 24 Feb 2015 18:25:11 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 36E8E176; Tue, 24 Feb 2015 18:25:11 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1OIP7k3004633 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 10:25:07 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1OIP7NI004632; Tue, 24 Feb 2015 10:25:07 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 10:25:07 -0800 From: John-Mark Gurney To: Alfred Perlstein Subject: Re: locks and kernel randomness... Message-ID: <20150224182507.GI46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ECBD4B.6000007@freebsd.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 10:25:07 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:25:11 -0000 Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 -0500: > On 2/24/15 12:40 PM, John-Mark Gurney wrote: > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > >> Then again, if you want to change random(), provide a weak_random() that???s > >> the traditional non-crypto thing that???s fast and lockless. That would make it easy > >> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > >> just needs to make different choices sometimes to ensure its notion of fairness. > > > > I do not support having a weak_random... If the consumer is sure > > enough that you don't need a secure random, then they can pick an LCG > > and implement it themselves and deal (or not) w/ the locking issues... > > > > It appears that the scheduler had an LCG but for some reason the authors > > didn't feel like using it here.. > > The way I read this argument is that no low quality sources of > randomness shall be allowed. No, I'm saying that the person who needs the predictable randomness needs to do extra work to get it... If they care that much about performance/predictability/etc, then a little extra work won't hurt them.. And if they don't know what an LCG is, then they aren't qualified to make the decision that a weaker RNG is correct for their situation.. > So we should get rid of rand(3)? When do we deprecate that? No, we should replace it w/ proper randomness like OpenBSD has... I'm willing to go that far and I think FreeBSD should... OpenBSD has done a lot of leg work in tracking down ports that correctly use rand(3), and letting them keep their deterministic randomness, while the remaining get real random.. > Your argument doesn't hold water. Sorry, you're argument sounds like it's from the 90's when we didn't know any better on how to make secure systems... Will you promise to audit all new uses of randomness in the system to make sure that they are using the correct, secure API? Considering that it's been recommended that people NOT use read_random(9) for 14 years, yet people continue to use it in new code, demonstrates that people do not know what they are doing (wrt randomness), and the only way to make sure they do the correct, secure thing is to only provide the secure API... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 18:30:54 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 987AAFF1 for ; Tue, 24 Feb 2015 18:30:54 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 56A811B3 for ; Tue, 24 Feb 2015 18:30:54 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1OIUpTm004681 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 10:30:51 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1OIUpGl004680; Tue, 24 Feb 2015 10:30:51 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 10:30:51 -0800 From: John-Mark Gurney To: Warner Losh Subject: Re: locks and kernel randomness... Message-ID: <20150224183051.GJ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 10:30:51 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:30:54 -0000 Warner Losh wrote this message on Tue, Feb 24, 2015 at 11:03 -0700: > > > On Feb 24, 2015, at 10:40 AM, John-Mark Gurney wrote: > > > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > >> Then again, if you want to change random(), provide a weak_random() that???s > >> the traditional non-crypto thing that???s fast and lockless. That would make it easy > >> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > >> just needs to make different choices sometimes to ensure its notion of fairness. > > > > I do not support having a weak_random... If the consumer is sure > > enough that you don't need a secure random, then they can pick an LCG > > and implement it themselves and deal (or not) w/ the locking issues... > > > > It appears that the scheduler had an LCG but for some reason the authors > > didn't feel like using it here.. > > Why don???t you support having a common random routine that???s to mix the > pot, but not cryptographically secure? Lots of algorithms use them, and having > a common one would keep us from reinventing the wheel. Why can't these algorithms use a cryptographically secure RNG instead? No one has truely answered that point.. Everyone says they want to use an insecure RNG, but the real question is, why can't/shouldn't these algorithms use a CSPRNG? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 18:53:17 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 443247A4 for ; Tue, 24 Feb 2015 18:53:17 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E51726A3 for ; Tue, 24 Feb 2015 18:53:16 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id F0457E45F for ; Tue, 24 Feb 2015 13:53:14 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.867 X-Spam-Level: X-Spam-Status: No, score=-3.867 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.160, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id alUhNW924xXF for ; Tue, 24 Feb 2015 13:53:14 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 90F59E460 for ; Tue, 24 Feb 2015 13:53:14 -0500 (EST) Message-ID: <54ECC89B.9030805@astrodoggroup.com> Date: Tue, 24 Feb 2015 10:53:15 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> In-Reply-To: <20150224182507.GI46794@funkthat.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 18:53:17 -0000 On 02/24/15 10:25, John-Mark Gurney wrote: > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 > -0500: >> On 2/24/15 12:40 PM, John-Mark Gurney wrote: >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 >>> -0700: >>>> Then again, if you want to change random(), provide a >>>> weak_random() that???s the traditional non-crypto thing >>>> that???s fast and lockless. That would make it easy to audit >>>> in our tree. The scheduler doesn???t need cryptographic >>>> randomness, it just needs to make different choices sometimes >>>> to ensure its notion of fairness. >>> >>> I do not support having a weak_random... If the consumer is >>> sure enough that you don't need a secure random, then they can >>> pick an LCG and implement it themselves and deal (or not) w/ >>> the locking issues... >>> >>> It appears that the scheduler had an LCG but for some reason >>> the authors didn't feel like using it here.. >> >> The way I read this argument is that no low quality sources of >> randomness shall be allowed. > > No, I'm saying that the person who needs the predictable > randomness needs to do extra work to get it... If they care that > much about performance/predictability/etc, then a little extra work > won't hurt them.. And if they don't know what an LCG is, then they > aren't qualified to make the decision that a weaker RNG is correct > for their situation.. > >> So we should get rid of rand(3)? When do we deprecate that? > > No, we should replace it w/ proper randomness like OpenBSD has... > I'm willing to go that far and I think FreeBSD should... OpenBSD > has done a lot of leg work in tracking down ports that correctly > use rand(3), and letting them keep their deterministic randomness, > while the remaining get real random.. > >> Your argument doesn't hold water. > > Sorry, you're argument sounds like it's from the 90's when we > didn't know any better on how to make secure systems... Will you > promise to audit all new uses of randomness in the system to make > sure that they are using the correct, secure API? > > Considering that it's been recommended that people NOT use > read_random(9) for 14 years, yet people continue to use it in new > code, demonstrates that people do not know what they are doing > (wrt randomness), and the only way to make sure they do the > correct, secure thing is to only provide the secure API... > I think a large part of this issue is, in the end, terminology. Developers call "random()" when they actually mean a whole lot of different things, from ULE's "I need a number here that changes some so this value jitters" to "I'm trying to create an OTP to protect this letter from the NSA and don't have time to break out my geiger counter." A single function simply won't meet all of these needs. I believe the default behavior *should* be a CSPRNG, simply so that if someone hasn't fully considered what they need, their code will still be "safe" (as far as random numbers go, anyway). That being said, if someone *has* considered the problem, there is no particular reason to force them to write yet another implementation of something thousands of other people have written. I propose converting random() to be a true, SMP-safe, CSPRNG so that the default behavior is as secure as possible, and the creation of "fauxrandom()" (Or some such thing... hopefully someone has a better name) for those circumstances where one can accept the degradation in random number randomness, in exchange for a significant performance improvement. By having both options available, and documenting the difference between them, we can generally get the best of both worlds. I don't believe there is a programming solution to solving the problem of a developer who is wrong also being sure about what sort of "random" they need. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 19:09:19 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D97C5D38 for ; Tue, 24 Feb 2015 19:09:18 +0000 (UTC) Received: from pmta1.delivery2.ore.mailhop.org (pmta1.delivery2.ore.mailhop.org [54.149.210.130]) by mx1.freebsd.org (Postfix) with ESMTP id B29E7865 for ; Tue, 24 Feb 2015 19:09:18 +0000 (UTC) Received: from smtp2.ore.mailhop.org (172.31.18.134) by pmta1.delivery1.ore.mailhop.org id htj65q20r84j for ; Tue, 24 Feb 2015 19:05:28 +0000 (envelope-from ) Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp2.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YQKnP-0003Kz-GB; Tue, 24 Feb 2015 19:05:27 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t1OJ5MeJ011556; Tue, 24 Feb 2015 12:05:22 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1/NRGAiGgi9+YfjJ/ls2UiE Message-ID: <1424804722.3293.12.camel@freebsd.org> Subject: Re: locks and kernel randomness... From: Ian Lepore To: John-Mark Gurney Date: Tue, 24 Feb 2015 12:05:22 -0700 In-Reply-To: <20150224183051.GJ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> <20150224183051.GJ46794@funkthat.com> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.8 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 19:09:19 -0000 On Tue, 2015-02-24 at 10:30 -0800, John-Mark Gurney wrote: > Warner Losh wrote this message on Tue, Feb 24, 2015 at 11:03 -0700: > > > > > On Feb 24, 2015, at 10:40 AM, John-Mark Gurney wrote: > > > > > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > > >> Then again, if you want to change random(), provide a weak_random() that???s > > >> the traditional non-crypto thing that???s fast and lockless. That would make it easy > > >> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > > >> just needs to make different choices sometimes to ensure its notion of fairness. > > > > > > I do not support having a weak_random... If the consumer is sure > > > enough that you don't need a secure random, then they can pick an LCG > > > and implement it themselves and deal (or not) w/ the locking issues... > > > > > > It appears that the scheduler had an LCG but for some reason the authors > > > didn't feel like using it here.. > > > > Why don???t you support having a common random routine that???s to mix the > > pot, but not cryptographically secure? Lots of algorithms use them, and having > > a common one would keep us from reinventing the wheel. > > Why can't these algorithms use a cryptographically secure RNG instead? > No one has truely answered that point.. Everyone says they want to use > an insecure RNG, but the real question is, why can't/shouldn't these > algorithms use a CSPRNG? > Because of performance. Not everything needs crypto-strength randomness. The problem here is that you are never going to accept the validity of that statement no matter how many of us complain over and over again about this kind of thing. You've got some religion that the rest of us don't buy into, and you've got it fervently enough that you'll just keep saying the same mindless things over and over again until we all get bored and wander away. And freebsd gets a little more bloated and a little slower and a lot less able to run on anything except the very fastest hardware that was just released last week. Thanks for "improving" it in that way. -- Ian From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 19:44:37 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 16B39876; Tue, 24 Feb 2015 19:44:37 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DD78BCCC; Tue, 24 Feb 2015 19:44:36 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1OJiWV4005178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 11:44:32 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1OJiVUe005177; Tue, 24 Feb 2015 11:44:31 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 11:44:31 -0800 From: John-Mark Gurney To: Ian Lepore Subject: Re: locks and kernel randomness... Message-ID: <20150224194431.GM46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> <20150224183051.GJ46794@funkthat.com> <1424804722.3293.12.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424804722.3293.12.camel@freebsd.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 11:44:32 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 19:44:37 -0000 Ian Lepore wrote this message on Tue, Feb 24, 2015 at 12:05 -0700: > On Tue, 2015-02-24 at 10:30 -0800, John-Mark Gurney wrote: > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 11:03 -0700: > > > > > > > On Feb 24, 2015, at 10:40 AM, John-Mark Gurney wrote: > > > > > > > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > > > >> Then again, if you want to change random(), provide a weak_random() that???s > > > >> the traditional non-crypto thing that???s fast and lockless. That would make it easy > > > >> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > > > >> just needs to make different choices sometimes to ensure its notion of fairness. > > > > > > > > I do not support having a weak_random... If the consumer is sure > > > > enough that you don't need a secure random, then they can pick an LCG > > > > and implement it themselves and deal (or not) w/ the locking issues... > > > > > > > > It appears that the scheduler had an LCG but for some reason the authors > > > > didn't feel like using it here.. > > > > > > Why don???t you support having a common random routine that???s to mix the > > > pot, but not cryptographically secure? Lots of algorithms use them, and having > > > a common one would keep us from reinventing the wheel. > > > > Why can't these algorithms use a cryptographically secure RNG instead? > > No one has truely answered that point.. Everyone says they want to use > > an insecure RNG, but the real question is, why can't/shouldn't these > > algorithms use a CSPRNG? > > Because of performance. Not everything needs crypto-strength > randomness. LOL... If you need something that gives you more than 500MB/sec of randomness, using an LCG is the least of your worries, and you need to be implementing you're own anyways... > The problem here is that you are never going to accept the validity of > that statement no matter how many of us complain over and over again > about this kind of thing. You've got some religion that the rest of us Give me examples of code in the tree that needs high performance, low quality randomness, and I will agree with you and provide the interface, but as long as you continue to create strawmen, I won't... The example we've been discussing of the scheduler is called LESS than once a second! OMG! soo many cycles wasted! > don't buy into, and you've got it fervently enough that you'll just keep > saying the same mindless things over and over again until we all get > bored and wander away. Just like you keep mindless saying that people need an LCG because performance, trust us! Performance! Performance! > And freebsd gets a little more bloated and a little slower and a lot > less able to run on anything except the very fastest hardware that was > just released last week. Thanks for "improving" it in that way. Again, give me an example of calling random(9) more than 100 times per second, or testing w/ my patch, and showing more than .01% cpu in random and I'll have an honest discussion about this... But you're speculating on performance issues w/o numbers... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 19:45:46 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F67E930 for ; Tue, 24 Feb 2015 19:45:46 +0000 (UTC) Received: from mail-pa0-f46.google.com (mail-pa0-f46.google.com [209.85.220.46]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D343CDB for ; Tue, 24 Feb 2015 19:45:46 +0000 (UTC) Received: by pabkq14 with SMTP id kq14so38455683pab.3 for ; Tue, 24 Feb 2015 11:45:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=NTu2Y9PGr/c4fbh+NdT3+JK330H/BB0dYSJAOvzYo0U=; b=WdERFpPQmPf7VetrnxXJ9y0N2xk/bcRr9vp93D9yank7xSYI6yn4XWxR+FozjT/Z49 I+I50YlJLE6i6YWmIkRgAkC8GTp8rzdlcEQH7+VhagVweD4L5p0c7CtmOEmnsFfK8olV kNFD74mVOCKeZVOlU32cQuPnSen3aD7jVozCF8fTuuH1hd7Ze43QK1+ZDJLrGjMMkBOV jeQFpM5XrFTUNxfCAR6FsCxxVyZ3V6043/nEwX0KqaDb+BUaZ29qYJExcrQCEqch03Af p4dCuXYGLVs6hCbbwvF3nVJnNK7xkkdIVff8GlZlzjDj7Mo8gw0bWCGjfpg6xTdJaZYU ItbQ== X-Gm-Message-State: ALoCoQnEkdJd49qOeOxKI8kjyFkohKMdmwaetJD9F6OE/R/6TJW55lZeCyG6+MigTfgzZV5/DVpq X-Received: by 10.70.140.130 with SMTP id rg2mr1998365pdb.49.1424807145367; Tue, 24 Feb 2015 11:45:45 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id pa6sm30056613pac.45.2015.02.24.11.45.42 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 11:45:44 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150224183051.GJ46794@funkthat.com> Date: Tue, 24 Feb 2015 12:45:41 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8157A5FC-C402-4C77-8535-AAF73BB64E8E@bsdimp.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> <20150224183051.GJ46794@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 19:45:46 -0000 > On Feb 24, 2015, at 11:30 AM, John-Mark Gurney = wrote: >=20 > Warner Losh wrote this message on Tue, Feb 24, 2015 at 11:03 -0700: >>=20 >>> On Feb 24, 2015, at 10:40 AM, John-Mark Gurney = wrote: >>>=20 >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: >>>> Then again, if you want to change random(), provide a weak_random() = that???s >>>> the traditional non-crypto thing that???s fast and lockless. That = would make it easy >>>> to audit in our tree. The scheduler doesn???t need cryptographic = randomness, it >>>> just needs to make different choices sometimes to ensure its notion = of fairness. >>>=20 >>> I do not support having a weak_random... If the consumer is sure >>> enough that you don't need a secure random, then they can pick an = LCG >>> and implement it themselves and deal (or not) w/ the locking = issues... >>>=20 >>> It appears that the scheduler had an LCG but for some reason the = authors >>> didn't feel like using it here.. >>=20 >> Why don???t you support having a common random routine that???s to = mix the >> pot, but not cryptographically secure? Lots of algorithms use them, = and having >> a common one would keep us from reinventing the wheel. >=20 > Why can't these algorithms use a cryptographically secure RNG instead? > No one has truely answered that point.. Everyone says they want to = use > an insecure RNG, but the real question is, why can't/shouldn't these > algorithms use a CSPRNG? They could, assuming that no locks are needed to get this and the = computation isn=E2=80=99t too large because this is in the fast path of the kernel. = They just don=E2=80=99t need it to be that strong. Not having any other interactions with the rest of = the system is also desirable. Historically, a CSPRNG is spelled rand() or random(). So by calling = those functions, they are saying they want that. Some callers need more, others do not. Warner= From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 20:06:48 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F044FDB for ; Tue, 24 Feb 2015 20:06:48 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id D1F41EF3 for ; Tue, 24 Feb 2015 20:06:47 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1OK6isl005322 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 12:06:44 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1OK6hrZ005321; Tue, 24 Feb 2015 12:06:43 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 12:06:43 -0800 From: John-Mark Gurney To: Warner Losh Subject: Re: locks and kernel randomness... Message-ID: <20150224200643.GN46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> <20150224183051.GJ46794@funkthat.com> <8157A5FC-C402-4C77-8535-AAF73BB64E8E@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8157A5FC-C402-4C77-8535-AAF73BB64E8E@bsdimp.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 12:06:44 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 20:06:48 -0000 Warner Losh wrote this message on Tue, Feb 24, 2015 at 12:45 -0700: > > > On Feb 24, 2015, at 11:30 AM, John-Mark Gurney wrote: > > > > Warner Losh wrote this message on Tue, Feb 24, 2015 at 11:03 -0700: > >> > >>> On Feb 24, 2015, at 10:40 AM, John-Mark Gurney wrote: > >>> > >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > >>>> Then again, if you want to change random(), provide a weak_random() that???s > >>>> the traditional non-crypto thing that???s fast and lockless. That would make it easy > >>>> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > >>>> just needs to make different choices sometimes to ensure its notion of fairness. > >>> > >>> I do not support having a weak_random... If the consumer is sure > >>> enough that you don't need a secure random, then they can pick an LCG > >>> and implement it themselves and deal (or not) w/ the locking issues... > >>> > >>> It appears that the scheduler had an LCG but for some reason the authors > >>> didn't feel like using it here.. > >> > >> Why don???t you support having a common random routine that???s to mix the > >> pot, but not cryptographically secure? Lots of algorithms use them, and having > >> a common one would keep us from reinventing the wheel. > > > > Why can't these algorithms use a cryptographically secure RNG instead? > > No one has truely answered that point.. Everyone says they want to use > > an insecure RNG, but the real question is, why can't/shouldn't these > > algorithms use a CSPRNG? > > They could, assuming that no locks are needed to get this and the computation > isn???t too large because this is in the fast path of the kernel. They just don???t need > it to be that strong. Not having any other interactions with the rest of the system > is also desirable. I agree about having no interations w/ other parts of the system, which is why I posted the original email.. Asking for help/advice w/ the problem... Instead of help, all I've received is but you'll make my system slow, because and other less helpful comments... > Historically, a CSPRNG is spelled rand() or random(). So by calling those functions, > they are saying they want that. Some callers need more, others do not. Citation please? In my copy of the C99 specification, the rand function says nothing about being cryptographicly secure.. and the srand function specificly states that after calling srand, rand will be seeded w/ a unsigned int, or 32bits, so by definition not CSPRNG.. Also, Single UNIX Specification: http://pubs.opengroup.org/onlinepubs/007908799/xsh/rand.html has the same definition. As for random() from our own man page: The random() function uses a non-linear additive feedback random number generator employing a default table of size 31 long integers to return successive pseudo-random numbers in the range from 0 to (2**31)-1. The oh, and immediately before that, it says: The functions described in this manual page are not cryptographically secure. Cryptographic applications should use arc4random(3) instead. So, I really would like to know where you get the idea the rand() and random() are CSPRNG.. Though I'm fine w/ making them so.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 20:20:53 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B6C070C for ; Tue, 24 Feb 2015 20:20:53 +0000 (UTC) Received: from mail-pd0-f169.google.com (mail-pd0-f169.google.com [209.85.192.169]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 382EAD7 for ; Tue, 24 Feb 2015 20:20:52 +0000 (UTC) Received: by pdjp10 with SMTP id p10so35884296pdj.3 for ; Tue, 24 Feb 2015 12:20:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=hqnzeQ7/tIc4cGXTpsPpX7MdfIeRVtxA33nLHPWEdKc=; b=fWRoboxX7xQhqU2lUiURrWL5kPhMt3KIWOI1WqE2OOHfqAIedq9BFBddGMDat7rjG1 7KcBRXv1A1XU8CkPph+Mq5Z2vM8PATFRdDwpYjZfJSLvmO7XESHpmXju1qNkALrlHzsw LSHvPNDcWI83mrT64e+z0JjcanZBWt/1SVpuBXC/1kEpWV3Wzuc2N5iCjDuBDwQdbu73 UpoOw8rNKA2S16EPDpw+kqraHDDgoJ9AdHEl3n9ws0ZfiXln8HCbLEekABCpD+kwtMzr IAwyUms4zxVJmdgMiS4kK4VYKPDetckZB4u1ziTE88/VypBsKHHdi4IrwkNNidIhv9/u /bUA== X-Gm-Message-State: ALoCoQkWLM3JK4vMlqEfcxGVNYQZcOWOR5wCoGe2FxkeHNw/5Q0x2ZUfwgLI+VaUsFNDIFQg7x80 X-Received: by 10.66.124.225 with SMTP id ml1mr31609827pab.142.1424809245849; Tue, 24 Feb 2015 12:20:45 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id of14sm4660396pdb.50.2015.02.24.12.20.44 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 12:20:45 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150224200643.GN46794@funkthat.com> Date: Tue, 24 Feb 2015 13:20:42 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <1E4A5E62-6E06-48BA-B5C5-9BD05811CDEF@bsdimp.com> <20150224183051.GJ46794@funkthat.com> <8157A5FC-C402-4C77-8535-AAF73BB64E8E@bsdimp.com> <20150224200643.GN46794@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 20:20:53 -0000 > On Feb 24, 2015, at 1:06 PM, John-Mark Gurney = wrote: >=20 >> Historically, a CSPRNG is spelled rand() or random(). So by calling = those functions, >> they are saying they want that. Some callers need more, others do = not. >=20 > Citation please? In my copy of the C99 specification, the rand = function > says nothing about being cryptographicly secure.. and the srand = function > specificly states that after calling srand, rand will be seeded w/ > a unsigned int, or 32bits, so by definition not CSPRNG.. >=20 > Also, Single UNIX Specification: > http://pubs.opengroup.org/onlinepubs/007908799/xsh/rand.html >=20 > has the same definition. >=20 > As for random() from our own man page: > The random() function uses a non-linear additive feedback random = number > generator employing a default table of size 31 long integers to = return > successive pseudo-random numbers in the range from 0 to (2**31)-1. = The >=20 > oh, and immediately before that, it says: > The functions described in this manual page are not = cryptographically > secure. Cryptographic applications should use arc4random(3) = instead. >=20 > So, I really would like to know where you get the idea the rand() and > random() are CSPRNG.. Though I'm fine w/ making them so.. Historically algorithmic PRNG is spelled random(). My brain thought that = and typed CSPRNG. Warner From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 21:13:24 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 750EE975 for ; Tue, 24 Feb 2015 21:13:24 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5E21F946 for ; Tue, 24 Feb 2015 21:13:24 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.133.26.10]) by elvis.mu.org (Postfix) with ESMTPSA id 7BAB4341F910; Tue, 24 Feb 2015 13:13:18 -0800 (PST) Message-ID: <54ECEA43.2080008@freebsd.org> Date: Tue, 24 Feb 2015 16:16:51 -0500 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: John-Mark Gurney Subject: Re: locks and kernel randomness... References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> In-Reply-To: <20150224182507.GI46794@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 21:13:24 -0000 On 2/24/15 1:25 PM, John-Mark Gurney wrote: > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 -0500: >> On 2/24/15 12:40 PM, John-Mark Gurney wrote: >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: >>>> Then again, if you want to change random(), provide a weak_random() that???s >>>> the traditional non-crypto thing that???s fast and lockless. That would make it easy >>>> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it >>>> just needs to make different choices sometimes to ensure its notion of fairness. >>> >>> I do not support having a weak_random... If the consumer is sure >>> enough that you don't need a secure random, then they can pick an LCG >>> and implement it themselves and deal (or not) w/ the locking issues... >>> >>> It appears that the scheduler had an LCG but for some reason the authors >>> didn't feel like using it here.. >> >> The way I read this argument is that no low quality sources of >> randomness shall be allowed. > > No, I'm saying that the person who needs the predictable randomness > needs to do extra work to get it... If they care that much about > performance/predictability/etc, then a little extra work won't hurt > them.. And if they don't know what an LCG is, then they aren't > qualified to make the decision that a weaker RNG is correct for their > situation.. > >> So we should get rid of rand(3)? When do we deprecate that? > > No, we should replace it w/ proper randomness like OpenBSD has... > I'm willing to go that far and I think FreeBSD should... OpenBSD has > done a lot of leg work in tracking down ports that correctly use > rand(3), and letting them keep their deterministic randomness, while > the remaining get real random.. > >> Your argument doesn't hold water. > > Sorry, you're argument sounds like it's from the 90's when we didn't > know any better on how to make secure systems... Will you promise to > audit all new uses of randomness in the system to make sure that they > are using the correct, secure API? > > Considering that it's been recommended that people NOT use > read_random(9) for 14 years, yet people continue to use it in new code, > demonstrates that people do not know what they are doing (wrt > randomness), and the only way to make sure they do the correct, secure > thing is to only provide the secure API... That speaks to more of the drive-by czars we have in BSD land that take an area with a hard lock and then go away. Also, do not want to attempt to be like openbsd, learn from for sure, but to be like, no way. -Alfred From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 23:19:25 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B5A4523A; Tue, 24 Feb 2015 23:19:25 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B38EA88; Tue, 24 Feb 2015 23:19:25 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1ONJM5w006597 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 15:19:22 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1ONJLDX006596; Tue, 24 Feb 2015 15:19:21 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 15:19:21 -0800 From: John-Mark Gurney To: Alfred Perlstein Subject: Re: locks and kernel randomness... Message-ID: <20150224231921.GQ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ECEA43.2080008@freebsd.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 15:19:22 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 23:19:25 -0000 Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 16:16 -0500: > On 2/24/15 1:25 PM, John-Mark Gurney wrote: > > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 -0500: > >> On 2/24/15 12:40 PM, John-Mark Gurney wrote: > >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > >>>> Then again, if you want to change random(), provide a weak_random() that???s > >>>> the traditional non-crypto thing that???s fast and lockless. That would make it easy > >>>> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > >>>> just needs to make different choices sometimes to ensure its notion of fairness. > >>> > >>> I do not support having a weak_random... If the consumer is sure > >>> enough that you don't need a secure random, then they can pick an LCG > >>> and implement it themselves and deal (or not) w/ the locking issues... > >>> > >>> It appears that the scheduler had an LCG but for some reason the authors > >>> didn't feel like using it here.. > >> > >> The way I read this argument is that no low quality sources of > >> randomness shall be allowed. > > > > No, I'm saying that the person who needs the predictable randomness > > needs to do extra work to get it... If they care that much about > > performance/predictability/etc, then a little extra work won't hurt > > them.. And if they don't know what an LCG is, then they aren't > > qualified to make the decision that a weaker RNG is correct for their > > situation.. > > > >> So we should get rid of rand(3)? When do we deprecate that? > > > > No, we should replace it w/ proper randomness like OpenBSD has... > > I'm willing to go that far and I think FreeBSD should... OpenBSD has > > done a lot of leg work in tracking down ports that correctly use > > rand(3), and letting them keep their deterministic randomness, while > > the remaining get real random.. > > > >> Your argument doesn't hold water. > > > > Sorry, you're argument sounds like it's from the 90's when we didn't > > know any better on how to make secure systems... Will you promise to > > audit all new uses of randomness in the system to make sure that they > > are using the correct, secure API? > > > > Considering that it's been recommended that people NOT use > > read_random(9) for 14 years, yet people continue to use it in new code, > > demonstrates that people do not know what they are doing (wrt > > randomness), and the only way to make sure they do the correct, secure > > thing is to only provide the secure API... > > That speaks to more of the drive-by czars we have in BSD land that take > an area with a hard lock and then go away. It also speaks to the airchair quarterbacking that stops people from wanting to contribute... Someone comes along and tries to make an improvement, then x number of people raise their arms about oh, I still use grdc (sorry dteske, not trying to pick on you) as tcp keep alive, and then the person abandons or leaves incomplete the work that they started... I was very close to NOT posting the email to -arch, but after various questions from twitter, and adrian's continued pleas to talk changes more publicly, I decided to do so... If people continue to react this way, it just demonstrates that doing things publicly is NOT a way to get things to move forward in FreeBSD, and people will continue to do things in private... Luckily, I'm consulting, so I have a few more hours (for now) to fight these fights, but if it continues to be an issue, we'll continue to have this problem of czars that come in, drop a bunch of code and then leave, because dealing w/ this becomes too expensive... So far, only ONE person has commented on the patch on reviews, and that is delphij... > Also, do not want to attempt to be like openbsd, learn from for sure, > but to be like, no way. I'm fine not being like OpenBSD, but as you said, we should learn from them, and leverage their work... Though I agree w/ OpenBSD's work to replace random(3), it also isn't who FreeBSD is, but if we want to continue to be relevant, we do need to take security seriously, and IMO, this is one of those steps. If someone does find a performance issue w/ my patch, I WILL work with them on a solution, but I will not work w/ people who make unfounded claims about the impact of this work... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Tue Feb 24 23:33:04 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D70C849C; Tue, 24 Feb 2015 23:33:04 +0000 (UTC) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 52D39C63; Tue, 24 Feb 2015 23:33:04 +0000 (UTC) Received: by labgq15 with SMTP id gq15so285905lab.6; Tue, 24 Feb 2015 15:33:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=cdSCoiAAjGkc6eE4t4Cy5yDSqKv759juqdEIDmPJTF4=; b=E4yQI/W7+aZ0ImoFApg3YxMiQbZkQvcsrhDG6fb3cTg+BFDN5chjgkvD8HFrHRqKUc w5vPTi0mFCRcWJ4It5dXLiWvIDgXcgAgX+M3EU1UCWLhCdFIBwC5TJ9EM/BY7OqLvB4z BQ4BuVx05KaE3Z08mu9fwWE/uAbeJkMso8FZ8gDSLFj/Yu3XJGrsvnSgzP0eygUY0WDe eqdJDOojZCzmIi/RL1bx0ehXUGikaosj9XE0pq69rljdk9ANCpntVFH9GrS9Ha8C326e pChNlr4J3wKQ/y7uBFINrAsCxWsczBOr0t6dyKxFtTUTbauMWpyUD01SqWKDQjQUvrCs GDOw== MIME-Version: 1.0 X-Received: by 10.112.138.233 with SMTP id qt9mr349470lbb.44.1424820782457; Tue, 24 Feb 2015 15:33:02 -0800 (PST) Sender: kmacybsd@gmail.com Received: by 10.25.77.74 with HTTP; Tue, 24 Feb 2015 15:33:02 -0800 (PST) In-Reply-To: <20150224231921.GQ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> Date: Tue, 24 Feb 2015 15:33:02 -0800 X-Google-Sender-Auth: bQFMyivKMFMMULgjpQ2QZubnE4Q Message-ID: Subject: Re: locks and kernel randomness... From: "K. Macy" To: John-Mark Gurney Content-Type: text/plain; charset=UTF-8 Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2015 23:33:04 -0000 > If someone does find a performance issue w/ my patch, I WILL work with > them on a solution, but I will not work w/ people who make unfounded > claims about the impact of this work... > ... The concerns may be exaggerated, but they aren't unfounded. Not quite the same thing, but no one wants to spend the cycles doing a SHA256 because it's "The Right Thing"(tm) when their use case only requires a fletcher2. If it doesn't already exist, it might also be worth looking in to a more scalable CSPRNG implementation not requiring locking in the common case. For example, each core is seeded separately periodically so that has a private pool that is protected by a critical section. The private pool would be regularly refreshed by cpu-local callout. Thus, a lock would only be acquired if the local entropy were depleted. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 00:02:15 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0723B97F; Wed, 25 Feb 2015 00:02:15 +0000 (UTC) Received: from pmta1.delivery9.ore.mailhop.org (pmta1.delivery9.ore.mailhop.org [54.186.172.23]) by mx1.freebsd.org (Postfix) with ESMTP id D3156EF6; Wed, 25 Feb 2015 00:02:14 +0000 (UTC) Received: from smtp6.ore.mailhop.org (172.31.18.134) by pmta1.delivery1.ore.mailhop.org id htk8g820r84t; Wed, 25 Feb 2015 00:02:05 +0000 (envelope-from ) Received: from c-73-34-117-227.hsd1.co.comcast.net ([73.34.117.227] helo=ilsoft.org) by smtp6.ore.mailhop.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.82) (envelope-from ) id 1YQPQU-0005Gi-Uo; Wed, 25 Feb 2015 00:02:07 +0000 Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240]) by ilsoft.org (8.14.9/8.14.9) with ESMTP id t1P022TS012284; Tue, 24 Feb 2015 17:02:02 -0700 (MST) (envelope-from ian@freebsd.org) X-Mail-Handler: DuoCircle Outbound SMTP X-Originating-IP: 73.34.117.227 X-Report-Abuse-To: abuse@duocircle.com (see https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information for abuse reporting information) X-MHO-User: U2FsdGVkX1/q2XrpaqKzqC2vmgoXC++8 Message-ID: <1424822522.1328.11.camel@freebsd.org> Subject: Re: locks and kernel randomness... From: Ian Lepore To: John-Mark Gurney Date: Tue, 24 Feb 2015 17:02:02 -0700 In-Reply-To: <20150224231921.GQ46794@funkthat.com> References: <20150224012026.GY46794@funkthat.com> <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> Content-Type: text/plain; charset="us-ascii" X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 00:02:15 -0000 On Tue, 2015-02-24 at 15:19 -0800, John-Mark Gurney wrote: > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 16:16 -0500: > > On 2/24/15 1:25 PM, John-Mark Gurney wrote: > > > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 -0500: > > >> On 2/24/15 12:40 PM, John-Mark Gurney wrote: > > >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > > >>>> Then again, if you want to change random(), provide a weak_random() that???s > > >>>> the traditional non-crypto thing that???s fast and lockless. That would make it easy > > >>>> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > > >>>> just needs to make different choices sometimes to ensure its notion of fairness. > > >>> > > >>> I do not support having a weak_random... If the consumer is sure > > >>> enough that you don't need a secure random, then they can pick an LCG > > >>> and implement it themselves and deal (or not) w/ the locking issues... > > >>> > > >>> It appears that the scheduler had an LCG but for some reason the authors > > >>> didn't feel like using it here.. > > >> > > >> The way I read this argument is that no low quality sources of > > >> randomness shall be allowed. > > > > > > No, I'm saying that the person who needs the predictable randomness > > > needs to do extra work to get it... If they care that much about > > > performance/predictability/etc, then a little extra work won't hurt > > > them.. And if they don't know what an LCG is, then they aren't > > > qualified to make the decision that a weaker RNG is correct for their > > > situation.. > > > > > >> So we should get rid of rand(3)? When do we deprecate that? > > > > > > No, we should replace it w/ proper randomness like OpenBSD has... > > > I'm willing to go that far and I think FreeBSD should... OpenBSD has > > > done a lot of leg work in tracking down ports that correctly use > > > rand(3), and letting them keep their deterministic randomness, while > > > the remaining get real random.. > > > > > >> Your argument doesn't hold water. > > > > > > Sorry, you're argument sounds like it's from the 90's when we didn't > > > know any better on how to make secure systems... Will you promise to > > > audit all new uses of randomness in the system to make sure that they > > > are using the correct, secure API? > > > > > > Considering that it's been recommended that people NOT use > > > read_random(9) for 14 years, yet people continue to use it in new code, > > > demonstrates that people do not know what they are doing (wrt > > > randomness), and the only way to make sure they do the correct, secure > > > thing is to only provide the secure API... > > > > That speaks to more of the drive-by czars we have in BSD land that take > > an area with a hard lock and then go away. > > It also speaks to the airchair quarterbacking that stops people from > wanting to contribute... Someone comes along and tries to make an > improvement, then x number of people raise their arms about oh, I > still use grdc (sorry dteske, not trying to pick on you) as tcp keep > alive, and then the person abandons or leaves incomplete the work that > they started... > > I was very close to NOT posting the email to -arch, but after various > questions from twitter, and adrian's continued pleas to talk changes > more publicly, I decided to do so... If people continue to react this > way, it just demonstrates that doing things publicly is NOT a way to > get things to move forward in FreeBSD, and people will continue to do > things in private... Luckily, I'm consulting, so I have a few more > hours (for now) to fight these fights, but if it continues to be an > issue, we'll continue to have this problem of czars that come in, drop > a bunch of code and then leave, because dealing w/ this becomes too > expensive... > > So far, only ONE person has commented on the patch on reviews, and that > is delphij... > > > Also, do not want to attempt to be like openbsd, learn from for sure, > > but to be like, no way. > > I'm fine not being like OpenBSD, but as you said, we should learn from > them, and leverage their work... Though I agree w/ OpenBSD's work to > replace random(3), it also isn't who FreeBSD is, but if we want to > continue to be relevant, we do need to take security seriously, and > IMO, this is one of those steps. > > If someone does find a performance issue w/ my patch, I WILL work with > them on a solution, but I will not work w/ people who make unfounded > claims about the impact of this work... > Yeah, the problem could all that. Or it could be people who "collaborate" by saying I'm going to make this change. I'm not going to justify it in any way, and if anybody disagrees I'm just going to dismiss their concerns and demand that THEY hold the burden of proof that my unnecessary change is harmful, and if they don't, then screw the collobaration thing, I'm just going to do it anyway. -- Ian From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 00:23:04 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4529CE1; Wed, 25 Feb 2015 00:23:04 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EFAAC180; Wed, 25 Feb 2015 00:23:03 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1P0N1jp007020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 16:23:01 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1P0N1wI007019; Tue, 24 Feb 2015 16:23:01 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 16:23:01 -0800 From: John-Mark Gurney To: "K. Macy" Subject: Re: locks and kernel randomness... Message-ID: <20150225002301.GS46794@funkthat.com> References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 16:23:01 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 00:23:04 -0000 K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 -0800: > > If someone does find a performance issue w/ my patch, I WILL work with > > them on a solution, but I will not work w/ people who make unfounded > > claims about the impact of this work... > > > > ... The concerns may be exaggerated, but they aren't > unfounded. Not quite the same thing, but no one wants to spend the Till someone shows me code in the kernel tree where this is even close to a performance problem, it is unfounded... I've asked, and no one has > cycles doing a SHA256 because it's "The Right Thing"(tm) when their > use case only requires a fletcher2. Depends upon what you're doing.. I haven't proposed changing ZFS's default to sha256, so stop w/ the false equivalences... > If it doesn't already exist, it might also be worth looking in to a > more scalable CSPRNG implementation not requiring locking in the > common case. For example, each core is seeded separately periodically > so that has a private pool that is protected by a critical section. > The private pool would be regularly refreshed by cpu-local callout. > Thus, a lock would only be acquired if the local entropy were > depleted. I'm not discussing this until you read and reply to my original email, since it's clear that my original email's contents has been ignored in this thread... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 00:30:00 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 077FF2CF; Wed, 25 Feb 2015 00:30:00 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id ADA221B7; Wed, 25 Feb 2015 00:29:59 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1P0TvbD007063 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 16:29:57 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1P0Tune007062; Tue, 24 Feb 2015 16:29:56 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 16:29:56 -0800 From: John-Mark Gurney To: Ian Lepore Subject: Re: locks and kernel randomness... Message-ID: <20150225002956.GT46794@funkthat.com> References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1424822522.1328.11.camel@freebsd.org> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 16:29:57 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 00:30:00 -0000 Ian Lepore wrote this message on Tue, Feb 24, 2015 at 17:02 -0700: > On Tue, 2015-02-24 at 15:19 -0800, John-Mark Gurney wrote: > > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 16:16 -0500: > > > On 2/24/15 1:25 PM, John-Mark Gurney wrote: > > > > Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 -0500: > > > >> On 2/24/15 12:40 PM, John-Mark Gurney wrote: > > > >>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 -0700: > > > >>>> Then again, if you want to change random(), provide a weak_random() that???s > > > >>>> the traditional non-crypto thing that???s fast and lockless. That would make it easy > > > >>>> to audit in our tree. The scheduler doesn???t need cryptographic randomness, it > > > >>>> just needs to make different choices sometimes to ensure its notion of fairness. > > > >>> > > > >>> I do not support having a weak_random... If the consumer is sure > > > >>> enough that you don't need a secure random, then they can pick an LCG > > > >>> and implement it themselves and deal (or not) w/ the locking issues... > > > >>> > > > >>> It appears that the scheduler had an LCG but for some reason the authors > > > >>> didn't feel like using it here.. > > > >> > > > >> The way I read this argument is that no low quality sources of > > > >> randomness shall be allowed. > > > > > > > > No, I'm saying that the person who needs the predictable randomness > > > > needs to do extra work to get it... If they care that much about > > > > performance/predictability/etc, then a little extra work won't hurt > > > > them.. And if they don't know what an LCG is, then they aren't > > > > qualified to make the decision that a weaker RNG is correct for their > > > > situation.. > > > > > > > >> So we should get rid of rand(3)? When do we deprecate that? > > > > > > > > No, we should replace it w/ proper randomness like OpenBSD has... > > > > I'm willing to go that far and I think FreeBSD should... OpenBSD has > > > > done a lot of leg work in tracking down ports that correctly use > > > > rand(3), and letting them keep their deterministic randomness, while > > > > the remaining get real random.. > > > > > > > >> Your argument doesn't hold water. > > > > > > > > Sorry, you're argument sounds like it's from the 90's when we didn't > > > > know any better on how to make secure systems... Will you promise to > > > > audit all new uses of randomness in the system to make sure that they > > > > are using the correct, secure API? > > > > > > > > Considering that it's been recommended that people NOT use > > > > read_random(9) for 14 years, yet people continue to use it in new code, > > > > demonstrates that people do not know what they are doing (wrt > > > > randomness), and the only way to make sure they do the correct, secure > > > > thing is to only provide the secure API... > > > > > > That speaks to more of the drive-by czars we have in BSD land that take > > > an area with a hard lock and then go away. > > > > It also speaks to the airchair quarterbacking that stops people from > > wanting to contribute... Someone comes along and tries to make an > > improvement, then x number of people raise their arms about oh, I > > still use grdc (sorry dteske, not trying to pick on you) as tcp keep > > alive, and then the person abandons or leaves incomplete the work that > > they started... > > > > I was very close to NOT posting the email to -arch, but after various > > questions from twitter, and adrian's continued pleas to talk changes > > more publicly, I decided to do so... If people continue to react this > > way, it just demonstrates that doing things publicly is NOT a way to > > get things to move forward in FreeBSD, and people will continue to do > > things in private... Luckily, I'm consulting, so I have a few more > > hours (for now) to fight these fights, but if it continues to be an > > issue, we'll continue to have this problem of czars that come in, drop > > a bunch of code and then leave, because dealing w/ this becomes too > > expensive... > > > > So far, only ONE person has commented on the patch on reviews, and that > > is delphij... > > > > > Also, do not want to attempt to be like openbsd, learn from for sure, > > > but to be like, no way. > > > > I'm fine not being like OpenBSD, but as you said, we should learn from > > them, and leverage their work... Though I agree w/ OpenBSD's work to > > replace random(3), it also isn't who FreeBSD is, but if we want to > > continue to be relevant, we do need to take security seriously, and > > IMO, this is one of those steps. > > > > If someone does find a performance issue w/ my patch, I WILL work with > > them on a solution, but I will not work w/ people who make unfounded > > claims about the impact of this work... > > Yeah, the problem could all that. > > Or it could be people who "collaborate" by saying I'm going to make this > change. I'm not going to justify it in any way, and if anybody I have justified it... > disagrees I'm just going to dismiss their concerns and demand that THEY > hold the burden of proof that my unnecessary change is harmful, and if How many audits of the random() calls in the kernel have you done? You've raised concers, I've said I've looked and don't see any, how can I prove a negative? What can I do to convince you that you're wrong? All you have to do to convince me I'm wrong is show me a place in the kernel where it is a performance issue. Is it really that hard to come up w/ one? > they don't, then screw the collobaration thing, I'm just going to do it > anyway. It goes both ways, I see it that you're objecting w/o complete intformation, and no mater what evidence or work I do, you'll just ignore it, and still say it isn't correct or that there's this unprovable codition that prevents the work for going in... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 00:52:18 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8B08281D; Wed, 25 Feb 2015 00:52:18 +0000 (UTC) Received: from mail-lb0-x22f.google.com (mail-lb0-x22f.google.com [IPv6:2a00:1450:4010:c04::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 03E5565D; Wed, 25 Feb 2015 00:52:18 +0000 (UTC) Received: by lbjb6 with SMTP id b6so611644lbj.2; Tue, 24 Feb 2015 16:52:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=hpoCm6Yh7KrLkgM3ABml2S4PfEcz1dpHLCL0A/1ZcQQ=; b=BAlL+kfM1hFS/1rJcHJMx/pWUdpCiztw8qO3U89AX2zMD58vg/WFzbkjV4+qmSe2ye r1Wksicu78rY/5rWhmF65E5ACbIpHc+nsJmVHRUVXhPeRJr+Q59we5y4gUpDxfbRAJ0Q rxv1tCLVsDmOy3hM2APfm0RiH1Cl46rYpWC5h/NAeAlpNnTH5C/N3Xf4ZcvT5TA2Zvoz h7fvMn6YvyTzqdO9u5/Ef5U01Ef/2Vs/A7XtGHzOxpvBWuj1lwdUDXmR8/Dic1g9NF9R n070gTJwOUR11a67TK+98O1uypjJ+/2esx2JAzYRRuQySKuvJnVy+rvONe04n0Qa+XY6 jxpA== MIME-Version: 1.0 X-Received: by 10.152.10.66 with SMTP id g2mr549726lab.44.1424825536205; Tue, 24 Feb 2015 16:52:16 -0800 (PST) Sender: kmacybsd@gmail.com Received: by 10.25.77.74 with HTTP; Tue, 24 Feb 2015 16:52:16 -0800 (PST) In-Reply-To: <20150225002301.GS46794@funkthat.com> References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> Date: Tue, 24 Feb 2015 16:52:16 -0800 X-Google-Sender-Auth: bH6SiaG-Ms9VvxhEBYzIB_1UrDk Message-ID: Subject: Re: locks and kernel randomness... From: "K. Macy" To: John-Mark Gurney Content-Type: text/plain; charset=UTF-8 Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 00:52:18 -0000 On Tue, Feb 24, 2015 at 4:23 PM, John-Mark Gurney wrote: > K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 -0800: >> > If someone does find a performance issue w/ my patch, I WILL work with >> > them on a solution, but I will not work w/ people who make unfounded >> > claims about the impact of this work... >> > >> >> ... The concerns may be exaggerated, but they aren't >> unfounded. Not quite the same thing, but no one wants to spend the > > Till someone shows me code in the kernel tree where this is even close > to a performance problem, it is unfounded... I've asked, and no one > has In these sorts of situations the thing to do is to ask for case where performance would be adversely effected and the burden would be on you to show that they're not. If there are none forthcoming, then you're done. > > I'm not discussing this until you read and reply to my original email, > since it's clear that my original email's contents has been ignored in > this thread... > A patch for review on phabricator would do much to alleviate any concerns. I apologize if I missed the reference to that. -K From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 01:01:15 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 324EAB6F for ; Wed, 25 Feb 2015 01:01:15 +0000 (UTC) Received: from mail-pa0-f49.google.com (mail-pa0-f49.google.com [209.85.220.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F064E7C8 for ; Wed, 25 Feb 2015 01:01:14 +0000 (UTC) Received: by padet14 with SMTP id et14so827562pad.11 for ; Tue, 24 Feb 2015 17:01:08 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=H2UwMSj67y6qzU4Dyor85OSkqQEgxvfCQQqI2An5tYo=; b=W6VeGINnAQIx3V7gqoIT4+60Z6D0F+ho0fnHMs3PMlJRI837hBigJq+rJauVioXbu8 uWaa+K3SwCrhjYh64tH//n2+Ah3tYHk1Bbt3plyc98x2pQOWBMmrJyRp+VD0nNyaOsV6 +8+rLCACC/EwqgaVV8v3tLHo3L4RTNe1eEij7o8wLv4tzL1wbeOjF+I9OITo6/OgxG4S b5ki+xfk7FY9x2oBahSoSycmAdD3YS9l0pIrU9LbbcFFT6+uM4ye0xZvKRNR+3RYjIBx BSD0bdUhe7uGeIrOO/nb4kEdwuMHpHWw9kda+3KLUV/QcALDLAOBmccDm9zTFF0bJM1w Wjsw== X-Gm-Message-State: ALoCoQmL97YvQnRv29QBGyp4IpgGvYY+6M494zYgSzoBZSKw20CI10L/TQoAm1LPCpaO2U14g0UM X-Received: by 10.68.201.168 with SMTP id kb8mr790054pbc.89.1424826068168; Tue, 24 Feb 2015 17:01:08 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id cf12sm15082219pdb.43.2015.02.24.17.01.05 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 24 Feb 2015 17:01:06 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150225002956.GT46794@funkthat.com> Date: Tue, 24 Feb 2015 18:01:03 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> To: John-Mark Gurney X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 01:01:15 -0000 > On Feb 24, 2015, at 5:29 PM, John-Mark Gurney = wrote: >=20 > Ian Lepore wrote this message on Tue, Feb 24, 2015 at 17:02 -0700: >> On Tue, 2015-02-24 at 15:19 -0800, John-Mark Gurney wrote: >>> Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 16:16 = -0500: >>>> On 2/24/15 1:25 PM, John-Mark Gurney wrote: >>>>> Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at 13:04 = -0500: >>>>>> On 2/24/15 12:40 PM, John-Mark Gurney wrote: >>>>>>> Warner Losh wrote this message on Tue, Feb 24, 2015 at 07:56 = -0700: >>>>>>>> Then again, if you want to change random(), provide a = weak_random() that???s >>>>>>>> the traditional non-crypto thing that???s fast and lockless. = That would make it easy >>>>>>>> to audit in our tree. The scheduler doesn???t need = cryptographic randomness, it >>>>>>>> just needs to make different choices sometimes to ensure its = notion of fairness. >>>>>>>=20 >>>>>>> I do not support having a weak_random... If the consumer is = sure >>>>>>> enough that you don't need a secure random, then they can pick = an LCG >>>>>>> and implement it themselves and deal (or not) w/ the locking = issues... >>>>>>>=20 >>>>>>> It appears that the scheduler had an LCG but for some reason the = authors >>>>>>> didn't feel like using it here.. >>>>>>=20 >>>>>> The way I read this argument is that no low quality sources of >>>>>> randomness shall be allowed. >>>>>=20 >>>>> No, I'm saying that the person who needs the predictable = randomness >>>>> needs to do extra work to get it... If they care that much about >>>>> performance/predictability/etc, then a little extra work won't = hurt >>>>> them.. And if they don't know what an LCG is, then they aren't >>>>> qualified to make the decision that a weaker RNG is correct for = their >>>>> situation.. >>>>>=20 >>>>>> So we should get rid of rand(3)? When do we deprecate that? >>>>>=20 >>>>> No, we should replace it w/ proper randomness like OpenBSD has... >>>>> I'm willing to go that far and I think FreeBSD should... OpenBSD = has >>>>> done a lot of leg work in tracking down ports that correctly use >>>>> rand(3), and letting them keep their deterministic randomness, = while >>>>> the remaining get real random.. >>>>>=20 >>>>>> Your argument doesn't hold water. >>>>>=20 >>>>> Sorry, you're argument sounds like it's from the 90's when we = didn't >>>>> know any better on how to make secure systems... Will you promise = to >>>>> audit all new uses of randomness in the system to make sure that = they >>>>> are using the correct, secure API? >>>>>=20 >>>>> Considering that it's been recommended that people NOT use >>>>> read_random(9) for 14 years, yet people continue to use it in new = code, >>>>> demonstrates that people do not know what they are doing (wrt >>>>> randomness), and the only way to make sure they do the correct, = secure >>>>> thing is to only provide the secure API... >>>>=20 >>>> That speaks to more of the drive-by czars we have in BSD land that = take=20 >>>> an area with a hard lock and then go away. >>>=20 >>> It also speaks to the airchair quarterbacking that stops people from >>> wanting to contribute... Someone comes along and tries to make an >>> improvement, then x number of people raise their arms about oh, I >>> still use grdc (sorry dteske, not trying to pick on you) as tcp keep >>> alive, and then the person abandons or leaves incomplete the work = that >>> they started... >>>=20 >>> I was very close to NOT posting the email to -arch, but after = various >>> questions from twitter, and adrian's continued pleas to talk changes >>> more publicly, I decided to do so... If people continue to react = this >>> way, it just demonstrates that doing things publicly is NOT a way to >>> get things to move forward in FreeBSD, and people will continue to = do >>> things in private... Luckily, I'm consulting, so I have a few more >>> hours (for now) to fight these fights, but if it continues to be an >>> issue, we'll continue to have this problem of czars that come in, = drop >>> a bunch of code and then leave, because dealing w/ this becomes too >>> expensive... >>>=20 >>> So far, only ONE person has commented on the patch on reviews, and = that >>> is delphij... >>>=20 >>>> Also, do not want to attempt to be like openbsd, learn from for = sure,=20 >>>> but to be like, no way. >>>=20 >>> I'm fine not being like OpenBSD, but as you said, we should learn = from >>> them, and leverage their work... Though I agree w/ OpenBSD's work = to >>> replace random(3), it also isn't who FreeBSD is, but if we want to >>> continue to be relevant, we do need to take security seriously, and >>> IMO, this is one of those steps. >>>=20 >>> If someone does find a performance issue w/ my patch, I WILL work = with >>> them on a solution, but I will not work w/ people who make unfounded >>> claims about the impact of this work... >>=20 >> Yeah, the problem could all that. >>=20 >> Or it could be people who "collaborate" by saying I'm going to make = this >> change. I'm not going to justify it in any way, and if anybody >=20 > I have justified it=E2=80=A6 I think you should explain what you explained to me on IRC. Specifically, through a timing attack, you can find (by default) the = lower 7 bits of the value returned by random(). Since random() is not MP safe, it can sometimes return the same value twice (through some race that may or may not have been lost). This means other users can see this data. In this instance, it isn=E2=80=99t so much what sched_ule is doing, but = rather what others are able to glean from it. Now, it isn=E2=80=99t clear that these = 7 bits are a big deal since you also have to lose the race and know the race was lost. = Other things in the system might care if you expose this state. Also, in this specific case, it can use the current random generator in = sched_ule to get this number as well. It=E2=80=99s run on a time scale of ticks, = with some jitter. In this specific case, it doesn=E2=80=99t need to be using random(), but = it isn=E2=80=99t clear if the get_cyclecount() stuff provides enough low-order bits that are = random enough to meet sched_ule=E2=80=99s needs. But it isn=E2=80=99t clear = that it doesn=E2=80=99t (only cause for concern is if there=E2=80=99s a beat pattern for a cycle count = that=E2=80=99s low-resolution, but I don=E2=80=99t think we have any of these on SMP work loads). Ideally, since there=E2=80=99s a small chance of a performance = regression, we should find some benchmark to run that would exercise this code path and see if a regression can be measured or not. After looking at the code, I=E2=80=99= m skeptical that there would be one. But data would settle this once and for all, = since this is an interaction with the scheduler, which historically has made people = very nervous.=20 >> disagrees I'm just going to dismiss their concerns and demand that = THEY >> hold the burden of proof that my unnecessary change is harmful, and = if >=20 > How many audits of the random() calls in the kernel have you done? >=20 > You've raised concers, I've said I've looked and don't see any, how = can > I prove a negative? What can I do to convince you that you're wrong? > All you have to do to convince me I'm wrong is show me a place in the > kernel where it is a performance issue. Is it really that hard to > come up w/ one? You can prove a negative with benchmarks. Then we=E2=80=99d be arguing = over the efficacy of them, but at least that would be progress :) Or you can = strongly suggest a negative by failing to reject the null hypothesis of no = change. That too would be progress. >> they don't, then screw the collobaration thing, I'm just going to do = it >> anyway. >=20 > It goes both ways, I see it that you're objecting w/o complete > intformation, and no mater what evidence or work I do, you'll just > ignore it, and still say it isn't correct or that there's this > unprovable codition that prevents the work for going in=E2=80=A6 Data is going to break this log-jam. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 02:22:24 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DE858EFB; Wed, 25 Feb 2015 02:22:24 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B0BE4FDE; Wed, 25 Feb 2015 02:22:23 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1P2MKOo007822 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 24 Feb 2015 18:22:20 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1P2MK5Y007821; Tue, 24 Feb 2015 18:22:20 -0800 (PST) (envelope-from jmg) Date: Tue, 24 Feb 2015 18:22:20 -0800 From: John-Mark Gurney To: Warner Losh Subject: Re: locks and kernel randomness... Message-ID: <20150225022219.GW46794@funkthat.com> References: <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 24 Feb 2015 18:22:20 -0800 (PST) Cc: Konstantin Belousov , Harrison Grundy , Alfred Perlstein , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 02:22:25 -0000 Warner Losh wrote this message on Tue, Feb 24, 2015 at 18:01 -0700: > > > On Feb 24, 2015, at 5:29 PM, John-Mark Gurney wrote: > > It goes both ways, I see it that you're objecting w/o complete > > intformation, and no mater what evidence or work I do, you'll just > > ignore it, and still say it isn't correct or that there's this > > unprovable codition that prevents the work for going in??? > > Data is going to break this log-jam. If I do a buildworld benchmark, is that enough for you and Ian? -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 04:58:09 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 88F45304; Wed, 25 Feb 2015 04:58:09 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A8121BF; Wed, 25 Feb 2015 04:58:08 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id ED8E7E47C; Tue, 24 Feb 2015 23:58:00 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.867 X-Spam-Level: X-Spam-Status: No, score=-3.867 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.160, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IsoIeF5l9DOA; Tue, 24 Feb 2015 23:58:00 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id D471DE471; Tue, 24 Feb 2015 23:57:59 -0500 (EST) Message-ID: <54ED5656.50607@astrodoggroup.com> Date: Tue, 24 Feb 2015 20:57:58 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: Warner Losh , John-Mark Gurney Subject: Re: locks and kernel randomness... References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> In-Reply-To: <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Cc: Konstantin Belousov , Alfred Perlstein , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 04:58:09 -0000 On 02/24/15 17:01, Warner Losh wrote: > >> On Feb 24, 2015, at 5:29 PM, John-Mark Gurney >> wrote: >> >> Ian Lepore wrote this message on Tue, Feb 24, 2015 at 17:02 >> -0700: >>> On Tue, 2015-02-24 at 15:19 -0800, John-Mark Gurney wrote: >>>> Alfred Perlstein wrote this message on Tue, Feb 24, 2015 at >>>> 16:16 -0500: >>>>> On 2/24/15 1:25 PM, John-Mark Gurney wrote: >>>>>> Alfred Perlstein wrote this message on Tue, Feb 24, 2015 >>>>>> at 13:04 -0500: >>>>>>> On 2/24/15 12:40 PM, John-Mark Gurney wrote: >>>>>>>> Warner Losh wrote this message on Tue, Feb 24, 2015 >>>>>>>> at 07:56 -0700: >>>>>>>>> Then again, if you want to change random(), provide >>>>>>>>> a weak_random() that???s the traditional non-crypto >>>>>>>>> thing that???s fast and lockless. That would make >>>>>>>>> it easy to audit in our tree. The scheduler >>>>>>>>> doesn???t need cryptographic randomness, it just >>>>>>>>> needs to make different choices sometimes to ensure >>>>>>>>> its notion of fairness. >>>>>>>> >>>>>>>> I do not support having a weak_random... If the >>>>>>>> consumer is sure enough that you don't need a secure >>>>>>>> random, then they can pick an LCG and implement it >>>>>>>> themselves and deal (or not) w/ the locking >>>>>>>> issues... >>>>>>>> >>>>>>>> It appears that the scheduler had an LCG but for some >>>>>>>> reason the authors didn't feel like using it here.. >>>>>>> >>>>>>> The way I read this argument is that no low quality >>>>>>> sources of randomness shall be allowed. >>>>>> >>>>>> No, I'm saying that the person who needs the predictable >>>>>> randomness needs to do extra work to get it... If they >>>>>> care that much about performance/predictability/etc, then >>>>>> a little extra work won't hurt them.. And if they don't >>>>>> know what an LCG is, then they aren't qualified to make >>>>>> the decision that a weaker RNG is correct for their >>>>>> situation.. >>>>>> >>>>>>> So we should get rid of rand(3)? When do we deprecate >>>>>>> that? >>>>>> >>>>>> No, we should replace it w/ proper randomness like >>>>>> OpenBSD has... I'm willing to go that far and I think >>>>>> FreeBSD should... OpenBSD has done a lot of leg work in >>>>>> tracking down ports that correctly use rand(3), and >>>>>> letting them keep their deterministic randomness, while >>>>>> the remaining get real random.. >>>>>> >>>>>>> Your argument doesn't hold water. >>>>>> >>>>>> Sorry, you're argument sounds like it's from the 90's >>>>>> when we didn't know any better on how to make secure >>>>>> systems... Will you promise to audit all new uses of >>>>>> randomness in the system to make sure that they are using >>>>>> the correct, secure API? >>>>>> >>>>>> Considering that it's been recommended that people NOT >>>>>> use read_random(9) for 14 years, yet people continue to >>>>>> use it in new code, demonstrates that people do not know >>>>>> what they are doing (wrt randomness), and the only way to >>>>>> make sure they do the correct, secure thing is to only >>>>>> provide the secure API... >>>>> >>>>> That speaks to more of the drive-by czars we have in BSD >>>>> land that take an area with a hard lock and then go away. >>>> >>>> It also speaks to the airchair quarterbacking that stops >>>> people from wanting to contribute... Someone comes along and >>>> tries to make an improvement, then x number of people raise >>>> their arms about oh, I still use grdc (sorry dteske, not >>>> trying to pick on you) as tcp keep alive, and then the person >>>> abandons or leaves incomplete the work that they started... >>>> >>>> I was very close to NOT posting the email to -arch, but after >>>> various questions from twitter, and adrian's continued pleas >>>> to talk changes more publicly, I decided to do so... If >>>> people continue to react this way, it just demonstrates that >>>> doing things publicly is NOT a way to get things to move >>>> forward in FreeBSD, and people will continue to do things in >>>> private... Luckily, I'm consulting, so I have a few more >>>> hours (for now) to fight these fights, but if it continues to >>>> be an issue, we'll continue to have this problem of czars >>>> that come in, drop a bunch of code and then leave, because >>>> dealing w/ this becomes too expensive... >>>> >>>> So far, only ONE person has commented on the patch on >>>> reviews, and that is delphij... >>>> >>>>> Also, do not want to attempt to be like openbsd, learn from >>>>> for sure, but to be like, no way. >>>> >>>> I'm fine not being like OpenBSD, but as you said, we should >>>> learn from them, and leverage their work... Though I agree >>>> w/ OpenBSD's work to replace random(3), it also isn't who >>>> FreeBSD is, but if we want to continue to be relevant, we do >>>> need to take security seriously, and IMO, this is one of >>>> those steps. >>>> >>>> If someone does find a performance issue w/ my patch, I WILL >>>> work with them on a solution, but I will not work w/ people >>>> who make unfounded claims about the impact of this work... >>> >>> Yeah, the problem could all that. >>> >>> Or it could be people who "collaborate" by saying I'm going to >>> make this change. I'm not going to justify it in any way, and >>> if anybody >> >> I have justified it… > > I think you should explain what you explained to me on IRC. > > Specifically, through a timing attack, you can find (by default) > the lower 7 bits of the value returned by random(). Since random() > is not MP safe, it can sometimes return the same value twice > (through some race that may or may not have been lost). This means > other users can see this data. > > In this instance, it isn’t so much what sched_ule is doing, but > rather what others are able to glean from it. Now, it isn’t clear > that these 7 bits are a big deal since you also have to lose the > race and know the race was lost. Other things in the system might > care if you expose this state. > > Also, in this specific case, it can use the current random > generator in sched_ule to get this number as well. It’s run on a > time scale of ticks, with some jitter. In this specific case, it > doesn’t need to be using random(), but it isn’t clear if the > get_cyclecount() stuff provides enough low-order bits that are > random enough to meet sched_ule’s needs. But it isn’t clear that it > doesn’t (only cause for concern is if there’s a beat pattern for a > cycle count that’s low-resolution, but I don’t think we have any of > these on SMP work loads). > <... snip ...> The timing attack I talked to you about on IRC works like this: A userland process creates as many threads as there are CPUs, and by manipulating the load they generate, gets it so they're all flagged as interactive and at the same priority. (alternating spin and sleep with a 2% duty cycle would work, for instance) It would also be possible to coerce a userland process, like apache to behave this way. These threads now have the ability to preempt all timeshare tasks on all CPUs for slice_size time, by waking up and spinning at the same time. This means they can get very precise knowledge about scheduling, by timing when they get to run, versus when they have to wait. By watching CPU0, one of these threads can measure balance_ticks. This is important because balance_ticks directly exposes the last 7 bits it gets back from random(). (The value gets applied to balance_interval to keep the balancer from running on exactly the same interval) This means that if an attacker can trigger the use of random, or is willing to wait long enough for a race, they can determine the value of those bits that were passed along to anyone who called random() at the same time. It also means that they can eventually discover the state of the RNG, and predict future values. The security implications of disclosing the values this way isn't as severe as it might seem, simply because random() isn't really used in any cryptographically sensitive areas, but there are definite consequences, like predicting firewall port values, and NFS client transaction IDs. It is, however, surprising to learn that the balance_interval sysctl has security implications. --- Harrison <... snip ...> > > You can prove a negative with benchmarks. Then we’d be arguing > over the efficacy of them, but at least that would be progress :) > Or you can strongly suggest a negative by failing to reject the > null hypothesis of no change. That too would be progress. > >>> they don't, then screw the collobaration thing, I'm just going >>> to do it anyway. >> >> It goes both ways, I see it that you're objecting w/o complete >> intformation, and no mater what evidence or work I do, you'll >> just ignore it, and still say it isn't correct or that there's >> this unprovable codition that prevents the work for going in… > > Data is going to break this log-jam. > > Warner > From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 07:55:21 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CA71285; Wed, 25 Feb 2015 07:55:21 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 73519B4F; Wed, 25 Feb 2015 07:55:20 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.133.26.10]) by elvis.mu.org (Postfix) with ESMTPSA id ABC57341F8B1; Tue, 24 Feb 2015 23:55:19 -0800 (PST) Message-ID: <54ED80BD.1080603@freebsd.org> Date: Wed, 25 Feb 2015 02:58:53 -0500 From: Alfred Perlstein Organization: FreeBSD User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: John-Mark Gurney , "K. Macy" Subject: Re: locks and kernel randomness... References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> In-Reply-To: <20150225002301.GS46794@funkthat.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: Konstantin Belousov , Harrison Grundy , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 07:55:21 -0000 On 2/24/15 7:23 PM, John-Mark Gurney wrote: > K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 -0800: >>> If someone does find a performance issue w/ my patch, I WILL work with >>> them on a solution, but I will not work w/ people who make unfounded >>> claims about the impact of this work... >>> >> ... The concerns may be exaggerated, but they aren't >> unfounded. Not quite the same thing, but no one wants to spend the > Till someone shows me code in the kernel tree where this is even close > to a performance problem, it is unfounded... I've asked, and no one > has > >> cycles doing a SHA256 because it's "The Right Thing"(tm) when their >> use case only requires a fletcher2. > Depends upon what you're doing.. I haven't proposed changing ZFS's > default to sha256, so stop w/ the false equivalences... > >> If it doesn't already exist, it might also be worth looking in to a >> more scalable CSPRNG implementation not requiring locking in the >> common case. For example, each core is seeded separately periodically >> so that has a private pool that is protected by a critical section. >> The private pool would be regularly refreshed by cpu-local callout. >> Thus, a lock would only be acquired if the local entropy were >> depleted. > I'm not discussing this until you read and reply to my original email, > since it's clear that my original email's contents has been ignored in > this thread... > What is final proposal? More spinlocks? That is not a good idea. Doing a single buildworld is not enough. Ask netflix or someone with a real load of 1000s of threads/processing to do testing for you if you truly want to touch scheduler. -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 08:29:31 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 64324B76 for ; Wed, 25 Feb 2015 08:29:31 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F391BE87 for ; Wed, 25 Feb 2015 08:29:30 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 46C98E087 for ; Wed, 25 Feb 2015 03:29:29 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.866 X-Spam-Level: X-Spam-Status: No, score=-3.866 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.159, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id k4-shd69UYLd for ; Wed, 25 Feb 2015 03:29:26 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id D93A7E089 for ; Wed, 25 Feb 2015 03:29:25 -0500 (EST) Message-ID: <54ED87E9.8030706@astrodoggroup.com> Date: Wed, 25 Feb 2015 00:29:29 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224015721.GT74514@kib.kiev.ua> <54EBDC1C.3060007@astrodoggroup.com> <20150224024250.GV74514@kib.kiev.ua> <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> In-Reply-To: <54ED80BD.1080603@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 08:29:31 -0000 On 02/24/15 23:58, Alfred Perlstein wrote: > > On 2/24/15 7:23 PM, John-Mark Gurney wrote: >> K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 -0800: >>>> If someone does find a performance issue w/ my patch, I WILL >>>> work with them on a solution, but I will not work w/ people >>>> who make unfounded claims about the impact of this work... >>>> >>> ... The concerns may be exaggerated, but they aren't >>> unfounded. Not quite the same thing, but no one wants to spend >>> the >> Till someone shows me code in the kernel tree where this is even >> close to a performance problem, it is unfounded... I've asked, >> and no one has >> >>> cycles doing a SHA256 because it's "The Right Thing"(tm) when >>> their use case only requires a fletcher2. >> Depends upon what you're doing.. I haven't proposed changing >> ZFS's default to sha256, so stop w/ the false equivalences... >> >>> If it doesn't already exist, it might also be worth looking in >>> to a more scalable CSPRNG implementation not requiring locking >>> in the common case. For example, each core is seeded separately >>> periodically so that has a private pool that is protected by a >>> critical section. The private pool would be regularly refreshed >>> by cpu-local callout. Thus, a lock would only be acquired if >>> the local entropy were depleted. >> I'm not discussing this until you read and reply to my original >> email, since it's clear that my original email's contents has >> been ignored in this thread... >> > What is final proposal? More spinlocks? That is not a good idea. > > Doing a single buildworld is not enough. Ask netflix or someone > with a real load of 1000s of threads/processing to do testing for > you if you truly want to touch scheduler. sched_ule runs this code once every .5 to 1.5 seconds, depending on the value of random, so using a CSPRNG there wouldn't actually be noticeable. (We're talking about a few thousand cycles, when the existing implementation has to make a remote memory read/write numpackages-1/numpackages percent of the time, which costs tens of thousands of cycles. Switching to a per-CPU CSPRNG is actually faster in those cases.) That being said, I believe the plan is to remove random() from sched_ule entirely. It doesn't need it to perform the balancing, and we can just use the LCG from cpu_search, if get_cyclecount isn't viable. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 08:57:12 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F27B255F for ; Wed, 25 Feb 2015 08:57:11 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7A68E24D for ; Wed, 25 Feb 2015 08:57:11 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1P8v06g044840 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 10:57:00 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1P8v06g044840 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1P8uxe4044839; Wed, 25 Feb 2015 10:56:59 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 25 Feb 2015 10:56:59 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150225085659.GA74514@kib.kiev.ua> References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> <54ED87E9.8030706@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ED87E9.8030706@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 08:57:12 -0000 On Wed, Feb 25, 2015 at 12:29:29AM -0800, Harrison Grundy wrote: > > > On 02/24/15 23:58, Alfred Perlstein wrote: > > > > On 2/24/15 7:23 PM, John-Mark Gurney wrote: > >> K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 -0800: > >>>> If someone does find a performance issue w/ my patch, I WILL > >>>> work with them on a solution, but I will not work w/ people > >>>> who make unfounded claims about the impact of this work... > >>>> > >>> ... The concerns may be exaggerated, but they aren't > >>> unfounded. Not quite the same thing, but no one wants to spend > >>> the > >> Till someone shows me code in the kernel tree where this is even > >> close to a performance problem, it is unfounded... I've asked, > >> and no one has > >> > >>> cycles doing a SHA256 because it's "The Right Thing"(tm) when > >>> their use case only requires a fletcher2. > >> Depends upon what you're doing.. I haven't proposed changing > >> ZFS's default to sha256, so stop w/ the false equivalences... > >> > >>> If it doesn't already exist, it might also be worth looking in > >>> to a more scalable CSPRNG implementation not requiring locking > >>> in the common case. For example, each core is seeded separately > >>> periodically so that has a private pool that is protected by a > >>> critical section. The private pool would be regularly refreshed > >>> by cpu-local callout. Thus, a lock would only be acquired if > >>> the local entropy were depleted. > >> I'm not discussing this until you read and reply to my original > >> email, since it's clear that my original email's contents has > >> been ignored in this thread... > >> > > What is final proposal? More spinlocks? That is not a good idea. > > > > Doing a single buildworld is not enough. Ask netflix or someone > > with a real load of 1000s of threads/processing to do testing for > > you if you truly want to touch scheduler. > > sched_ule runs this code once every .5 to 1.5 seconds, depending on > the value of random, so using a CSPRNG there wouldn't actually be > noticeable. (We're talking about a few thousand cycles, when the > existing implementation has to make a remote memory read/write > numpackages-1/numpackages percent of the time, which costs tens of > thousands of cycles. Switching to a per-CPU CSPRNG is actually faster > in those cases.) The cost of the proposed patch, of course, is not the several thousands of instructions in the rebalance. The problem with it is the introduction of the new spinlock, which will be used in many places after the introduction. The cost of the new and often used spinlock is the increase of both interrupt latency and interrupt handler jitter and cpu switch jitter. So neither buildworld timing, nor network throughput are adequate to estimate the change. It is system unresponsivness and loss of the realtime behaviour up to some degree. I thought that it was obvious, at least after spinlocks were mentioned, but apparently it is not, since proposals to measure the patch effect by benchmarking buildworld or passing the traffic are made. > > That being said, I believe the plan is to remove random() from > sched_ule entirely. It doesn't need it to perform the balancing, and > we can just use the LCG from cpu_search, if get_cyclecount isn't viable. I agree with this, but please see my other response. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 09:06:48 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68AC986F; Wed, 25 Feb 2015 09:06:48 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0714737C; Wed, 25 Feb 2015 09:06:47 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1P96cR5047097 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 11:06:38 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1P96cR5047097 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1P96cDB047096; Wed, 25 Feb 2015 11:06:38 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 25 Feb 2015 11:06:38 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150225090638.GB74514@kib.kiev.ua> References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ED5656.50607@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: John-Mark Gurney , Alfred Perlstein , Ian Lepore , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 09:06:48 -0000 On Tue, Feb 24, 2015 at 08:57:58PM -0800, Harrison Grundy wrote: > <... snip ...> > > The timing attack I talked to you about on IRC works like this: > > A userland process creates as many threads as there are CPUs, and by > manipulating the load they generate, gets it so they're all flagged as > interactive and at the same priority. (alternating spin and sleep with > a 2% duty cycle would work, for instance) > > It would also be possible to coerce a userland process, like apache to > behave this way. > > These threads now have the ability to preempt all timeshare tasks on > all CPUs for slice_size time, by waking up and spinning at the same > time. This means they can get very precise knowledge about scheduling, > by timing when they get to run, versus when they have to wait. Ok, this is definitely not impossible. > > By watching CPU0, one of these threads can measure balance_ticks. > > This is important because balance_ticks directly exposes the last 7 > bits it gets back from random(). (The value gets applied to > balance_interval to keep the balancer from running on exactly the same > interval) > > This means that if an attacker can trigger the use of random, or is > willing to wait long enough for a race, they can determine the value > of those bits that were passed along to anyone who called random() at > the same time. > > It also means that they can eventually discover the state of the RNG, > and predict future values. > > The security implications of disclosing the values this way isn't as > severe as it might seem, simply because random() isn't really used in > any cryptographically sensitive areas, but there are definite > consequences, like predicting firewall port values, and NFS client > transaction IDs. > > It is, however, surprising to learn that the balance_interval sysctl > has security implications. So this is an argument to remove the current random() call from the sched_balance(). There is no implications for use of e.g. get_cyclecount() in the sched_balance(), since on x86 userspace has the ability to read the underlying counter directly. On other architectures, where counter backing get_cyclecount() is not accessible to userspace, it is still feasible to use in sched_balance(), simply because counter is ticking. Do you agree with these statements ? Also, as I understand from your other responses, you did tested the patch to use get_cyclecount() on non-x86 machines ? I try to understand what testing was done for the get_cyclecount() for sched_balance() patch, i.e. is it ready for commit. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 09:11:00 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5388E999 for ; Wed, 25 Feb 2015 09:11:00 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 06E0B62B for ; Wed, 25 Feb 2015 09:10:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id C0CF6E476 for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.865 X-Spam-Level: X-Spam-Status: No, score=-3.865 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.158, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vkmaY1yetDdh for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 29156E472 for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) Message-ID: <54ED91A5.8080106@astrodoggroup.com> Date: Wed, 25 Feb 2015 01:11:01 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> In-Reply-To: <20150225085659.GA74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 09:11:00 -0000 On 02/25/15 00:56, Konstantin Belousov wrote: > On Wed, Feb 25, 2015 at 12:29:29AM -0800, Harrison Grundy wrote: >> >> >> On 02/24/15 23:58, Alfred Perlstein wrote: >>> >>> On 2/24/15 7:23 PM, John-Mark Gurney wrote: >>>> K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 >>>> -0800: >>>>>> If someone does find a performance issue w/ my patch, I >>>>>> WILL work with them on a solution, but I will not work w/ >>>>>> people who make unfounded claims about the impact of this >>>>>> work... >>>>>> >>>>> ... The concerns may be exaggerated, but they >>>>> aren't unfounded. Not quite the same thing, but no one >>>>> wants to spend the >>>> Till someone shows me code in the kernel tree where this is >>>> even close to a performance problem, it is unfounded... I've >>>> asked, and no one has >>>> >>>>> cycles doing a SHA256 because it's "The Right Thing"(tm) >>>>> when their use case only requires a fletcher2. >>>> Depends upon what you're doing.. I haven't proposed changing >>>> ZFS's default to sha256, so stop w/ the false >>>> equivalences... >>>> >>>>> If it doesn't already exist, it might also be worth looking >>>>> in to a more scalable CSPRNG implementation not requiring >>>>> locking in the common case. For example, each core is >>>>> seeded separately periodically so that has a private pool >>>>> that is protected by a critical section. The private pool >>>>> would be regularly refreshed by cpu-local callout. Thus, a >>>>> lock would only be acquired if the local entropy were >>>>> depleted. >>>> I'm not discussing this until you read and reply to my >>>> original email, since it's clear that my original email's >>>> contents has been ignored in this thread... >>>> >>> What is final proposal? More spinlocks? That is not a good >>> idea. >>> >>> Doing a single buildworld is not enough. Ask netflix or >>> someone with a real load of 1000s of threads/processing to do >>> testing for you if you truly want to touch scheduler. >> >> sched_ule runs this code once every .5 to 1.5 seconds, depending >> on the value of random, so using a CSPRNG there wouldn't actually >> be noticeable. (We're talking about a few thousand cycles, when >> the existing implementation has to make a remote memory >> read/write numpackages-1/numpackages percent of the time, which >> costs tens of thousands of cycles. Switching to a per-CPU CSPRNG >> is actually faster in those cases.) > The cost of the proposed patch, of course, is not the several > thousands of instructions in the rebalance. The problem with it is > the introduction of the new spinlock, which will be used in many > places after the introduction. The cost of the new and often used > spinlock is the increase of both interrupt latency and interrupt > handler jitter and cpu switch jitter. > > So neither buildworld timing, nor network throughput are adequate > to estimate the change. It is system unresponsivness and loss of > the realtime behaviour up to some degree. > > I thought that it was obvious, at least after spinlocks were > mentioned, but apparently it is not, since proposals to measure the > patch effect by benchmarking buildworld or passing the traffic are > made. > I was referring to the sched_ule case only. That analysis will certainly need to be done for each use of random(), though. >> >> That being said, I believe the plan is to remove random() from >> sched_ule entirely. It doesn't need it to perform the balancing, >> and we can just use the LCG from cpu_search, if get_cyclecount >> isn't viable. > > I agree with this, but please see my other response. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 09:16:19 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D14DFB7A for ; Wed, 25 Feb 2015 09:16:19 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7821C676 for ; Wed, 25 Feb 2015 09:16:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 0FD0FE458 for ; Wed, 25 Feb 2015 04:16:18 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.865 X-Spam-Level: X-Spam-Status: No, score=-3.865 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.158, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XO8szVm+sDlg for ; Wed, 25 Feb 2015 04:16:17 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 8908CE45C for ; Wed, 25 Feb 2015 04:16:17 -0500 (EST) Message-ID: <54ED92E5.4010803@astrodoggroup.com> Date: Wed, 25 Feb 2015 01:16:21 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> In-Reply-To: <20150225090638.GB74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 09:16:19 -0000 On 02/25/15 01:06, Konstantin Belousov wrote: > On Tue, Feb 24, 2015 at 08:57:58PM -0800, Harrison Grundy wrote: >> <... snip ...> >> >> The timing attack I talked to you about on IRC works like this: >> >> A userland process creates as many threads as there are CPUs, and by >> manipulating the load they generate, gets it so they're all flagged as >> interactive and at the same priority. (alternating spin and sleep with >> a 2% duty cycle would work, for instance) >> >> It would also be possible to coerce a userland process, like apache to >> behave this way. >> >> These threads now have the ability to preempt all timeshare tasks on >> all CPUs for slice_size time, by waking up and spinning at the same >> time. This means they can get very precise knowledge about scheduling, >> by timing when they get to run, versus when they have to wait. > Ok, this is definitely not impossible. > >> >> By watching CPU0, one of these threads can measure balance_ticks. >> >> This is important because balance_ticks directly exposes the last 7 >> bits it gets back from random(). (The value gets applied to >> balance_interval to keep the balancer from running on exactly the same >> interval) >> >> This means that if an attacker can trigger the use of random, or is >> willing to wait long enough for a race, they can determine the value >> of those bits that were passed along to anyone who called random() at >> the same time. >> >> It also means that they can eventually discover the state of the RNG, >> and predict future values. >> >> The security implications of disclosing the values this way isn't as >> severe as it might seem, simply because random() isn't really used in >> any cryptographically sensitive areas, but there are definite >> consequences, like predicting firewall port values, and NFS client >> transaction IDs. >> >> It is, however, surprising to learn that the balance_interval sysctl >> has security implications. > > So this is an argument to remove the current random() call from > the sched_balance(). There is no implications for use of e.g. > get_cyclecount() in the sched_balance(), since on x86 userspace has the > ability to read the underlying counter directly. > > On other architectures, where counter backing get_cyclecount() is not > accessible to userspace, it is still feasible to use in sched_balance(), > simply because counter is ticking. > > Do you agree with these statements ? Yes. sched_balance itself does not need any sort of non-public randomness. The worst thing an attacker can do is gain a few extra cycles on a CPU by only running on longer balance intervals. Given the many other ways load gets transferred in ULE, there's not much utility there. > > Also, as I understand from your other responses, you did tested the > patch to use get_cyclecount() on non-x86 machines ? I try to understand > what testing was done for the get_cyclecount() for sched_balance() patch, > i.e. is it ready for commit. I have not tested this on other arches. I spoke to some of the committers active on them to get an idea of what get_cyclecount does. I'm currently testing a patch that creates "sched_random()", using the random number generator from cpu_search. This should provide good enough jitter for the balancer, and other potential scheduler uses of random(); I'll add it to the PR, and send a note out here after I've run it for a bit. --- Harrison > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 09:47:57 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 37F5D8BD for ; Wed, 25 Feb 2015 09:47:57 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D2F8FA12 for ; Wed, 25 Feb 2015 09:47:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 2F2C6E3AC for ; Wed, 25 Feb 2015 04:47:55 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.864 X-Spam-Level: X-Spam-Status: No, score=-3.864 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.157, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mQru7TU+WWuh for ; Wed, 25 Feb 2015 04:47:54 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id AC4F9E23B for ; Wed, 25 Feb 2015 04:47:54 -0500 (EST) Message-ID: <54ED9A4B.4060802@astrodoggroup.com> Date: Wed, 25 Feb 2015 01:47:55 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> <54ED92E5.4010803@astrodoggroup.com> In-Reply-To: <54ED92E5.4010803@astrodoggroup.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 09:47:57 -0000 On 02/25/15 01:16, Harrison Grundy wrote: > > > On 02/25/15 01:06, Konstantin Belousov wrote: >> On Tue, Feb 24, 2015 at 08:57:58PM -0800, Harrison Grundy wrote: >>> <... snip ...> >>> >>> The timing attack I talked to you about on IRC works like this: >>> >>> A userland process creates as many threads as there are CPUs, and by >>> manipulating the load they generate, gets it so they're all flagged as >>> interactive and at the same priority. (alternating spin and sleep with >>> a 2% duty cycle would work, for instance) >>> >>> It would also be possible to coerce a userland process, like apache to >>> behave this way. >>> >>> These threads now have the ability to preempt all timeshare tasks on >>> all CPUs for slice_size time, by waking up and spinning at the same >>> time. This means they can get very precise knowledge about scheduling, >>> by timing when they get to run, versus when they have to wait. >> Ok, this is definitely not impossible. >> >>> >>> By watching CPU0, one of these threads can measure balance_ticks. >>> >>> This is important because balance_ticks directly exposes the last 7 >>> bits it gets back from random(). (The value gets applied to >>> balance_interval to keep the balancer from running on exactly the same >>> interval) >>> >>> This means that if an attacker can trigger the use of random, or is >>> willing to wait long enough for a race, they can determine the value >>> of those bits that were passed along to anyone who called random() at >>> the same time. >>> >>> It also means that they can eventually discover the state of the RNG, >>> and predict future values. >>> >>> The security implications of disclosing the values this way isn't as >>> severe as it might seem, simply because random() isn't really used in >>> any cryptographically sensitive areas, but there are definite >>> consequences, like predicting firewall port values, and NFS client >>> transaction IDs. >>> >>> It is, however, surprising to learn that the balance_interval sysctl >>> has security implications. >> >> So this is an argument to remove the current random() call from >> the sched_balance(). There is no implications for use of e.g. >> get_cyclecount() in the sched_balance(), since on x86 userspace has the >> ability to read the underlying counter directly. >> >> On other architectures, where counter backing get_cyclecount() is not >> accessible to userspace, it is still feasible to use in sched_balance(), >> simply because counter is ticking. >> >> Do you agree with these statements ? > > Yes. sched_balance itself does not need any sort of non-public > randomness. The worst thing an attacker can do is gain a few extra > cycles on a CPU by only running on longer balance intervals. Given the > many other ways load gets transferred in ULE, there's not much utility > there. > >> >> Also, as I understand from your other responses, you did tested the >> patch to use get_cyclecount() on non-x86 machines ? I try to understand >> what testing was done for the get_cyclecount() for sched_balance() patch, >> i.e. is it ready for commit. > > I have not tested this on other arches. I spoke to some of the > committers active on them to get an idea of what get_cyclecount does. > > I'm currently testing a patch that creates "sched_random()", using the > random number generator from cpu_search. This should provide good enough > jitter for the balancer, and other potential scheduler uses of random(); > > I'll add it to the PR, and send a note out here after I've run it for a bit. > Three choices here are attached here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 The only remaining one I don't have a patch for is putting a "real" PRNG in ULE. At this point, as far as ULE goes, It just comes down to picking from one of those approaches. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 10:05:26 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 300861AB for ; Wed, 25 Feb 2015 10:05:26 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id ABA71C50 for ; Wed, 25 Feb 2015 10:05:25 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1PA5Cah017047 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 12:05:13 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1PA5Cah017047 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1PA5Cb9017031; Wed, 25 Feb 2015 12:05:12 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 25 Feb 2015 12:05:12 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150225100512.GC74514@kib.kiev.ua> References: <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> <54ED92E5.4010803@astrodoggroup.com> <54ED9A4B.4060802@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54ED9A4B.4060802@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 10:05:26 -0000 On Wed, Feb 25, 2015 at 01:47:55AM -0800, Harrison Grundy wrote: > Three choices here are attached here: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 > > The only remaining one I don't have a patch for is putting a "real" PRNG > in ULE. > > At this point, as far as ULE goes, It just comes down to picking from > one of those approaches. The third patch, ' Creates sched_random, using the system used in cpu_search.', seems to miss updating the dpcpu randomval in sched_random(), isn't it ? From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 10:17:10 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CC283651 for ; Wed, 25 Feb 2015 10:17:10 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7F666D77 for ; Wed, 25 Feb 2015 10:17:10 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 09E26E43B for ; Wed, 25 Feb 2015 05:17:09 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.863 X-Spam-Level: X-Spam-Status: No, score=-3.863 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.156, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xBrSziFWyqq9 for ; Wed, 25 Feb 2015 05:17:08 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 9F3D4E438 for ; Wed, 25 Feb 2015 05:17:08 -0500 (EST) Message-ID: <54EDA128.4000107@astrodoggroup.com> Date: Wed, 25 Feb 2015 02:17:12 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> <54ED92E5.4010803@astrodoggroup.com> <54ED9A4B.4060802@astrodoggroup.com> <20150225100512.GC74514@kib.kiev.ua> In-Reply-To: <20150225100512.GC74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 10:17:10 -0000 On 02/25/15 02:05, Konstantin Belousov wrote: > On Wed, Feb 25, 2015 at 01:47:55AM -0800, Harrison Grundy wrote: >> Three choices here are attached here: >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 >> >> The only remaining one I don't have a patch for is putting a >> "real" PRNG in ULE. >> >> At this point, as far as ULE goes, It just comes down to picking >> from one of those approaches. > > The third patch, ' Creates sched_random, using the system used in > cpu_search.', seems to miss updating the dpcpu randomval in > sched_random(), isn't it ? > It does exactly what cpu_search does. I really think the scheduler does not actually need randomness in these locations. I've been running for the past few days on a few systems here that way for testing purposes without issue. I'll post a separate call for testers for a patch that overtly removes them. ULE has a ton of different methods for balancing load between cores (which is why you can turn off the long term balancer entirely). --- Harrison From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 10:21:25 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E0F22865 for ; Wed, 25 Feb 2015 10:21:25 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E5DDD9B for ; Wed, 25 Feb 2015 10:21:25 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1PALDjS047905 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 25 Feb 2015 12:21:13 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1PALDjS047905 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t1PALDno047904; Wed, 25 Feb 2015 12:21:13 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 25 Feb 2015 12:21:13 +0200 From: Konstantin Belousov To: Harrison Grundy Subject: Re: locks and kernel randomness... Message-ID: <20150225102113.GD74514@kib.kiev.ua> References: <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> <54ED92E5.4010803@astrodoggroup.com> <54ED9A4B.4060802@astrodoggroup.com> <20150225100512.GC74514@kib.kiev.ua> <54EDA128.4000107@astrodoggroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54EDA128.4000107@astrodoggroup.com> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 10:21:26 -0000 On Wed, Feb 25, 2015 at 02:17:12AM -0800, Harrison Grundy wrote: > > > On 02/25/15 02:05, Konstantin Belousov wrote: > > On Wed, Feb 25, 2015 at 01:47:55AM -0800, Harrison Grundy wrote: > >> Three choices here are attached here: > >> > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 > >> > >> The only remaining one I don't have a patch for is putting a > >> "real" PRNG in ULE. > >> > >> At this point, as far as ULE goes, It just comes down to picking > >> from one of those approaches. > > > > The third patch, ' Creates sched_random, using the system used in > > cpu_search.', seems to miss updating the dpcpu randomval in > > sched_random(), isn't it ? > > > > It does exactly what cpu_search does. I do not see how it does exactly what cpu_search does. cpu_search is updating *rndptr: rnd = (*rndptr = *rndptr * 69069 + 5) >> 26; while code in patch is: + rndptr = DPCPU_PTR(randomval); + rnd = (*rndptr * 69069 + 5) >> 26; where is the write to *rndptr ? > > I really think the scheduler does not actually need randomness in > these locations. I've been running for the past few days on a few > systems here that way for testing purposes without issue. Why doing the arithmetic then ? > > I'll post a separate call for testers for a patch that overtly removes > them. ULE has a ton of different methods for balancing load between > cores (which is why you can turn off the long term balancer entirely). From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 10:23:32 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1E3D5980 for ; Wed, 25 Feb 2015 10:23:32 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C5B7EE36 for ; Wed, 25 Feb 2015 10:23:31 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 20835E448 for ; Wed, 25 Feb 2015 05:23:30 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.863 X-Spam-Level: X-Spam-Status: No, score=-3.863 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.156, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id byJcTmoHhP6E for ; Wed, 25 Feb 2015 05:23:30 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id C6254E446 for ; Wed, 25 Feb 2015 05:23:29 -0500 (EST) Message-ID: <54EDA2A6.9080209@astrodoggroup.com> Date: Wed, 25 Feb 2015 02:23:34 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224231921.GQ46794@funkthat.com> <1424822522.1328.11.camel@freebsd.org> <20150225002956.GT46794@funkthat.com> <2F49527F-2F58-4BD2-B8BE-1B1190CCD4D0@bsdimp.com> <54ED5656.50607@astrodoggroup.com> <20150225090638.GB74514@kib.kiev.ua> <54ED92E5.4010803@astrodoggroup.com> <54ED9A4B.4060802@astrodoggroup.com> <20150225100512.GC74514@kib.kiev.ua> <54EDA128.4000107@astrodoggroup.com> <20150225102113.GD74514@kib.kiev.ua> In-Reply-To: <20150225102113.GD74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 10:23:32 -0000 On 02/25/15 02:21, Konstantin Belousov wrote: > On Wed, Feb 25, 2015 at 02:17:12AM -0800, Harrison Grundy wrote: >> >> >> On 02/25/15 02:05, Konstantin Belousov wrote: >>> On Wed, Feb 25, 2015 at 01:47:55AM -0800, Harrison Grundy >>> wrote: >>>> Three choices here are attached here: >>>> >>>> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=197922 >>>> >>>> The only remaining one I don't have a patch for is putting a >>>> "real" PRNG in ULE. >>>> >>>> At this point, as far as ULE goes, It just comes down to >>>> picking from one of those approaches. >>> >>> The third patch, ' Creates sched_random, using the system used >>> in cpu_search.', seems to miss updating the dpcpu randomval in >>> sched_random(), isn't it ? >>> >> >> It does exactly what cpu_search does. > I do not see how it does exactly what cpu_search does. > > cpu_search is updating *rndptr: rnd = (*rndptr = *rndptr * 69069 + > 5) >> 26; > > while code in patch is: + rndptr = DPCPU_PTR(randomval); + > rnd = (*rndptr * 69069 + 5) >> 26; > > where is the write to *rndptr ? My mistake. I'll update the patch accordingly. Interestingly enough, ULE doesn't seem to care about losing randomness there either. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 14:07:35 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 79AFE713 for ; Wed, 25 Feb 2015 14:07:35 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 61ECCC54 for ; Wed, 25 Feb 2015 14:07:35 +0000 (UTC) Received: from AlfredMacbookAir.local (unknown [12.133.26.10]) by elvis.mu.org (Postfix) with ESMTPSA id EAEFA341F906; Wed, 25 Feb 2015 06:07:33 -0800 (PST) Message-ID: <54EDD7F9.9030608@mu.org> Date: Wed, 25 Feb 2015 09:11:05 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Konstantin Belousov , Harrison Grundy Subject: Re: locks and kernel randomness... References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> In-Reply-To: <20150225085659.GA74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 14:07:35 -0000 On 2/25/15 3:56 AM, Konstantin Belousov wrote: > The cost of the proposed patch, of course, is not the several > thousands of instructions in the rebalance. The problem with it is the > introduction of the new spinlock, which will be used in many places > after the introduction. The cost of the new and often used spinlock is > the increase of both interrupt latency and interrupt handler jitter and > cpu switch jitter. > > So neither buildworld timing, nor network throughput are adequate > to estimate the change. It is system unresponsivness and loss of > the realtime behaviour up to some degree. > > I thought that it was obvious, at least after spinlocks were mentioned, > but apparently it is not, since proposals to measure the patch effect > by benchmarking buildworld or passing the traffic are made. > Thank you. -Alfred From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 15:09:52 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4287FD2B for ; Wed, 25 Feb 2015 15:09:52 +0000 (UTC) Received: from mail-pa0-f41.google.com (mail-pa0-f41.google.com [209.85.220.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F6C9375 for ; Wed, 25 Feb 2015 15:09:51 +0000 (UTC) Received: by pablf10 with SMTP id lf10so5861874pab.6 for ; Wed, 25 Feb 2015 07:09:45 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=OavwdixP7bB85A/vyNLW1YNcun0WuuvLONwrn3rgWFc=; b=LtMO69UiD7J8TxTvmT+92ZZs9vsnxeR6xwf5bdCC2U3+MlYosw8cQh1r/6VDAjKP6z iQ6/WWd90kLbsFUTj9Mp8Hiyn4+IutTgI8w06833hnjBz2Xw/oqsupTYIv/oEzlq8lSD IWDqMmngnOtU7/jwJUuZwdNwstXuSwJjalUpzrVtfEVWUaTlPq24JXU5r4GmJoZrGd2M jkicnz/59VZwKuqyaETg+SXQtZfjKnUl6DzTI0BAreIUQTA9+yWKHtqm2452o884Wc+J hk4yvjsJjw3pneD89zFyhanISgkoPyzW3JJpWcMYBYCckrM3iUPZwkvwCQvYCOVxBwRO 44Qw== X-Gm-Message-State: ALoCoQleCzOZ0xnlDW5LtqvTTgSmLUsZL9u+qBivS8LwfrQ7Syt+3TJZBp5jFtx5OSdTJ7D0+y3Q X-Received: by 10.69.27.15 with SMTP id jc15mr6200532pbd.126.1424876984948; Wed, 25 Feb 2015 07:09:44 -0800 (PST) Received: from macintosh-3c0754232d17.corp.netflix.com ([69.53.236.236]) by mx.google.com with ESMTPSA id mm9sm41519877pbc.76.2015.02.25.07.09.43 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 25 Feb 2015 07:09:43 -0800 (PST) Sender: Warner Losh Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Subject: Re: locks and kernel randomness... From: Warner Losh In-Reply-To: <20150225085659.GA74514@kib.kiev.ua> Date: Wed, 25 Feb 2015 08:09:39 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <3A406DF7-E6A1-494D-9B7D-3666D41DBC2B@bsdimp.com> References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.2070.6) Cc: Harrison Grundy , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 15:09:52 -0000 > On Feb 25, 2015, at 1:56 AM, Konstantin Belousov = wrote: > So neither buildworld timing, nor network throughput are adequate > to estimate the change. It is system unresponsivness and loss of > the realtime behaviour up to some degree. Yes. You need to look at the changes affects, if any, on outlier = behavior under load. Without careful monitoring of that, you won=E2=80=99t see the bad = effects. I=E2=80=99ve made several changes over the years to improve performance that I later (usually much = later) had to back out or modify because it made the outlier behavior much worse. So data here isn=E2=80=99t =E2=80=9CI did 3 build worlds and the time = was about the same=E2=80=9D even if ministat(8) says there=E2=80=99s no difference. Warner From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 21:50:13 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3C6CC82F for ; Wed, 25 Feb 2015 21:50:13 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14605E2C for ; Wed, 25 Feb 2015 21:50:13 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 24AB1B9AD; Wed, 25 Feb 2015 16:50:12 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... Date: Wed, 25 Feb 2015 15:20:52 -0500 Message-ID: <1774232.7ZAkabLA24@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <20150225085659.GA74514@kib.kiev.ua> References: <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 25 Feb 2015 16:50:12 -0500 (EST) Cc: Konstantin Belousov X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 21:50:13 -0000 On Wednesday, February 25, 2015 10:56:59 AM Konstantin Belousov wrote: > The cost of the proposed patch, of course, is not the several > thousands of instructions in the rebalance. The problem with it is the > introduction of the new spinlock, which will be used in many places > after the introduction. The cost of the new and often used spinlock is > the increase of both interrupt latency and interrupt handler jitter and > cpu switch jitter. > > So neither buildworld timing, nor network throughput are adequate > to estimate the change. It is system unresponsivness and loss of > the realtime behaviour up to some degree. > > I thought that it was obvious, at least after spinlocks were mentioned, > but apparently it is not, since proposals to measure the patch effect > by benchmarking buildworld or passing the traffic are made. +1 The only thing I will add is that in general this makes the system more fragile and complex as well. Please just stay with a regular mutex and change the scheduler to not use random() (which seems to be in progress?). I'm not sure why we needed the extra 40 messages in this thread after that was effectively said the first time. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 22:11:27 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E78598E2; Wed, 25 Feb 2015 22:11:26 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6488A142; Wed, 25 Feb 2015 22:11:25 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMBL4B025160 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:11:21 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMBLut025159; Thu, 26 Feb 2015 01:11:21 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:11:21 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150225221120.GC17947@FreeBSD.org> References: <201502170150.t1H1ouxM020621@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502170150.t1H1ouxM020621@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , George Neville-Neil , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:11:27 -0000 On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: M> Well, I developed the prototype as I had planned, using a 64-bit media M> word, and found that I got about 100 files in GENERIC that didn't compile; M> they attempted to store "media words" in an int. My kingdom for a typedef. M> That didn't meet my goal of KPI compatibility, so I went to Plan B. M> M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media M> type. I then used the variant/subtype field to store the extended type. M> Effectively, the previously unused bit doubles the effective size of the M> subtype field. Given that the previous 5-bit field lasted us 18 years, M> I figured that doubling it would last a while. I also changed the M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended M> types are all mapped to IFM_OTHER (31) using the old interface, but M> are visible using the new one. M> M> With these changes, I modified one driver (vtnet) to use an extended type, M> and the rest of GENERIC is happy. The changes to ifconfig are also fairly M> small. The patch is appended, where email programs will screw it up, M> or at ftp://ftp.karels.net/outgoing/if_media.patch. M> M> The VFAST subtype is a throw-away for testing. M> M> This seems like a reasonably pragmatic change to support the new 40 Gb/s M> media types until someone wants to design an improved but non-backward- M> compatible interface. I think it meets the goal of suitability for M> back-porting; it could be MFCed. I will dare to vote against the crowd. We can't and don't plan to preserve the driver KPI for the 11 branch. The plan, that I hope to accomplish by 11 is to provide a driver KPI, where drivers do not about struct ifnet, and other network stack stuff. Of course, that's a huge change in KPI. But we do it for the sake to avoid future changes. So, all this tricks with one extra bit seem unnecessary to me. I'd suggest to introduce new 'struct ifmedia' with enough space, and of course put extra space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code to handle it, without any extended bit tricks. For the sake of userland API, save old current 'struct ifmedia' as 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and tries to convert from ifmedia to oifmedia, To summarise: the patch adds tricks to just double the ifmedia name space, not solving the problem forever. New API is introduced, but old limited one doesn't have foreseable obsolete plan, since new is tied to it. All tricks are performed for the sake of driver KPI stability, which isn't planned to be kept for this major release cycle. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 22:17:47 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 59044F9A; Wed, 25 Feb 2015 22:17:47 +0000 (UTC) Received: from mail-la0-x22e.google.com (mail-la0-x22e.google.com [IPv6:2a00:1450:4010:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB5A91E9; Wed, 25 Feb 2015 22:17:46 +0000 (UTC) Received: by lams18 with SMTP id s18so7119192lam.11; Wed, 25 Feb 2015 14:17:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ehVDh+dZz5XzbqTPgyNVhQMDB6B0hGYGlnQNl5aXvzg=; b=J0pFXpfd+Niavq4VJufPKlkKMhkXmHkT6YrZv+EETpcZUTlOiYILMgidLMNd8CSaVh C4MywqW9MlpvNgLqjmqhDB+ya/3j0AJJOLOuGTTzCOXY40+YpyIJl4AD6IAtpHLC3aMR M8fQtXH2pLQjE7UVjkzUuzJnGZ0+jNn6Ifgirp93SEDb8JF7bhQL2rFTRsIXmAoOq0Gh dcXvIbd7CNTBdbEbeaj2+CrZd19golYkhg9Z41quD2RxHD7V7+kdvM+LhJodmqzM/9sb PAbOFZsMAelfr7YaK3kYF7MCPCC/ze/irU3LcB2EFn68Zk6HcMuWw0rUZ2EHgWC/F7gi Sv/A== MIME-Version: 1.0 X-Received: by 10.112.169.39 with SMTP id ab7mr4662205lbc.85.1424902664816; Wed, 25 Feb 2015 14:17:44 -0800 (PST) Received: by 10.112.173.199 with HTTP; Wed, 25 Feb 2015 14:17:44 -0800 (PST) In-Reply-To: <20150225221120.GC17947@FreeBSD.org> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> Date: Wed, 25 Feb 2015 14:17:44 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:17:47 -0000 So we have products coming soon that need to extend the media, if you have grandiose plans in the future you can worry about things then, Linux can handle the extended media TODAY, wouldn't it be nice to do so as well? Also, this solution is something that can be MFC'd into 10 STABLE, if you do something big in the 11 stream I will be that wont be possible, so I'd say lets do this as a first step. Jack On Wed, Feb 25, 2015 at 2:11 PM, Gleb Smirnoff wrote: > On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: > M> Well, I developed the prototype as I had planned, using a 64-bit media > M> word, and found that I got about 100 files in GENERIC that didn't > compile; > M> they attempted to store "media words" in an int. My kingdom for a > typedef. > M> That didn't meet my goal of KPI compatibility, so I went to Plan B. > M> > M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media > M> type. I then used the variant/subtype field to store the extended type. > M> Effectively, the previously unused bit doubles the effective size of the > M> subtype field. Given that the previous 5-bit field lasted us 18 years, > M> I figured that doubling it would last a while. I also changed the > M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended > M> types are all mapped to IFM_OTHER (31) using the old interface, but > M> are visible using the new one. > M> > M> With these changes, I modified one driver (vtnet) to use an extended > type, > M> and the rest of GENERIC is happy. The changes to ifconfig are also > fairly > M> small. The patch is appended, where email programs will screw it up, > M> or at ftp://ftp.karels.net/outgoing/if_media.patch. > M> > M> The VFAST subtype is a throw-away for testing. > M> > M> This seems like a reasonably pragmatic change to support the new 40 Gb/s > M> media types until someone wants to design an improved but non-backward- > M> compatible interface. I think it meets the goal of suitability for > M> back-porting; it could be MFCed. > > I will dare to vote against the crowd. > > We can't and don't plan to preserve the driver KPI for the 11 branch. The > plan, that I hope to accomplish by 11 is to provide a driver KPI, where > drivers do not about struct ifnet, and other network stack stuff. Of > course, that's a huge change in KPI. But we do it for the sake to avoid > future changes. > > So, all this tricks with one extra bit seem unnecessary to me. I'd suggest > to introduce new 'struct ifmedia' with enough space, and of course put > extra > space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code > to handle it, without any extended bit tricks. > > For the sake of userland API, save old current 'struct ifmedia' as > 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. > Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and > tries to convert from ifmedia to oifmedia, > > To summarise: the patch adds tricks to just double the ifmedia name space, > not solving the problem forever. New API is introduced, but old limited one > doesn't have foreseable obsolete plan, since new is tied to it. All tricks > are performed for the sake of driver KPI stability, which isn't planned > to be kept for this major release cycle. > > -- > Totus tuus, Glebius. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 22:29:05 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DB48E661; Wed, 25 Feb 2015 22:29:05 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 59F9135C; Wed, 25 Feb 2015 22:29:04 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMT2EP025280 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:29:02 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMT2Z6025279; Thu, 26 Feb 2015 01:29:02 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:29:02 +0300 From: Gleb Smirnoff To: Jack Vogel Subject: Re: Adding new media types to if_media.h Message-ID: <20150225222902.GD17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:29:06 -0000 On Wed, Feb 25, 2015 at 02:17:44PM -0800, Jack Vogel wrote: J> So we have products coming soon that need to extend the media, if you have J> grandiose plans in the future you can worry about things then, Linux can J> handle the extended media TODAY, wouldn't it be nice to do so as well? I didn't suggest to wait, did I? J> Also, this solution is something that can be MFC'd into 10 STABLE, J> if you do something big in the 11 stream I will be that wont be possible, J> so I'd say lets do this as a first step. That would mean that new SIOCXGIFMEDIA needs to be supported from now and forever, still using the old structure and still limited to 62 types. Alternative is: In head: - new if_media, new value for SIOCGIFMEDIA, old API preserved via keeping oif_media, OSIOCGIFMEDIA, keeping it contained under ifdef BURN_BRIDGES. With a possibility to remove it some foreseable future. Merging to stable/10: - new if_media by the name of nif_media, new SIOCGIFMEDIA by name of NSIOCGIFMEDIA - old API untouched This way we still are able to introduce new feature to stable/10. But the ugly compat code is put into the branch with limited lifetime, instead of carrying it to future. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 22:42:57 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6198D5D; Wed, 25 Feb 2015 22:42:57 +0000 (UTC) Received: from mail-pa0-x231.google.com (mail-pa0-x231.google.com [IPv6:2607:f8b0:400e:c03::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E1A6784; Wed, 25 Feb 2015 22:42:57 +0000 (UTC) Received: by pabkx10 with SMTP id kx10so8785779pab.0; Wed, 25 Feb 2015 14:42:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=synNKbI10QmCXiRS13RbWOTJKfeuehlof2Uwl9DaGbQ=; b=Ffy0o7Zk6e8wOYkUJNMxojCvdcn3Gy2GvdtAjOgEFDWMd3WX+CL+5vLjNzs2jrslX8 b9yjC4+rtce5c65fnMd6V/yQ2TmU+dB8n/DcmIKJtQzJJnCGUktdHMx5y2qTMdDIj5gL jaPKiE3Kp56P3wtU8kYCaRHMPM8oSrlgXSY1HO7qwnn1+zNY+qe4/IKta6dmciUDnozc 1dALhF/lSLN3cKiY+59xlIpXb14Smq9OYCz5E43mL5CLiqih7fWEzAn7kvhVi8ugQNdk wvTRKy3z2oAlaU/dqWKzi+lBk840mCVS30Yn6OHJNZDyTxYvIrOzu5PwLNcchXM3aNol K7Fw== X-Received: by 10.70.46.65 with SMTP id t1mr9378775pdm.128.1424904176925; Wed, 25 Feb 2015 14:42:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.70.67.104 with HTTP; Wed, 25 Feb 2015 14:42:16 -0800 (PST) In-Reply-To: <20150225222902.GD17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> <20150225222902.GD17947@glebius.int.ru> From: Eric Joyner Date: Wed, 25 Feb 2015 14:42:16 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h To: Gleb Smirnoff Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Jack Vogel , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:42:57 -0000 Tbh, I respect Gleb's approach, but developing such a thing would take a while; the fix Mike proposed would be a fix now. I mean, I'd like to see a decoupling of media types and speeds from "standard" names, and maybe have both an ability to query what modules a device supports and what speeds it supports given its current module, instead of relying on the clunky media type list now. And then it'd be nice to set flow control from ifconfig, too, without having to couple those modes to media types. I'm still all for having a new system in the future. But in changing the KPI so much, it'd be important to consider the "do we need an ethtool-equivalent" discussion. And I think we've lost a bunch of people who were in the original discussion from the to:/cc: list. --- - Eric Joyner On Wed, Feb 25, 2015 at 2:29 PM, Gleb Smirnoff wrote: > On Wed, Feb 25, 2015 at 02:17:44PM -0800, Jack Vogel wrote: > J> So we have products coming soon that need to extend the media, if you > have > J> grandiose plans in the future you can worry about things then, Linux can > J> handle the extended media TODAY, wouldn't it be nice to do so as well? > > I didn't suggest to wait, did I? > > J> Also, this solution is something that can be MFC'd into 10 STABLE, > J> if you do something big in the 11 stream I will be that wont be > possible, > J> so I'd say lets do this as a first step. > > That would mean that new SIOCXGIFMEDIA needs to be supported from now and > forever, still using the old structure and still limited to 62 types. > > Alternative is: > > In head: > - new if_media, new value for SIOCGIFMEDIA, old API preserved via > keeping oif_media, OSIOCGIFMEDIA, keeping it contained under > ifdef BURN_BRIDGES. With a possibility to remove it some foreseable > future. > > Merging to stable/10: > - new if_media by the name of nif_media, new SIOCGIFMEDIA by name of > NSIOCGIFMEDIA > - old API untouched > > This way we still are able to introduce new feature to stable/10. But > the ugly compat code is put into the branch with limited lifetime, > instead of carrying it to future. > > -- > Totus tuus, Glebius. > _______________________________________________ > freebsd-arch@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-arch > To unsubscribe, send any mail to "freebsd-arch-unsubscribe@freebsd.org" > From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 22:50:54 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 62F2ACB; Wed, 25 Feb 2015 22:50:54 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B8C79833; Wed, 25 Feb 2015 22:50:53 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1PMop0u025388 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 26 Feb 2015 01:50:51 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1PMopYp025387; Thu, 26 Feb 2015 01:50:51 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Thu, 26 Feb 2015 01:50:51 +0300 From: Gleb Smirnoff To: Eric Joyner Subject: Re: Adding new media types to if_media.h Message-ID: <20150225225051.GE17947@glebius.int.ru> References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> <20150225222902.GD17947@glebius.int.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Jack Vogel , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 22:50:54 -0000 On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: E> Tbh, I respect Gleb's approach, but developing such a thing would take a E> while; the fix Mike proposed would be a fix now. E> E> I mean, I'd like to see a decoupling of media types and speeds from E> "standard" names, and maybe have both an ability to query what modules a E> device supports and what speeds it supports given its current module, E> instead of relying on the clunky media type list now. And then it'd be nice E> to set flow control from ifconfig, too, without having to couple those E> modes to media types. I'm still all for having a new system in the future. E> E> But in changing the KPI so much, it'd be important to consider the "do we E> need an ethtool-equivalent" discussion. E> E> And I think we've lost a bunch of people who were in the original E> discussion from the to:/cc: list. Actually the amount of code for my approach is approximately the same as with Mike's. The only thing we must sit down and think without a hurry are the required and spare fields for new if_media. We definitely need input from Adrian on his net80211 requirements, and input from all involved parties. I'm willing to code this if we all agree on the topic, so that you will code all done and commited before 10.2-RELEASE. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 23:07:47 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 712833AC; Wed, 25 Feb 2015 23:07:47 +0000 (UTC) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5A6649FA; Wed, 25 Feb 2015 23:07:47 +0000 (UTC) Received: from AlfredMacbookAir.local (3.sub-70-208-71.myvzw.com [70.208.71.3]) by elvis.mu.org (Postfix) with ESMTPSA id C34CC341F90F; Wed, 25 Feb 2015 15:07:39 -0800 (PST) Message-ID: <54EE5692.5070402@mu.org> Date: Wed, 25 Feb 2015 18:11:14 -0500 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: Gleb Smirnoff , Mike Karels Subject: Re: Adding new media types to if_media.h References: <201502170150.t1H1ouxM020621@mail.karels.net> <20150225221120.GC17947@FreeBSD.org> In-Reply-To: <20150225221120.GC17947@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , George Neville-Neil , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 23:07:47 -0000 On 2/25/15 5:11 PM, Gleb Smirnoff wrote: > On Mon, Feb 16, 2015 at 07:50:56PM -0600, Mike Karels wrote: > M> Well, I developed the prototype as I had planned, using a 64-bit media > M> word, and found that I got about 100 files in GENERIC that didn't compile; > M> they attempted to store "media words" in an int. My kingdom for a typedef. > M> That didn't meet my goal of KPI compatibility, so I went to Plan B. > M> > M> Plan B is to steal an unused bit (RFU) to indicate an "extended" media > M> type. I then used the variant/subtype field to store the extended type. > M> Effectively, the previously unused bit doubles the effective size of the > M> subtype field. Given that the previous 5-bit field lasted us 18 years, > M> I figured that doubling it would last a while. I also changed the > M> SIOGGIFMEDIA ioctl, splitting it for binary compatibility; extended > M> types are all mapped to IFM_OTHER (31) using the old interface, but > M> are visible using the new one. > M> > M> With these changes, I modified one driver (vtnet) to use an extended type, > M> and the rest of GENERIC is happy. The changes to ifconfig are also fairly > M> small. The patch is appended, where email programs will screw it up, > M> or at ftp://ftp.karels.net/outgoing/if_media.patch. > M> > M> The VFAST subtype is a throw-away for testing. > M> > M> This seems like a reasonably pragmatic change to support the new 40 Gb/s > M> media types until someone wants to design an improved but non-backward- > M> compatible interface. I think it meets the goal of suitability for > M> back-porting; it could be MFCed. > > I will dare to vote against the crowd. > > We can't and don't plan to preserve the driver KPI for the 11 branch. The > plan, that I hope to accomplish by 11 is to provide a driver KPI, where > drivers do not about struct ifnet, and other network stack stuff. Of > course, that's a huge change in KPI. But we do it for the sake to avoid > future changes. > > So, all this tricks with one extra bit seem unnecessary to me. I'd suggest > to introduce new 'struct ifmedia' with enough space, and of course put extra > space in there. Give a new value to SIOCGIFMEDIA. Write a new clear code > to handle it, without any extended bit tricks. > > For the sake of userland API, save old current 'struct ifmedia' as > 'struct oifmedia', and take old value of ioctl to OSIOCIGIFMEDIA. > Write a function under BURN_BRIDGES that handles OSIOCIGIFMEDIA and > tries to convert from ifmedia to oifmedia, > > To summarise: the patch adds tricks to just double the ifmedia name space, > not solving the problem forever. New API is introduced, but old limited one > doesn't have foreseable obsolete plan, since new is tied to it. All tricks > are performed for the sake of driver KPI stability, which isn't planned > to be kept for this major release cycle. +1, rip the bandaid off. -Alfred From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 00:18:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CB4E5200 for ; Thu, 26 Feb 2015 00:18:22 +0000 (UTC) Received: from mail-pa0-f54.google.com (mail-pa0-f54.google.com [209.85.220.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 997FD10D for ; Thu, 26 Feb 2015 00:18:22 +0000 (UTC) Received: by paceu11 with SMTP id eu11so9224668pac.10 for ; Wed, 25 Feb 2015 16:18:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:subject:mime-version:content-type:from :in-reply-to:date:cc:message-id:references:to; bh=3O2r3Gy2W4/yrfRaFn33+YJ54Zem4RfAzJuwpWI67Zg=; b=XGJw/oDYeXZqp6EY6Oif+fGiXbsjdzpkoNWL+dCJgYTADb/L4v9Mgbn9GQ/uVNJ1PE D4a8/rzXvbjHGoNkbr5jaelH61cZSeLw/vPt+okS2fv1O4elpO4T74yaj7Yzip+so1Dw za3rCEjk6xfnPmgXp2nhX8Iy0JqliyZM/0OIyPOV6VYWNcZBChaOvYSx4t0ERjQ0C6Wf 8Uf+DZgWWtI1yz3GETqeCPDhMsAugbOMV4i9ipLWXUhNSdWcLzB7FuazE2EUSrnzzgiQ +yJ86GupiVw0eBtAffZJtZhAy6Hz/W/zk8xmfXcB180bC5bno6rO9uyA3LfHvks9hCVe DZxg== X-Gm-Message-State: ALoCoQkpiaHz3ZvKUSGGCRePNAIhdAG1ZN3ba2GXdz7ZQcArbaB4wpzv4/vCWNwipIER0TRem3Mp X-Received: by 10.67.10.47 with SMTP id dx15mr9909009pad.139.1424909894329; Wed, 25 Feb 2015 16:18:14 -0800 (PST) Received: from [10.64.25.63] ([69.53.236.236]) by mx.google.com with ESMTPSA id qp4sm42379950pbc.63.2015.02.25.16.18.12 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 25 Feb 2015 16:18:13 -0800 (PST) Sender: Warner Losh Subject: Re: locks and kernel randomness... Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2070.6\)) Content-Type: multipart/signed; boundary="Apple-Mail=_CE406D6A-343F-4088-9141-E6CFE14BA61E"; protocol="application/pgp-signature"; micalg=pgp-sha512 X-Pgp-Agent: GPGMail 2.5b5 From: Warner Losh In-Reply-To: <1774232.7ZAkabLA24@ralph.baldwin.cx> Date: Wed, 25 Feb 2015 17:18:10 -0700 Message-Id: <871FF097-7372-4901-8202-003398B6ABE3@bsdimp.com> References: <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> <1774232.7ZAkabLA24@ralph.baldwin.cx> To: John Baldwin X-Mailer: Apple Mail (2.2070.6) Cc: Konstantin Belousov , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 00:18:22 -0000 --Apple-Mail=_CE406D6A-343F-4088-9141-E6CFE14BA61E Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Feb 25, 2015, at 1:20 PM, John Baldwin wrote: >=20 > On Wednesday, February 25, 2015 10:56:59 AM Konstantin Belousov wrote: >> The cost of the proposed patch, of course, is not the several >> thousands of instructions in the rebalance. The problem with it is = the >> introduction of the new spinlock, which will be used in many places >> after the introduction. The cost of the new and often used spinlock = is >> the increase of both interrupt latency and interrupt handler jitter = and >> cpu switch jitter. >>=20 >> So neither buildworld timing, nor network throughput are adequate >> to estimate the change. It is system unresponsivness and loss of >> the realtime behaviour up to some degree. >>=20 >> I thought that it was obvious, at least after spinlocks were = mentioned, >> but apparently it is not, since proposals to measure the patch effect >> by benchmarking buildworld or passing the traffic are made. >=20 > +1 >=20 > The only thing I will add is that in general this makes the system = more > fragile and complex as well. Please just stay with a regular mutex = and change > the scheduler to not use random() (which seems to be in progress?). = I'm not > sure why we needed the extra 40 messages in this thread after that was > effectively said the first time. If you=E2=80=99d like to be helpful, please review = https://reviews.freebsd.org/D1968 Harrison Grundy wrote it, and I put it in phabricator for him while he = sorts out his own account. In a fit of passive aggressiveness, I=E2=80=99ve added everybody who = commented on this thread as reviewers, except Bruce and Harrison who don=E2=80=99t = have accounts yet. Warner --Apple-Mail=_CE406D6A-343F-4088-9141-E6CFE14BA61E Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQIcBAEBCgAGBQJU7mZCAAoJEGwc0Sh9sBEA3VsP/iNIgP9IbRVC8ZYh2yUx/9fX PRLC4Iqsj2ZakIdPaP3YnRXHEmrwEN8l2G6Zcf/rtJu1EZojyhdzAwW8cgdh6kfj 2AWmR2IjEph3FLiFFo2endZTwFvdE2SBj34uda45xV2ow4q98UvdliOKE9+ba0z1 7eaOSXjqYY3J1mCETHxfEDc08lf1qiRwxVtN2QwaCmzH0/l7a9QgLFvgbbD4sxyh /zf8xvnUG9CAj7HxCFrBs3r9NkFE9ifx6SBZxxK2mycv/YZKHF4ESp3XUXngy31Y hsJjC/m2SQQlCmoV4tACZQ4wL7TqI9wobBejr0wUGV6pbdGGhhrdhxDsfQHnJb8v W4gRQCBAIbNvYztyPql0LZuSC2ZQftCSOW/KHel0eZnvbIS/+wUAbzgrQgt4U0Ps tf2nX73vVCOh7SiW09ybezpK71F9NDFiKfQmYYYzquYJ6v8gHigjBnSaZlrnh2IC iY2s69wB+Huge6v/VMg4HsoDD6tXpAtf9Z/D9Lv4Ona17aLW9EQi2F0CPXTskxfX BJauYUzGPMyMWxqU3rbCXGAAzSF4ybxC6QE+e9n2ag6JpErqH3BLX9Ck6jfUQplk 19AU1DPFvde9Qt5Agj+Qnnde8jMhOmfv071GjQLDNWnrWncQFG8xzwXzLCe6cUME OQ96bfjh5fjZi/Uwi744 =EXES -----END PGP SIGNATURE----- --Apple-Mail=_CE406D6A-343F-4088-9141-E6CFE14BA61E-- From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 02:58:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2B744252; Thu, 26 Feb 2015 02:58:22 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id AC4299DA; Thu, 26 Feb 2015 02:58:21 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1Q2wKNK054143; Wed, 25 Feb 2015 20:58:20 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502260258.t1Q2wKNK054143@mail.karels.net> To: Gleb Smirnoff From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 01:50:51 +0300. <20150225225051.GE17947@glebius.int.ru> Date: Wed, 25 Feb 2015 20:58:20 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 02:58:22 -0000 > On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: > E> Tbh, I respect Gleb's approach, but developing such a thing would take a > E> while; the fix Mike proposed would be a fix now. > E> > E> I mean, I'd like to see a decoupling of media types and speeds from > E> "standard" names, and maybe have both an ability to query what modules a > E> device supports and what speeds it supports given its current module, > E> instead of relying on the clunky media type list now. And then it'd be nice > E> to set flow control from ifconfig, too, without having to couple those > E> modes to media types. I'm still all for having a new system in the future. > E> > E> But in changing the KPI so much, it'd be important to consider the "do we > E> need an ethtool-equivalent" discussion. > E> > E> And I think we've lost a bunch of people who were in the original > E> discussion from the to:/cc: list. > Actually the amount of code for my approach is approximately the same > as with Mike's. The only thing we must sit down and think without a hurry > are the required and spare fields for new if_media. We definitely need > input from Adrian on his net80211 requirements, and input from all involved > parties. I'm not sure what would be different about your approach; you mentioned "n" versions rather than "x" versions of the ioctls, but I don't know what you have in mind for encoding. Any compatible version would be limited to int. In terms of a "real" fix ("ripping the bandaid off"), I think that if_media is basically wrong, and widening it won't fix it. There should be a generic structure that reports the media type (e.g. Ethernet), perhaps the speed and some generic status ("active"). Then there should be media-specific structures that encode the appropriate things including attachment type. 802.11 apparently already has an extension, and Ethernet should have a similar extension. The KPI should be media-type-specific. I don't see something like this being designed soon, and certainly wouldn't be able to be MFC'd. Meanwhile, many of us need to support 40 Gb/s Ethernet on non-current (or non-future) systems. > I'm willing to code this if we all agree on the topic, so that you will > code all done and commited before 10.2-RELEASE. I'd be interested in a sketch or more extended description sometime before 10.2. Back on the subject of the review: the VFAST entries should be removed, and the other entries should be moved down. Mike From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 09:38:18 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9E12F4DF; Thu, 26 Feb 2015 09:38:18 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5CA0B809; Thu, 26 Feb 2015 09:38:18 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 45EEC1FE022; Thu, 26 Feb 2015 10:38:15 +0100 (CET) Message-ID: <54EEE9B6.6090106@selasky.org> Date: Thu, 26 Feb 2015 10:39:02 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net, Gleb Smirnoff Subject: Re: Adding new media types to if_media.h References: <201502260258.t1Q2wKNK054143@mail.karels.net> In-Reply-To: <201502260258.t1Q2wKNK054143@mail.karels.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 09:38:18 -0000 Hi, I'm doing some work for Mellanox and we need some 100GBase types for coming hardware products too. I think we are not using the 32-bits of "ifm_media" well enough. Has it been discussed to add more bits to "IFM_NMASK" and have more ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... Currently 5 IFM types are defined. If 2 more bits can be added to IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it should be possible to define "(32 - 5) * 32 = 864" more ethernet types, which I think should be enough for now - or we add even one more bit to IFM_NMASK ? --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 14:21:49 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 124D819C; Thu, 26 Feb 2015 14:21:49 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id 97314C33; Thu, 26 Feb 2015 14:21:48 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1QELflw056051; Thu, 26 Feb 2015 08:21:42 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502261421.t1QELflw056051@mail.karels.net> To: Hans Petter Selasky From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 10:39:02 +0100. <54EEE9B6.6090106@selasky.org> Date: Thu, 26 Feb 2015 08:21:41 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:21:49 -0000 > I'm doing some work for Mellanox and we need some 100GBase types for > coming hardware products too. > I think we are not using the 32-bits of "ifm_media" well enough. > Has it been discussed to add more bits to "IFM_NMASK" and have more > ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... > Currently 5 IFM types are defined. If 2 more bits can be added to > IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it > should be possible to define "(32 - 5) * 32 = 864" more ethernet types, > which I think should be enough for now - or we add even one more bit to > IFM_NMASK ? Did you have specific bits in mind? I'm fairly sure they are all assigned to something now. The adjacent bits are used for the subtype/variant and options. Most of the options are used, maybe not all. I haven't checked whether the "instance" field is still used, though. It was for MII PHY numbers, I believe. If we had more bits, it seems better to put them directly into the subtype field rather than the type field. Mike From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 14:23:09 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E644740E for ; Thu, 26 Feb 2015 14:23:09 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1075CDC for ; Thu, 26 Feb 2015 14:23:09 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id DC5DDE722 for ; Thu, 26 Feb 2015 09:23:07 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.856 X-Spam-Level: X-Spam-Status: No, score=-3.856 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.149, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NDv7SsThGuYg for ; Thu, 26 Feb 2015 09:23:07 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 8B1C1E71E for ; Thu, 26 Feb 2015 09:23:07 -0500 (EST) Message-ID: <54EF2C54.7030207@astrodoggroup.com> Date: Thu, 26 Feb 2015 06:23:16 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Minor ULE changes and optimizations Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:23:10 -0000 In addition to the random() replacement patch, I have two other minor ULE patches: https://reviews.freebsd.org/D1970 This fixes the comment in sched_balance_pair to reflect what ULE is actually doing, and moves the load comparison outside of tdq_lock to avoid taking the lock unless there is actually a load imbalance between the two tdqs. https://reviews.freebsd.org/D1969 This allows a non-migratable thread to pin itself to a CPU if it is already running on that CPU. I've been running these patches for the past week or so without issue. Any additional testing or comments would be greatly appreciated. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 14:34:38 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A485D6EB; Thu, 26 Feb 2015 14:34:38 +0000 (UTC) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6275DE0B; Thu, 26 Feb 2015 14:34:38 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 234CD1FE022; Thu, 26 Feb 2015 15:34:35 +0100 (CET) Message-ID: <54EF2F2A.8060304@selasky.org> Date: Thu, 26 Feb 2015 15:35:22 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net Subject: Re: Adding new media types to if_media.h References: <201502261421.t1QELflw056051@mail.karels.net> In-Reply-To: <201502261421.t1QELflw056051@mail.karels.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:34:38 -0000 On 02/26/15 15:21, Mike Karels wrote: >> I'm doing some work for Mellanox and we need some 100GBase types for >> coming hardware products too. > >> I think we are not using the 32-bits of "ifm_media" well enough. > >> Has it been discussed to add more bits to "IFM_NMASK" and have more >> ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, IFM_ETHER_3 .... > >> Currently 5 IFM types are defined. If 2 more bits can be added to >> IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it >> should be possible to define "(32 - 5) * 32 = 864" more ethernet types, >> which I think should be enough for now - or we add even one more bit to >> IFM_NMASK ? > > Did you have specific bits in mind? I'm fairly sure they are all assigned > to something now. The adjacent bits are used for the subtype/variant and > options. Most of the options are used, maybe not all. > > I haven't checked whether the "instance" field is still used, though. It > was for MII PHY numbers, I believe. > > If we had more bits, it seems better to put them directly into the subtype > field rather than the type field. > > Mike > Hi, There are 6 token ring bits, which I presume are available when token ring is not selected. #define IFM_TOK_ETR 0x00000200 /* Early token release */ #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing features */ #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route bcast */ #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic token ring */ Maybe these can be used for other purposes when the type is equal to ethernet? --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 14:50:59 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 90FDECB; Thu, 26 Feb 2015 14:50:59 +0000 (UTC) Received: from mail.turbocat.net (heidi.turbocat.net [88.198.202.214]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D4E7FA4; Thu, 26 Feb 2015 14:50:58 +0000 (UTC) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id 071C51FE022; Thu, 26 Feb 2015 15:50:49 +0100 (CET) Message-ID: <54EF32F9.1020009@selasky.org> Date: Thu, 26 Feb 2015 15:51:37 +0100 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: mike@karels.net Subject: Re: Adding new media types to if_media.h References: <201502261421.t1QELflw056051@mail.karels.net> <54EF2F2A.8060304@selasky.org> In-Reply-To: <54EF2F2A.8060304@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 14:50:59 -0000 On 02/26/15 15:35, Hans Petter Selasky wrote: > On 02/26/15 15:21, Mike Karels wrote: >>> I'm doing some work for Mellanox and we need some 100GBase types for >>> coming hardware products too. >> >>> I think we are not using the 32-bits of "ifm_media" well enough. >> >>> Has it been discussed to add more bits to "IFM_NMASK" and have more >>> ethernet types like IFM_ETHER_0, IFM_ETHER_1, IFM_ETHER_2, >>> IFM_ETHER_3 .... >> >>> Currently 5 IFM types are defined. If 2 more bits can be added to >>> IFM_NMASK we have 5 bits total giving us 2**5 = 32 IFM types. Then it >>> should be possible to define "(32 - 5) * 32 = 864" more ethernet types, >>> which I think should be enough for now - or we add even one more bit to >>> IFM_NMASK ? >> >> Did you have specific bits in mind? I'm fairly sure they are all >> assigned >> to something now. The adjacent bits are used for the subtype/variant and >> options. Most of the options are used, maybe not all. >> >> I haven't checked whether the "instance" field is still used, though. It >> was for MII PHY numbers, I believe. >> >> If we had more bits, it seems better to put them directly into the >> subtype >> field rather than the type field. >> >> Mike >> > > Hi, > > There are 6 token ring bits, which I presume are available when token > ring is not selected. > > #define IFM_TOK_ETR 0x00000200 /* Early token release */ > #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing > features */ > #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route > bcast */ > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > token ring */ > > Maybe these can be used for other purposes when the type is equal to > ethernet? > > --HPS > Hi Mike, My proposal is, convert: > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic Into different network types: #define IFM_TOKEN_NONE #define IFM_TOKEN_DTR #define IFM_TOKEN_CLASSIC #define IFM_TOKEN_AUTO and extend the IFM_NMASK like this: #define IFM_NMASK 0x0000e0e0 /* Network type */ --HPS From owner-freebsd-arch@FreeBSD.ORG Thu Feb 26 23:00:35 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D3FDB9F3; Thu, 26 Feb 2015 23:00:35 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 5941F69D; Thu, 26 Feb 2015 23:00:34 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1QN0WMC031904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 02:00:32 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1QN0Vxt031903; Fri, 27 Feb 2015 02:00:31 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 02:00:31 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150226230031.GN17947@glebius.int.ru> References: <20150225225051.GE17947@glebius.int.ru> <201502260258.t1Q2wKNK054143@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502260258.t1Q2wKNK054143@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Feb 2015 23:00:36 -0000 On Wed, Feb 25, 2015 at 08:58:20PM -0600, Mike Karels wrote: M> > On Wed, Feb 25, 2015 at 02:42:16PM -0800, Eric Joyner wrote: M> > E> Tbh, I respect Gleb's approach, but developing such a thing would take a M> > E> while; the fix Mike proposed would be a fix now. M> > E> M> > E> I mean, I'd like to see a decoupling of media types and speeds from M> > E> "standard" names, and maybe have both an ability to query what modules a M> > E> device supports and what speeds it supports given its current module, M> > E> instead of relying on the clunky media type list now. And then it'd be nice M> > E> to set flow control from ifconfig, too, without having to couple those M> > E> modes to media types. I'm still all for having a new system in the future. M> > E> M> > E> But in changing the KPI so much, it'd be important to consider the "do we M> > E> need an ethtool-equivalent" discussion. M> > E> M> > E> And I think we've lost a bunch of people who were in the original M> > E> discussion from the to:/cc: list. M> M> > Actually the amount of code for my approach is approximately the same M> > as with Mike's. The only thing we must sit down and think without a hurry M> > are the required and spare fields for new if_media. We definitely need M> > input from Adrian on his net80211 requirements, and input from all involved M> > parties. M> M> I'm not sure what would be different about your approach; you mentioned "n" M> versions rather than "x" versions of the ioctls, but I don't know what you M> have in mind for encoding. Any compatible version would be limited to int. The difference is that I suggest to go with a completely new interface. Yep, as you say, if_media is basically wrong. So new ioctl will use new non-wrong structure as argument. And we achieve new feature in 10.2 by merging new ioctl back there, where it will coexist with old unmodified interface. While in head, we no longer need to carry forth the wrong if_media. M> In terms of a "real" fix ("ripping the bandaid off"), I think that M> if_media is basically wrong, and widening it won't fix it. There should M> be a generic structure that reports the media type (e.g. Ethernet), M> perhaps the speed and some generic status ("active"). Then there should M> be media-specific structures that encode the appropriate things including M> attachment type. 802.11 apparently already has an extension, and Ethernet M> should have a similar extension. The KPI should be media-type-specific. M> I don't see something like this being designed soon, and certainly wouldn't M> be able to be MFC'd. Meanwhile, many of us need to support 40 Gb/s Ethernet M> on non-current (or non-future) systems. M> M> > I'm willing to code this if we all agree on the topic, so that you will M> > code all done and commited before 10.2-RELEASE. M> M> I'd be interested in a sketch or more extended description sometime before M> 10.2. I will try to show smth soon. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 04:11:33 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54977946; Fri, 27 Feb 2015 04:11:33 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id E0E49BC5; Fri, 27 Feb 2015 04:11:32 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1R4BTee058023; Thu, 26 Feb 2015 22:11:30 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502270411.t1R4BTee058023@mail.karels.net> To: Hans Petter Selasky From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Thu, 26 Feb 2015 15:51:37 +0100. <54EF32F9.1020009@selasky.org> Date: Thu, 26 Feb 2015 22:11:29 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:11:33 -0000 > > Hi, > > > > There are 6 token ring bits, which I presume are available when token > > ring is not selected. > > > > #define IFM_TOK_ETR 0x00000200 /* Early token release */ > > #define IFM_TOK_SRCRT 0x00000400 /* Enable source routing > > features */ > > #define IFM_TOK_ALLR 0x00000800 /* All routes / Single route > > bcast */ > > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > > token ring */ > > > > Maybe these can be used for other purposes when the type is equal to > > ethernet? These are the "type-specific options". Ethernet uses three of these (see IFM_ETH_*). I hadn't thought about it, but some of the remaining five bits could be used for extended Ethernet subtypes. > Hi Mike, > My proposal is, convert: > > #define IFM_TOK_DTR 0x00002000 /* Dedicated token ring */ > > #define IFM_TOK_CLASSIC 0x00004000 /* Classic token ring */ > > #define IFM_TOK_AUTO 0x00008000 /* Automatic Dedicate/Classic > Into different network types: > #define IFM_TOKEN_NONE > #define IFM_TOKEN_DTR > #define IFM_TOKEN_CLASSIC > #define IFM_TOKEN_AUTO > and extend the IFM_NMASK like this: > #define IFM_NMASK 0x0000e0e0 /* Network type */ I don't think any change is required for token ring. Finding hardware to test would be a challenge in any case. Simply using some of the options bits for subtype would be simpler. I could outline this if anyone wants. Mike From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 04:17:04 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D6FAA9D; Fri, 27 Feb 2015 04:17:04 +0000 (UTC) Received: from mail.karels.net (mail.karels.net [63.231.190.5]) by mx1.freebsd.org (Postfix) with ESMTP id BBCB6BE7; Fri, 27 Feb 2015 04:17:03 +0000 (UTC) Received: from mail.karels.net (localhost [127.0.0.1]) by mail.karels.net (8.14.7/8.14.7) with ESMTP id t1R4H37Y058057; Thu, 26 Feb 2015 22:17:03 -0600 (CST) (envelope-from mike@karels.net) Message-Id: <201502270417.t1R4H37Y058057@mail.karels.net> To: Gleb Smirnoff From: Mike Karels Reply-to: mike@karels.net Subject: Re: Adding new media types to if_media.h In-reply-to: Your message of Fri, 27 Feb 2015 02:00:31 +0300. <20150226230031.GN17947@glebius.int.ru> Date: Thu, 26 Feb 2015 22:17:03 -0600 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:17:04 -0000 > M> I'm not sure what would be different about your approach; you mentioned "n" > M> versions rather than "x" versions of the ioctls, but I don't know what you > M> have in mind for encoding. Any compatible version would be limited to int. > The difference is that I suggest to go with a completely new interface. Yep, > as you say, if_media is basically wrong. So new ioctl will use new non-wrong > structure as argument. I think that part of the wrongness of if_media is to try to create a one-size-fits-all-network-types interface. If the replacement is a single ioctl, I don't think it's enough of an improvement to break the driver KPI. > And we achieve new feature in 10.2 by merging new ioctl back there, where > it will coexist with old unmodified interface. While in head, we no longer > need to carry forth the wrong if_media. I would think that this would be a huge problem for driver modules. And I think the old if_media will need to be supported for the user-level API for some time, unless someone is going to fix a lot of ports. Also, many of us are backporting much before 10.2. Mike From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 04:26:00 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C5291C3F; Fri, 27 Feb 2015 04:26:00 +0000 (UTC) Received: from mail-ie0-x236.google.com (mail-ie0-x236.google.com [IPv6:2607:f8b0:4001:c03::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89C97CB9; Fri, 27 Feb 2015 04:26:00 +0000 (UTC) Received: by iecrp18 with SMTP id rp18so25578697iec.9; Thu, 26 Feb 2015 20:26:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=eQ7qCGqmKvS8aCXViddXsmkEP9pmB7eBbMcA2OU7M8I=; b=zGqIq1aGvxP1meZUYAwiGyNk1SXvJJJMpXnzeqzYQklNhQi/rQFtutECbvs1Twq1S9 GUH8mz05MTFBsuvrdfk6eomJYbsE2J6Epo4dMxGj92uGMUteL16MEb3HK51A4R+UER/0 tpXgFJy+ZPU+vSuG46+ER+x7D8YpUkfZQNDYWoqCaYRUZImc4XpSdfnWGDJ2AYcoiEuq NcUb6IQoVJ/O8rTA6jU3niYWllc4OgbawMXsekzWsQK7k4wllM5IzwP+Ms+XOrutJEPK S6LZCHpXmnw92X51kPVMEyb7pmZLODSf65thiqefkrrRx+E1OZPnXRWg/s6pXiic34ie GGGg== MIME-Version: 1.0 X-Received: by 10.50.66.170 with SMTP id g10mr484956igt.49.1425011160083; Thu, 26 Feb 2015 20:26:00 -0800 (PST) Sender: adrian.chadd@gmail.com Received: by 10.36.17.66 with HTTP; Thu, 26 Feb 2015 20:25:59 -0800 (PST) In-Reply-To: <201502270417.t1R4H37Y058057@mail.karels.net> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> Date: Thu, 26 Feb 2015 20:25:59 -0800 X-Google-Sender-Auth: niK7omPtjjM7VOy2dINawBj6xxw Message-ID: Subject: Re: Adding new media types to if_media.h From: Adrian Chadd To: mike@karels.net Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 04:26:00 -0000 [snip] I think Mike's approach is good - it makes it easy to MFC to 10.2 since there's extended lifecycle stuff to do there - and then we can plan out how do the "betterer" fix after it's landed and churned things. -a From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 05:56:36 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2620D861; Fri, 27 Feb 2015 05:56:36 +0000 (UTC) Received: from mail-we0-x22e.google.com (mail-we0-x22e.google.com [IPv6:2a00:1450:400c:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8B6E6E0; Fri, 27 Feb 2015 05:56:35 +0000 (UTC) Received: by wesp10 with SMTP id p10so15237352wes.12; Thu, 26 Feb 2015 21:56:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=ZW4Szw2euCIuED6BM8osB3JmUUvhYMIBLB7qFEx2GfY=; b=c4NPLOEJqabcrF9Keuqllg6ZXW1xWxG6jEwVWjxvKUOZPnfmz0Shzoz6k+/PnFs1W5 nrvWzHb1N5dw0B8KxrDPZhGeQg6CNByPWCx7OV9qwrhf3EW4lDcdjvuaSP+JKeCTr+C4 7VahJseHjqqrI3htbxxgX/oH3hlsLFAhf5TVCTloxKZ5MJzsikkl72ztwCWnVk4GqUbW m7dTPw9RVsrRBwtFs75bNsfj/SR4u6HxrAF1RxHnahpb8P5iQY+DQiZ5M9R9b1mXjynm 7wuQA7/avrdUf+7PCH5YAgBCPltry0By3Mhvr3hFoAahk8kxrQKrZllQcpKx8iYgyE77 j63w== MIME-Version: 1.0 X-Received: by 10.180.38.76 with SMTP id e12mr2989387wik.76.1425016594048; Thu, 26 Feb 2015 21:56:34 -0800 (PST) Received: by 10.194.101.106 with HTTP; Thu, 26 Feb 2015 21:56:33 -0800 (PST) In-Reply-To: References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> Date: Thu, 26 Feb 2015 21:56:33 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Adrian Chadd Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Eric Joyner , Gleb Smirnoff , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 05:56:36 -0000 I agree, like I said, we have hardware coming this year that will need support, and as we just saw 11 isn't until 2016. Eric tested Mike's code with our needed types and it worked. If something truly revolutionary wants to be done for 11 it can be done after this and that not backed into the 10.X stream. Jack On Thu, Feb 26, 2015 at 8:25 PM, Adrian Chadd wrote: > [snip] > > I think Mike's approach is good - it makes it easy to MFC to 10.2 > since there's extended lifecycle stuff to do there - and then we can > plan out how do the "betterer" fix after it's landed and churned > things. > > > > -a > From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 15:21:22 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2D658402 for ; Fri, 27 Feb 2015 15:21:22 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 07033A5B for ; Fri, 27 Feb 2015 15:21:22 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 1E4B9B960; Fri, 27 Feb 2015 10:21:21 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Minor ULE changes and optimizations Date: Fri, 27 Feb 2015 09:14:10 -0500 Message-ID: <2311645.BNIPBaFv2E@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <54EF2C54.7030207@astrodoggroup.com> References: <54EF2C54.7030207@astrodoggroup.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 27 Feb 2015 10:21:21 -0500 (EST) Cc: Harrison Grundy X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 15:21:22 -0000 On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy wrote: > https://reviews.freebsd.org/D1969 > This allows a non-migratable thread to pin itself to a CPU if it is > already running on that CPU. > > I've been running these patches for the past week or so without issue. > Any additional testing or comments would be greatly appreciated. Can you explain the reason / use case for this? This seems to be allowing an API violation. sched_pin() was designed to be a lower-level API than sched_bind(), so you wouldn't call sched_bind() if you were already pinned. In addition, sched_pin() is sometimes used by code that assumes it won't migrate until sched_unpin() (e.g. temporary mappings inside an sfbuf). If you allow sched_bind() to move a thread that is pinned you will allow someone to unintentionally break those sort of things instead of getting an assertion failure panic. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 15:51:50 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58DDEF55 for ; Fri, 27 Feb 2015 15:51:50 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 113D4D10 for ; Fri, 27 Feb 2015 15:51:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 3A920E3E2 for ; Fri, 27 Feb 2015 10:51:42 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.855 X-Spam-Level: X-Spam-Status: No, score=-3.855 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.148, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id GIbWSYbD-8TH for ; Fri, 27 Feb 2015 10:51:42 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id E02AEE3D9 for ; Fri, 27 Feb 2015 10:51:41 -0500 (EST) Message-ID: <54F0925F.30002@astrodoggroup.com> Date: Fri, 27 Feb 2015 07:50:55 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: Minor ULE changes and optimizations References: <54EF2C54.7030207@astrodoggroup.com> <2311645.BNIPBaFv2E@ralph.baldwin.cx> In-Reply-To: <2311645.BNIPBaFv2E@ralph.baldwin.cx> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 15:51:50 -0000 On 02/27/15 06:14, John Baldwin wrote: > On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy wrote: >> https://reviews.freebsd.org/D1969 This allows a non-migratable >> thread to pin itself to a CPU if it is already running on that >> CPU. >> >> I've been running these patches for the past week or so without >> issue. Any additional testing or comments would be greatly >> appreciated. > > Can you explain the reason / use case for this? This seems to be > allowing an API violation. sched_pin() was designed to be a > lower-level API than sched_bind(), so you wouldn't call > sched_bind() if you were already pinned. In addition, sched_pin() > is sometimes used by code that assumes it won't migrate until > sched_unpin() (e.g. temporary mappings inside an sfbuf). If you > allow sched_bind() to move a thread that is pinned you will allow > someone to unintentionally break those sort of things instead of > getting an assertion failure panic. > For a pinned thread, the underlying idea is that if you're already on the CPU you pinned to, calling sched_bind with that CPU specified allows you to set TSF_BOUND without calling sched_unpin first. If a pinned thread were to call sched_bind for a CPU it isn't pinned to, it would still hit the assert and fail. For any unpinned thread, if you're already running on the correct CPU, you can skip the THREAD_CAN_MIGRATE check and the call to mi_switch. --- Harrison From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 19:19:20 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EBD2F94B; Fri, 27 Feb 2015 19:19:20 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 72FECBD1; Fri, 27 Feb 2015 19:19:19 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1RJJHQe036826 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 22:19:17 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1RJJGWq036825; Fri, 27 Feb 2015 22:19:16 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 22:19:16 +0300 From: Gleb Smirnoff To: Mike Karels Subject: Re: Adding new media types to if_media.h Message-ID: <20150227191916.GQ17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201502270417.t1R4H37Y058057@mail.karels.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:19:21 -0000 On Thu, Feb 26, 2015 at 10:17:03PM -0600, Mike Karels wrote: M> > M> I'm not sure what would be different about your approach; you mentioned "n" M> > M> versions rather than "x" versions of the ioctls, but I don't know what you M> > M> have in mind for encoding. Any compatible version would be limited to int. M> M> > The difference is that I suggest to go with a completely new interface. Yep, M> > as you say, if_media is basically wrong. So new ioctl will use new non-wrong M> > structure as argument. M> M> I think that part of the wrongness of if_media is to try to create a M> one-size-fits-all-network-types interface. If the replacement is a M> single ioctl, I don't think it's enough of an improvement to break M> the driver KPI. The KPI for drivers in 11 will be changed very significantly anyway[1], so right now we got a chance to make everything properly. Why not to utilize this chance for ifmedia? Imagine you do this from scratch, and implement it the right way. And this way it will go into 11. Then shims for 10-merge can be done. And it would be okay if shims look ugly, since we won't take them with us into the future. M> > And we achieve new feature in 10.2 by merging new ioctl back there, where M> > it will coexist with old unmodified interface. While in head, we no longer M> > need to carry forth the wrong if_media. M> M> I would think that this would be a huge problem for driver modules. Why huge? The new functionality is required for a couple of drivers only. And module KPI is guaranteed to be forward compatible only: one can load older module on newer kernel, but not vice versa. So, if newer module requires some functionality from the kernel, its entirely okay. M> And I think the old if_media will need to be supported for the user-level M> API for some time, unless someone is going to fix a lot of ports. M> M> Also, many of us are backporting much before 10.2. Of course old if_media API should remain for ports. [1] See projects/ifnet subversion branch. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 19:23:12 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E451CC50; Fri, 27 Feb 2015 19:23:12 +0000 (UTC) Received: from cell.glebius.int.ru (glebius.int.ru [81.19.69.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cell.glebius.int.ru", Issuer "cell.glebius.int.ru" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6B878CA5; Fri, 27 Feb 2015 19:23:11 +0000 (UTC) Received: from cell.glebius.int.ru (localhost [127.0.0.1]) by cell.glebius.int.ru (8.14.9/8.14.9) with ESMTP id t1RJNAwQ036864 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 27 Feb 2015 22:23:10 +0300 (MSK) (envelope-from glebius@FreeBSD.org) Received: (from glebius@localhost) by cell.glebius.int.ru (8.14.9/8.14.9/Submit) id t1RJNAVU036863; Fri, 27 Feb 2015 22:23:10 +0300 (MSK) (envelope-from glebius@FreeBSD.org) X-Authentication-Warning: cell.glebius.int.ru: glebius set sender to glebius@FreeBSD.org using -f Date: Fri, 27 Feb 2015 22:23:10 +0300 From: Gleb Smirnoff To: Adrian Chadd Subject: Re: Adding new media types to if_media.h Message-ID: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) Cc: "freebsd-net@freebsd.org" , Eric Joyner , Jack Vogel , mike@karels.net, "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 19:23:13 -0000 On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: A> [snip] A> A> I think Mike's approach is good - it makes it easy to MFC to 10.2 A> since there's extended lifecycle stuff to do there - and then we can A> plan out how do the "betterer" fix after it's landed and churned A> things. ... and we will be ought to support the "betterer" fix along with the "not so betterer" for a very long time. The rock on which we split in this argument is that some developers write their code for stable/x and then forward-port it to head, focused on quality of result for stable/x; while other developers do the opposite: write code to head, then consider or not consider merging it stable/x. -- Totus tuus, Glebius. From owner-freebsd-arch@FreeBSD.ORG Fri Feb 27 21:58:28 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9CAAFFFC; Fri, 27 Feb 2015 21:58:28 +0000 (UTC) Received: from mail-wg0-x232.google.com (mail-wg0-x232.google.com [IPv6:2a00:1450:400c:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B0D3F6B; Fri, 27 Feb 2015 21:58:28 +0000 (UTC) Received: by wgha1 with SMTP id a1so22938255wgh.12; Fri, 27 Feb 2015 13:58:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=bZpkrHrM/VKQFp5R5pZ95k9XQsUoWNg60KMvdAkvR4I=; b=jqe2Af445fTtxB9pSgVCgheqJy2S2UkifLHG8ugqLaX4qqYWbyo9TcgFuz0LFSqNS7 LCpymDDj/YVtI64AFX3dll2N6qQn7cuOinUbAMvoD6EVMTHP1v23rKfaxYujmZy4WCkH Fmeh1s5CYCMEc6a//TpVujFjXDLq390fxbexG4GK21B4YzId4wK2YXleNEH5LF0+RKTA TFdB7Us71nWOgiA5j2dq+7yzRu7kXbqTqxbNvUePSlrVIbEI0n/eEWxDzZwwx03r6j1S ZAutwcC5pYtEbm8klwuVPrlcOhPESGByajOWEdLRj4DXmPyOyuMN/mlsdRmwZGa4EHwV umOg== MIME-Version: 1.0 X-Received: by 10.180.38.76 with SMTP id e12mr10565678wik.76.1425074306512; Fri, 27 Feb 2015 13:58:26 -0800 (PST) Received: by 10.194.101.106 with HTTP; Fri, 27 Feb 2015 13:58:26 -0800 (PST) In-Reply-To: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <201502270417.t1R4H37Y058057@mail.karels.net> <20150227192310.GR17947@glebius.int.ru> Date: Fri, 27 Feb 2015 13:58:26 -0800 Message-ID: Subject: Re: Adding new media types to if_media.h From: Jack Vogel To: Gleb Smirnoff Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-net@freebsd.org" , Adrian Chadd , Eric Joyner , Mike Karels , "freebsd-arch@freebsd.org" X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Feb 2015 21:58:28 -0000 On Fri, Feb 27, 2015 at 11:23 AM, Gleb Smirnoff wrote: > On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: > A> [snip] > A> > A> I think Mike's approach is good - it makes it easy to MFC to 10.2 > A> since there's extended lifecycle stuff to do there - and then we can > A> plan out how do the "betterer" fix after it's landed and churned > A> things. > > ... and we will be ought to support the "betterer" fix along with > the "not so betterer" for a very long time. > > The rock on which we split in this argument is that some developers > write their code for stable/x and then forward-port it to head, > focused on quality of result for stable/x; while other developers > do the opposite: write code to head, then consider or not consider > merging it stable/x. > > I think this is oversimplified. In my 10 years in this role at Intel I've had cases when, in response to a customer issue, I had to work from an existing code base to solve a problem, or add support for a new feature. But then there are other times when I've been working on a new driver, and its been totally developed from HEAD. It depends on what's right for the circumstance, but as I said, on this issue we have real product/customer needs that are short term, and the competition (Linux and Windows) is prepared to handle the new media today, I think its in FreeBSD's interest to address this ASAP. Jack From owner-freebsd-arch@FreeBSD.ORG Sat Feb 28 12:41:30 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8053BD5E; Sat, 28 Feb 2015 12:41:30 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5699BB4; Sat, 28 Feb 2015 12:41:30 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 5401FB9A3; Sat, 28 Feb 2015 07:41:29 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Adding new media types to if_media.h Date: Sat, 28 Feb 2015 07:28:14 -0500 Message-ID: <1919032.aFEK3un8ig@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <20150227192310.GR17947@glebius.int.ru> References: <20150226230031.GN17947@glebius.int.ru> <20150227192310.GR17947@glebius.int.ru> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 28 Feb 2015 07:41:29 -0500 (EST) Cc: Adrian Chadd , mike@karels.net, "freebsd-net@freebsd.org" , Gleb Smirnoff , Jack Vogel , Eric Joyner X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 12:41:30 -0000 On Friday, February 27, 2015 10:23:10 PM Gleb Smirnoff wrote: > On Thu, Feb 26, 2015 at 08:25:59PM -0800, Adrian Chadd wrote: > A> [snip] > A> > A> I think Mike's approach is good - it makes it easy to MFC to 10.2 > A> since there's extended lifecycle stuff to do there - and then we can > A> plan out how do the "betterer" fix after it's landed and churned > A> things. > > ... and we will be ought to support the "betterer" fix along with > the "not so betterer" for a very long time. > > The rock on which we split in this argument is that some developers > write their code for stable/x and then forward-port it to head, > focused on quality of result for stable/x; while other developers > do the opposite: write code to head, then consider or not consider > merging it stable/x. No, this is not quite true. Some folks have to write drivers on HEAD but also support running those drivers on older branches. The MFC's get harder when you have very different APIs on the different branches. It's already harder to test stat changes now since it requires completely different patches for <= 10 (the only thing people are supposed to use in production) vs head due to if_getcounter() and friends. Also, since 11 won't be out until 2016, that is far, far too long to wait for more media types. The stuff we need to support is already shipping in products today. We can't not support these in 10 (and possibly 9). -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Feb 28 12:41:31 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 171A2D65 for ; Sat, 28 Feb 2015 12:41:31 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E3E1BB6 for ; Sat, 28 Feb 2015 12:41:30 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-54-116-245.nwrknj.fios.verizon.net [173.54.116.245]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 077E5B9B1; Sat, 28 Feb 2015 07:41:30 -0500 (EST) From: John Baldwin To: freebsd-arch@freebsd.org Subject: Re: Minor ULE changes and optimizations Date: Sat, 28 Feb 2015 07:24:30 -0500 Message-ID: <1547642.s3cC06khRt@ralph.baldwin.cx> User-Agent: KMail/4.14.2 (FreeBSD/10.1-STABLE; KDE/4.14.2; amd64; ; ) In-Reply-To: <54F0925F.30002@astrodoggroup.com> References: <54EF2C54.7030207@astrodoggroup.com> <2311645.BNIPBaFv2E@ralph.baldwin.cx> <54F0925F.30002@astrodoggroup.com> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Sat, 28 Feb 2015 07:41:30 -0500 (EST) Cc: Harrison Grundy X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 12:41:31 -0000 On Friday, February 27, 2015 07:50:55 AM Harrison Grundy wrote: > On 02/27/15 06:14, John Baldwin wrote: > > On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy wrote: > >> https://reviews.freebsd.org/D1969 This allows a non-migratable > >> thread to pin itself to a CPU if it is already running on that > >> CPU. > >> > >> I've been running these patches for the past week or so without > >> issue. Any additional testing or comments would be greatly > >> appreciated. > > > > Can you explain the reason / use case for this? This seems to be > > allowing an API violation. sched_pin() was designed to be a > > lower-level API than sched_bind(), so you wouldn't call > > sched_bind() if you were already pinned. In addition, sched_pin() > > is sometimes used by code that assumes it won't migrate until > > sched_unpin() (e.g. temporary mappings inside an sfbuf). If you > > allow sched_bind() to move a thread that is pinned you will allow > > someone to unintentionally break those sort of things instead of > > getting an assertion failure panic. > > For a pinned thread, the underlying idea is that if you're already on > the CPU you pinned to, calling sched_bind with that CPU specified > allows you to set TSF_BOUND without calling sched_unpin first. > > If a pinned thread were to call sched_bind for a CPU it isn't pinned > to, it would still hit the assert and fail. > > For any unpinned thread, if you're already running on the correct CPU, > you can skip the THREAD_CAN_MIGRATE check and the call to mi_switch. Ah, ok, so you aren't allowing migration in theory. However, I'm still curious as to why you want/need this. This makes the API usage a bit more complex to reason about (sched_bind() can sometimes be called while pinned but not always after this change), so I think that extra complexity needs a reason to exist. -- John Baldwin From owner-freebsd-arch@FreeBSD.ORG Sat Feb 28 15:46:12 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 993AE220 for ; Sat, 28 Feb 2015 15:46:12 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4F29032A for ; Sat, 28 Feb 2015 15:46:11 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id 3C6DEE4E4 for ; Sat, 28 Feb 2015 10:46:10 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.854 X-Spam-Level: X-Spam-Status: No, score=-3.854 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.147, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NyNyrMowlR-9 for ; Sat, 28 Feb 2015 10:46:09 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 9BACFE4E2 for ; Sat, 28 Feb 2015 10:46:09 -0500 (EST) Message-ID: <54F1E25F.5040905@astrodoggroup.com> Date: Sat, 28 Feb 2015 07:44:31 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: Minor ULE changes and optimizations References: <54EF2C54.7030207@astrodoggroup.com> <2311645.BNIPBaFv2E@ralph.baldwin.cx> <54F0925F.30002@astrodoggroup.com> <1547642.s3cC06khRt@ralph.baldwin.cx> In-Reply-To: <1547642.s3cC06khRt@ralph.baldwin.cx> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 28 Feb 2015 15:46:12 -0000 On 02/28/15 04:24, John Baldwin wrote: > On Friday, February 27, 2015 07:50:55 AM Harrison Grundy wrote: >> On 02/27/15 06:14, John Baldwin wrote: >>> On Thursday, February 26, 2015 06:23:16 AM Harrison Grundy >>> wrote: >>>> https://reviews.freebsd.org/D1969 This allows a >>>> non-migratable thread to pin itself to a CPU if it is already >>>> running on that CPU. >>>> >>>> I've been running these patches for the past week or so >>>> without issue. Any additional testing or comments would be >>>> greatly appreciated. >>> >>> Can you explain the reason / use case for this? This seems to >>> be allowing an API violation. sched_pin() was designed to be >>> a lower-level API than sched_bind(), so you wouldn't call >>> sched_bind() if you were already pinned. In addition, >>> sched_pin() is sometimes used by code that assumes it won't >>> migrate until sched_unpin() (e.g. temporary mappings inside an >>> sfbuf). If you allow sched_bind() to move a thread that is >>> pinned you will allow someone to unintentionally break those >>> sort of things instead of getting an assertion failure panic. >> >> For a pinned thread, the underlying idea is that if you're >> already on the CPU you pinned to, calling sched_bind with that >> CPU specified allows you to set TSF_BOUND without calling >> sched_unpin first. >> >> If a pinned thread were to call sched_bind for a CPU it isn't >> pinned to, it would still hit the assert and fail. >> >> For any unpinned thread, if you're already running on the correct >> CPU, you can skip the THREAD_CAN_MIGRATE check and the call to >> mi_switch. > > Ah, ok, so you aren't allowing migration in theory. However, I'm > still curious as to why you want/need this. This makes the API > usage a bit more complex to reason about (sched_bind() can > sometimes be called while pinned but not always after this change), > so I think that extra complexity needs a reason to exist. Primarily, it allows those threads already on a CPU to skip the call to mi_switch and get out of sched_bind a bit faster. Additionally, it allows a driver to call sched_pin, then bind to that same cpu later without having to write something like "critical_enter(); sched_unpin(); sched_bind(foo, bar); critical_exit();", since otherwise it could be migrated/preempted between unpin and bind.