From owner-freebsd-arch@FreeBSD.ORG Wed Feb 25 09:11:00 2015 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5388E999 for ; Wed, 25 Feb 2015 09:11:00 +0000 (UTC) Received: from mailout.easymail.ca (mailout.easymail.ca [64.68.201.169]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 06E0B62B for ; Wed, 25 Feb 2015 09:10:59 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mailout.easymail.ca (Postfix) with ESMTP id C0CF6E476 for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) X-Virus-Scanned: Debian amavisd-new at mailout.easymail.ca X-Spam-Flag: NO X-Spam-Score: -3.865 X-Spam-Level: X-Spam-Status: No, score=-3.865 required=5 tests=[ALL_TRUSTED=-1.8, AWL=-0.158, BAYES_00=-2.599, DNS_FROM_AHBL_RHSBL=0.692] Received: from mailout.easymail.ca ([127.0.0.1]) by localhost (easymail-mailout.easydns.vpn [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id vkmaY1yetDdh for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) Received: from bsddt1241.lv01.astrodoggroup.com (unknown [40.141.24.126]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mailout.easymail.ca (Postfix) with ESMTPSA id 29156E472 for ; Wed, 25 Feb 2015 04:10:58 -0500 (EST) Message-ID: <54ED91A5.8080106@astrodoggroup.com> Date: Wed, 25 Feb 2015 01:11:01 -0800 From: Harrison Grundy User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.3.0 MIME-Version: 1.0 To: freebsd-arch@freebsd.org Subject: Re: locks and kernel randomness... References: <20150224174053.GG46794@funkthat.com> <54ECBD4B.6000007@freebsd.org> <20150224182507.GI46794@funkthat.com> <54ECEA43.2080008@freebsd.org> <20150224231921.GQ46794@funkthat.com> <20150225002301.GS46794@funkthat.com> <54ED80BD.1080603@freebsd.org> <54ED87E9.8030706@astrodoggroup.com> <20150225085659.GA74514@kib.kiev.ua> In-Reply-To: <20150225085659.GA74514@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Feb 2015 09:11:00 -0000 On 02/25/15 00:56, Konstantin Belousov wrote: > On Wed, Feb 25, 2015 at 12:29:29AM -0800, Harrison Grundy wrote: >> >> >> On 02/24/15 23:58, Alfred Perlstein wrote: >>> >>> On 2/24/15 7:23 PM, John-Mark Gurney wrote: >>>> K. Macy wrote this message on Tue, Feb 24, 2015 at 15:33 >>>> -0800: >>>>>> If someone does find a performance issue w/ my patch, I >>>>>> WILL work with them on a solution, but I will not work w/ >>>>>> people who make unfounded claims about the impact of this >>>>>> work... >>>>>> >>>>> ... The concerns may be exaggerated, but they >>>>> aren't unfounded. Not quite the same thing, but no one >>>>> wants to spend the >>>> Till someone shows me code in the kernel tree where this is >>>> even close to a performance problem, it is unfounded... I've >>>> asked, and no one has >>>> >>>>> cycles doing a SHA256 because it's "The Right Thing"(tm) >>>>> when their use case only requires a fletcher2. >>>> Depends upon what you're doing.. I haven't proposed changing >>>> ZFS's default to sha256, so stop w/ the false >>>> equivalences... >>>> >>>>> If it doesn't already exist, it might also be worth looking >>>>> in to a more scalable CSPRNG implementation not requiring >>>>> locking in the common case. For example, each core is >>>>> seeded separately periodically so that has a private pool >>>>> that is protected by a critical section. The private pool >>>>> would be regularly refreshed by cpu-local callout. Thus, a >>>>> lock would only be acquired if the local entropy were >>>>> depleted. >>>> I'm not discussing this until you read and reply to my >>>> original email, since it's clear that my original email's >>>> contents has been ignored in this thread... >>>> >>> What is final proposal? More spinlocks? That is not a good >>> idea. >>> >>> Doing a single buildworld is not enough. Ask netflix or >>> someone with a real load of 1000s of threads/processing to do >>> testing for you if you truly want to touch scheduler. >> >> sched_ule runs this code once every .5 to 1.5 seconds, depending >> on the value of random, so using a CSPRNG there wouldn't actually >> be noticeable. (We're talking about a few thousand cycles, when >> the existing implementation has to make a remote memory >> read/write numpackages-1/numpackages percent of the time, which >> costs tens of thousands of cycles. Switching to a per-CPU CSPRNG >> is actually faster in those cases.) > The cost of the proposed patch, of course, is not the several > thousands of instructions in the rebalance. The problem with it is > the introduction of the new spinlock, which will be used in many > places after the introduction. The cost of the new and often used > spinlock is the increase of both interrupt latency and interrupt > handler jitter and cpu switch jitter. > > So neither buildworld timing, nor network throughput are adequate > to estimate the change. It is system unresponsivness and loss of > the realtime behaviour up to some degree. > > I thought that it was obvious, at least after spinlocks were > mentioned, but apparently it is not, since proposals to measure the > patch effect by benchmarking buildworld or passing the traffic are > made. > I was referring to the sched_ule case only. That analysis will certainly need to be done for each use of random(), though. >> >> That being said, I believe the plan is to remove random() from >> sched_ule entirely. It doesn't need it to perform the balancing, >> and we can just use the LCG from cpu_search, if get_cyclecount >> isn't viable. > > I agree with this, but please see my other response.