Date: Tue, 08 Nov 2011 22:47:15 +0200 From: Andriy Gapon <avg@FreeBSD.org> To: Jeff Roberson <jroberson@jroberson.net> Cc: freebsd-current@FreeBSD.org Subject: Re: couple of sched_ule issues Message-ID: <4EB99553.5010005@FreeBSD.org> In-Reply-To: <alpine.BSF.2.00.1111031015180.1995@desktop> References: <4E720CB6.3070103@FreeBSD.org> <alpine.BSF.2.00.1111031015180.1995@desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
on 03/11/2011 22:17 Jeff Roberson said the following:
> On Thu, 15 Sep 2011, Andriy Gapon wrote:
>
>>
>> This is more of a "just for the record" email.
>> I think I've already stated the following observations, but I suspect that they
>> drowned in the noise of a thread in which I mentioned them.
>>
>> 1. Incorrect topology is built for single-package SMP systems.
>> That topology has two levels ("shared nothing" and "shared package") with exactly
>> the same CPU sets. That doesn't work well with the rebalancing algorithm which
>> assumes that each level is a proper/strict subset of its parent.
>>
>> 2. CPU load comparison algorithms are biased towards lower logical CPU IDs.
>> With all other things being equal the algorithms will always pick a CPU with a
>> lower ID. This creates certain load asymmetry and predictable patterns in load
>> distribution.
>
> If all other things truly are equal why does selecting a lower cpu number matter?
Honestly, I am not sure.
This bias could definitely produce an "inequality" from the point of view of how
many threads each CPU gets. But I can not say if there could be any
"inequality" in how much CPU time each thread gets. But I don't rule out that
that might be possible...
Please also see this post of mine:
http://thread.gmane.org/gmane.os.freebsd.current/133763/focus=134192
>> Another observation.
>> It seems that ULE makes a decision about thread-to-CPU affinity at the time
>> when a
>> thread gets switched out. This looks logical from the implementation point of
>> view. But it doesn't seem logical from a general point of view - when the thread
>> will be becoming running again its affinity profile may become completely
>> different. I think that it would depend on how much a thread actually spends not
>> running.
>
> The decision is made at sched_add() time. sched_pickcpu() does the work and
> selects the run-queue we will be added to. We consider the CPU that the thread
> was last running on but the decision is made at the time that a run queue must
> be selected.
Ah, yes. But, OTOH, it doesn't look like sched_add would be called for a
CPU-bound thread (no voluntary sleeps) ?
--
Andriy Gapon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4EB99553.5010005>