From owner-freebsd-stable@FreeBSD.ORG Tue Dec 13 23:39:14 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09B051065670; Tue, 13 Dec 2011 23:39:14 +0000 (UTC) (envelope-from fidaj@ukr.net) Received: from fsm2.ukr.net (fsm2.ukr.net [195.214.192.121]) by mx1.freebsd.org (Postfix) with ESMTP id 8A7C18FC08; Tue, 13 Dec 2011 23:39:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=fsm; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References:In-Reply-To:Message-ID:Subject:Cc:To:From:Date; bh=5F70nZqRcYgm7bijrHgHyKB7UoBju6OC5/bO8dW/88o=; b=UCCsjgOajR4VEgZZUmFT8w1hirtKquOO5Imc+Y+MxOaCwFvWJHpSRzawb4RG8UtOwO39O8ln9yMHmfUQFZfk4sHuyDdTgiD+e5YJGQ9o6LPDgvP/+XA1/4abE7Fxon5FSIdselgvFv3ZKh+9DCi7ybFm7aRNXgx+H0jDZkIHJv0=; Received: from [178.137.138.140] (helo=nonamehost.) by fsm2.ukr.net with esmtpsa ID 1RabwD-0003M3-1I ; Wed, 14 Dec 2011 01:39:09 +0200 Date: Wed, 14 Dec 2011 01:39:06 +0200 From: Ivan Klymenko To: Jilles Tjoelker Message-ID: <20111214013906.068f69df@nonamehost.> In-Reply-To: <20111213230441.GB42285@stack.nl> References: <4EE1EAFE.3070408@m5p.com> <4EE22421.9060707@gmail.com> <4EE6060D.5060201@mail.zedat.fu-berlin.de> <4EE69C5A.3090005@FreeBSD.org> <20111213104048.40f3e3de@nonamehost.> <20111213230441.GB42285@stack.nl> X-Mailer: Claws Mail 3.7.10 (GTK+ 2.24.6; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "O. Hartmann" , Doug Barton , freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, Current FreeBSD Subject: Re: SCHED_ULE should not be the default X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 23:39:14 -0000 =D0=92 Wed, 14 Dec 2011 00:04:42 +0100 Jilles Tjoelker =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > On Tue, Dec 13, 2011 at 10:40:48AM +0200, Ivan Klymenko wrote: > > If the algorithm ULE does not contain problems - it means the > > problem has Core2Duo, or in a piece of code that uses the ULE > > scheduler. I already wrote in a mailing list that specifically in > > my case (Core2Duo) partially helps the following patch: > > --- sched_ule.c.orig 2011-11-24 18:11:48.000000000 +0200 > > +++ sched_ule.c 2011-12-10 22:47:08.000000000 +0200 > > @@ -794,7 +794,8 @@ > > * 1.5 * balance_interval. > > */ > > balance_ticks =3D max(balance_interval / 2, 1); > > - balance_ticks +=3D random() % balance_interval; > > +// balance_ticks +=3D random() % balance_interval; > > + balance_ticks +=3D ((int)random()) % balance_interval; > > if (smp_started =3D=3D 0 || rebalance =3D=3D 0) > > return; > > tdq =3D TDQ_SELF(); >=20 > This avoids a 64-bit division on 64-bit platforms but seems to have no > effect otherwise. Because this function is not called very often, the > change seems unlikely to help. Yes, this section does not apply to this problem :) Just I posted the latest patch which i using now... >=20 > > @@ -2118,13 +2119,21 @@ > > struct td_sched *ts; > > =20 > > THREAD_LOCK_ASSERT(td, MA_OWNED); > > + if (td->td_pri_class & PRI_FIFO_BIT) > > + return; > > + ts =3D td->td_sched; > > + /* > > + * We used up one time slice. > > + */ > > + if (--ts->ts_slice > 0) > > + return; >=20 > This skips most of the periodic functionality (long term load > balancer, saving switch count (?), insert index (?), interactivity > score update for long running thread) if the thread is not going to > be rescheduled right now. >=20 > It looks wrong but it is a data point if it helps your workload. Yes, I did it for as long as possible to delay the execution of the code in= section: ... #ifdef SMP /* * We run the long term load balancer infrequently on the first cpu. */ if (balance_tdq =3D=3D tdq) { if (balance_ticks && --balance_ticks =3D=3D 0) sched_balance(); } #endif ... >=20 > > tdq =3D TDQ_SELF(); > > #ifdef SMP > > /* > > * We run the long term load balancer infrequently on the > > first cpu. */ > > - if (balance_tdq =3D=3D tdq) { > > - if (balance_ticks && --balance_ticks =3D=3D 0) > > + if (balance_ticks && --balance_ticks =3D=3D 0) { > > + if (balance_tdq =3D=3D tdq) > > sched_balance(); > > } > > #endif >=20 > The main effect of this appears to be to disable the long term load > balancer completely after some time. At some point, a CPU other than > the first CPU (which uses balance_tdq) will set balance_ticks =3D 0, and > sched_balance() will never be called again. >=20 That is, for the same reason as above in the text... > It also introduces a hypothetical race condition because the access to > balance_ticks is no longer restricted to one CPU under a spinlock. >=20 > If the long term load balancer may be causing trouble, try setting > kern.sched.balance_interval to a higher value with unpatched code. I checked it in the first place - but it did not help fix the situation... The impression of malfunction rebalancing... It seems that the thread is passed on to the same core that is loaded and s= o... Perhaps this is a consequence of an incorrect definition of the topology CP= U? >=20 > > @@ -2144,9 +2153,6 @@ > > if > > (TAILQ_EMPTY(&tdq->tdq_timeshare.rq_queues[tdq->tdq_ridx])) > > tdq->tdq_ridx =3D tdq->tdq_idx; } > > - ts =3D td->td_sched; > > - if (td->td_pri_class & PRI_FIFO_BIT) > > - return; > > if (PRI_BASE(td->td_pri_class) =3D=3D PRI_TIMESHARE) { > > /* > > * We used a tick; charge it to the thread so > > @@ -2157,11 +2163,6 @@ > > sched_priority(td); > > } > > /* > > - * We used up one time slice. > > - */ > > - if (--ts->ts_slice > 0) > > - return; > > - /* > > * We're out of time, force a requeue at userret(). > > */ > > ts->ts_slice =3D sched_slice; >=20 > > and refusal to use options FULL_PREEMPTION > > But no one has unsubscribed to my letter, my patch helps or not in > > the case of Core2Duo... > > There is a suspicion that the problems stem from the sections of > > code associated with the SMP... > > Maybe I'm in something wrong, but I want to help in solving this > > problem ... >=20