From owner-freebsd-stable@FreeBSD.ORG Tue Dec 13 12:47:10 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76FED106564A for ; Tue, 13 Dec 2011 12:47:10 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 1FEF78FC13 for ; Tue, 13 Dec 2011 12:47:09 +0000 (UTC) Received: from omta05.westchester.pa.mail.comcast.net ([76.96.62.43]) by qmta01.westchester.pa.mail.comcast.net with comcast id 8cn81i0010vyq2s51cnADx; Tue, 13 Dec 2011 12:47:10 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta05.westchester.pa.mail.comcast.net with comcast id 8cn81i0191t3BNj3Rcn9GG; Tue, 13 Dec 2011 12:47:10 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 56D23102C19; Tue, 13 Dec 2011 04:47:07 -0800 (PST) Date: Tue, 13 Dec 2011 04:47:07 -0800 From: Jeremy Chadwick To: "O. Hartmann" Message-ID: <20111213124707.GA75440@icarus.home.lan> References: <4EE1EAFE.3070408@m5p.com> <4EE22421.9060707@gmail.com> <4EE6060D.5060201@mail.zedat.fu-berlin.de> <4EE619FC.4000601@unsane.co.uk> <4EE73366.7080304@mail.zedat.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4EE73366.7080304@mail.zedat.fu-berlin.de> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-performance@freebsd.org, Current FreeBSD , freebsd-stable@freebsd.org, Vincent Hoffman Subject: Re: SCHED_ULE should not be the default X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2011 12:47:10 -0000 On Tue, Dec 13, 2011 at 12:13:42PM +0100, O. Hartmann wrote: > On 12/12/11 16:13, Vincent Hoffman wrote: > > > > On 12/12/2011 13:47, O. Hartmann wrote: > > > >>> Not fully right, boinc defaults to run on idprio 31 so this isn't an > >>> issue. And yes, there are cases where SCHED_ULE shows much better > >>> performance then SCHED_4BSD. [...] > > > >> Do we have any proof at hand for such cases where SCHED_ULE performs > >> much better than SCHED_4BSD? Whenever the subject comes up, it is > >> mentioned, that SCHED_ULE has better performance on boxes with a ncpu > > >> 2. But in the end I see here contradictionary statements. People > >> complain about poor performance (especially in scientific environments), > >> and other give contra not being the case. > > It all a little old now but some if the stuff in > > http://people.freebsd.org/~kris/scaling/ > > covers improvements that were seen. > > > > http://jeffr-tech.livejournal.com/5705.html > > shows a little too, reading though Jeffs blog is worth it as it has some > > interesting stuff on SHED_ULE. > > > > I thought there were some more benchmarks floating round but cant find > > any with a quick google. > > > > > > Vince > > > > > > Interesting, there seems to be a much more performant scheduler in 7.0, > called SCHED_SMP. I have some faint recalls on that ... where is this > beast gone? Boy I sure hope I remember this right. I strongly urge others to correct me where I'm wrong; thanks in advance! The classic scheduler, SCHED_4BSD, was implemented back before there was oxygen. sched_4bsd(4) mentions this. No need to discuss it. Jeff Robertson began working on the "first-generation ULE scheduler" during the days of FreeBSD 5.x (I believe 5.1), and a paper on it was presented at USENIX circa 2003: http://www.usenix.org/event/bsdcon03/tech/full_papers/roberson/roberson.pdf Over the following years, Jeff (and others I assume -- maybe folks like George Neville-Neil and/or Kirk McKusick?) adjusted and tinkered with some of the semantics and models/methods. If I remember right, some of these quirks/fixes were committed. All of this was happening under the scheduler that was then called SCHED_ULE, but it was "ULE 1.0" for lack of better terminology. This scheduler did not perform well, if I remember right, and Jeff was quite honest about that. From this point forward, Jeff began idealising and working on a scheduler which he called SCHED_SMP -- think of it as "ULE 2.0", again, for lack of better terminology. It was different than the existing SCHED_ULE scheduler, hence a different name. Jeff blogged about this in early 2007, using exactly that term ("ULE 2.0"): http://jeffr-tech.livejournal.com/3729.html In mid-2007, prior to FreeBSD 7.0-RELEASE, Jeff announced that effectively he wanted to make SCHED_ULE do what SCHED_SMP did, and provided a patch to SCHED_ULE to accomplish just that: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2007-07/msg00755.html Full thread is here (beware -- many replies): http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2007-07/threads.html#00755 The patch mentioned above was merged into HEAD on 2007/07/19. http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sched_ule.c#rev1.202 So in effect, as of 2007/07/19, SCHED_ULE became SCHED_SMP. FreeBSD 7.0-RELEASE was released on 2008/02/27, and the above commit/changes were available at that time as well (meaning: RELENG_7 and RELENG_7_0 at that moment in time should have included the patch from the above paragraph). The document released by Kris Kenneway hinted at those changes and performance improvements: http://people.freebsd.org/~kris/scaling/7.0%20Preview.pdf Keep in mind, however, that at that time kernel configuration files (GENERIC, etc.) still defaulted to SCHED_4BSD. The default scheduler in kernel config files (GENERIC, etc.) for i386 and amd64 (not sure about others) was changed in 2007/10/19: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/conf/GENERIC#rev1.475 http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/amd64/conf/GENERIC#rev1.485 This was done *prior* to FreeBSD 7.1-RELEASE. So, it first became available as the default scheduler "for the masses" when 7.1-RELEASE came out on 2009/01/05. "All of the answers", in a roundabout and non-user-friendly way, are available by examining the commit history for src/sys/kern/sched_ule.c. It's hard to follow especially given that you have to consider all the releases/branchpoints that took place over time, but: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/kern/sched_ule.c Are we having fun yet? :-) -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |