From owner-freebsd-stable@FreeBSD.ORG Thu Dec 15 02:42:53 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2779106564A for ; Thu, 15 Dec 2011 02:42:53 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id 5C9068FC0C for ; Thu, 15 Dec 2011 02:42:53 +0000 (UTC) Received: from omta03.westchester.pa.mail.comcast.net ([76.96.62.27]) by qmta10.westchester.pa.mail.comcast.net with comcast id 9Ei91i0020bG4ec5AEisZt; Thu, 15 Dec 2011 02:42:52 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta03.westchester.pa.mail.comcast.net with comcast id 9Eir1i00b1t3BNj3PEirrR; Thu, 15 Dec 2011 02:42:52 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id D7F8A102C19; Wed, 14 Dec 2011 18:42:49 -0800 (PST) Date: Wed, 14 Dec 2011 18:42:49 -0800 From: Jeremy Chadwick To: Oliver Pinter Message-ID: <20111215024249.GA13557@icarus.home.lan> References: <4EE1EAFE.3070408@m5p.com> <4EE2AE64.9060802@m5p.com> <4EE88343.2050302@m5p.com> <4EE933C6.4020209@zedat.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Tom Evans , Attilio Rao , George Mitchell , "O. Hartmann" , freebsd-stable@freebsd.org Subject: Re: SCHED_ULE should not be the default X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 02:42:53 -0000 On Thu, Dec 15, 2011 at 03:05:12AM +0100, Oliver Pinter wrote: > On 12/15/11, O. Hartmann wrote: > > On 12/14/11 18:54, Tom Evans wrote: > >> On Wed, Dec 14, 2011 at 11:06 AM, George Mitchell > >> wrote: > >>> > >>> Dear Secret Masters of FreeBSD: Can we have a decision on whether to > >>> change back to SCHED_4BSD while SCHED_ULE gets properly fixed? > >>> > >> > >> Please do not do this. This thread has shown that ULE performs poorly > >> in very specific scenarios where the server is loaded with NCPU+1 CPU > >> bound processes, and brought forward more complaints about > >> interactivity in X (I've never noticed this, and use a FreeBSD desktop > >> daily). > > > > I would highly appreciate a decission against SCHED_ULE as the default > > scheduler! SCHED_4BSD is considered a more mature entity and obviously > > it seems that SCHED_ULE needs some refinements to achieve a better level > > of quality. > > > >> > >> On the other hand, we have very many benchmarks showing how poorly > >> 4BSD scales on things like postgresql. We get much more load out of > >> our 8.1 ULE DB and web servers than we do out of our 7.0 ones. It's > >> easy to look at what you do and say "well, what suits my environment > >> is clearly the best default", but I think there are probably more > >> users typically running IO bound processes than CPU bound processes. > > > > You compare SCHED_ULE on FBSD 8.1 with SCHED_4BSD on FBSD 7.0? Shouldn't > > you compare SCHED_ULE and SCHED_4BSD on the very same platform? > > > > Development of SCHED_ULE has been focused very much on DB like > > PostgreSQL, no wonder the performance benefit. But this is also a very > > specific scneario where SCHED_ULE shows a real benefit compared to > > SCHED_4BSD. > > > >> > >> I believe the correct thing to do is to put some extra documentation > >> into the handbook about scheduler choice, noting the potential issues > >> with loading NCPU+1 CPU bound processes. Perhaps making it easier to > >> switch scheduler would also help? > > > > Many people more experst in the issue than myself revealed some issues > > in the code of both SCHED_ULE and even SCHED_4BSD. It would be a pitty > > if all the discussions get flushed away like a "toilette-busisness" as > > it has been done all the way in the past. > > > > > > Well, I'd like to see a kind of "standardized" benchmark. Like on > > openbenchmark.org or at phoronix.com. I know that Phoronix' way of > > performing benchmarks is questionable and do not reveal much of the > > issues, but it is better than nothing. I'm always surprised by the worse > > performance of FreeBSD when it comes to threaded I/O. The differences > > between Linux and FreeBSD of the same development maturity are > > tremendous and scaring! > > > > It is a long time since I saw a SPEC benchmark on a FreeBSD driven HPC > > box. Most benchmark around for testing hardware are performed with Linux > > and Linux seems to make the race in nearly every scenario. It would be > > highly appreciable and interesting to see how Linux and FreeBSD would > > perform in SPEC on the same hardware platform. This is only an idea. > > Without a suitable benchmark with a codebase understood the discussion > > is in many aspects pointless -both ways. > > > > > >> > >> Cheers > >> > >> Tom > >> > >> References: > >> > >> http://people.freebsd.org/~kris/scaling/mysql-freebsd.png > >> http://suckit.blog.hu/2009/10/05/freebsd_8_is_it_worth_to_upgrade > >> _______________________________________________ > > Hi! > > Can you try with this settings: > op@opn ~> sysctl kern.sched. I'm replying with a list of each setting which differs compared to RELENG_8 stock on our ULE systems. Note that our ULE systems are 1 physical CPU with 4 cores. > kern.sched.cpusetsize: 8 I see no such tunable/sysctl on any of our RELENG_8 and RELENG_7 systems. Nor do I find any references to it in /usr/src (on any system). Is this a RELENG_9 setting? Please explain where it comes from. I hope it's not a custom kernel patch... > kern.sched.preemption: 0 This differs; default value is 1. > kern.sched.name: ULE > kern.sched.slice: 13 > kern.sched.interact: 30 > kern.sched.preempt_thresh: 224 This differs; default value is 64. The "magic value" of 224 has been discussed in the past, in this thread even. > kern.sched.static_boost: 152 This differs; on our systems it's 160. > kern.sched.idlespins: 10000 > kern.sched.idlespinthresh: 16 This differs; on our systems it's 4. > Most of them from 7-STABLE settings, and with this, "works for me". > This an laptop with core2 duo cpu (with enabled powerd), and my kernel > config is here: > http://oliverp.teteny.bme.hu/freebsd/kernel_conf -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |