From owner-freebsd-current@FreeBSD.ORG Thu Jun 17 13:02:18 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9489516A4CE; Thu, 17 Jun 2004 13:02:18 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B139443D41; Thu, 17 Jun 2004 13:02:12 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i5HD08t4029767; Thu, 17 Jun 2004 09:00:08 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i5HD08oR029764; Thu, 17 Jun 2004 09:00:08 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Thu, 17 Jun 2004 09:00:08 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Jon Noack In-Reply-To: <40D13BB6.3020709@alumni.rice.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: mike cc: Julian Elischer cc: freebsd-threads@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: Possible Threading problem with -CURRENT / MySQL? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jun 2004 13:02:18 -0000 On Thu, 17 Jun 2004, Jon Noack wrote: > On 06/17/04 01:07, Brad Knowles wrote: > > At 1:31 AM -0400 2004-06-17, Robert Watson wrote: > >> - Removing Giant from UNIX domain sockets > > > > Out of curiosity, is this something that could be relatively safely > > done in general? Any ideas on what the plan is for doing this as the > > default on -CURRENT? > > Wasn't this already done? See this commit: > http://lists.freebsd.org/pipermail/cvs-src/2004-June/025082.html The locks have been introduced to protect UNIX domain sockets, but the merge of locking for the cross-protocol socket infrastructure isn't complete yet. This means that if you disable Giant, the UNIX domain bits work fine, but the socket buffers, socket state, and socket event handling will probably corrupt themselves rapidly on SMP. Per an earlier e-mail, I'm working to complete that merge, but took a break to run some benchmarks. Looks for more information here in about a week and a half. > >> - Disabling HTT - Using ADAPTIVE_MUTEXES > > > > These both sound like typical improvements, based on what I've seen > > on this list. Any ideas on when they might become the default? > > ADAPTIVE_MUTEXES was already enabled by default on amd64. See this > commit (read thread for discussion): > http://lists.freebsd.org/pipermail/cvs-src/2004-June/025234.html > > It appears no comprehensive testing has been done to check whether it > really does improve performance. Many signs do point that way, though. Agreed. I think we need to run a broad range of benchmarks across a number of hardware configurations in order to make this decision. Results on a couple of interesting benchmarks, though, are very suggestive, so we should probably do this sooner rather than later. > >> - Running with SCHED_4BSD instead of SCHED_ULE > > > > This is the only one that really concerns me. This shows that we > > clearly need more work on ULE. Is there one particular thing that we > > seem to be tripping up on, or is it a multitude of things? > > I think I'll switch to 4BSD until I see more work being done on ULE (I > read cvs-src as a hobby already so I'll know when to switch back). I > noticed a performance drop from 5.2.1-p8 (with 4BSD) to -CURRENT (with > ULE) when I upgraded, but I attributed it to other things. However, the > system still feels slower than before despite having since disabled > INVARIANTS, WITNESS, and userspace malloc debugging flags. ULE seems to do a very good job of scheduling interactive tasks over other workloads, resulting in a very "snappy" feel on my boxes, despite heavy CPU load from background builds, etc. The workload I looked at had no real "interactive" component, although it was a latency-centric RPC test, so timely hand-off as well as high throughput would be important. I know that Jeff's measurement work on ULE had a substantial focus on deadlines -- whether or not ULE was timely in scheduling tasks, etc, and that he demonstrated that it was much stronger than most other available schedulers in this area. One of the next obvious steps in optimizing either ULE or 4BSD is going to be to spend a lot of time sitting with KTR(4) and looking at context switch traces for "dumb things", such as bouncing between CPUs, rapid switches back and forth, undo multiple wakeups, etc. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Senior Research Scientist, McAfee Research