From owner-freebsd-arch@FreeBSD.ORG Thu Oct 27 10:31:11 2005 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FBF516A41F; Thu, 27 Oct 2005 10:31:11 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 499AD43D46; Thu, 27 Oct 2005 10:31:11 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id E20E346B0C; Thu, 27 Oct 2005 06:31:10 -0400 (EDT) Date: Thu, 27 Oct 2005 11:31:10 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: David Xu In-Reply-To: <436027E6.8080604@freebsd.org> Message-ID: <20051027112622.O32255@fledge.watson.org> References: <20051025120538.K52058@fledge.watson.org> <435E2DCF.6080809@freebsd.org> <20051025134834.GB62148@stack.nl> <435EBD49.7090207@freebsd.org> <20051026120219.V32255@fledge.watson.org> <436027E6.8080604@freebsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Marc Olzheim , Daniel Eischen , arch@freebsd.org Subject: Re: libc_r is deprecated X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Oct 2005 10:31:11 -0000 On Thu, 27 Oct 2005, David Xu wrote: > You sample program is nothing to do with libpthread or libthr, you never > have userland locking code, in kernel they should be same, in fact. I > have done a simple test, on my dual PIII machine, libthr is faster than > libpthread, if I use ULE, libpthread is far behind than libthr, I > recommend you to try different file sizes, I have got very different > result, for small file size, e.g /usr/src/sys/sys/umtx.h, libthr beats > libpthread, I can not test big file, because my 100M network is very > easy to be saturated. > > If you suspect fd locking overhead, try to use fork() model, though it > will introduce more context switch overhead, but it still worthy to do. Well, I think the significant difference is the scheduling of when work is requested: specifically, that libthr with our current scheduler necessarily results with a process competing more with itself for locks associated with the process than libpthread, which tries not to run additional threads unless there's a gap in current execution by a current thread. The question, I suppose, is whether the answer to this problem lies in: - Helping the scheduler do a better job of discouraging that scenario. - Working to change the file descriptor model to reduce the impact of contention. - Accepting that it's inevitable in the threading model and looking for other things to improve. I would like to run the following additional tests: - Create a forked httpd microbenchmark to see what impact that has. I suspect a performance improvement. - Experiment with reducing contention on the file descriptor lock through hacking a bit on how file descriptor arrays work -- nothing production-worthy, but something that changes the potential contention enough to see if it helps. I.e., break the array into two parts locked using different locks. - Use ktrace, mutex profiling, or pmc to try and evaluate whether contention on the lock really is the cause. Alternatively, to roughly attempt to profile wait times for that lock. Robert N M Watson