From owner-freebsd-current@FreeBSD.ORG Mon Oct 31 09:26:13 2005 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2AD3916A41F; Mon, 31 Oct 2005 09:26:13 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFCFF43D45; Mon, 31 Oct 2005 09:26:12 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id B435546B9B; Mon, 31 Oct 2005 04:26:11 -0500 (EST) Date: Mon, 31 Oct 2005 09:26:11 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: freebsd-current@freebsd.org In-Reply-To: <20051031075843.GF39253@dragon.NUXI.org> Message-ID: <20051031083447.Y11619@fledge.watson.org> References: <30595.1130493297@critter.freebsd.dk> <20051028153457.d0wqgn2ask4sgw4k@netchild.homeip.net> <20051029195703.GB39253@dragon.NUXI.org> <43646AAC.2080107@freebsd.org> <20051030093718.GE39253@dragon.NUXI.org> <4364D90F.3090205@samsco.org> <20051031075843.GF39253@dragon.NUXI.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Alexander Leidinger , David Xu Subject: Re: TSC instead of ACPI: powerd doesn't work anymore (to be expected?) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Oct 2005 09:26:13 -0000 On Sun, 30 Oct 2005, David O'Brien wrote: >> ACPI-fast requires an ioport read which takes about 1us (according to >> Google). Do that 1000 times a second and you have each CPU spending 1% >> of its time doing nothing but reading the clock. Yikes. > > But we've lived with using the ACPI timercounter (vs. TSC) for quite a > while now. Why all of a sudden are the authors of this thread having an > issue with it now. I know about the recent MySQL thread - but with the > TSC being untrustable on MP and power managed systems, why is there such > a desire to use the TSC? Because in the past few weeks, several developers have noticed that if you change timercounters from ACPI-fast to TSC, you get significant performance boosts on real-world workloads. Drew reported this a couple of weeks ago on net@ after looking at loopback network performance, and I suggested he frob the timecounter on his system, which resulted in a significant measured performance increase. Then recently it was reported that MySQL also saw a significant boost from faster timekeeping, as it performs frequent time measurement. So we have at least two interesting cases: (1) We perform time measurement in the kernel context switch code for every context switch. Using cheaper time measurement here can make a big different in performance for a broad range of workloads involving frequent context switches -- high interrupt rates, loopback or local IPC, etc. (2) Many applications measure the time in order to self-profile, measure network timeouts, etc; it's quite possible that self-instrumentation with frequent time measurements is increasing due to the relatively low cost of time measurement on Linux, which has the effect of significantly reducing FreeBSD performance, where time measurement is much more expensive. For example, it was reported that MySQL is measuring time with every IPC operation. It has been suggested that the former can operate quite well with significantly reduced quality. It has alos been suggested that most applications could operate fine with somewhat reduced quality, but that the API definitions don't say anything about how to specify application quality requirements vs performance requirements for time measurement. Some performance results for loopback TCP on a UP kernel attached below. In the default case, a 12.8% performance improvement on loopback TCP is observed by switching from ACPI fast to TSC. With local changes to disable some unwise preemptions, I'm able to improve performance by about 29%, and adding on top of that the TSC change, 35.2%. In the nopreempt case, I've reduced the number of context switches, so the effect of changing time counters is reduced, as there are fewer time measurements. In this scenario, the performance improvement from nopreempt to nopreempt.tsc is 4.7%. These are non-trivial performance changes. FYI: measured on SMP, moving to TSC for this benchmark gives a 4.9% improvement in the preempt case, and a 2.5% improvement in the nopreempt case. On SMP, the preemption changes I'm working with reduce performance rather than improving it, possibly because the effects of preemption are still cheaper than the effects of migration. Regardless, the timecounter change is significant, although less so than on UP. Robert N M Watson x preempt + preempt.tsc * nopreempt % nopreempt.tsc +--------------------------------------------------------------------------+ | xx + * %% | | xx + ** %% | | xx ++ ** %% | |x xx +++ * ** **** %%% %%%| | |_A_| |A |__AM_| |_AM_|| +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 12 2123.5 2194.31 2186.03 2181.4983 19.144156 + 12 2444.03 2468.44 2463.22 2460.1725 6.5242305 Difference at 95.0% confidence 278.674 +/- 12.1092 12.7744% +/- 0.555084% (Student's t, pooled s = 14.3015) * 12 2750.12 2845.31 2829.98 2816.2325 31.188601 Difference at 95.0% confidence 634.734 +/- 21.9101 29.0962% +/- 1.00436% (Student's t, pooled s = 25.8769) % 12 2902.27 2979.3 2954.93 2949.0792 25.48312 Difference at 95.0% confidence 767.581 +/- 19.0828 35.1859% +/- 0.874755% (Student's t, pooled s = 22.5376)