Date: Sat, 17 Jun 2006 13:50:48 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: performance@FreeBSD.org Subject: HZ=100: not necessarily better? Message-ID: <20060617134402.O8526@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
Scott asked me if I could take a look at the impact of changing HZ for some
simple TCP performance tests. I ran the first couple, and got some results
that were surprising, so I thought I'd post about them and ask people who are
interested if they could do some investigation also. The short of it is that
we had speculated that the increased CPU overhead of a higher HZ would be
significant when it came to performance measurement, but in fact, I measure
improved performance under high HTTP load with a higher HZ. This was, of
course, the reason we first looked at increasing HZ: improving timer
granularity helps improve the performance of network protocols, such as TCP.
Recent popular opinion has swung in the opposite direction, that higher HZ
overhead outweighs this benefit, and I think we should be cautious and do a
lot more investigating before assuming that is true.
Simple performance results below. Two boxes on a gig-e network with if_em
ethernet cards, one running a simple web server hosting 100 byte pages, and
the other downloading them in parallel (netrate/http and netrate/httpd). The
performance difference is marginal, but at least in the SMP case, likely more
than a measurement error or cache alignment fluke. Results are
transactions/second sustained over a 30 second test -- bigger is better; box
is a dual xeon p4 with HTT; 'vendor.*' are the default 7-CURRENT HZ setting
(1000) and 'hz.*' are the HZ=100 versions of the same kernels. Regardless,
there wasn't an obvious performance improvement by reducing HZ from 1000 to
100. Results may vary, use only as directed.
What we might want to explore is using a programmable timer to set up high
precision timeouts, such as TCP timers, while keeping base statistics
profiling and context switching at 100hz. I think phk has previously proposed
doing this with the HPET timer.
I'll run some more diverse tests today, such as raw bandwidth tests, pps on
UDP, and so on, and see where things sit. The reduced overhead should be
measurable in cases where the test is CPU-bound and there's no clear benefit
to more accurate timing, such as with TCP, but it would be good to confirm
that.
Robert N M Watson
Computer Laboratory
University of Cambridge
peppercorn:~/tmp/netperf/hz> ministat *SMP
x hz.SMP
+ vendor.SMP
+--------------------------------------------------------------------------+
|xx x xx x xx x + + + + + ++ + ++|
| |_______A________| |_____________A___M________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 10 13715 13793 13750 13751.1 29.319883
+ 10 13813 13970 13921 13906.5 47.551726
Difference at 95.0% confidence
155.4 +/- 37.1159
1.13009% +/- 0.269913%
(Student's t, pooled s = 39.502)
peppercorn:~/tmp/netperf/hz> ministat *UP
x hz.UP
+ vendor.UP
+--------------------------------------------------------------------------+
|x x xx x xx+ ++x+ ++ * + + +|
| |_________M_A_______|___|______M_A____________| |
+--------------------------------------------------------------------------+
N Min Max Median Avg Stddev
x 10 14067 14178 14116 14121.2 31.279386
+ 10 14141 14257 14170 14175.9 33.248058
Difference at 95.0% confidence
54.7 +/- 30.329
0.387361% +/- 0.214776%
(Student's t, pooled s = 32.2787)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060617134402.O8526>