Date: Thu, 17 Jan 2002 12:29:42 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Michal Mertl <mime@traveller.cz> Cc: arch@freebsd.org Subject: Re: 64 bit counters again Message-ID: <3C473436.BBBABF6B@mindspring.com> References: <Pine.BSF.4.41.0201171742520.5459-100000@prg.traveller.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
Michal Mertl wrote: > I wan't to inform you that I tried to look at some system pushing data > with different size/implementation network counters. I did my last test on > dual PIII 750. I don't know, of any good way to measure the load, so I > just run vmstat -w1 (and calculated average idle) while pushing the data > and also looked at the throughput at 100Mbit Full-Duplex. System was > performing about 10000 interrupts and 15000 packets per second. I didn't > notice any difference between using 32 bits non atomic operations (3 > clocks per op) or 64 bit atomic (lock;cmpxchg8b - 50 clocks). I did also > measure it on single Duron 800 with the same result. 1) Use a gigabit interface, not an FXP. 2) Use 1K payload data. 3) Tune your system so that it can push data at wire speed on the gigabit; this isn't hard, but it is a test of whether you can recognize overhead when you see it. 4) Measure CPU overhead as well as I/O overhead. 5) Measure total number of mbufs and clusters in use, 6) Use an SMP system, make sure that you have a sender on both CPUs, and measure TLB shootdown and page mapping turnover to ensure you get that overhead in there, too (plus the lock overhead). 7) Make sure you are sending data already in the kernel, so you aren't including copy overhead in the CPU cost, since practically no one implements servers with copy overhead these days. If you push data at 100Mbit, and not even at full throttle at that, you can't reasonably expect to see a slowdown when you have other bottlenecks between you and the changes. In particular, you're not going to see things like the pool size go up because of increased pool retention time, etc., due to the overhead of doing the calculations. Also, realize that even though simply pushing data doesn't use up a lot of CPU on FreeBSD if you do it right, even 2% or 4% increase in CPU overhead overall is enough to cause problems for already CPU-bound applications (i.e. that's ~40 less SSL connections per server). -- Personally, I don't consider 3 and a half million clocks to be peanuts, and if you ran at wire speed on the 100Mbit, I would expect that to become 35 million clocks on a gigabit. But we can wait for your effects on the mbuf count high watermark and CPU utilization values before jumping to any conclusions... -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C473436.BBBABF6B>