Date: Mon, 17 Feb 2003 11:27:18 -0800 From: Terry Lambert <tlambert2@mindspring.com> To: Jonathan Lemon <jlemon@flugsvamp.com> Cc: current@freebsd.org Subject: Re: Polygraph Considered Evil 8^) (was: Re: 5-STABLE Roadmap) Message-ID: <3E513796.12DD2C33@mindspring.com> References: <local.mail.freebsd-current/20030216184257.GZ10767@garage.freebsd.pl> <local.mail.freebsd-current/3E4FFDD3.9050802@btc.adaptec.com> <local.mail.freebsd-current/20030216214322.GB10767@garage.freebsd.pl> <local.mail.freebsd-current/Pine.BSF.4.53.0302162130370.46493@measurement-factory.com> <200302171902.h1HJ2tnm040767@mail.flugsvamp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Jonathan Lemon wrote: > In article <local.mail.freebsd-current/3E511E7A.8225ABA9@mindspring.com> you write: > >Alex Rousskov wrote: > >One issue I have with Polygraph is that it intentionally works > >for a very long time to get worst case performance out of caches; > >basically, it cache-busts on purpose. Then the test runs. This > >seems to be an editorial comment on end-to-end guarantees, much > >more than it seems a valid measurement of actual cache performance. > > This is a realistic scenario; in the real-world, caches never start > empty, but are constantly full. However, polygraph numbers are biased > towards misses, and don't quite reflect real-world traffic. Particularly for something like a reverse proxy cache, where the intent is to offload static content traffic from a web server farm so that all it has to worry about is dynamic content and/or CGI processing. In that case, you get hits on everything by non-cacheable content. It's not like SQUID or other standard proxy caches, where they are, effectively, trying to cache the Internet. It's unfortunately that Slashdot doesn't put one in front of the sites they pound with traffic. Of course, though, that's one of the points, to be able to claim to drive that level of traffic; if they cached the content, where would be the fun in that? 8-). > >> > net.inet.tcp.msl=3000 > > > >And this seems designed to get a bad one. You are aware that, by > >default, NT systems cheat on the MSL, right? For gigabit, this is > >a larger number than you want, I think. > > Polygraph checks the MSL of the target system and complains if it > is < 3sec (TCP spec violation). Yep. > Also, the tuning above is for the polygraph *client* machine, not > the cache machine under test. Good point; it didn't occur to me, in the context of a discussion of FreeBSD benchmarking, but, of course, you are right. > >The hash is a reasonable modification; it'd probably be better handled > >through the routing code, since it has to be hashed there anyway, if > >you planned on using a lot of IP aliases. > > This patch is not needed any longer. A hash table lookup to handle > large # of IP aliases was added circa 4.4R. Yes. I was thinking in terms of 4.3, which the patches were against. This is somewhat like the callout wheel for the timers, to my mind; the hash list isn't big enough except for small loads. At the top end, the overhead gets large, quickly. > >I haven't looked at the client code, but you are aware that adding > >IP aliases doesn't really do anything, unless you managed your > >port space your self, manually, with a couple of clever tricks? In > >other words, you are going to be limited to your total number of > >outbound connections as your ports space (e.g. ~40K), because the > >port autoallocation takes place in the same space as the INADDR_ANY > >space? I guess this doesn't matter, if your maxopenfiles is only 16K, > >since that's going to end up bounding you well before you run out of > >ports... > > The goal here is to stress the neighbor/route code on the cache machine, > this is done by generating packets from many different source IP addresses. This makes sense as well; again, I was looking at FreeBSD as the system being benchmarked, from the context of the thread. That was why I thought the hash should have been integrated into the routing code. If you're going to be doing that many connections, as well, then there are other areas in FreeBSD that tend to have exponential cost, as time goes on, too (e.g. the per-process open file table, some of the kevent implemention, etc.). > >I don't think that anything you do in this regard is going to be able > >to give you iMimic or NetApp level numbers, which are created by > >professional benchmark-wranglers, so any comparison values you get > >will liekly be poor, compared to commercial offerings. > > You're not going to get commercial quality numbers with squid. Nope. Or with FreeBSD. I only mentioned SQUID because of the easy cheat for the page replacement policy allowing incredibly better numbers than, say, SQUID on Linux. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3E513796.12DD2C33>