Date: Fri, 30 Nov 2001 12:58:39 -0500 From: Leo Bicknell <bicknell@ufp.org> To: freebsd-hackers@freebsd.org Subject: TCP Performance Graphs Message-ID: <20011130125839.A88302@ussenterprise.ufp.org>
next in thread | raw e-mail | index | archive | help
Since the topic has come up again, I'll provide some graphs, and go back to my suggestion to see if it gets some traction this time around. http://www.ufp.org/~bicknell/fbsdtcp.png This graph shows the theoretical maximum performance of FreeBSD's TCP stack (assuming a network with ample free bandwidth, no router buffering, no dropped packets, etc). The red curve is with the existing (16k) window. I've used a scale of 0 to 100ms RTT, as I think that's the range you should find in the contentional US in the real world. Obviously higher values would be needed to make transoceanic hops, satellite hops, or other cases work. As you can see, we should be able to fill a T1 up to about 83 MS RTT, Ethernet up to about 16ms RTT, and DS-3 up to about 3ms. My 'rough estimate' on the real world is you can get about 75% of those figures across what we know and love as "the Internet", so you could fill a T1 over a connection with an RTT of about 62ms. The question that immediately comes to mind is, why not simply use as big a value as possible? The problem comes down to buffering the data, and busy servers may have to buffer a lot of data. Having a 1 meg window size may have you buffer 1 meg per connection. Note that FreeBSD's current buffer management is particularly stupid in that it will _always_ buffer 1 Meg, need it or not. Until we fix this we need an interim solution. Most of the commercial Unix vendors as well as Linux have moved to a 32k default. This is the green line marked 'proposal' in the graph. This will, on average double the network memory used. If you want to see the impact of this on your own systems, run 'netstat -M', and consider the worst case of doubled usage. I suspect virtually all server admins won't care about the additional memory if it means additional performance. 32k windows, as the graph shows, let you saturate a T1 with a nice buffer. With T1 DSL and Cable modems being common now I feel very strongly that out-of-the-box ability to saturate these links is essential to make people believe FreeBSD is a good performer. It also provides a nice boost (double throughput, imagine that) to users off Ethernet hubs behind higher speed connections. They can now get full Ethernet speed up to about 32 ms, which opens up a significant number of network sites. I've also included 64k, the largest value that can be used without sliding window support. For now, I would consider 64k to be the largest default we should even think about, and it may not be a good idea due to the larger memory footprint. That said, DRAM prices are at an all time low, so now may be the time to poke people to buy more if they want real performance. I don't know who can move this forward, but I'd really like to see 32k windows be the default in the next release. I think 32k could go into current immediately, and stable nearly immediately to start to get some feedback and insure there are no major issues. Finally, many people keep replying that applications can set larger window sizes on their own, so this is unnecessary. While true, this is completely impractical for a number of reasons: * End users won't. They expect it to work out of the box. Tweaking a setting is unacceptable. * Every "bulk transfer" application would have to be modified. Take a look in ports, see if you think that's a good idea. * Non-bulk transfer applications can become bulk transfer applications. For instance, is an ssh session an interactive session, or really an scp of a large file? * Hard coding these values into thousands of programs will make future upgrades (when network speed and memory allow) infinitely harder. If Linux and most of the commercial vendors have found 32k to be an acceptable value I think it's time FreeBSD join them. We should be leading, not last to adopt. (Note, for those curious in another view, try http://www.ufp.org/~bicknell/fbsdtcp3d.png) -- Leo Bicknell - bicknell@ufp.org - CCIE 3440 PGP keys at http://www.ufp.org/~bicknell/ Read TMBG List - tmbg-list-request@tmbg.org, www.tmbg.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20011130125839.A88302>