Date: Thu, 31 Oct 2002 13:56:01 -0500 From: Fran Lawas-Grodek <Fran.Lawas-Grodek@grc.nasa.gov> To: freebsd-net@freebsd.org Cc: Cindy.Tran@grc.nasa.gov, Mark.Allman@grc.nasa.gov Subject: Problem in High Speed and Long Delay with FreeBSD Message-ID: <20021031135601.B23802@grc.nasa.gov>
next in thread | raw e-mail | index | archive | help
Hello, Hopefully someone might have some advice on our problem. We are setting up a testbed consisting of FreeBSD 4.1 on the sender and receiver machines. (This older version of FreeBSD is necessary due to subsequent TCP development patches that are to be tested.) The problem that we are seeing is that on a 100Mbps link with a 250millisecond delay, our overall throughput does not exceed 32Mbps. We expect to see at least 60Mbps with the 250ms delay. Without a delay, throughput is acceptable at 87Mbps. Tcptrace output shows no retransmissions and no out-of-order packets under the delay. With the delay, we are setting up a 3MByte sender and receiver socket buffer size, but through the time sequence plot, we only see about 1MB used of the buffer on the sender side. Additional tests were run with an 80ms delay and requesting a 1MB buffer, and yet it appears only are allowed to use up to 500KB of the window. Raising the requested socket buffer size shows no affect -- we keep running into some invisible threshold that is much smaller than the requested socket buffer size. A hacked version of the ttcp application prints out that what we request in the -b buffer size via setsockopt, is what we get (via getsockopt). Unfortunately, the xplots show that our transfers reach a lower threshold (1MB instead of 3MB, 500KB instead of 1MB). Testing commands used: receiver> ttcp -b 312500 -l 1024 -r -s sender> ttcp -b 312500 -l 1024 -n 100000 -s -t receiverhost The -b above is set to the value of the Bandwidth Delay Product for a 100Mbps link and 250ms delay. Per the Pittsburgh Supercomputing site, we have already increased the maxsockbuf, nmbclusters, memory, tcp_sendspace, tcp_recvspace sysctl values, but are still resulting in this low throughput. The network cards that we are using are 3Com95-TX 100BaseT cards. The machines with the FreeBSD installations are Pentium II 400Mhz with Xeon chips, 400 Mb RAM each. Using two other non-FreeBSD machines, we have verified that it is not the intermediary network routers or delay emulator equipment, as the non-FreeBSD machines give the expected throughput under delayed and no delay conditions. (60+Mbps under 250ms delay) Has anyone else any experiences in this type of testing? Perhaps there might be something wrong with our network card driver? Any other suggested network cards to try? Does anyone know of a limit that needs to be tweaked in the TCP stack or FreeBSD operating system that would allow us to actually use more than this invisible socket buffer limit? One thought is that there is some sort of calculated limit that won't allow us to send more packets than our requested socket buffer will allow, but not having any the kernel expertise, we are not sure where to look. Thanks in advance for any suggestions, Fran Lawas-Grodek Cindy Tran NASA Glenn Research Center Cleveland OH USA (ps: We have already consulted with Mark Allman at our lab, and he is just as stumped. The feeling is that the cause of the problem is buried somewhere in the kernel, not due to the network cards.) -- ________________________________________________________________ Frances J. Lawas-Grodek | NASA Glenn Research Center | phone: (216) 433-5052 21000 Brookpark Rd, MS 142-4 | fax : (216) 433-8000 Cleveland, Ohio 44135 | email: fran@grc.nasa.gov ________________________________________________________________ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021031135601.B23802>