From owner-freebsd-net Thu Jun 6 9:42:10 2002 Delivered-To: freebsd-net@freebsd.org Received: from bug.udcast.com (ANice-101-2-1-104.abo.wanadoo.fr [193.251.10.104]) by hub.freebsd.org (Postfix) with ESMTP id ED91C37B40B for ; Thu, 6 Jun 2002 09:41:38 -0700 (PDT) Received: from udcast.com (IDENT:eduros@localhost [127.0.0.1]) by bug.udcast.com (8.11.0/8.11.0) with ESMTP id g56Gf8Y16710 for ; Thu, 6 Jun 2002 18:41:08 +0200 Message-ID: <3CFF90A3.7000400@udcast.com> Date: Thu, 06 Jun 2002 18:41:07 +0200 From: Emmanuel Duros User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.9) Gecko/20020516 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-net@freebsd.org Subject: Weird TCP behavior with RTT of 1000 ms Content-Type: multipart/mixed; boundary="------------010601000702080005010400" Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org This is a multi-part message in MIME format. --------------010601000702080005010400 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Dear All, I am currently evaluating TCP with Freebsd 4.4 boxes simulating a very high latency link. For some reasons, I can get a TCP throughput extremely close to what is expected and sometimes, *without* changing any setting in my configuration I can get very poor performance ! Here is my setup: box 1 Router box 2 | | | | ------------------ ------------------- LAN 1 (100BT) LAN 2 (100BT) box 1, Router, and box 2 use FreeBSD 4.4. Router adds 1000ms (1 second) delay to all packets going *from* LAN 1 *to* LAN 2 using dummynet. I changed the TCP window sizes on box 1 and box 2 using sysctl command: box 1 box 2 sendspace 200000 default recvspace default 200000 I use ttcp to transfer data over tcp from box 1 to box 2. (1) When the transfer goes well, I get a 200Kbyte throughput after the slowstart period. This can be seen here with a 'netstat -I xl0 -w 1' on box 2: input (xl0) output packets errs bytes packets errs bytes colls 0 0 0 0 0 0 0 1 0 74 1 0 74 0 2 0 1580 2 0 132 0 2 0 3028 1 0 66 0 3 0 4542 2 0 132 0 5 0 7570 4 0 264 0 9 0 13626 6 0 396 0 15 0 22710 10 0 660 0 25 0 37850 17 0 1122 0 42 0 63588 28 0 1848 0 70 0 105980 47 0 3102 0 117 0 177138 78 0 5148 0 138 0 208932 92 0 6072 0 137 0 207418 91 0 6006 0 137 0 207418 92 0 6072 0 138 0 208932 92 0 6072 0 (2) However, very often the transfer does not happen so well... The transfer starts and gets stuck at 3Kbytes/s, and this can go up to 2 minutes and the slowstart operates again and the troughtput reaches 200KB/s. The netstat looks like: input (xl0) output packets errs bytes packets errs bytes colls 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 74 1 0 74 0 2 0 1580 2 0 132 0 3 0 4542 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 2 0 3028 2 0 132 0 [... for 2 minutes or more ! and then ...] 2 0 3028 1 0 66 0 3 0 4542 2 0 132 0 5 0 7570 4 0 264 0 input (xl0) output packets errs bytes packets errs bytes colls 9 0 13626 6 0 396 0 15 0 22710 10 0 660 0 25 0 37850 17 0 1122 0 42 0 63588 28 0 1848 0 70 0 105980 47 0 3102 0 117 0 177138 78 0 5148 0 138 0 208932 92 0 6072 0 137 0 207418 91 0 6006 0 138 0 208932 92 0 6072 0 134 0 202876 89 0 5874 0 I really want to point out that I am not changing anything in my settings between 2 transfers. I am just starting ttcp again and again on my boxes and TCP behave like (1) or (2). It looks like this behavior is not deterministic. I attached the tcpdump ouput that showes the beginning of the transfer on box 1. As you can see there is no packet loss and this cannot be the reason why the congestion window does not increase... I also noticed that with a 500ms delay instead of 1000ms I have not seen this problem. Is this a bug, a 'feature', ... ??? Thanks a lot and regards, Emmanuel -- Emmanuel Duros http://www.udcast.com 2455 Route des Dolines BP355 | Tel : +33 (0)4 93 00 16 60 06906 Sophia Antipolis France | Fax : +33 (0)4 93 00 16 61 ** Full IP over Broadcast Media ** --------------010601000702080005010400 Content-Type: text/plain; name="tcp-output.txt" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="tcp-output.txt" tcpdump -n -i fxp0 host 192.168.132.2 tcpdump: listening on fxp0 17:34:09.151317 192.168.99.102.1416 > 192.168.132.2.5001: S 3748748564:3748748564(0) win 16384 (DF) 17:34:10.143720 192.168.132.2.5001 > 192.168.99.102.1416: S 2021053759:2021053759(0) ack 3748748565 win 65535 (DF) 17:34:10.143792 192.168.99.102.1416 > 192.168.132.2.5001: . ack 1 win 17376 (DF) 17:34:10.144883 192.168.99.102.1416 > 192.168.132.2.5001: . 1:1449(1448) ack 1 win 17376 (DF) 17:34:11.143652 192.168.132.2.5001 > 192.168.99.102.1416: . ack 1 win 50318 (DF) 17:34:11.230230 192.168.99.102.1416 > 192.168.132.2.5001: . 1:1449(1448) ack 1 win 17376 (DF) 17:34:11.234687 192.168.132.2.5001 > 192.168.99.102.1416: . ack 1449 win 50318 (DF) 17:34:11.234737 192.168.99.102.1416 > 192.168.132.2.5001: . 1449:2897(1448) ack 1 win 17376 (DF) 17:34:11.234760 192.168.99.102.1416 > 192.168.132.2.5001: . 2897:4345(1448) ack 1 win 17376 (DF) 17:34:12.223754 192.168.132.2.5001 > 192.168.99.102.1416: . ack 1449 win 50318 (DF) 17:34:12.233813 192.168.132.2.5001 > 192.168.99.102.1416: . ack 4345 win 49956 (DF) 17:34:12.233870 192.168.99.102.1416 > 192.168.132.2.5001: . 4345:5793(1448) ack 1 win 17376 (DF) 17:34:12.233894 192.168.99.102.1416 > 192.168.132.2.5001: . 5793:7241(1448) ack 1 win 17376 (DF) 17:34:13.353855 192.168.132.2.5001 > 192.168.99.102.1416: . ack 5793 win 50318 (DF) 17:34:13.353940 192.168.99.102.1416 > 192.168.132.2.5001: . 7241:8689(1448) ack 1 win 17376 (DF) 17:34:13.444788 192.168.132.2.5001 > 192.168.99.102.1416: . ack 7241 win 50318 (DF) 17:34:13.444835 192.168.99.102.1416 > 192.168.132.2.5001: . 8689:10137(1448) ack 1 win 17376 (DF) 17:34:14.443749 192.168.132.2.5001 > 192.168.99.102.1416: . ack 10137 win 49956 (DF) 17:34:14.443811 192.168.99.102.1416 > 192.168.132.2.5001: . 10137:11585(1448) ack 1 win 17376 (DF) 17:34:14.443835 192.168.99.102.1416 > 192.168.132.2.5001: . 11585:13033(1448) ack 1 win 17376 (DF) 17:34:15.443816 192.168.132.2.5001 > 192.168.99.102.1416: . ack 11585 win 50318 (DF) 17:34:15.443870 192.168.99.102.1416 > 192.168.132.2.5001: . 13033:14481(1448) ack 1 win 17376 (DF) 17:34:15.534760 192.168.132.2.5001 > 192.168.99.102.1416: . ack 13033 win 50318 (DF) 17:34:15.534803 192.168.99.102.1416 > 192.168.132.2.5001: . 14481:15929(1448) ack 1 win 17376 (DF) 17:34:16.533773 192.168.132.2.5001 > 192.168.99.102.1416: . ack 15929 win 49956 (DF) 17:34:16.533831 192.168.99.102.1416 > 192.168.132.2.5001: . 15929:17377(1448) ack 1 win 17376 (DF) 17:34:16.533854 192.168.99.102.1416 > 192.168.132.2.5001: . 17377:18825(1448) ack 1 win 17376 (DF) 17:34:17.533793 192.168.132.2.5001 > 192.168.99.102.1416: . ack 17377 win 50318 (DF) 17:34:17.533846 192.168.99.102.1416 > 192.168.132.2.5001: . 18825:20273(1448) ack 1 win 17376 (DF) 17:34:17.624800 192.168.132.2.5001 > 192.168.99.102.1416: . ack 18825 win 50318 (DF) 17:34:17.624851 192.168.99.102.1416 > 192.168.132.2.5001: . 20273:21721(1448) ack 1 win 17376 (DF) 17:34:18.623755 192.168.132.2.5001 > 192.168.99.102.1416: . ack 21721 win 49956 (DF) 17:34:18.623882 192.168.99.102.1416 > 192.168.132.2.5001: . 21721:23169(1448) ack 1 win 17376 (DF) 17:34:18.623910 192.168.99.102.1416 > 192.168.132.2.5001: . 23169:24617(1448) ack 1 win 17376 (DF) 17:34:19.623854 192.168.132.2.5001 > 192.168.99.102.1416: . ack 23169 win 50318 (DF) 17:34:19.623913 192.168.99.102.1416 > 192.168.132.2.5001: . 24617:26065(1448) ack 1 win 17376 (DF) 17:34:19.714848 192.168.132.2.5001 > 192.168.99.102.1416: . ack 24617 win 50318 (DF) 17:34:19.714903 192.168.99.102.1416 > 192.168.132.2.5001: . 26065:27513(1448) ack 1 win 17376 (DF) 17:34:20.713755 192.168.132.2.5001 > 192.168.99.102.1416: . ack 27513 win 49956 (DF) 17:34:20.713810 192.168.99.102.1416 > 192.168.132.2.5001: . 27513:28961(1448) ack 1 win 17376 (DF) 17:34:20.713831 192.168.99.102.1416 > 192.168.132.2.5001: . 28961:30409(1448) ack 1 win 17376 (DF) 17:34:21.713862 192.168.132.2.5001 > 192.168.99.102.1416: . ack 28961 win 50318 (DF) 17:34:21.713923 192.168.99.102.1416 > 192.168.132.2.5001: . 30409:31857(1448) ack 1 win 17376 (DF) 17:34:21.811503 192.168.132.2.5001 > 192.168.99.102.1416: . ack 30409 win 50318 (DF) 17:34:21.811550 192.168.99.102.1416 > 192.168.132.2.5001: . 31857:33305(1448) ack 1 win 17376 (DF) 17:34:22.803753 192.168.132.2.5001 > 192.168.99.102.1416: . ack 33305 win 49956 (DF) 17:34:22.803813 192.168.99.102.1416 > 192.168.132.2.5001: . 33305:34753(1448) ack 1 win 17376 (DF) 17:34:22.803838 192.168.99.102.1416 > 192.168.132.2.5001: . 34753:36201(1448) ack 1 win 17376 (DF) 17:34:23.803873 192.168.132.2.5001 > 192.168.99.102.1416: . ack 34753 win 50318 (DF) 17:34:23.803979 192.168.99.102.1416 > 192.168.132.2.5001: . 36201:37649(1448) ack 1 win 17376 (DF) 17:34:23.894946 192.168.132.2.5001 > 192.168.99.102.1416: . ack 36201 win 50318 (DF) 17:34:23.894997 192.168.99.102.1416 > 192.168.132.2.5001: . 37649:39097(1448) ack 1 win 17376 (DF) --------------010601000702080005010400-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message