Date: Thu, 07 Feb 2002 22:22:20 +0530 From: "murthy kn" <knmurthy30@hotmail.com> To: silby@silby.com Cc: freebsd-net@FreeBSD.ORG, tinguely@web.cs.ndsu.nodak.edu Subject: Re: Duplicate Acks and Fast Retransmit Message-ID: <F15115fyy6EcpOSvjbe0000a4d6@hotmail.com>
next in thread | raw e-mail | index | archive | help
Hello all,
Thanks for the response.
I have attached the "tcpdump" collected at the receiver,
and "netstat -p tcp" at the sender as well as the receiver.
I run the tests on a isolated pair of machines (FreeBSD 4.3), connected
over a pair of back-to-back crossover cables and these
links are made to appear as a single aggregated link
using a "virtual aggregation device".
In order to "simulate" more reordering, I have a sysctl variable
controlling the "Fast Retransmit Threshold" and I set it to 1 and
do a TCP transfer from "sender" to "receiver".
1. In my understanding, this should make the "sending" TCP get into Fast
Retransmit mode each time it receives 2 identical acks (or 1 duplicate ack).
However, I am not seeing this behaviour and would like
to get your advice on if I am missing something.
2. WRT to the netstat output at the receiver and sender below,
given that receiver has seen N out-of-order packets and
the sender has received say M duplicate ACKs, is it not
possible to infer that there have been reorders and
hence there should be Fast Retransmit (assuming Retransmit
threshold is 1 and on the testbed I have indicated above,
AFAIK, issues packet loss are practically not there).
3. I am also interested in knowing, in general, as you have
indicated, why we cannot draw inferences from "netstat".
For eg, in my case, once I start packet capture on the
sender machine, I will not see the duplicate acks or
reordering - this has mostly to do with the fact that
reordering also depends on the load u put (amongst other things).
Since I have a virtual interface, I do not know if it is
possible to use another machine in a passive monitoring mode
to capture the packets on the "virtual" interface of this
mahine - as there are 2 wires and not 1. (Apart from using
2 other machines and kind of merging the 2 dumps, is there any other way to
capture packets/analyse TCP behaviour in this kind of scenarios??)
--------------------------------------------------------
retransmit threshold value : 1
--------------------------------------------------------
netstat output at sender
tcp:
4529 packets sent
4526 data packets (6553600 bytes)
-----------> 0 data packets (0 bytes) retransmitted
0 resends initiated by MTU discovery
2 ack-only packets (0 delayed)
0 URG only packets
0 window probe packets
0 window update packets
1 control packet
4465 packets received
2262 acks (for 6553601 bytes)
------> 508 duplicate acks
0 acks for unsent data
0 packets (0 bytes) received in-sequence
0 completely duplicate packets (0 bytes)
0 old duplicate packets
0 packets with some dup. data (0 bytes duped)
1 out-of-order packet (0 bytes)
0 packets (0 bytes) of data after window
0 window probes
1657 window update packets
0 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
1 connection request
0 connection accepts
0 bad connection attempts
0 listen queue overflows
1 connection established (including accepts)
6 connections closed (including 0 drops)
1 connection updated cached RTT on close
1 connection updated cached RTT variance on close
0 connections updated cached ssthresh on close
0 embryonic connections dropped
2262 segments updated rtt (of 1723 attempts)
------> 0 retransmit timeouts
0 connections dropped by rexmit timeout
0 persist timeouts
0 connections dropped by persist timeout
0 keepalive timeouts
0 keepalive probes sent
0 connections dropped by keepalive
34 correct ACK header predictions
0 correct data packet header predictions
------------------------------------------------------------------
netstat output at receiver
tcp:
4465 packets sent
0 data packets (0 bytes)
0 data packets (0 bytes) retransmitted
0 resends initiated by MTU discovery
2780 ack-only packets (0 delayed)
0 URG only packets
0 window probe packets
1684 window update packets
1 control packet
4529 packets received
2 acks (for 2 bytes)
0 duplicate acks
0 acks for unsent data
3492 packets (5056416 bytes) received in-sequence
0 completely duplicate packets (0 bytes)
0 old duplicate packets
0 packets with some dup. data (0 bytes duped)
1034 out-of-order packets (1497184 bytes)
0 packets (0 bytes) of data after window
0 window probes
0 window update packets
0 packets received after close
0 discarded for bad checksums
0 discarded for bad header offset fields
0 discarded because packet too short
0 connection requests
1 connection accept
0 bad connection attempts
0 listen queue overflows
1 connection established (including accepts)
3 connections closed (including 0 drops)
0 connections updated cached RTT on close
0 connections updated cached RTT variance on close
0 connections updated cached ssthresh on close
0 embryonic connections dropped
2 segments updated rtt (of 2 attempts)
0 retransmit timeouts
0 connections dropped by rexmit timeout
0 persist timeouts
0 connections dropped by persist timeout
0 keepalive timeouts
0 keepalive probes sent
0 connections dropped by keepalive
0 correct ACK header predictions
3491 correct data packet header predictions
-------------------------------------------------------
tcpdump at the reciever
10:16:02.495530 10.10.10.1.1024 > 10.10.10.2.5001: S
3330616029:3330616029(0) win 65535 <mss 1460,nop,wscale 3,nop,nop,timestamp
103897 0> (DF)
10:16:02.496078 10.10.10.2.5001 > 10.10.10.1.1024: S
3041789324:3041789324(0) ack 3330616030 win 65535 <mss 1460,nop,wscale
1,nop,nop,timestamp 99398 103897> (DF)
10:16:02.496209 10.10.10.1.1024 > 10.10.10.2.5001: . ack 1 win 32942
<nop,nop,timestamp 103897 99398> (DF)
10:16:02.503477 10.10.10.1.1024 > 10.10.10.2.5001: . 1:1449(1448) ack 1 win
32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503501 10.10.10.1.1024 > 10.10.10.2.5001: . 1449:2897(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503553 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2897 win 31856
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.503613 10.10.10.1.1024 > 10.10.10.2.5001: . 2897:4345(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503629 10.10.10.1.1024 > 10.10.10.2.5001: . 4345:5793(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503678 10.10.10.2.5001 > 10.10.10.1.1024: . ack 5793 win 30408
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.503718 10.10.10.1.1024 > 10.10.10.2.5001: . 5793:7241(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503760 10.10.10.1.1024 > 10.10.10.2.5001: . 7241:8689(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503797 10.10.10.2.5001 > 10.10.10.1.1024: . ack 8689 win 29684
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.503841 10.10.10.1.1024 > 10.10.10.2.5001: . 8689:10137(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503869 10.10.10.1.1024 > 10.10.10.2.5001: . 10137:11585(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503904 10.10.10.2.5001 > 10.10.10.1.1024: . ack 11585 win 28960
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.503964 10.10.10.1.1024 > 10.10.10.2.5001: . 11585:13033(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.503993 10.10.10.1.1024 > 10.10.10.2.5001: . 13033:14481(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.504029 10.10.10.2.5001 > 10.10.10.1.1024: . ack 14481 win 29684
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504091 10.10.10.1.1024 > 10.10.10.2.5001: . 14481:15929(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99398> (DF)
10:16:02.504116 10.10.10.1.1024 > 10.10.10.2.5001: . 15929:17377(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504153 10.10.10.2.5001 > 10.10.10.1.1024: . ack 17377 win 30408
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504213 10.10.10.1.1024 > 10.10.10.2.5001: . 17377:18825(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504240 10.10.10.1.1024 > 10.10.10.2.5001: . 18825:20273(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504275 10.10.10.2.5001 > 10.10.10.1.1024: . ack 20273 win 31132
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504310 10.10.10.2.5001 > 10.10.10.1.1024: . ack 20273 win 33304
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504339 10.10.10.1.1024 > 10.10.10.2.5001: . 20273:21721(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504361 10.10.10.1.1024 > 10.10.10.2.5001: . 21721:23169(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504401 10.10.10.2.5001 > 10.10.10.1.1024: . ack 23169 win 31856
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504437 10.10.10.2.5001 > 10.10.10.1.1024: . ack 23169 win 33304
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504462 10.10.10.1.1024 > 10.10.10.2.5001: . 23169:24617(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504500 10.10.10.1.1024 > 10.10.10.2.5001: . 24617:26065(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504548 10.10.10.2.5001 > 10.10.10.1.1024: . ack 26065 win 31856
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504579 10.10.10.1.1024 > 10.10.10.2.5001: . 26065:27513(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504624 10.10.10.1.1024 > 10.10.10.2.5001: . 27513:28961(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504660 10.10.10.2.5001 > 10.10.10.1.1024: . ack 28961 win 31132
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504704 10.10.10.1.1024 > 10.10.10.2.5001: . 28961:30409(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504715 10.10.10.2.5001 > 10.10.10.1.1024: . ack 28961 win 33304
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504731 10.10.10.1.1024 > 10.10.10.2.5001: . 30409:31857(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
10:16:02.504774 10.10.10.2.5001 > 10.10.10.1.1024: . ack 31857 win 31856
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504810 10.10.10.2.5001 > 10.10.10.1.1024: . ack 31857 win 33304
<nop,nop,timestamp 99399 103898> (DF)
10:16:02.504827 10.10.10.1.1024 > 10.10.10.2.5001: . 31857:33305(1448) ack 1
win 32942 <nop,nop,timestamp 103898 99399> (DF)
............
PS : For the fear of size of the mail, I am removing some of the portions of
the dump (which I feel might not be very useful and hence
skipping to the "interesting" portion which I have commented
below. Hope this is not a problem. I can post the full dump if required).
................
10:16:02.513320 10.10.10.1.1024 > 10.10.10.2.5001: . 231681:233129(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513330 10.10.10.2.5001 > 10.10.10.1.1024: . ack 231681 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513344 10.10.10.1.1024 > 10.10.10.2.5001: P 233129:234577(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513387 10.10.10.2.5001 > 10.10.10.1.1024: . ack 234577 win 31856
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513422 10.10.10.2.5001 > 10.10.10.1.1024: . ack 234577 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513443 10.10.10.1.1024 > 10.10.10.2.5001: . 234577:236025(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513473 10.10.10.1.1024 > 10.10.10.2.5001: . 236025:237473(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513505 10.10.10.2.5001 > 10.10.10.1.1024: . ack 237473 win 31856
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513534 10.10.10.2.5001 > 10.10.10.1.1024: . ack 237473 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513580 10.10.10.1.1024 > 10.10.10.2.5001: . 237473:238921(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513599 10.10.10.1.1024 > 10.10.10.2.5001: P 238921:240369(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513647 10.10.10.2.5001 > 10.10.10.1.1024: . ack 240369 win 31856
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513687 10.10.10.1.1024 > 10.10.10.2.5001: . 240369:241817(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513700 10.10.10.2.5001 > 10.10.10.1.1024: . ack 240369 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513714 10.10.10.1.1024 > 10.10.10.2.5001: . 241817:243265(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513757 10.10.10.2.5001 > 10.10.10.1.1024: . ack 243265 win 31856
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513791 10.10.10.2.5001 > 10.10.10.1.1024: . ack 243265 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513841 10.10.10.1.1024 > 10.10.10.2.5001: P 244713:246161(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513889 10.10.10.2.5001 > 10.10.10.1.1024: . ack 243265 win 33304
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.513905 10.10.10.1.1024 > 10.10.10.2.5001: . 243265:244713(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.513944 10.10.10.2.5001 > 10.10.10.1.1024: . ack 246161 win 31856
<nop,nop,timestamp 99400 103899> (DF)
Above is an example of one of the many "out-of-order" segments
got at the receiver (indicated by the "netstat -p tcp" at receiver
and the consequent "Dup" Ack generated by receiver and received at
the sender (netstat -p tcp at the sender)
10:16:02.513966 10.10.10.1.1024 > 10.10.10.2.5001: P 247609:249057(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.514015 10.10.10.2.5001 > 10.10.10.1.1024: . ack 246161 win 31856
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.514030 10.10.10.1.1024 > 10.10.10.2.5001: . 246161:247609(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
Above is an example of one of the many "out-of-order" segments
got at the receiver (indicated by the "netstat -p tcp" at receiver
and the consequent "Dup" Ack generated by receiver and received at
the sender (netstat -p tcp at the sender)
10:16:02.514071 10.10.10.2.5001 > 10.10.10.1.1024: . ack 249057 win 30408
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.514091 10.10.10.1.1024 > 10.10.10.2.5001: . 250505:251953(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.514125 10.10.10.2.5001 > 10.10.10.1.1024: . ack 249057 win 30408
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.514143 10.10.10.1.1024 > 10.10.10.2.5001: . 249057:250505(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.514179 10.10.10.2.5001 > 10.10.10.1.1024: . ack 251953 win 28960
<nop,nop,timestamp 99400 103899> (DF)
10:16:02.514214 10.10.10.1.1024 > 10.10.10.2.5001: . 251953:253401(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.514226 10.10.10.1.1024 > 10.10.10.2.5001: P 253401:254849(1448) ack
1 win 32942 <nop,nop,timestamp 103899 99400> (DF)
10:16:02.514271 10.10.10.2.5001 > 10.10.10.1.1024: . ack 254849 win 27512
<nop,nop,timestamp 99400 103899> (DF)
..................
.................. Last few segments from the capture
10:16:02.647898 10.10.10.1.1024 > 10.10.10.2.5001: P 2213993:2215441(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.647941 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2215441 win 31856
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.647968 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2215441 win 33304
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648016 10.10.10.1.1024 > 10.10.10.2.5001: . 2216889:2218337(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648055 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2215441 win 33304
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648068 10.10.10.1.1024 > 10.10.10.2.5001: . 2215441:2216889(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648107 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2218337 win 31856
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648127 10.10.10.1.1024 > 10.10.10.2.5001: . 2218337:2219785(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648143 10.10.10.1.1024 > 10.10.10.2.5001: P 2219785:2221233(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648185 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2221233 win 30408
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648236 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2221233 win 33304
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648263 10.10.10.1.1024 > 10.10.10.2.5001: . 2221233:2222681(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648277 10.10.10.1.1024 > 10.10.10.2.5001: P 2222681:2224129(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648322 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2224129 win 31856
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648358 10.10.10.2.5001 > 10.10.10.1.1024: . ack 2224129 win 33304
<nop,nop,timestamp 99414 103912> (DF)
10:16:02.648378 10.10.10.1.1024 > 10.10.10.2.5001: . 2224129:2225577(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
10:16:02.648396 10.10.10.1.1024 > 10.10.10.2.5001: . 2225577:2227025(1448)
ack 1 win 32942 <nop,nop,timestamp 103912 99413> (DF)
Since I HAD to run the capture on the reciver, tcpdump reported
drops and some packets seem to have been dropped...
------------------------------------------------------------
Thanks in advance for all your help and advice.
Murthy
---------------------------------------------------------
>From: Mike Silbersack <silby@silby.com>
>To: murthy kn <knmurthy30@hotmail.com>
>CC: freebsd-net@FreeBSD.ORG
> > In order to understand the effect of reordering better,
>
>...
>
>If you really wish to understand how the tcp stack works better and / or
>fix bugs in it, you really need to use tcpdump to log what is actually
>passing over the wire. The statistics from netstat -s are only meant as
>an overview, and are not detailed enough to make inferences about how
>things are actually working.
>
>If you can show specific behaviors in a tcpdump that you're interested in,
>I'm sure that many people would be willing to help answer your questions.
>
>Mike "Silby" Silbersack
>
_________________________________________________________________
Chat with friends online, try MSN Messenger: http://messenger.msn.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?F15115fyy6EcpOSvjbe0000a4d6>