Date: Fri, 18 Oct 2002 14:20:07 -0700 From: Stephen Macmanus <stephenm@windriver.com> To: cjclark@alum.mit.edu Cc: net@freebsd.org, stephenm@mail.wrs.com Subject: Re: Strange FTP TCP Window Problem Message-ID: <200210182119.OAA25087@mail.wrs.com> In-Reply-To: Your message of "Fri, 18 Oct 2002 10:10:55 PDT." <20021018171055.GA45449@blossom.cjclark.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This type of behavior typically indicates a problem with the
application itself, above the socket layer, not TCP or the
other protocols. The window size continually decreases as
data arrives since TCP is storing the incoming data in the
receive buffer, but the target application is not reading it.
I have seen cases where this behavior occurs due to subtle
failures like CPU starvation due to scheduling priorities
or incorrect mutual exclusion.
I have also encountered simpler causes, such as creating a
socket for the purpose of sending data that uses a port which
also matches incoming data. In that case, the application simply
never read the unexpected data from the socket. Setting the
receive buffer to zero allowed TCP to discard the data and avoid
the problem. This possibility may not apply in your case, since the
client successfully handles other file transfers, but in my experience
the application is the most likely cause of the problem.
Stephen
--
------------------
Stephen Macmanus #include <std_disclaimer.h>
Technical Staff
Wind River Networks stephenm@wrs.com
> This has got me stumped. I must be missing something obvious. I am
> trying to download some files via FTP. There are four files. Three
> download just fine, one just won't go. Since the three work fine, I
> assume it's not PORT vs. PASV, NAT, or other firewall issues, that is,
> the usual suspects have been eliminated. But what is really strange is
> that it looks like _my client_ is the one causing the headaches.
>
> Here is a tcpdump(8) of the data connection. Keep an eye on the window
> values my client is sending back,
>
> 09:15:11.097800 a.b.c.136.29826 > d.e.f.17.18747: S [tcp sum ok] 2410623756:2410623756(0) win 16384 <mss 1460,nop,nop,sackOK,nop,wscale 0,nop,nop,timestamp 6027020 0> (ttl 64, id 50031, len 64)
> 09:15:11.118335 d.e.f.17.18747 > a.b.c.136.29826: S [tcp sum ok] 2932757104:2932757104(0) ack 2410623757 win 16384 <mss 512,nop,wscale 0,nop,nop,timestamp 1679619 6027020> (ttl 56, id 24108, len 60)
> 09:15:11.118443 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 1 win 16500 <nop,nop,timestamp 6027020 1679619> (ttl 64, id 43640, len 52)
> 09:15:11.156424 d.e.f.17.18747 > a.b.c.136.29826: . 1:501(500) ack 1 win 16384 <nop,nop,timestamp 1679619 6027020> [tos 0x8] (ttl 56, id 24111, len 552)
> 09:15:11.159468 d.e.f.17.18747 > a.b.c.136.29826: . 501:1001(500) ack 1 win 16384 <nop,nop,timestamp 1679619 6027020> [tos 0x8] (ttl 56, id 24112, len 552)
> 09:15:11.227371 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 1001 win 15500 <nop,nop,timestamp 6027020 1679619> [tos 0x8] (ttl 64, id 61006, len 52)
> 09:15:11.258205 d.e.f.17.18747 > a.b.c.136.29826: . 1001:1501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24113, len 552)
> 09:15:11.261225 d.e.f.17.18747 > a.b.c.136.29826: . 1501:2001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24114, len 552)
> 09:15:11.279923 d.e.f.17.18747 > a.b.c.136.29826: . 2001:2501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24115, len 552)
> 09:15:11.427395 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 2501 win 14000 <nop,nop,timestamp 6027021 1679620> [tos 0x8] (ttl 64, id 40196, len 52)
> 09:15:11.457865 d.e.f.17.18747 > a.b.c.136.29826: . 2501:3001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24116, len 552)
> 09:15:11.461105 d.e.f.17.18747 > a.b.c.136.29826: . 3001:3501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24117, len 552)
> 09:15:11.480485 d.e.f.17.18747 > a.b.c.136.29826: . 3501:4001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24118, len 552)
> 09:15:11.483702 d.e.f.17.18747 > a.b.c.136.29826: . 4001:4501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24119, len 552)
> 09:15:11.627707 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 4501 win 12000 <nop,nop,timestamp 6027021 1679620> [tos 0x8] (ttl 64, id 44302, len 52)
> 09:15:11.648692 d.e.f.17.18747 > a.b.c.136.29826: . 4501:5001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24120, len 552)
> 09:15:11.659731 d.e.f.17.18747 > a.b.c.136.29826: . 5001:5501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24121, len 552)
> 09:15:11.670448 d.e.f.17.18747 > a.b.c.136.29826: . 5501:6001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24122, len 552)
> 09:15:11.673581 d.e.f.17.18747 > a.b.c.136.29826: . 6001:6501(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24123, len 552)
> 09:15:11.692780 d.e.f.17.18747 > a.b.c.136.29826: . 6501:7001(500) ack 1 win 16384 <nop,nop,timestamp 1679620 6027020> [tos 0x8] (ttl 56, id 24124, len 552)
> 09:15:11.827393 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 7001 win 9500 <nop,nop,timestamp 6027021 1679620> [tos 0x8] (ttl 64, id 38068, len 52)
> 09:15:11.855900 d.e.f.17.18747 > a.b.c.136.29826: . 7001:7501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24125, len 552)
> 09:15:11.860077 d.e.f.17.18747 > a.b.c.136.29826: . 7501:8001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24126, len 552)
> 09:15:11.878834 d.e.f.17.18747 > a.b.c.136.29826: . 8001:8501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24127, len 552)
> 09:15:11.881856 d.e.f.17.18747 > a.b.c.136.29826: . 8501:9001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24128, len 552)
> 09:15:11.892382 d.e.f.17.18747 > a.b.c.136.29826: . 9001:9501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24129, len 552)
> 09:15:11.903415 d.e.f.17.18747 > a.b.c.136.29826: . 9501:10001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24130, len 552)
> 09:15:12.027385 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 10001 win 6500 <nop,nop,timestamp 6027022 1679621> [tos 0x8] (ttl 64, id 60342, len 52)
> 09:15:12.052757 d.e.f.17.18747 > a.b.c.136.29826: . 10001:10501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24131, len 552)
> 09:15:12.063589 d.e.f.17.18747 > a.b.c.136.29826: . 10501:11001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24132, len 552)
> 09:15:12.074515 d.e.f.17.18747 > a.b.c.136.29826: . 11001:11501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24133, len 552)
> 09:15:12.078151 d.e.f.17.18747 > a.b.c.136.29826: . 11501:12001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24134, len 552)
> 09:15:12.096930 d.e.f.17.18747 > a.b.c.136.29826: . 12001:12501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24135, len 552)
> 09:15:12.099980 d.e.f.17.18747 > a.b.c.136.29826: . 12501:13001(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24136, len 552)
> 09:15:12.110647 d.e.f.17.18747 > a.b.c.136.29826: . 13001:13501(500) ack 1 win 16384 <nop,nop,timestamp 1679621 6027020> [tos 0x8] (ttl 56, id 24137, len 552)
> 09:15:12.227383 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 13501 win 3000 <nop,nop,timestamp 6027022 1679621> [tos 0x8] (ttl 64, id 51576, len 52)
> 09:15:12.264837 d.e.f.17.18747 > a.b.c.136.29826: . 13501:14001(500) ack 1 win 16384 <nop,nop,timestamp 1679622 6027020> [tos 0x8] (ttl 56, id 24138, len 552)
> 09:15:12.268095 d.e.f.17.18747 > a.b.c.136.29826: . 14001:14501(500) ack 1 win 16384 <nop,nop,timestamp 1679622 6027020> [tos 0x8] (ttl 56, id 24139, len 552)
> 09:15:12.278653 d.e.f.17.18747 > a.b.c.136.29826: . 14501:15001(500) ack 1 win 16384 <nop,nop,timestamp 1679622 6027020> [tos 0x8] (ttl 56, id 24140, len 552)
> 09:15:12.289849 d.e.f.17.18747 > a.b.c.136.29826: . 15001:15501(500) ack 1 win 16384 <nop,nop,timestamp 1679622 6027020> [tos 0x8] (ttl 56, id 24141, len 552)
> 09:15:12.293592 d.e.f.17.18747 > a.b.c.136.29826: . 15501:16001(500) ack 1 win 16384 <nop,nop,timestamp 1679622 6027020> [tos 0x8] (ttl 56, id 24142, len 552)
> 09:15:12.427402 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16001 win 500 <nop,nop,timestamp 6027023 1679622> [tos 0x8] (ttl 64, id 41001, len 52)
> 09:15:17.202100 d.e.f.17.18747 > a.b.c.136.29826: . 16001:16501(500) ack 1 win 16384 <nop,nop,timestamp 1679631 6027020> [tos 0x8] (ttl 56, id 24157, len 552)
> 09:15:17.227476 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027032 1679631> [tos 0x8] (ttl 64, id 59052, len 52)
> 09:15:22.186435 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679641 6027020> [tos 0x8] (ttl 56, id 24183, len 53)
> 09:15:22.186625 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027042 1679641> [tos 0x8] (ttl 64, id 55942, len 52)
> 09:15:30.194393 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679657 6027020> [tos 0x8] (ttl 56, id 24221, len 53)
> 09:15:30.195913 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027058 1679657> [tos 0x8] (ttl 64, id 14020, len 52)
> 09:15:46.198335 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679689 6027020> [tos 0x8] (ttl 56, id 24333, len 53)
> 09:15:46.198583 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027090 1679689> [tos 0x8] (ttl 64, id 16087, len 52)
> 09:16:18.201164 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679753 6027020> [tos 0x8] (ttl 56, id 24500, len 53)
> 09:16:18.201409 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027154 1679753> [tos 0x8] (ttl 64, id 30338, len 52)
> 09:17:18.194176 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679873 6027020> [tos 0x8] (ttl 56, id 25061, len 53)
> 09:17:18.194430 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027274 1679873> [tos 0x8] (ttl 64, id 20615, len 52)
> 09:18:18.193763 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1679993 6027020> [tos 0x8] (ttl 56, id 25447, len 53)
> 09:18:18.194005 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027394 1679993> [tos 0x8] (ttl 64, id 12107, len 52)
> 09:19:18.186418 d.e.f.17.18747 > a.b.c.136.29826: . [tcp sum ok] 16501:16502(1) ack 1 win 16384 <nop,nop,timestamp 1680113 6027020> [tos 0x8] (ttl 56, id 25862, len 53)
> 09:19:18.187970 a.b.c.136.29826 > d.e.f.17.18747: . [tcp sum ok] 1:1(0) ack 16501 win 0 <nop,nop,timestamp 6027514 1680113> [tos 0x8] (ttl 64, id 51084, len 52)
>
> And this just hangs here forever.
>
> I can't see why the client is refusing to accept more data. Like I
> said, I'll bet I'm not seeing something obvious, but all of the SEQ
> and ACK numbers look good to me. I should note that this isn't just an
> issue with the FreeBSD ftp client. I get the same result with Windows,
> Solaris, Cygwin, and OpenBSD too.
>
> Why is it happening and how do I get around this?
> --
> Crist J. Clark | cjclark@alum.mit.edu
> | cjclark@jhu.edu
> http://people.freebsd.org/~cjc/ | cjc@freebsd.org
>
> To Unsubscribe: send mail to majordomo@FreeBSD.org
> with "unsubscribe freebsd-net" in the body of the message
--
------------------
Stephen Macmanus #include <std_disclaimer.h>
Technical Staff
Wind River Networks stephenm@wrs.com
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200210182119.OAA25087>