Date: Sat, 18 Nov 2006 20:02:48 -0600 From: Dan Nelson <dnelson@allantgroup.com> To: Dieter <freebsd@sopwith.solgatos.com> Cc: freebsd-questions@freebsd.org Subject: Re: TCP parameters and interpreting tcpdump output Message-ID: <20061119020247.GB15898@dan.emsphone.com> In-Reply-To: <200611182020.UAA03897@sopwith.solgatos.com> References: <200611182020.UAA03897@sopwith.solgatos.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In the last episode (Nov 18), Dieter said: > In the tcpdump output below, the src machine is sending data to the > bsd machine. At one point during this test, the bsd machine is > slowly falling behind, as shown in the smaller and smaller window > size. > > It looks like at one point, the bsd machine takes 5.5 seconds to ack > a packet. :-( Am I interpreting the -ttt delta time correctly? A shrinking window and no packet loss is an indication that the program the socket is connected to isn't reading data fast enough. If you're locally gzipping the output of a remote backup, for example, you'll see this. The receive window on the BSD box shrunk to 1051 free bytes, the sender decided not to send a partial packet, 5.5 seconds later the process on the BSD box finally read some data, and the kernel sent a couple of window update acks to the sending box letting it know to send more data. > Getsockopt() says bsd machine's send buffer = 33580, rec buffer = 197100 > Is there a way for the bsd machine to find out what the src machine's > send buffer size is? I doubt that it is large enough for 5.5 seconds' > worth of data, but it would be nice to know what the goal is. > > Towards the end of the log, it looks to me like both sides are a > bit quick to resend data and acks? Those acks are being resent because from the BSD machine's point of view, the sender has skipped some data starting at 1281784, so it'll keep asking for that byte until the sender sends it (which it eventually does). The receiver usually sends an ack for every other full-size incoming data packet, but I'm pretty sure when there's loss every one gets an ack (whether the incoming packet increments the receivers sequence number or not). The completely duplicated data packets from the sender, even before any perceived packet loss, are troubling. Either the sender decided to resend that data on its own, or the packet was duplicated by a router or switch in transit. Dumps of the same stream from both sender and receiver would help, as would enabling rfc 1323 extensions on both systems (which will put a timestamp value on each packet and enable SACK. It's enabled by default on FreeBSD). > 000016 IP bsd.12340 > src.rfe: . ack 1201508 win 65535 > 000641 IP src.rfe > bsd.12340: P 1201508:1202824(1316) ack 1 win 4096 > 000780 IP src.rfe > bsd.12340: P 1202824:1204140(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1204140 win 65535 > 000953 IP src.rfe > bsd.12340: P 1204140:1205456(1316) ack 1 win 4096 > 000938 IP src.rfe > bsd.12340: P 1205456:1206772(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1206772 win 65535 > 000640 IP src.rfe > bsd.12340: P 1206772:1208088(1316) ack 1 win 4096 > 000781 IP src.rfe > bsd.12340: P 1208088:1209404(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1209404 win 62903 > 001110 IP src.rfe > bsd.12340: P 1209404:1210720(1316) ack 1 win 4096 > 000780 IP src.rfe > bsd.12340: P 1210720:1212036(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1212036 win 60271 > 000641 IP src.rfe > bsd.12340: P 1212036:1213352(1316) ack 1 win 4096 > 000782 IP src.rfe > bsd.12340: P 1213352:1214668(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1214668 win 57639 > 000953 IP src.rfe > bsd.12340: P 1214668:1215984(1316) ack 1 win 4096 > 000941 IP src.rfe > bsd.12340: P 1215984:1217300(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1217300 win 55007 > 000952 IP src.rfe > bsd.12340: P 1217300:1218616(1316) ack 1 win 4096 > 000781 IP src.rfe > bsd.12340: P 1218616:1219932(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1219932 win 52375 > 000798 IP src.rfe > bsd.12340: P 1219932:1221248(1316) ack 1 win 4096 > 000794 IP src.rfe > bsd.12340: P 1221248:1222564(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1222564 win 49743 > 000646 IP src.rfe > bsd.12340: P 1222564:1223880(1316) ack 1 win 4096 > 000933 IP src.rfe > bsd.12340: P 1223880:1225196(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1225196 win 47111 > 000954 IP src.rfe > bsd.12340: P 1225196:1226512(1316) ack 1 win 4096 > 000625 IP src.rfe > bsd.12340: P 1226512:1227828(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1227828 win 44479 > 000798 IP src.rfe > bsd.12340: P 1227828:1229144(1316) ack 1 win 4096 > 000936 IP src.rfe > bsd.12340: P 1229144:1230460(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1230460 win 41847 > 000953 IP src.rfe > bsd.12340: P 1230460:1231776(1316) ack 1 win 4096 > 000624 IP src.rfe > bsd.12340: P 1231776:1233092(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1233092 win 39215 > 000797 IP src.rfe > bsd.12340: P 1233092:1234408(1316) ack 1 win 4096 > 000780 IP src.rfe > bsd.12340: P 1234408:1235724(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1235724 win 36583 > 000953 IP src.rfe > bsd.12340: P 1235724:1237040(1316) ack 1 win 4096 > 000937 IP src.rfe > bsd.12340: P 1237040:1238356(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1238356 win 33951 > 000641 IP src.rfe > bsd.12340: P 1238356:1239672(1316) ack 1 win 4096 > 001092 IP src.rfe > bsd.12340: P 1239672:1240988(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1240988 win 31319 > 000796 IP src.rfe > bsd.12340: P 1240988:1242304(1316) ack 1 win 4096 > 000626 IP src.rfe > bsd.12340: P 1242304:1243620(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1243620 win 28687 > 000952 IP src.rfe > bsd.12340: P 1243620:1244936(1316) ack 1 win 4096 > 000783 IP src.rfe > bsd.12340: P 1244936:1246252(1316) ack 1 win 4096 > 000015 IP bsd.12340 > src.rfe: . ack 1246252 win 26055 > 000797 IP src.rfe > bsd.12340: P 1246252:1247568(1316) ack 1 win 4096 > 000937 IP src.rfe > bsd.12340: P 1247568:1248884(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1248884 win 23423 > 000797 IP src.rfe > bsd.12340: P 1248884:1250200(1316) ack 1 win 4096 > 000938 IP src.rfe > bsd.12340: P 1250200:1251516(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1251516 win 20791 > 000642 IP src.rfe > bsd.12340: P 1251516:1252832(1316) ack 1 win 4096 > 000782 IP src.rfe > bsd.12340: P 1252832:1254148(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1254148 win 18159 > 000797 IP src.rfe > bsd.12340: P 1254148:1255464(1316) ack 1 win 4096 > 001251 IP src.rfe > bsd.12340: P 1255464:1256780(1316) ack 1 win 4096 > 000014 IP bsd.12340 > src.rfe: . ack 1256780 win 15527 > 000640 IP src.rfe > bsd.12340: P 1256780:1258096(1316) ack 1 win 4096 > 000937 IP src.rfe > bsd.12340: P 1258096:1259412(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1259412 win 12895 > 000797 IP src.rfe > bsd.12340: P 1259412:1260728(1316) ack 1 win 4096 > 000945 IP src.rfe > bsd.12340: P 1260728:1262044(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1262044 win 10263 > 000640 IP src.rfe > bsd.12340: P 1262044:1263360(1316) ack 1 win 4096 > 000952 IP src.rfe > bsd.12340: P 1263360:1264676(1316) ack 1 win 4096 > 000014 IP bsd.12340 > src.rfe: . ack 1264676 win 7631 > 000796 IP src.rfe > bsd.12340: P 1264676:1265992(1316) ack 1 win 4096 > 000938 IP src.rfe > bsd.12340: P 1265992:1267308(1316) ack 1 win 4096 > 000011 IP bsd.12340 > src.rfe: . ack 1267308 win 4999 > 000801 IP src.rfe > bsd.12340: P 1267308:1268624(1316) ack 1 win 4096 > 000936 IP src.rfe > bsd.12340: P 1268624:1269940(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1269940 win 2367 > 000798 IP src.rfe > bsd.12340: P 1269940:1271256(1316) ack 1 win 4096 > 099460 IP bsd.12340 > src.rfe: . ack 1271256 win 1051 <-- window smaller than 1316 > 5. 537213 IP bsd.12340 > src.rfe: . ack 1271256 win 4096 <-- does this mean 5.5 seconds? :-( > 000027 IP bsd.12340 > src.rfe: . ack 1271256 win 31744 > 000026 IP bsd.12340 > src.rfe: . ack 1271256 win 57088 > 000024 IP bsd.12340 > src.rfe: . ack 1271256 win 65535 > 000333 IP src.rfe > bsd.12340: P 1271256:1272572(1316) ack 1 win 4096 > 000006 IP src.rfe > bsd.12340: P 1272572:1273888(1316) ack 1 win 4096 > 000034 IP bsd.12340 > src.rfe: . ack 1273888 win 65535 > 000113 IP src.rfe > bsd.12340: P 1273888:1275204(1316) ack 1 win 4096 > 000157 IP src.rfe > bsd.12340: P 1275204:1276520(1316) ack 1 win 4096 > 000003 IP src.rfe > bsd.12340: P 1271256:1272572(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1276520 win 65535 > 000008 IP bsd.12340 > src.rfe: . ack 1276520 win 65535 > 000134 IP src.rfe > bsd.12340: P 1272572:1273888(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1276520 win 65535 > 000144 IP src.rfe > bsd.12340: P 1273888:1275204(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1276520 win 65535 > 000145 IP src.rfe > bsd.12340: P 1275204:1276520(1316) ack 1 win 4096 > 000003 IP src.rfe > bsd.12340: P 1276520:1277836(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1276520 win 65535 > 000142 IP src.rfe > bsd.12340: P 1277836:1279152(1316) ack 1 win 4096 > 000013 IP bsd.12340 > src.rfe: . ack 1279152 win 65535 > 000152 IP src.rfe > bsd.12340: P 1279152:1280468(1316) ack 1 win 4096 > 000149 IP src.rfe > bsd.12340: P 1280468:1281784(1316) ack 1 win 4096 > 000003 IP src.rfe > bsd.12340: P 1276520:1277836(1316) ack 1 win 4096 <--- dup data? > 000012 IP bsd.12340 > src.rfe: . ack 1281784 win 65535 > 000007 IP bsd.12340 > src.rfe: . ack 1281784 win 65535 <--- a dup ack after only 7 usec? > 000135 IP src.rfe > bsd.12340: P 1277836:1279152(1316) ack 1 win 4096 <--- dup data even though it was acked? > 000013 IP bsd.12340 > src.rfe: . ack 1281784 win 65535 <--- another dup ack? > 000144 IP src.rfe > bsd.12340: P 1279152:1280468(1316) ack 1 win 4096 <--- another dup data? > 000003 IP src.rfe > bsd.12340: P 1280468:1281784(1316) ack 1 win 4096 <--- another dup data? > 000012 IP bsd.12340 > src.rfe: . ack 1281784 win 65535 <--- and another dup ack? > 000005 IP bsd.12340 > src.rfe: . ack 1281784 win 65535 <--- yet another dup ack? > 000137 IP src.rfe > bsd.12340: P 1281784:1283100(1316) ack 1 win 4096 > 000158 IP src.rfe > bsd.12340: P 1283100:1284416(1316) ack 1 win 4096 > 000012 IP bsd.12340 > src.rfe: . ack 1284416 win 65535 > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" -- Dan Nelson dnelson@allantgroup.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061119020247.GB15898>