From owner-freebsd-stable@FreeBSD.ORG Wed Feb 13 13:01:02 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0157DC4E for ; Wed, 13 Feb 2013 13:01:02 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from qmta01.emeryville.ca.mail.comcast.net (qmta01.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:43:76:96:30:16]) by mx1.freebsd.org (Postfix) with ESMTP id DBEEED3 for ; Wed, 13 Feb 2013 13:01:01 +0000 (UTC) Received: from omta12.emeryville.ca.mail.comcast.net ([76.96.30.44]) by qmta01.emeryville.ca.mail.comcast.net with comcast id zovT1k00P0x6nqcA1p10Bx; Wed, 13 Feb 2013 13:01:00 +0000 Received: from koitsu.strangled.net ([67.180.84.87]) by omta12.emeryville.ca.mail.comcast.net with comcast id zp0z1k00L1t3BNj8Yp0z8R; Wed, 13 Feb 2013 13:01:00 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 34D4873A1C; Wed, 13 Feb 2013 05:00:59 -0800 (PST) Date: Wed, 13 Feb 2013 05:00:59 -0800 From: Jeremy Chadwick To: Eugene Grosbein Subject: Re: Unusual TCP/IP Packet Size Message-ID: <20130213130059.GA57337@icarus.home.lan> References: <96AE8BD1-79C2-4743-854F-B8386C54E4A1@lafn.org> <511B6B21.5030606@rdtc.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <511B6B21.5030606@rdtc.ru> User-Agent: Mutt/1.5.21 (2010-09-15) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1360760460; bh=bB4LtahistahWakz1xyRDtq0LftMJ3JflN4Tv68Vp7Y=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=YvXqvX6iJQNHIWOnBEWPmJzJeWo++JlfjFh87HoJIeIzhA7cBUW7n9+PBMFmG/WLR B+lMfGUDAx78CDs3qRPJunRUBBWXW+fRKV78X6fM4pL2gXERkvDViQ08CPDCpScoEk nX3SxNGqJV4OAbIqFMO8jCe19IKkB2vilT2SHTAUH4WCuklzbZgbcdHVBucia2PsOA 8EW3eeYfmS9AuR3fpGHJ1jSGnMG2bWzdj2qEd3AuWFlJzvidBXpnb4548uvUR/db2H wQPPBrsGPyFDsB5qziWnd3fi9cbGZehTT3LroEEVHyPTLGBG1jmBExp7gOe5AcxAdU gz8PM9Y67RCxg== Cc: freebsd-stable@freebsd.org, yongari@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Feb 2013 13:01:02 -0000 On Wed, Feb 13, 2013 at 05:29:53PM +0700, Eugene Grosbein wrote: > 13.02.2013 17:25, Doug Hardie ??????????: > > Monitoring a tcpdump between two systems, a FreeBSD 9.1 system has the following interface: > > > > msk0: flags=8843 metric 0 mtu 1500 > > options=c011b > > ether 00:11:2f:2a:c7:03 > > inet 10.0.1.199 netmask 0xffffff00 broadcast 10.0.1.255 > > inet6 fe80::211:2fff:fe2a:c703%msk0 prefixlen 64 scopeid 0x1 > > nd6 options=29 > > media: Ethernet autoselect (100baseTX ) > > status: active > > > > > > It sent the following packet: (data content abbreviated) > > > > 02:14:42.081617 IP 10.0.1.199.443 > 10.0.1.2.61258: Flags [P.], seq 930:4876, ack 846, win 1040, options [nop,nop,TS val 401838072 ecr 920110183], length 3946 > > 0x0000: 4500 0f9e ea89 4000 4006 2a08 0a00 01c7 E.....@.@.*..... > > 0x0010: 0a00 0102 01bb ef4a ece1 680b ae37 1bbc .......J..h..7.. > > 0x0020: 8018 0410 3407 0000 0101 080a 17f3 8ff8 ....4...??????. > > > > > > The indicated packet length is 3946 and the load of data shown is that size. The MTU on both interfaces is 1500. The receiving system received 3 packets. There is a router and switch between them. One of them fragmented that packet. This is part of a SSL/TLS exchange and one side or the other is hanging on this and just dropping the connection. I suspect the packet size is the issue. ssldump complains about the packet too and stops monitoring. Could this possibly be related to the hardware checksums? > > You have TSO enabled on the interface, so large outgoing TCP packet is pretty normal. > It will be split by the NIC. Disable TSO with ifconfig if it interferes with your ssldump. This is not the behaviour I see with em(4) on a 82573E with all defaults used (which includes TSO4). Note that Doug is using msk(4). I can provide packet captures on both ends of a LAN segment using both tcpdump (on the FreeBSD side) and Wireshark (on the Windows side) that show a difference in behaviour compared to what Doug sees. What I see on the FreeBSD side with tcpdump is repeated "bad-len 0" messages for payloads which are chunked or segmented as a result of TSO. I do not see a 1:1 ratio of "bad-len" entries to chunked payloads; I only see one "bad-len" entry for all chunks (up until the next ACK or PSH+ACK of course). The important part: I do not see captured TCP packets reporting a length greater than MTU (or MSS for that matter (remember: IP header + TCP header + MSS <= MTU). Also note for Doug: remember that if you're doing packet captures between two devices that have NAT involved, you may see different behaviour. Example case: 03:58:47.907582 IP 67.180.84.87.2983 > 206.125.172.42.80: Flags [.], ack 13419, win 64240, length 0 03:58:47.907649 IP 206.125.172.42.80 > 67.180.84.87.2983: Flags [.], seq 17799:19259, ack 292, win 1026, length 1460 03:58:47.907679 IP 206.125.172.42.80 > 67.180.84.87.2983: Flags [.], seq 19259:20719, ack 292, win 1026, length 1460 03:58:47.912546 IP 67.180.84.87.2983 > 206.125.172.42.80: Flags [.], ack 16339, win 64240, length 0 In the above example there's a Linux NAT router (67.180.84.87) with a client (192.168.1.50) behind it, talking to 206.125.172.42. MTU is 1500 (I obviously didn't include the initial SYN :-) ). -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |