Date: Tue, 21 Sep 2004 10:41:12 +0200 From: Aragon Gouveia <aragon@phat.za.net> To: freebsd-net@freebsd.org Subject: Re: Wierd tunnel+MTU issue Message-ID: <20040921084112.GA21160@phat.za.net> In-Reply-To: <20040921090313.2c49f2c6.dudu@diaspar.rdsnet.ro> References: <20040920184431.GA89606@phat.za.net> <20040921090313.2c49f2c6.dudu@diaspar.rdsnet.ro>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, No, it's not that. No filtering is taking place. I've figured out the problem, but I'm not sure how to solve it. Here's what I think is the problem. >From a tcpdump transcript: 09:56:37.652907 <host>.4185 > <notebook>.80: S 487952620:487952620(0) win 57344 <mss 1460,nop,wscale 0,nop,nop,timestamp 611364250 0> (DF) [tos 0x10] 09:56:37.653076 <notebook>.80 > <host>.4185: S 4069940133:4069940133(0) ack 487952621 win 65535 <mss 1452,nop,wscale 1,nop,nop,timestamp 317780676 611364250> (DF) <notebook> is my notebook running Apache. As can be seen above, it's negotiating an MSS of 1452 with the peer, which it should not be doing. The reason it's doing that is because my default route is via an interface with an MTU of 1492 - the tun interface opened by userland ppp for the PPPoE session over my ADSL bridge. As I said, I'm using ipfw fwd to source route packets from <notebook> (the vtun tunnel interface address) to the vtun tunnel's remote end-point. But I'm guessing MSS is chosen based on the host's routing table. Which makes perfect sense. So to prove my suspicion I added a route on my notebook as follows: route add -host <host> 196.15.a.y 196.15.a.y being the vtun tunnel's remote end-point. Now the tcpdump transcript looks like this: 10:10:21.227506 <host>.2404 > <notebook>.80: S 996010957:996010957(0) win 57344 <mss 1460,nop,wscale 0,nop,nop,timestamp 3560266528 0> (DF) [tos 0x10] 10:10:21.227717 <notebook>.80 > <host>.2404: S 2935622965:2935622965(0) ack 996010958 win 65535 <mss 1204,nop,wscale 1,nop,nop,timestamp 318604232 3560266528> (DF) The tunnel's interface MTU was set at 1256 when I did this. So the negotiated MSS is now correct and things are working. But I need to be able to route based on source address and ipfw fwd is the only way I know how to do it. Can anyone think of a clever workaround for this? Is there a way to force the TCP stack to use a set MSS regardless of what the routing table and interface MTU say? Thanks, Aragon | By Vlad GALU <dudu@diaspar.rdsnet.ro> | [ 2004-09-21 08:02 +0200 ] > On Mon, 20 Sep 2004 20:44:31 +0200 > Aragon Gouveia <aragon@phat.za.net> wrote: > > It looks like your PMTUD doesn't work. Have you somehow disabled > incoming ICMP traffic to the box you've noticed these issues ? > > > Hi, > > > > A while ago I setup a vtun tunnel between a FreeBSD 4.10-RELEASE > > machine and a 5.2.1-RELEASE-p9 machine. Initially everything appeared > > to work great, but I've just stumbled upon a seriously wierd problem > > that I can't figure out. > > > > I know this is not a support forum for the vtun package, but the > > problem I'm having is consistently reproducable with another VPN type > > package - OpenVPN. I'm beginning to think the problem is not related > > to vtun/OpenVPN and was hoping someone could shed some light on my > > problem. > > > > My setup is as follows. > > > > My notebook is running 5.2.1-REALEASE-p9 using userland ppp to > > establish a PPPoE session over an ADSL bridge. Above that, it runs > > vtun to establish a UDP tunnel to my VPN server sitting at an ISP. > > > > The VPN server is the 4.10-RELEASE machine also running vtun 1.6 > > configured in server mode. The link is configured for no compression, > > but encryption enabled. > > > > Now the link comes up 100% and passes most data fine. I've set my > > interface MTU down to 1200 on both sides. I'm also source routing on > > my notebook with ipfw fwd to make sure traffic flows out the links I > > want them to flow. This is working 100% too. > > > > The problem is this: I'm running Apache 2.0.50 on my notebook. If I > > request a page from my notebook from an outside machine via the VPN, > > request responses that exceed the interface MTU are simply not sent. > > > > For example, if I request a file sized at 1100 bytes, this is the > > tcpdump transcript running on my notebook, sniffing the VPN interface: > > > > tcpdump: listening on tun1 > > 20:31:01.892060 196.15.a.z.3159 > 196.15.a.x.80: S > > 4115956342:4115956342(0) win 57344 <mss 1460,nop,wscale > > 0,nop,nop,timestamp 606530904 0> (DF) [tos 0x10] 20:31:01.892228 > > 196.15.a.x.80 > 196.15.a.z.3159: S 1941194202:1941194202(0) ack > > 4115956343 win 65535 <mss 1452,nop,wscale 1,nop,nop,timestamp > > 269446043 606530904> (DF) 20:31:01.974087 196.15.a.z.3159 > > > 196.15.a.x.80: . ack 1 win 57600 <nop,nop,timestamp 606530914 > > 269446043> (DF) [tos 0x10] 20:31:08.417478 196.15.a.z.3159 > > > 196.15.a.x.80: P 1:21(20) ack 1 win 57600 <nop,nop,timestamp 606531558 > > 269446043> (DF) [tos 0x10] 20:31:08.517285 196.15.a.x.80 > > > 196.15.a.z.3159: . ack 21 win 33120 <nop,nop,timestamp 269452670 > > 606531558> (DF) 20:31:10.340468 196.15.a.z.3159 > 196.15.a.x.80: P > > 21:23(2) ack 1 win 57600 <nop,nop,timestamp 606531751 269452670> (DF) > > [tos 0x10] 20:31:10.341371 196.15.a.x.80 > 196.15.a.z.3159: P > > 1:286(285) ack 23 win 33120 <nop,nop,timestamp 269454494 606531751> > > (DF) 20:31:10.341412 196.15.a.x.80 > 196.15.a.z.3159: P 286:1386(1100) > > ack 23 win 33120 <nop,nop,timestamp 269454494 606531751> (DF) > > 20:31:10.342143 196.15.a.x.80 > 196.15.a.z.3159: F 1386:1386(0) ack 23 > > win 33120 <nop,nop,timestamp 269454495 606531751> (DF) 20:31:10.568480 > > 196.15.a.z.3159 > 196.15.a.x.80: . ack 286 win 57600 > > <nop,nop,timestamp 606531773 269454494> (DF) [tos 0x10] > > 20:31:10.618594 196.15.a.z.3159 > 196.15.a.x.80: . ack 1387 win 57600 > > <nop,nop,timestamp 606531779 269454494> (DF) [tos 0x10] > > 20:31:10.626417 196.15.a.z.3159 > 196.15.a.x.80: F 23:23(0) ack 1387 > > win 57600 <nop,nop,timestamp 606531779 269454494> (DF) [tos 0x10] > > 20:31:10.626532 196.15.a.x.80 > 196.15.a.z.3159: . ack 24 win 33119 > > <nop,nop,timestamp 269454779 606531779> (DF) > > > > > > The two important lines being: > > > > 20:31:10.341371 196.15.a.x.80 > 196.15.a.z.3159: P 1:286(285) ack 23 > > win 33120 <nop,nop,timestamp 269454494 606531751> (DF) 20:31:10.341412 > > 196.15.a.x.80 > 196.15.a.z.3159: P 286:1386(1100) ack 23 win 33120 > > <nop,nop,timestamp 269454494 606531751> (DF) > > > > The first of these two is the HTTP response header, and the second the > > actual requested data (1100 bytes as shown). > > > > If I try request a file 1400 bytes large (MTU is 1200): > > > > tcpdump: listening on tun1 > > 20:35:06.588068 196.15.a.z.3161 > 196.15.a.x.80: S > > 1461068997:1461068997(0) win 57344 <mss 1460,nop,wscale > > 0,nop,nop,timestamp 606555375 0> (DF) [tos 0x10] 20:35:06.588242 > > 196.15.a.x.80 > 196.15.a.z.3161: S 1654337904:1654337904(0) ack > > 1461068998 win 65535 <mss 1452,nop,wscale 1,nop,nop,timestamp > > 269690732 606555375> (DF) 20:35:06.659998 196.15.a.z.3161 > > > 196.15.a.x.80: . ack 1 win 57600 <nop,nop,timestamp 606555381 > > 269690732> (DF) [tos 0x10] 20:35:10.490089 196.15.a.z.3161 > > > 196.15.a.x.80: P 1:21(20) ack 1 win 57600 <nop,nop,timestamp 606555765 > > 269690732> (DF) [tos 0x10] 20:35:10.589617 196.15.a.x.80 > > > 196.15.a.z.3161: . ack 21 win 33120 <nop,nop,timestamp 269694734 > > 606555765> (DF) 20:35:11.506613 196.15.a.z.3161 > 196.15.a.x.80: P > > 21:23(2) ack 1 win 57600 <nop,nop,timestamp 606555867 269694734> (DF) > > [tos 0x10] 20:35:11.507306 196.15.a.x.80 > 196.15.a.z.3161: P > > 1:286(285) ack 23 win 33120 <nop,nop,timestamp 269695651 606555867> > > (DF) 20:35:11.716698 196.15.a.z.3161 > 196.15.a.x.80: . ack 286 win > > 57600 <nop,nop,timestamp 606555888 269695651> (DF) [tos 0x10] > > 20:35:16.619379 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 > > win 57600 <nop,nop,timestamp 606556378 269695651> (DF) [tos 0x10] > > 20:35:17.815936 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 > > win 57600 <nop,nop,timestamp 606556498 269695651> (DF) [tos 0x10] > > 20:35:20.017123 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 > > win 57600 <nop,nop,timestamp 606556718 269695651> (DF) [tos 0x10] > > 20:35:24.227404 196.15.a.z.3161 > 196.15.a.x.80: F 23:23(0) ack 286 > > win 57600 <nop,nop,timestamp 606557138 269695651> (DF) [tos 0x10] > > > > > > The two important lines being: > > > > 20:35:11.507306 196.15.a.x.80 > 196.15.a.z.3161: P 1:286(285) ack 23 > > win 33120 <nop,nop,timestamp 269695651 606555867> (DF) 20:35:11.716698 > > 196.15.a.z.3161 > 196.15.a.x.80: . ack 286 win 57600 > > <nop,nop,timestamp 606555888 269695651> (DF) [tos 0x10] > > > > > > The first line of the important ones is the HTTP response header. The > > second one obviously just the TCP acknowledgement. The expected next > > few packets that should follow the header never get sent. > > > > I'm stumped. As I said, I've tried OpenVPN as well but the behaviour > > is precisely the same. Does anyone have any ideas? > > > > > > Thanks, > > Aragon > > _______________________________________________ > > freebsd-net@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > > ---- > If it's there, and you can see it, it's real. > If it's not there, and you can see it, it's virtual. > If it's there, and you can't see it, it's transparent. > If it's not there, and you can't see it, you erased it. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040921084112.GA21160>