From owner-freebsd-net@FreeBSD.ORG Fri Oct 27 19:03:49 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D8BC16A403 for ; Fri, 27 Oct 2006 19:03:49 +0000 (UTC) (envelope-from khetan@os.org.za) Received: from gauntlet.os.org.za (gauntlet.os.org.za [196.35.70.242]) by mx1.FreeBSD.org (Postfix) with ESMTP id A6BBF43D45 for ; Fri, 27 Oct 2006 19:03:44 +0000 (GMT) (envelope-from khetan@os.org.za) Received: from localhost (localhost [127.0.0.1]) by gauntlet.os.org.za (Postfix) with ESMTP id 0E8486794B for ; Fri, 27 Oct 2006 21:03:42 +0200 (SAST) X-Virus-Scanned: amavisd-new at os.org.za Received: from gauntlet.os.org.za ([127.0.0.1]) by localhost (gauntlet.os.org.za [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 0Fyi17RRTJ-x for ; Fri, 27 Oct 2006 21:03:35 +0200 (SAST) Received: from gauntlet.os.org.za (gauntlet.os.org.za [196.35.70.242]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: khetan) by gauntlet.os.org.za (Postfix) with ESMTP id D302D67941 for ; Fri, 27 Oct 2006 21:03:35 +0200 (SAST) Date: Fri, 27 Oct 2006 21:03:35 +0200 (SAST) From: Khetan Gajjar To: freebsd-net@freebsd.org Message-ID: <20061027203322.X2293@gauntlet.os.org.za> X-Alternate-From: Khetan Gajjar X-Mobile: +27 82 885 4047 X-URL: http://khetan.gajjar.co.za/ X-Attribute-1: BOFH X-Attribute-2: the righteous bastard with a finger on The Switch X-Message-flag: This message sponsored by Internet Solutions. X-PGP-KeyID: 0x806AD0D9 X-PGP-Fingerprint: 19 29 68 D5 74 2B 6E E5 1B 88 45 3B 29 0B 8A 27 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Subject: Path MTU discovery broken in IPSec X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 27 Oct 2006 19:03:49 -0000 Hi. Summary; searching for this problem revealed another query, but no solution - http://lists.freebsd.org/pipermail/freebsd-net/2005-July/007899.html Explanation; I'm experiencing a broken path MTU discovery problem between two hosts connecting with each other via IPSec transport mode, exasperated by the fact that the two hosts are more than 600ms apart in terms of network latency. Host 1 and Host 2 both run FreeBSD 6.1-stable, circa Sep 7. Host 1's IPsec config looks like /etc/ipsec.conf: flush; spdflush; spdadd x.x.x.x y.y.y.y any -P out ipsec esp/transport//require; spdadd y.y.y.y x.x.x.x any -P in ipsec esp/transport//require; and its network config looks like em0: flags=9843 mtu 1500 options=b inet6 fe80::212:3fff:feec:d1ce%em0 prefixlen 64 scopeid 0x1 inet x.x.x.x netmask 0xffffff00 broadcast x.x.x.255 ether 00:12:3f:ec:d1:ce media: Ethernet 100baseTX status: active lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 Host 2's IPsec config looks like /etc/ipsec.conf: flush; spdflush; spdadd x.x.x.x y.y.y.y any -P in ipsec esp/transport//require; spdadd y.y.y.y x.x.x.x any -P out ipsec esp/transport//require; and its network config looks like fxp0: flags=9843 mtu 1500 options=b inet6 fe80::202:b3ff:feeb:21db%fxp0 prefixlen 64 scopeid 0x1 inet y.y.y.y netmask 0xfffffff8 broadcast y.y.y.z ether 00:02:b3:eb:21:db media: Ethernet 10baseT/UTP status: active plip0: flags=108810 mtu 1500 lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 Both machines are running the same kernel configs and the same sysctl configs. The sysctl's in play are net.inet.icmp.icmplim=500 net.inet.ip.ttl=128 net.inet.raw.maxdgram=57344 net.inet.raw.recvspace=65535 net.inet.tcp.always_keepalive=1 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.icmp.drop_redirect=1 net.inet.icmp.log_redirect=1 net.inet.ip.redirect=0 net.inet6.ip6.redirect=0 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.icmp.bmcastecho=0 net.inet.icmp.maskrepl=0 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.tcp.recvspace=65535 net.inet.udp.recvspace=65535 net.inet.udp.maxdgram=57344 net.local.stream.recvspace=65535 net.local.stream.sendspace=65535 racoon does its thing, and the ipsec tunnels come up. I can ping both sides, and there are no ipfw rules running. Connectivity via ssh and nfs seems to work fine, as do DNS zone transfers (for very small zones). Connectivity from host 2 to host 1 works perfectly. From host 1 to host 2 however, TCP sessions break / stall / timeout. I've tried reducing the MTU sizes from the default 1500 to 1492 on both interfaces, and that makes no difference. Are there any suggestions or additional debugging that could assist in solving this problem ? Khetan Gajjar. -- khetan@os.org.za +27 82 885 4047