Date: Mon, 11 Sep 2006 14:22:00 -0700 (PDT) From: Kelly Yancey <kbyanc@posi.net> To: Eugene Grosbein <eugen@grosbein.pp.ru> Cc: VANHULLEBUS Yvan <vanhu_bsd@zeninc.net>, net@freebsd.org Subject: Re: ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD Message-ID: <20060911131513.S27693@gateway.posi.net> In-Reply-To: <200609111341.k8BDfneZ020221@nkz.delikates-nk.ru> References: <200609111341.k8BDfneZ020221@nkz.delikates-nk.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 11 Sep 2006, Eugene Grosbein wrote: > > >Submitter-Id: current-users > >Originator: Eugene Grosbein > >Organization: Svyaz Service JSC > >Confidential: no > >Synopsis: ipsec with ipfw divert (not NAT) encodes a packet twice breaking PMTUD > >Severity: serious > >Priority: high > >Category: kern > >Class: sw-bug > >Release: FreeBSD 6.1-STABLE i386 > >Environment: > System: FreeBSD nkz.delikates-nk.ru 6.1-STABLE FreeBSD 6.1-STABLE #1: Thu Sep 7 13:31:53 KRAST 2006 root@nkz.delikates-nk.ru:/home/obj/home/src/sys/NKZ i386 > options IPDIVERT > options IPSEC > options IPSEC_ESP > > >Description: > When outgoing packet encoded due to corresponding IPSEC policy > is passed to divert socket (f.e. to ipacctd for accounting), > it is encoded second time with IPSEC then. Besides obvious > logic error, this also results in broken Path MTU Discovery. > > >How-To-Repeat: > > Use a kernel with options IPDIVERT, IPSEC, IPSEC_ESP > (my kernel also contains IPSEC_FILTERGIF, but this should not matter). > > Suppose there are two local nets numbered 192.168.1.0/24 > and 192.168.2.0/24, each has a FreeBSD router > (192.168.1.1 and 192.168.2.1). Routers make gif(4) tunnel between > and use IPSEC transport mode to encrypt its contents. > Their external IP addresses are 1.1.1.1 and 2.2.2.2 > Just FYI, when we implemented the enc interface for FreeBSD 4.10 for one of our products at work, we encountered a similar issue. The problem is that you need to add a flag to the sockaddr_in passed to the divert(4) consumer; when that consumer re-injects the packets into the network stack, ip_output() needs to check for the flag and goto skip_ipsec to avoid re-encapsulation. The next issue is that there is no room in the sockaddr_in structure for such a flag. We resorted to a hack (eventually, we re-implemented the divert interface to have its own sockaddr_div rather than overloading sockaddr_in, but that is another story). We stuck the flag indicating whether to skip IPsec encapsulation on input in the high bit of the first byte in the sin_zero array. This only works because natd(8) doesn't inspect the (partial) interface name stored in the sockaddr_in's sin_zero array. "Hack" doesn't really being to describe the hideousness of this workaround, but it will get the job done. It looks like the same effect might be able to be achieved by modifying natd to be able to set the policy on the divert socket to IPSEC_POLICY_NONE via the IP_IPSEC_POLICY socket option, but I don't know enough about that code path to say for sure. Good luck, Kelly -- Kelly Yancey - kbyanc@posi.net - kelly@nttmcl.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060911131513.S27693>