Date: Fri, 4 Mar 2005 18:26:18 GMT From: Stas <stas_k_freebsd@tiger.unisquad.com> To: freebsd-gnats-submit@FreeBSD.org Subject: bin/78424: Internal IPs on router, natd/libalias break PMTUD Message-ID: <200503041826.j24IQIjS028762@www.freebsd.org> Resent-Message-ID: <200503041830.j24IUNuD060290@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 78424 >Category: bin >Synopsis: Internal IPs on router, natd/libalias break PMTUD >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Mar 04 18:30:22 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Stas >Release: 4.10-RELEASE >Organization: >Environment: FreeBSD YYY.YYYYYYYY.YYY 4.10-RELEASE FreeBSD 4.10-RELEASE #0: Thu Aug 5 16:24: 31 EEST 2004 root@YYY.YYYYYYYY.YYY:/usr/src/sys/compile/YYY i386 >Description: Hi. There is a trouble when the following items are combined: 1. Path MTU Discovery 2. RFC1918 IPs for router 3. natd, and probably anything else linked with libalias Details: PC 10.1.1.2 ^ | V 10.1.1.1 Router ( has real IP X.X.X.X used for NAT) 192.168.1.2 ^ | V 192.168.1.1 Router2 ^ | Internet | V Server Y.Y.Y.Y The link between PC and router has low MTU, so when Server sends big packet to PC, Router sends ICMP Fragmentation Needed packet to Server. But since it has 192.168.1.2 on interface, it sends ICMP packet from this IP. 192.168.1.2 -> Y.Y.Y.Y ICMP: 10.1.1.2 unreachable: frag needed but DF set natd does not rewrite Router (192.168.1.2) address, only PC (10.1.1.2), this was done because of PR 20712 in lib/libalias/alias.c 1.23 Packet becomes: 192.168.1.2 -> Y.Y.Y.Y ICMP: X.X.X.X unreachable: frag needed but DF set Of course this packet gets dropped somewhere in the Net, because it has unrouteable source IP. And PMTUD gets broken: Server doesn't get "Fragmentation Needed" ICMP message, doesn't use lower packet size, connection times out,... I've modified libalias to have an exclusion for ICMP NEEDFRAG packets, and it seems to work for me. But my modification is very ugly, and you will want to make your own. OS is 4.10-RELEASE. Unfortunately, I can't install 4.11 or 5.3 on that router to check if this is fixed already. But I took a look at the last revision of alias.c and it looks to me that this behaviour was not modified. >How-To-Repeat: 1. Set up network pictured above, don't forget low MTU. 2. Connect from "PC" to "Server", and make Server to send big packet (eg connect to POP3 server and ask for E-mail of size >1500 bytes). >Fix: 1. Patch libalias. Mine is below, but it is ugly, and wont apply to recent libalias. It will probably need fixing tabs before applying. 2. Use real IPs for all routers, if you got them. --- lib/libalias/alias.c.orig Fri Mar 4 16:30:47 2005 +++ lib/libalias/alias.c Fri Mar 4 18:22:47 2005 @@ -632,6 +632,125 @@ return(PKT_ALIAS_IGNORED); } +static int +IcmpAliasOut3(struct ip *pip) +{ +/* + Alias outgoing ICMP error messages containing + IP header and first 64 bits of datagram. +*/ + struct ip *ip; + struct icmp *ic, *ic2; + struct udphdr *ud; + struct tcphdr *tc; + struct alias_link *link; + + ic = (struct icmp *) ((char *) pip + (pip->ip_hl << 2)); + ip = &ic->icmp_ip; + + ud = (struct udphdr *) ((char *) ip + (ip->ip_hl <<2)); + tc = (struct tcphdr *) ud; + ic2 = (struct icmp *) ud; + + if (ip->ip_p == IPPROTO_UDP) + link = FindUdpTcpOut(ip->ip_dst, ip->ip_src, + ud->uh_dport, ud->uh_sport, + IPPROTO_UDP, 0); + else if (ip->ip_p == IPPROTO_TCP) + link = FindUdpTcpOut(ip->ip_dst, ip->ip_src, + tc->th_dport, tc->th_sport, + IPPROTO_TCP, 0); + else if (ip->ip_p == IPPROTO_ICMP) { + if (ic2->icmp_type == ICMP_ECHO || ic2->icmp_type == ICMP_TSTAMP) + link = FindIcmpOut(ip->ip_dst, ip->ip_src, ic2->icmp_id, 0); + else + link = NULL; + } else + link = NULL; + + if (link != NULL) + { + if (ip->ip_p == IPPROTO_UDP || ip->ip_p == IPPROTO_TCP) + { + u_short *sptr; + int accumulate; + struct in_addr alias_address; + u_short alias_port; + + alias_address = GetAliasAddress(link); + alias_port = GetAliasPort(link); + +/* Adjust ICMP checksum */ + sptr = (u_short *) &(ip->ip_dst); + accumulate = *sptr++; + accumulate += *sptr; + sptr = (u_short *) &alias_address; + accumulate -= *sptr++; + accumulate -= *sptr; + accumulate += ud->uh_dport; + accumulate -= alias_port; + ADJUST_CHECKSUM(accumulate, ic->icmp_cksum); + +/* + * Alias address in IP header if it comes from the host + * the original TCP/UDP packet was destined for. + */ + //if (pip->ip_src.s_addr == ip->ip_dst.s_addr) { + DifferentialChecksum(&pip->ip_sum, + (u_short *) &alias_address, + (u_short *) &pip->ip_src, + 2); + pip->ip_src = alias_address; + //} + +/* Alias address and port number of original IP packet +fragment contained in ICMP data section */ + ip->ip_dst = alias_address; + ud->uh_dport = alias_port; + } + else if (ip->ip_p == IPPROTO_ICMP) + { + u_short *sptr; + int accumulate; + struct in_addr alias_address; + u_short alias_id; + + alias_address = GetAliasAddress(link); + alias_id = GetAliasPort(link); + +/* Adjust ICMP checksum */ + sptr = (u_short *) &(ip->ip_dst); + accumulate = *sptr++; + accumulate += *sptr; + sptr = (u_short *) &alias_address; + accumulate -= *sptr++; + accumulate -= *sptr; + accumulate += ic2->icmp_id; + accumulate -= alias_id; + ADJUST_CHECKSUM(accumulate, ic->icmp_cksum); + +/* + * Alias address in IP header if it comes from the host + * the original ICMP message was destined for. + */ + //if (pip->ip_src.s_addr == ip->ip_dst.s_addr) { + DifferentialChecksum(&pip->ip_sum, + (u_short *) &alias_address, + (u_short *) &pip->ip_src, + 2); + pip->ip_src = alias_address; + //} + +/* Alias address of original IP packet and sequence number of + embedded ICMP datagram */ + ip->ip_dst = alias_address; + ic2->icmp_id = alias_id; + } + return(PKT_ALIAS_OK); + } + return(PKT_ALIAS_IGNORED); +} + static int IcmpAliasOut(struct ip *pip) @@ -656,6 +775,10 @@ } break; case ICMP_UNREACH: + if (ic->icmp_code == 4) { + iresult = IcmpAliasOut3(pip); + break; + } case ICMP_SOURCEQUENCH: case ICMP_TIMXCEED: case ICMP_PARAMPROB: >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200503041826.j24IQIjS028762>