From owner-freebsd-ipfw@freebsd.org Sun Jun 10 21:00:08 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67B6D1004BBC for ; Sun, 10 Jun 2018 21:00:08 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 01BA9779C4 for ; Sun, 10 Jun 2018 21:00:08 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id B509E1004BBA; Sun, 10 Jun 2018 21:00:07 +0000 (UTC) Delivered-To: ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A38181004BB7 for ; Sun, 10 Jun 2018 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46754779B0 for ; Sun, 10 Jun 2018 21:00:07 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6CBDC20F66 for ; Sun, 10 Jun 2018 21:00:06 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w5AL06MB065059 for ; Sun, 10 Jun 2018 21:00:06 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w5AL06mq065054 for ipfw@FreeBSD.org; Sun, 10 Jun 2018 21:00:06 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201806102100.w5AL06mq065054@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: ipfw@FreeBSD.org Subject: Problem reports for ipfw@FreeBSD.org that need special attention Date: Sun, 10 Jun 2018 21:00:06 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Jun 2018 21:00:08 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 215875 | [ipfw] ipfw lookup tables do not support mbuf_tag 1 problems total for which you should take action. From owner-freebsd-ipfw@freebsd.org Wed Jun 13 17:16:13 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 99778101F175; Wed, 13 Jun 2018 17:16:13 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from mx.allycomm.com (mx.allycomm.com [138.68.30.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A5FB6EA4E; Wed, 13 Jun 2018 17:16:12 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from JKLETSKY1-MBP15.local (184-23-190-180.vpn.dynamic.sonic.net [184.23.190.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.allycomm.com (Postfix) with ESMTPSA id 5600428364; Wed, 13 Jun 2018 10:16:04 -0700 (PDT) From: Jeff Kletsky Subject: In-kernel NAT [ipfw] dropping large UDP return packets To: freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org Message-ID: Date: Wed, 13 Jun 2018 10:16:04 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 17:16:13 -0000 When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC tunnel to the T-Mobile provisioning servers, the reassembled, 4640-byte return packet is silently dropped by the in-kernel NAT, even though it "matches" the outbound packet from less than 100 ms prior. All other operations of the firewall seem to be functioning as expected. This includes iPhones using "WiFi Calling" which utilizes similar IPSEC connections to T-Mobile servers (though fragmentation has not been seen on those connections). The connection for the femto-cell can be handled by a Linux/netfilter NAT. Proper reassembly of the packet fragments within the firewall ("reass ip from any to any"), at the start of the rule set, prior to NAT, has been confirmed with ngtee and wireshark. Is there a known issue with large packets and in-kernel NAT? The only sysctl that I found that seemed related was the UDP timeout. For good measure I upped it to 30 (seconds), but that did not change the behavior. Are there known causes and/or resolutions for this behavior? Is there a way to be able to "monitor" the NAT table? (I didn't see anything obvious in the ipfw, natd, or libalias man pages.) Thanks! Jeff 11.1-RELEASE-p9 FreeBSD 11.1-RELEASE-p9 #0: Tue Apr  3 16:59:16 UTC 2018 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 Additional details at https://forums.freebsd.org/threads/in-kernel-nat-dropping-large-udp-return-packets.66262/ pcapng files and ipfw rule set available on request to freebsd {a} wagsky {.} com From owner-freebsd-ipfw@freebsd.org Wed Jun 13 17:22:58 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03839101FC67 for ; Wed, 13 Jun 2018 17:22:58 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9B0196F039 for ; Wed, 13 Jun 2018 17:22:57 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: by mail-qk0-x233.google.com with SMTP id c198-v6so1954349qkg.12 for ; Wed, 13 Jun 2018 10:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=cFC39+OMAiXZdH12Vuiljj0dcQXYmJ2GoHASfP6OwgA=; b=LZvzN/YmhgLdRIVcbYIcJNlUQX10p3pHDVP7cXevPt+RojTIMUh0Px3fuAAK9YcSyk 3K20+cE/63A8AQU/Km019dZV2Up0hYeUeIfhbybFoRk9gs4MqNnzKVmGVIPVDRff5neR ys43Dlp9rcHtExPgFZEIA1T54KQ3fz5UVwGk+RmrFhT0OEb1dRnUO7eFmgKFuBhuInJ1 Pz8CAolTan7OV7T8iRZk3ybccNXMIK7wh+T4rkbnqCjwHM6PqsjUmBZhb0yjhqKO46yI 5d9gAfBc0ALimUBQth1Xa/ETsUQrJK4OLIeGpVs/J7zUIQ3yMS3aF6QdDuwzBiS60YQo hyVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=cFC39+OMAiXZdH12Vuiljj0dcQXYmJ2GoHASfP6OwgA=; b=lEH+ejJozg8n803EHHneZlqdY7vWc1eFr5CNHabr1WihTBGIPYnlhhyUfSj7rLOGji D5ZqT+tp7DwNXsxWa6YQ4NmEsmsBw/wAgY+946COtv2/fkm83gBsd38vVvTgkxdzjKsw IBQvyJx3VCEtoi/MUlpzatrk3+JRcwcAEeElypOyxCGNvnPZhOQ8XDIMjaeGwNOrhaIJ Z6j75BrYX4+IMA+mP8pqNQMEC2x2+9k0mGrMmABxAWZZNocHV3I5IjaPGWML4Mkf/Lce t/+qrQS3QWa0X2kvOdUaag390WVo4dcsJKlbS3PLqVU+s+K/iQkhi+/OUtMavO1yD4al COjQ== X-Gm-Message-State: APt69E34ranEDdv0lFDWjEkceX0zAw9qcGETU+VgmPir8dN49z6QYL// 9VUopT1B0uuRt19eTtrPenuRRdAIdNtM9Zy1M7oFfQ== X-Google-Smtp-Source: ADUXVKKuG6uE5WmaA+QYYJO4iDkpUVHcTN2j5valB303M6a6yoG6ksolne918LBnVDEFMxzxPDVRZhmSRwJorJaToQo= X-Received: by 2002:a37:108f:: with SMTP id 15-v6mr5098545qkq.345.1528910576789; Wed, 13 Jun 2018 10:22:56 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac8:297d:0:0:0:0:0 with HTTP; Wed, 13 Jun 2018 10:22:56 -0700 (PDT) In-Reply-To: References: From: Michael Sierchio Date: Wed, 13 Jun 2018 10:22:56 -0700 Message-ID: Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: "freebsd-net@freebsd.org" , "freebsd-ipfw@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 17:22:58 -0000 On Wed, Jun 13, 2018 at 10:16 AM, Jeff Kletsky wrote: When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC tunnel > to the T-Mobile provisioning servers, the reassembled, 4640-byte return > packet is silently dropped by the in-kernel NAT, even though it "matches" > the outbound packet from less than 100 ms prior. Do you have a 'reass' rule before applying nat on inbound traffic? - M --=20 "Well," Brahma said, "even after ten thousand explanations, a fool is no wiser, but an intelligent person requires only two thousand five hundred." - The Mah=C4=81bh=C4=81rata From owner-freebsd-ipfw@freebsd.org Wed Jun 13 17:32:07 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20B21102092A; Wed, 13 Jun 2018 17:32:07 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from mx.allycomm.com (mx.allycomm.com [138.68.30.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B98416F739; Wed, 13 Jun 2018 17:32:06 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from JKLETSKY1-MBP15.local (184-23-190-180.vpn.dynamic.sonic.net [184.23.190.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.allycomm.com (Postfix) with ESMTPSA id 5C3E92837C; Wed, 13 Jun 2018 10:32:04 -0700 (PDT) Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: "freebsd-net@freebsd.org" , "freebsd-ipfw@freebsd.org" References: From: Jeff Kletsky Message-ID: <918b13e0-aef5-add2-6f5c-530bb5850a3a@wagsky.com> Date: Wed, 13 Jun 2018 10:32:04 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 17:32:07 -0000 On 6/13/18 10:22 AM, Michael Sierchio wrote: > On Wed, Jun 13, 2018 at 10:16 AM, Jeff Kletsky wrote: > > When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC tunnel >> to the T-Mobile provisioning servers, the reassembled, 4640-byte return >> packet is silently dropped by the in-kernel NAT, even though it "matches" >> the outbound packet from less than 100 ms prior. > > > Do you have a 'reass' rule before applying nat on inbound traffic? > > - M Yes, at the start of the rule set. Reassembly confirmed to be working by wireshark examination of the ngtee "taps" shown $ sudo ipfw list 00001 deny ip from any to any recv ng* 00004 ngtee 100 ip from any to any proto udp dst-port 500,4500 in 00004 ngtee 100 ip from any to any proto udp frag in 00004 ngtee 110 ip from any to any proto udp dst-port 500,4500 out 00004 ngtee 110 ip from any to any proto udp frag out 00005 reass ip from any to any 00006 ngtee 101 ip from any to any proto udp dst-port 500,4500 in // reassembled in 00006 ngtee 101 ip from any to any proto udp frag in // never should be frags after reass 00006 ngtee 111 ip from any to any proto udp dst-port 500,4500 out // reass out 00006 ngtee 111 ip from any to any proto udp frag out // never should be frage after reass [...] From owner-freebsd-ipfw@freebsd.org Wed Jun 13 17:41:27 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5817B102164E for ; Wed, 13 Jun 2018 17:41:27 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-qt0-x241.google.com (mail-qt0-x241.google.com [IPv6:2607:f8b0:400d:c0d::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E800B6FE3F for ; Wed, 13 Jun 2018 17:41:26 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: by mail-qt0-x241.google.com with SMTP id y20-v6so3171485qto.8 for ; Wed, 13 Jun 2018 10:41:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=iiQI7UTufGoAq3bUn4wXw/SKP4bwMvcYQkSwKmKBk6k=; b=TWdk3/Jkkzpz4QuquatVidx8h9gqGZg9qP2YotJQ7gbW4qMYSjWgigufy3Rb+fbIh6 iqPNi85uGjm0PkWJuRSUQbtDCJT2hiWuCtJry0ami6WAaM3spdWS8qV5TljQ/QCcaJNv jEBEdQDD7aI3oxJdtKxF7WLEG9YoKrswRSBi8Ovna0m2vfK967uUA0SqUJEHbMGrItKs FObXO6TXzowfptwFu+P0aGrMg1Mjvpv+N7fEltr3FiVZZ4I2GhxHA4WG0nZ15UCgFa/l wmbirN763AlVL23SgdXA/6vEu/6HA812C87xVMiidjRa/5GsEe7zi0wA4rGHb9EOKZWj gARQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=iiQI7UTufGoAq3bUn4wXw/SKP4bwMvcYQkSwKmKBk6k=; b=Y9YhzlumdqM2yS5OS0Exzw5FVfUKYyDaD8HsAdOkzCgqAnUIfshAjU756bAaxZYHo3 xg2cfRUG0RspHpM+ZEnQMGl4fPOsb/DDiS/x7b557q+RT7X0MeQJ1yMhuvI5wa6N9Vrp 2KH/dfu0M23o2ojuL9zT3EZGwDnmwKitpiaIzKHFsd3x9sIXCOw1bYgxx773rmn4OEWw moWOhXuGHOcEDp1Di1AsronCPjxYdv7IqAQtfECeIuiRMOZ9ZYtNyilMNh5wiSlzwfGn d2pXkSNDsZyHxIRKVa3yjPG8OCc2N2sxQh+b/MLrhIS4o5UsgP3HnDTLxjxv5IV0EzFp 9Mng== X-Gm-Message-State: APt69E2byCIh//ZDam8K71qPvL+mrRugQK0VKfJpLrJGm9Z2QkLYXXIo ZVRPh85cFMBl2mBNriVYyy9irfT5t/eY5iYTRHjB4Q== X-Google-Smtp-Source: ADUXVKL5dd+x8e/3NsNh266Cdo3hnt+/UdRl9wPljA2SDx2eHqTD85XeJaQGPF3SiQv1KUlTD+AzwymIABWj0WJ3oAs= X-Received: by 2002:ac8:18d7:: with SMTP id o23-v6mr5692801qtk.28.1528911686232; Wed, 13 Jun 2018 10:41:26 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac8:297d:0:0:0:0:0 with HTTP; Wed, 13 Jun 2018 10:41:25 -0700 (PDT) In-Reply-To: <918b13e0-aef5-add2-6f5c-530bb5850a3a@wagsky.com> References: <918b13e0-aef5-add2-6f5c-530bb5850a3a@wagsky.com> From: Michael Sierchio Date: Wed, 13 Jun 2018 10:41:25 -0700 Message-ID: Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: "freebsd-net@freebsd.org" , "freebsd-ipfw@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 17:41:27 -0000 I see you have a case of Netgraph. Perhaps Julian will chime in. On Wed, Jun 13, 2018 at 10:32 AM, Jeff Kletsky wrote: > On 6/13/18 10:22 AM, Michael Sierchio wrote: > > On Wed, Jun 13, 2018 at 10:16 AM, Jeff Kletsky wrote= : >> >> When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC tunn= el >> >>> to the T-Mobile provisioning servers, the reassembled, 4640-byte return >>> packet is silently dropped by the in-kernel NAT, even though it "matche= s" >>> the outbound packet from less than 100 ms prior. >>> >> >> >> Do you have a 'reass' rule before applying nat on inbound traffic? >> >> - M >> > Yes, at the start of the rule set. > > Reassembly confirmed to be working by wireshark examination of the ngtee > "taps" shown > > $ sudo ipfw list > 00001 deny ip from any to any recv ng* > 00004 ngtee 100 ip from any to any proto udp dst-port 500,4500 in > 00004 ngtee 100 ip from any to any proto udp frag in > 00004 ngtee 110 ip from any to any proto udp dst-port 500,4500 out > 00004 ngtee 110 ip from any to any proto udp frag out > 00005 reass ip from any to any > 00006 ngtee 101 ip from any to any proto udp dst-port 500,4500 in // > reassembled in > 00006 ngtee 101 ip from any to any proto udp frag in // never should be > frags after reass > 00006 ngtee 111 ip from any to any proto udp dst-port 500,4500 out // > reass out > 00006 ngtee 111 ip from any to any proto udp frag out // never should be > frage after reass > [...] > > > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > --=20 "Well," Brahma said, "even after ten thousand explanations, a fool is no wiser, but an intelligent person requires only two thousand five hundred." - The Mah=C4=81bh=C4=81rata From owner-freebsd-ipfw@freebsd.org Wed Jun 13 19:03:58 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A35B11007B33; Wed, 13 Jun 2018 19:03:58 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward100p.mail.yandex.net (forward100p.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2029A7497A; Wed, 13 Jun 2018 19:03:57 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback15g.mail.yandex.net (mxback15g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:94]) by forward100p.mail.yandex.net (Yandex) with ESMTP id A490C5103F07; Wed, 13 Jun 2018 22:03:47 +0300 (MSK) Received: from smtp4j.mail.yandex.net (smtp4j.mail.yandex.net [2a02:6b8:0:1619::15:6]) by mxback15g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id WgdgOFwEzu-3kIWcSZU; Wed, 13 Jun 2018 22:03:46 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1528916626; bh=8tLZ5QyfkJMNZYUYwP6L8aKgoSTah4DUM2jhZkzzW3k=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=cadGSbnqJg+1GweYghau9SIeiOWi24VrYpNG/+qCCc+1LOvPTisH9wFiX90KClKeF 6WIV0x5Q2qoTluPMOnW44z73BuIpsFKLFuYH2OUiY+AQhZHFsNVpDkEFkbIDJGfR26 IhMdGxzsK9WeVKYDzFPrMp/XI9X7JSnH6Exqa39o= Received: by smtp4j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id SpR23JFOHu-3jWidHNV; Wed, 13 Jun 2018 22:03:45 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1528916625; bh=8tLZ5QyfkJMNZYUYwP6L8aKgoSTah4DUM2jhZkzzW3k=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=kxLJivG0M7weXWPL1l927ok/V/kZXgt8ERewN0qeBkpuMXSApzCjzuomWB5bAPQyD Z309JF065eVcZEXIGR/z6Efgew5feZtMRtxLeD+kLYWp5R4HkwH8Lk1vjXAYEaDuZ8 IbQTa7bCDc9+JkOw3h2v2jYNUGz0rMvRFqM6/8yE= Authentication-Results: smtp4j.mail.yandex.net; dkim=pass header.i=@yandex.ru Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org References: From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= xsBNBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAHNIkFuZHJleSBWLiBFbHN1a292IDxhZUBmcmVlYnNkLm9yZz7CwHsEEwECACUCGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheABQJMB/ruAhkBAAoJEAHF6gQQyKF6MLwH/3Ri/TZl9uo0 SepYWXOnxL6EaDVXDA+dLb1eLKC4PRBBjX29ttQ0KaWapiE6y5/AfzOPmRtHLrHYHjd/aiHX GMLHcYRXD+5GvdkK8iMALrZ28X0JXyuuZa8rAxWIWmCbYHNSBy2unqWgTI04Erodk90IALgM 9JeHN9sFqTM6zalrMnTzlcmel4kcjT3lyYw3vOKgoYLtsLhKZSbJoVVVlvRlGBpHFJI5AoYJ SyfXoN0rcX6k9X7Isp2K50YjqxV4v78xluh1puhwZyC0p8IShPrmrp9Oy9JkMX90o6UAXdGU KfdExJuGJfUZOFBTtNIMNIAKfMTjhpRhxONIr0emxxDOwE0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAcLAXwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> Date: Wed, 13 Jun 2018 22:01:03 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="Op1gBlBgWtNgb01qlw9LkD6zK3MniembP" X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 19:03:58 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Op1gBlBgWtNgb01qlw9LkD6zK3MniembP Content-Type: multipart/mixed; boundary="K7OfjEwH7U5tMcHP5vEfgZ0pIQlISHnGM"; protected-headers="v1" From: "Andrey V. Elsukov" To: Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org Message-ID: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets References: In-Reply-To: --K7OfjEwH7U5tMcHP5vEfgZ0pIQlISHnGM Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 13.06.2018 20:16, Jeff Kletsky wrote: > When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC > tunnel to the T-Mobile provisioning servers, the reassembled, 4640-byte= > return packet is silently dropped by the in-kernel NAT, even though it > "matches" the outbound packet from less than 100 ms prior. > Are there known causes and/or resolutions for this behavior? >=20 > Is there a way to be able to "monitor" the NAT table? >=20 > (I didn't see anything obvious in the ipfw, natd, or libalias man pages= =2E) The kernel version of libalias uses m_megapullup() function to make single contiguous buffer. m_megapullup() uses m_get2() function to allocate mbuf of appropriate size. If size of packet greater than 4k it will fail. So, if you use MTU greater than 4k or if after fragments reassembly you get a packet with length greater than 4k, ipfw_nat() function will drop this packet. --=20 WBR, Andrey V. Elsukov --K7OfjEwH7U5tMcHP5vEfgZ0pIQlISHnGM-- --Op1gBlBgWtNgb01qlw9LkD6zK3MniembP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAlshae8ACgkQAcXqBBDI oXrHfwf/SOQV9IYt3CHnSosFsD7fn1F/IN9VtlPHMQuO2euyOlKcx1m3Vu9Tx5TD t73yBJ+8/Dp12l3y6RLJm0mrCU9TehothrrAbnAzFyTeLOT/QLbbcK3S/SxT/gkH nqpTGL8RkeEGUzM2eTf0gTn2Ib290+aSE60I5r266KP28VHdzdBRENmE0v+vyopZ M56HKQ315padOXYNuXyVachxQ0cYRI7WPJMy0SvQrdXdNp260DfewbBaFygsUPEO 2zBhOq1MnNi2CjpDZXXrFAGG9J5LZROROHqHa6oh1lKF0QCmTyyI7K/vqbvIAoFL MboWWv3vuxeDh86NHDLu1cZlowWK6w== =knFf -----END PGP SIGNATURE----- --Op1gBlBgWtNgb01qlw9LkD6zK3MniembP-- From owner-freebsd-ipfw@freebsd.org Wed Jun 13 20:04:25 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4473D100E7B2; Wed, 13 Jun 2018 20:04:25 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from mx.allycomm.com (mx.allycomm.com [138.68.30.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D629877A64; Wed, 13 Jun 2018 20:04:24 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from JKLETSKY1-MBP15.local (184-23-191-106.vpn.dynamic.sonic.net [184.23.191.106]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.allycomm.com (Postfix) with ESMTPSA id 7A33F284C9; Wed, 13 Jun 2018 13:04:22 -0700 (PDT) Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: "Andrey V. Elsukov" , Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org References: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> From: Jeff Kletsky Message-ID: Date: Wed, 13 Jun 2018 13:04:22 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 20:04:25 -0000 On 6/13/18 12:01 PM, Andrey V. Elsukov wrote: > On 13.06.2018 20:16, Jeff Kletsky wrote: >> When a T-Mobile "femto-cell" is trying to establish its IPv4, IPSEC >> tunnel to the T-Mobile provisioning servers, the reassembled, 4640-byte >> return packet is silently dropped by the in-kernel NAT, even though it >> "matches" the outbound packet from less than 100 ms prior. >> Are there known causes and/or resolutions for this behavior? >> >> Is there a way to be able to "monitor" the NAT table? >> >> (I didn't see anything obvious in the ipfw, natd, or libalias man pages.) > The kernel version of libalias uses m_megapullup() function to make > single contiguous buffer. m_megapullup() uses m_get2() function to > allocate mbuf of appropriate size. If size of packet greater than 4k it > will fail. So, if you use MTU greater than 4k or if after fragments > reassembly you get a packet with length greater than 4k, ipfw_nat() > function will drop this packet. > Thanks!! Mystery solved... /usr/src/sys/netinet/libalias/alias.c #ifdef _KERNEL /*  * m_megapullup() - this function is a big hack.  * Thankfully, it's only used in ng_nat and ipfw+nat. suggests that the "old school" approach of natd might resolve this. I'll give it a try when I'm close enough to the box to resolve it when I make a configuration error. Jeff From owner-freebsd-ipfw@freebsd.org Wed Jun 13 20:31:05 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D4F41012112; Wed, 13 Jun 2018 20:31:05 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward106p.mail.yandex.net (forward106p.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E52E5799DF; Wed, 13 Jun 2018 20:31:04 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback15g.mail.yandex.net (mxback15g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:94]) by forward106p.mail.yandex.net (Yandex) with ESMTP id 5CC8E2D81412; Wed, 13 Jun 2018 23:31:01 +0300 (MSK) Received: from smtp3o.mail.yandex.net (smtp3o.mail.yandex.net [2a02:6b8:0:1a2d::27]) by mxback15g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id hBhlp0Gmo0-V1IWgWOq; Wed, 13 Jun 2018 23:31:01 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1528921861; bh=cY2MBNtl1CrpFFN660VxOOv31sdM6f6AwUvGa8EfPy4=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=ZcCxJ5j56gYcJX1kikxnE18wEBIFW59sASbjfzeZN+4qV9kE2RPOxbqsD7gL45va+ FFn0dXGB/4NJdxDxD2E+ED6Uzn2XEQGOmndT+Bj86OF7WYqGUmHo6d2jWwc4m3jkzT 1S93kEFXJNKCpi2/9rikfZ2zBhzKUUTgOt+xI8B8= Received: by smtp3o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id EOF5bUA0uU-V0rCQuMJ; Wed, 13 Jun 2018 23:31:00 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1528921860; bh=cY2MBNtl1CrpFFN660VxOOv31sdM6f6AwUvGa8EfPy4=; h=Subject:To:References:From:Message-ID:Date:In-Reply-To; b=DRwuUuCaJ04PV/wBMyS7YOPZmZRERDd+b2p3KZILqhI4gc+CGt0DPLIPvy3a97o1a THvFTHGG3Oqz/2cbzljPvag9wiWRiANVRzclpQST+zBU+PZLWeUOH2/cfdW0cfma2k BxKKrmmcNlGFvogKPKOJntb5rEUFZhYEnjOJ5xE0= Authentication-Results: smtp3o.mail.yandex.net; dkim=pass header.i=@yandex.ru Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org References: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= xsBNBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAHNIkFuZHJleSBWLiBFbHN1a292IDxhZUBmcmVlYnNkLm9yZz7CwHsEEwECACUCGwMGCwkI BwMCBhUIAgkKCwQWAgMBAh4BAheABQJMB/ruAhkBAAoJEAHF6gQQyKF6MLwH/3Ri/TZl9uo0 SepYWXOnxL6EaDVXDA+dLb1eLKC4PRBBjX29ttQ0KaWapiE6y5/AfzOPmRtHLrHYHjd/aiHX GMLHcYRXD+5GvdkK8iMALrZ28X0JXyuuZa8rAxWIWmCbYHNSBy2unqWgTI04Erodk90IALgM 9JeHN9sFqTM6zalrMnTzlcmel4kcjT3lyYw3vOKgoYLtsLhKZSbJoVVVlvRlGBpHFJI5AoYJ SyfXoN0rcX6k9X7Isp2K50YjqxV4v78xluh1puhwZyC0p8IShPrmrp9Oy9JkMX90o6UAXdGU KfdExJuGJfUZOFBTtNIMNIAKfMTjhpRhxONIr0emxxDOwE0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAcLAXwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: Date: Wed, 13 Jun 2018 23:28:13 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="W9zmlRhr6HsD1pP9kGTQBXqhK1hohmu8o" X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 20:31:05 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --W9zmlRhr6HsD1pP9kGTQBXqhK1hohmu8o Content-Type: multipart/mixed; boundary="zKCnThgmpuso9SJk2ZHT0tm5KgUw0pTAS"; protected-headers="v1" From: "Andrey V. Elsukov" To: Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org Message-ID: Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets References: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> In-Reply-To: --zKCnThgmpuso9SJk2ZHT0tm5KgUw0pTAS Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 13.06.2018 23:04, Jeff Kletsky wrote: >> The kernel version of libalias uses m_megapullup() function to make >> single contiguous buffer. m_megapullup() uses m_get2() function to >> allocate mbuf of appropriate size. If size of packet greater than 4k i= t >> will fail. So, if you use MTU greater than 4k or if after fragments >> reassembly you get a packet with length greater than 4k, ipfw_nat() >> function will drop this packet. >> > Thanks!! >=20 > Mystery solved... >=20 > /usr/src/sys/netinet/libalias/alias.c >=20 > #ifdef _KERNEL > /* > =C2=A0* m_megapullup() - this function is a big hack. > =C2=A0* Thankfully, it's only used in ng_nat and ipfw+nat. >=20 > suggests that the "old school" approach of natd might resolve this. I'l= l > give it a try when I'm close enough to the box to resolve it when I mak= e > a configuration error. I didn't look at the rest of libalias, but you, probably, can improve this hack to use 9k or 16k mbufs. You can replace m_get2() call in m_megapullup() with the following code: if (len <=3D MJUMPAGESIZE) mcl =3D m_get2(len, M_NOWAIT, MT_DATA, M_PKTHDR); else if (len <=3D MJUM9BYTES) mcl =3D m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM9BYTES); else if (len <=3D MJUM16BYTES) mcl =3D m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM16BYTES); else goto bad; --=20 WBR, Andrey V. Elsukov --zKCnThgmpuso9SJk2ZHT0tm5KgUw0pTAS-- --W9zmlRhr6HsD1pP9kGTQBXqhK1hohmu8o Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAlshfl0ACgkQAcXqBBDI oXqsWgf+JCiWPm8RMV6aLLEDjEw6mqJS630ELX6QKdyoo3wAQQ7OlNylgzC/cSsD W38t7fVtK5kCQFteF0Rr6GrOBjPVJdvJYo2NeG+SqbsRaU17+xlB/Vdup+LXGKi+ jlemwOkLUUSaG36H5vPC5otUnIXua74rmrvsmhFOvrYpEnm/XX+p0Tj3ioV60s8a SmqCXGN75Wb/FRra07i3fUc5hBmsKMDHAPQwMqUuv7cgdEwDjNEJg0uQVjY0z42+ aEGbtgqdyLNrdwjDIgjH0X43mVX3tuYjo8kvvKF2WQvF1kemCLKbT+XQGftlxP9o xdFRbUscXz3oad/OQdFAlR6XNIYQ6Q== =0Y86 -----END PGP SIGNATURE----- --W9zmlRhr6HsD1pP9kGTQBXqhK1hohmu8o-- From owner-freebsd-ipfw@freebsd.org Wed Jun 13 23:44:56 2018 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 329071002294; Wed, 13 Jun 2018 23:44:56 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from mx.allycomm.com (mx.allycomm.com [138.68.30.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CCE8182D83; Wed, 13 Jun 2018 23:44:55 +0000 (UTC) (envelope-from jmk@wagsky.com) Received: from JKLETSKY1-MBP15.local (184-23-191-241.vpn.dynamic.sonic.net [184.23.191.241]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx.allycomm.com (Postfix) with ESMTPSA id 93E9E28666; Wed, 13 Jun 2018 16:44:53 -0700 (PDT) Subject: Re: In-kernel NAT [ipfw] dropping large UDP return packets To: "Andrey V. Elsukov" , Jeff Kletsky , freebsd-net@freebsd.org, freebsd-ipfw@freebsd.org References: <48e750c1-e38c-5376-a937-dcbb2d871256@yandex.ru> From: Jeff Kletsky Message-ID: <3b9b426e-8276-bc79-2624-60b66f04b344@wagsky.com> Date: Wed, 13 Jun 2018 16:44:53 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jun 2018 23:44:56 -0000 On 6/13/18 1:28 PM, Andrey V. Elsukov wrote: > On 13.06.2018 23:04, Jeff Kletsky wrote: >>> The kernel version of libalias uses m_megapullup() function to make >>> single contiguous buffer. m_megapullup() uses m_get2() function to >>> allocate mbuf of appropriate size. If size of packet greater than 4k it >>> will fail. So, if you use MTU greater than 4k or if after fragments >>> reassembly you get a packet with length greater than 4k, ipfw_nat() >>> function will drop this packet. >>> >> Thanks!! >> >> Mystery solved... >> >> /usr/src/sys/netinet/libalias/alias.c >> >> #ifdef _KERNEL >> /* >>  * m_megapullup() - this function is a big hack. >>  * Thankfully, it's only used in ng_nat and ipfw+nat. >> >> suggests that the "old school" approach of natd might resolve this. I'll >> give it a try when I'm close enough to the box to resolve it when I make >> a configuration error. > I didn't look at the rest of libalias, but you, probably, can improve > this hack to use 9k or 16k mbufs. You can replace m_get2() call in > m_megapullup() with the following code: > > if (len <= MJUMPAGESIZE) > mcl = m_get2(len, M_NOWAIT, MT_DATA, M_PKTHDR); > else if (len <= MJUM9BYTES) > mcl = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM9BYTES); > else if (len <= MJUM16BYTES) > mcl = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM16BYTES); > else > goto bad; > Tested and "works for me" on 11.1-RELEASE-p10 with GENERIC kernconf 8< --- alias.c.orig    2017-07-20 16:42:02.000000000 -0700 +++ alias.c    2018-06-13 15:41:46.862121000 -0700 @@ -1758,7 +1758,14 @@      if (m->m_next == NULL && M_WRITABLE(m))          return (m); -    mcl = m_get2(len, M_NOWAIT, MT_DATA, M_PKTHDR); +    if (len <= MJUMPAGESIZE) +        mcl = m_get2(len, M_NOWAIT, MT_DATA, M_PKTHDR); +    else if (len <= MJUM9BYTES) +        mcl = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM9BYTES); +    else if (len <= MJUM16BYTES) +        mcl = m_getjcl(M_NOWAIT, MT_DATA, M_PKTHDR, MJUM16BYTES); +    else +        goto bad;      if (mcl == NULL)          goto bad;      m_align(mcl, len); >8 Thanks again! Jeff