From owner-freebsd-ipfw@freebsd.org Thu Dec 8 22:58:03 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC4DAC6DE21 for ; Thu, 8 Dec 2016 22:58:03 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A94B616 for ; Thu, 8 Dec 2016 22:58:02 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id B4F063564 for ; Thu, 8 Dec 2016 16:49:38 -0600 (CST) To: freebsd-ipfw@freebsd.org From: Karl Denninger Subject: IPFW problem with passing IPSEC through in-kernel NAT Message-ID: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> Date: Thu, 8 Dec 2016 16:49:32 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020402030305090906000203" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Dec 2016 22:58:03 -0000 This is a cryptographically signed message in MIME format. --------------ms020402030305090906000203 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi folks; I have a fairly complex configuration here with IPSEC on a gateway machine, which is working fine. However, I also wish to pass through *client* IPSEC setup requests (which happen to be coming from cellphones that want to use WiFi calling) as well, and have run into a problem. T-Mobile's WiFi calling appears to set up an IPSEC tunnel back to the company when turned on. The issue I'm running into is that while this is *supposed* to work with a device behind a NAT router (and does in other locations around the area) my FreeBSD gateway (which also happens to run the IPSEC gateway) always appears to pass the *internal* address (!) for the phone outbound, and refuses to put the setup packets through NAT. If I shut down IPSEC on the gateway machine and remove all of its ipfw rules it still doesn't work; I get authentication errors returned (when looking at the data stream with tcpdump to and from the phone device) which implies that the packets sent to the host are being tampered with -- along with some untranslated transmissions as well. Does anyone have a sample configuration that works with T-Mobile's WiFi calling and FreeBSD's internal kernel NAT solution? That might be enough for me to figure out what's going on... FreeBSD 11.0-STABLE #13 r307318M: if the rev matters.... Thanks in advance! --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020402030305090906000203 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEyMDgyMjQ5MzJaME8GCSqGSIb3DQEJBDFCBEAK sUVXXqO2vJpy4iYrFOPbgcsCLDmRBx/d0SaIEJ7hQd6on5SKPBDeb0MAOuSP44M5JEBWXY2D kstLGIlkikh5MGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAoeiSWDUx 4u3kAZhfbMrcLuRQzFmBryxSOXSZ/LbclvE4uoULLu+j1edCRkqVeW70SvRHHXI2TZX6hTgw ptjKCZ9ZuMzwDkPwX1zkg1T4e+KeBSudOdn2FAARnbWEiKy8moPNEbyIBvsKXtiTnKfp+7wE cWhYpPyPaE+WB6q7wc7cebduX/QnPeDVlrpNEGzfQC5VZISDx6OencfPFdm4orGQ7HlPOfuX l2OkP5jTUoaA5m7YNNtT4JYehPyZ9ddq4sxQTPMIRi2/gmvgdePyEgwaWs9CY7GcRPt2/svP 6/A0mdVkdzR4jl7idchhmNXHol+Q6LwZxzp5WpT6A3vAdY6Arc7nhprtB0uKWvnHcymkltjN cZh1C9mKTbSbOn+O3BW6DR4jDj/rRBj/nPqKrn9qwjwGeyChdoHMV8jmOMyXeq4PwbWdPTx2 UGRtw5enEDL5Pn/4GON1U+UIPnO1mrxdXj7LhqEH04uHwGMiB63rQ+GYBoHkbwVF2lL6zEFI YVf6IXO9zgxjNQZqJYQo8vJ6uc1bHZHxbeBcyXMimc0nmOTIajb+3jiLejNcbErlLwBkjrw/ OTyQY4vh4x5RQ8CBQgDVX04KOB6tsD60AwIcHnhLd/TXBN52owQWRjcZJE527wovtC0ah503 x3GL+IspHCsokKNbgfTD2knGcVsAAAAAAAA= --------------ms020402030305090906000203-- From owner-freebsd-ipfw@freebsd.org Fri Dec 9 04:11:47 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96919C6DAFB for ; Fri, 9 Dec 2016 04:11:47 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3DD2717BD for ; Fri, 9 Dec 2016 04:11:46 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 0BD42661E2 for ; Thu, 8 Dec 2016 22:11:44 -0600 (CST) Subject: Re: IPFW problem with passing IPSEC through in-kernel NAT To: freebsd-ipfw@freebsd.org References: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> From: Karl Denninger Message-ID: <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> Date: Thu, 8 Dec 2016 22:11:38 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms060600010006020500070005" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 04:11:47 -0000 This is a cryptographically signed message in MIME format. --------------ms060600010006020500070005 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 12/8/2016 16:49, Karl Denninger wrote: > Hi folks; > > I have a fairly complex configuration here with IPSEC on a gateway > machine, which is working fine. However, I also wish to pass through > *client* IPSEC setup requests (which happen to be coming from cellphone= s > that want to use WiFi calling) as well, and have run into a problem. > > T-Mobile's WiFi calling appears to set up an IPSEC tunnel back to the > company when turned on. The issue I'm running into is that while this > is *supposed* to work with a device behind a NAT router (and does in > other locations around the area) my FreeBSD gateway (which also happens= > to run the IPSEC gateway) always appears to pass the *internal* address= > (!) for the phone outbound, and refuses to put the setup packets throug= h > NAT. If I shut down IPSEC on the gateway machine and remove all of its= > ipfw rules it still doesn't work; I get authentication errors returned > (when looking at the data stream with tcpdump to and from the phone > device) which implies that the packets sent to the host are being > tampered with -- along with some untranslated transmissions as well. > > Does anyone have a sample configuration that works with T-Mobile's WiFi= > calling and FreeBSD's internal kernel NAT solution? That might be > enough for me to figure out what's going on... > > FreeBSD 11.0-STABLE #13 r307318M: if the rev matters.... > > Thanks in advance! > Some more information on this issue.... I suspect that something is getting mangled somewhere in the IP stack, perhaps related to hardware checksumming or similar -- or in the ipfw code. If I do not block (explicitly) UDP packets from the phone's IP from getting out that have not been translated, I get some IPSEC (but no other) packets that get through the NAT level (!). If I block those then the ones that DO go out facially might be ok and might not; here's an example of one that DOES translate and the reply back from the other e= nd: 21:40:59.400170 IP (tos 0x0, ttl 253, id 5901, offset 0, flags [DF], proto UDP (17), length 388) denninger.net.56595 > m043536d0.tmodns.net.isakmp: [udp sum ok] isakmp 2.0 msgid 00000000 cookie f4db8a5ed1c6fb54->0000000000000000: parent_sa ikev2_init[I]: (sa: len=3D112 (p: #1 protoid=3Disakmp transform=3D12 len=3D112 (t: #1 type=3Dencr id=3D1des ) (t: #2 type=3Dencr id=3D3des ) (t: #3 type=3Dencr id=3Daes (type=3Dkeylen value=3D0080)) (t: #4 type=3Dencr id=3Daes (type=3Dkeylen value=3D0100)) (t: #5 type=3Dinteg id=3Dhmac-md5 ) (t: #6 type=3Dinteg id=3Dhmac-sha ) (t: #7 type=3Dinteg id=3Daes-xcbc ) (t: #8 type=3Dprf id=3Dhmac-sha ) (t: #9 type=3Dprf id=3Daes128_xcbc ) (t: #10 type=3Ddh id=3Dmodp1024 ) (t: #11 type=3Ddh id=3Dmodp1536 ) (t: #12 type=3Ddh id=3Dmodp2048 ))) (v2ke: len=3D128 group=3Dmodp1024) (nonce: len=3D20 nonce=3D(7ab63afdd30a89278e3c0fac813dc44e05becba2) )= (n: prot_id=3D#0 type=3D16388(nat_detection_source_ip)) (n: prot_id=3D#0 type=3D16389(nat_detection_destination_ip)) 21:40:59.455522 IP (tos 0xc0, ttl 54, id 17839, offset 0, flags [DF], proto UDP (17), length 66) m043536d0.tmodns.net.isakmp > denninger.net.56595: [udp sum ok] isakmp 2.0 msgid 00000000 cookie f4db8a5ed1c6fb54->f497c06d5e456d11: parent_sa ikev2_init[R]: (n: prot_id=3D#0 type=3D17(invalid_ke_payload)) The other end replies immediately with a complaint about the requested keying..... it never gets any further. If I *remove* the explicit block on the internal source address here: ${fwcmd} add 10999 deny log udp from 192.168.1.25 to any Which I stuck directly above the two lines that allow these UDP targeted packets to get out into the wild: # Allow T-Mobile's WiFi Calling to work ${fwcmd} add 11000 pass udp from any to any 500 keep-state ${fwcmd} add 11010 pass udp from any to any 4500 keep-state Then despite THIS line further up: ${fwcmd} add 2800 nat 100 udp from 192.168.1.0/24 any to not 192.168.0.0/16 isakmp out via ${nat_interface} Which *should* force anything coming from that address and pointed at any address for ipsec initiation through NAT I start seeing tcpdump entries like this show up immediately on the interface: 21:47:44.748334 IP (tos 0x0, ttl 253, id 5999, offset 0, flags [DF], proto UDP (17), length 388) D15.Denninger.Net.34453 > m043536d0.tmodns.net.isakmp: [udp sum ok] isakmp 2.0 msgid 00000000 cookie d5c4bc3dcbe650cc->0000000000000000: parent_sa ikev2_init[I]: (sa: len=3D112 (p: #1 protoid=3Disakmp transform=3D12 len=3D112 (t: #1 type=3Dencr id=3D1des ) (t: #2 type=3Dencr id=3D3des ) (t: #3 type=3Dencr id=3Daes (type=3Dkeylen value=3D0080)) (t: #4 type=3Dencr id=3Daes (type=3Dkeylen value=3D0100)) (t: #5 type=3Dinteg id=3Dhmac-md5 ) (t: #6 type=3Dinteg id=3Dhmac-sha ) (t: #7 type=3Dinteg id=3Daes-xcbc ) (t: #8 type=3Dprf id=3Dhmac-sha ) (t: #9 type=3Dprf id=3Daes128_xcbc ) (t: #10 type=3Ddh id=3Dmodp1024 ) (t: #11 type=3Ddh id=3Dmodp1536 ) (t: #12 type=3Ddh id=3Dmodp2048 ))) (v2ke: len=3D128 group=3Dmodp1024) (nonce: len=3D20 nonce=3D(ec204213e7501c0988dfb8ae84421b1224d10f7f) )= (n: prot_id=3D#0 type=3D16388(nat_detection_source_ip)) (n: prot_id=3D#0 type=3D16389(nat_detection_destination_ip)) That shouldn't be possible since it should not be possible for an untranslated internal IP address to get through the NAT part of the ipfw table with an internal source and 500 destination port, and in fact some of the packets that go into there DO get translated -- but apparently incorrectly. All other packet traffic works as expected both from that device and many others. IPSEC_NAT_T is in the kernel. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms060600010006020500070005 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEyMDkwNDExMzhaME8GCSqGSIb3DQEJBDFCBECc UWmjgIYKZYzPU6p7kJBUN+9Ap+imMq9tHtWWRjH5zEeeP0jXCPdvOr/QLvH2MOyzMF3CAPo+ 9WkTtmXZ3FAaMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAakN3GzE/ FUE3CBJ6dUbiGrLVWzqLm+Kltvzbsb7iRXiWhm7mtYrRSHzAjf02gPmY0Zf4WgAbIY+ysNEi 6keBtT1a2sBKHw9UF0kH8pCEoj6OuVE/3k8AWn205sE45tGLdFBIlrjjPV+im8lM/zMNYOfu e9dE9KSk+5jFs07Mk3JFK5gybPoJGJl0+MFNwlk6frXMmuYvANFaj6wa/dATpB6Ey83oub10 XvX2eUvQY3wvbINQXWS/K3cDDfEJPVVoXV3xlMnRsdLFioRGqtcmuwu0D9ykZA6kAiq1mQNB +CErlSamydWxj7q4XN1WT+IoSCCc9z8yyU1JhSggghvnE2HcP4KvuY9W/9G73X8C2qlJOJtJ J0LwXjisijlOOat9a3WOKSTM0PLOmZiFfdvnRS0+R5XBC6vuiB+cixqUq3oOzI+ngMVfG0sh CTFUH1Jt0v5A7Wp4bLY8QY3mH2IYlutODORVId6T02fcYUe7n/1EdkgyiFC03h+kQnyM+lAY dFNIUzlt4+kHsBGdXNwxMOjvfIrb1hmFb8bdziwrP4/coSEj/YvOxEqP348rmsyW00hKxBre w4hfkjHXJCk65WErxgr2ZfSp//PQf/WmkxdvAkmg5PZdQO6NNYR9kY423OYBI1WtDbpbzVG8 xA8e7KsbeG0wOQylrp7H0Hl27cEAAAAAAAA= --------------ms060600010006020500070005-- From owner-freebsd-ipfw@freebsd.org Fri Dec 9 12:55:40 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE626C6AF4C for ; Fri, 9 Dec 2016 12:55:40 +0000 (UTC) (envelope-from rj@obsigna.com) Received: from mo6-p00-ob.smtp.rzone.de (mo6-p00-ob.smtp.rzone.de [IPv6:2a01:238:20a:202:5300::4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.smtp.rzone.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4C55716E1 for ; Fri, 9 Dec 2016 12:55:39 +0000 (UTC) (envelope-from rj@obsigna.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; t=1481288136; l=1166; s=domk; d=obsigna.com; h=To:References:Content-Transfer-Encoding:Cc:Date:In-Reply-To:From: Subject:Mime-Version:Content-Type; bh=GV6uxe1h4mYyKd4LPuLt7kUtEY9sGuLC+mDWproCFAU=; b=EjK6VXtr8dofAwz3UlFeeeViLiqjdw5WBXOvJvrl3HMrYm2pqhxzNQk2HfzNDZcOvH UXn9eWoU4viEmJCrg8KFRJ6ii3ttuZ43Zyi8L+jpElmc6Qvh606qzBqTzSSMHmpjYp4i S2Un5NCvuQ217zbDW8FkPBYPSqnZQbL6sM+LU= X-RZG-AUTH: :O2kGeEG7b/pS1EK7WHa0hxqKZr4lnx6UhT0M0o35iAdWtoM07Gt3wQHFGh0h99HjZzw= X-RZG-CLASS-ID: mo00 Received: from mail.obsigna.com (bb02b667.virtua.com.br [187.2.182.103]) by smtp.strato.de (RZmta 39.10 DYNA|AUTH) with ESMTPSA id c06d32sB9CtZMRb (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (curve secp521r1 with 521 ECDH bits, eq. 15360 bits RSA)) (Client did not present a certificate); Fri, 9 Dec 2016 13:55:35 +0100 (CET) Received: from rolf.projectworld.net (rolf.projectworld.net [192.168.222.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.obsigna.com (Postfix) with ESMTPSA id BC583746A1B1; Fri, 9 Dec 2016 10:18:26 -0200 (BRST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.1 \(3251\)) Subject: Re: IPFW problem with passing IPSEC through in-kernel NAT From: "Dr. Rolf Jansen" In-Reply-To: <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> Date: Fri, 9 Dec 2016 10:18:26 -0200 Cc: Karl Denninger Content-Transfer-Encoding: quoted-printable Message-Id: <156E272C-0EFA-4A15-8544-C580AAEB6033@obsigna.com> References: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> To: freebsd-ipfw@freebsd.org X-Mailer: Apple Mail (2.3251) X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 12:55:40 -0000 > Am 09.12.2016 um 02:11 schrieb Karl Denninger : > ... > Some more information on this issue.... I suspect that something is > getting mangled somewhere in the IP stack, perhaps related to hardware > checksumming or similar -- or in the ipfw code. I had always ran into IPsec-NAT-UDP checksumming issues since I started = working with FreeBSD, that tim v8.0. With a rather simple change in the = respective kernel source file at least my issue can be resolved. This = may be related to your issue or even not, anyway, I guess it is worth to = give it a try. I am now running FreeBSD 11-RELEASE-p5. On line 462 of file = /usr/src/sys/netinet/udp_usrreq.c, I replaced: if (uh->uh_sum) { with: if (uh->uh_sum && uh->uh_dport !=3D htons(1701) && uh->uh_dport !=3D htons(4500)) { This effectively skips extended UDP checksumming for certain UDP ports = -- here the L2TP and IPsec-NAT-T ports. When I investigated the issue, I = found in one related RFC, that IPsec-NAT-T isn't supposed to do UDP = checksumming on the encapsulated packets anyway, and my patch enforces = this behaviour. Best regards Rolf= From owner-freebsd-ipfw@freebsd.org Fri Dec 9 13:16:56 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CBBDC6BD79 for ; Fri, 9 Dec 2016 13:16:56 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E988A1802 for ; Fri, 9 Dec 2016 13:16:55 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id DE1063F56 for ; Fri, 9 Dec 2016 07:16:53 -0600 (CST) Subject: Re: IPFW problem with passing IPSEC through in-kernel NAT To: freebsd-ipfw@freebsd.org References: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> <156E272C-0EFA-4A15-8544-C580AAEB6033@obsigna.com> From: Karl Denninger Message-ID: <01fbc965-f5bc-0f62-eb89-02e097e03cf7@denninger.net> Date: Fri, 9 Dec 2016 07:16:47 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <156E272C-0EFA-4A15-8544-C580AAEB6033@obsigna.com> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms000703010200020200040300" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 09 Dec 2016 13:16:56 -0000 This is a cryptographically signed message in MIME format. --------------ms000703010200020200040300 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 12/9/2016 06:18, Dr. Rolf Jansen wrote: >> Am 09.12.2016 um 02:11 schrieb Karl Denninger : >> ... >> Some more information on this issue.... I suspect that something is >> getting mangled somewhere in the IP stack, perhaps related to hardware= >> checksumming or similar -- or in the ipfw code. > I had always ran into IPsec-NAT-UDP checksumming issues since I started= working with FreeBSD, that tim v8.0. With a rather simple change in the = respective kernel source file at least my issue can be resolved. This may= be related to your issue or even not, anyway, I guess it is worth to giv= e it a try. > > I am now running FreeBSD 11-RELEASE-p5. On line 462 of file /usr/src/sy= s/netinet/udp_usrreq.c, I replaced: > > if (uh->uh_sum) { > > with: > > if (uh->uh_sum && > uh->uh_dport !=3D htons(1701) && > uh->uh_dport !=3D htons(4500)) { > > This effectively skips extended UDP checksumming for certain UDP ports = -- here the L2TP and IPsec-NAT-T ports. When I investigated the issue, I = found in one related RFC, that IPsec-NAT-T isn't supposed to do UDP check= summing on the encapsulated packets anyway, and my patch enforces this be= haviour. > > Best regards > > Rolf > In this case is that I never get to the use of port 4500 (there are no packets emitted on that port that I can find); the initial key exchange on port 500 is failing, and in-kernel NAT appears to be involved in some fashion because I'm getting inside addresses that are (in some cases) not being NATted at all despite the fact that as far as I can tell they *should* be. I'm going to spend some time refactoring the IPFW rule set to compartmentalize the various paths through it more-fully. Perhaps that will shed some more light on the problem, or at least make more-reasonable an attempt to trace it. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms000703010200020200040300 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEyMDkxMzE2NDdaME8GCSqGSIb3DQEJBDFCBEB1 OL80TyTZNmzydiO1KXDUnUwtTCli6VqQRONAvpGjBNTVVEOrvEWZlAuIyfmBM1OVF2FYShW1 1Pxfw2NDbqVzMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAV6xcJs3t eh5AIT3cax75PSL3AhcWz7USiP1iwPVIql3jepmPzYZzSsySmcPoiZZXhf6JjM24LoxP4yyj QxH5e28Q13AjxuNANooGA7dL4P2HJzS1MqL+B3nIv4TGduYunBa4mMipFRhs+T7mThEUwWiL D1mYiBmynh0NLfBI4k8VKRSRvWwZJ1PRw1UB4LkucPnzv+RUEsT1HyL6Rj+2w4iR+PicJ0xY a8Cc2vwpkEpkHcxRagV0hCMLKsUKRg/iBfNL8w/hifQpG05OPLuWLDAuQJuYwVufcRT7KewB 575tX89JpYDadCM5w9IKQa1LH+USNnDS0yTIXpWnnGXbs+5E1CsOHt69ZbYuLVcnXqKT7PfJ KwOWjKMwD8T1AT9dq0XowTCj+X+mTf6EnlZnkjwS9c74SYYpGF3o7OlXfJ0FF66u9ppZQdRo W9CuPnq8+8PVdH46kDcqeO4Jt78wcDlHAPjnGE++nIas9dt+qCsJd0QKfN8aKZwgqxNG/0Q5 5RSrc/jy1LqsrygLZI4kTK8PS+3euHG6aVR8v9B5QMEaxs4v2K+pGVgmhPXqOCKuUtwhOOzb S5w3GSi2Xh6dKenb/mCnOi1X5LJSwtP0g5LV7VILfhfWOCkghsrbhC6LpXbmY/VfTBPtcjlN PwiFrNCGmM+HOEBpwghiyJH88mQAAAAAAAA= --------------ms000703010200020200040300-- From owner-freebsd-ipfw@freebsd.org Sat Dec 10 03:23:16 2016 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22B70C6F11B for ; Sat, 10 Dec 2016 03:23:16 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EAC4E1959 for ; Sat, 10 Dec 2016 03:23:15 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id EECFC66275 for ; Fri, 9 Dec 2016 21:23:07 -0600 (CST) Subject: Re: IPFW problem with passing IPSEC through in-kernel NAT To: freebsd-ipfw@freebsd.org References: <099203a1-f601-bb79-548d-27c62fcbf556@denninger.net> <005b34c8-2217-fa06-5584-6999022481a3@denninger.net> <156E272C-0EFA-4A15-8544-C580AAEB6033@obsigna.com> <01fbc965-f5bc-0f62-eb89-02e097e03cf7@denninger.net> From: Karl Denninger Message-ID: Date: Fri, 9 Dec 2016 21:23:01 -0600 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: <01fbc965-f5bc-0f62-eb89-02e097e03cf7@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090906050805080905000008" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Dec 2016 03:23:16 -0000 This is a cryptographically signed message in MIME format. --------------ms090906050805080905000008 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 12/9/2016 07:16, Karl Denninger wrote: > On 12/9/2016 06:18, Dr. Rolf Jansen wrote: >>> Am 09.12.2016 um 02:11 schrieb Karl Denninger : >>> ... >>> Some more information on this issue.... I suspect that something is >>> getting mangled somewhere in the IP stack, perhaps related to hardwar= e >>> checksumming or similar -- or in the ipfw code. >> I had always ran into IPsec-NAT-UDP checksumming issues since I starte= d working with FreeBSD, that tim v8.0. With a rather simple change in the= respective kernel source file at least my issue can be resolved. This ma= y be related to your issue or even not, anyway, I guess it is worth to gi= ve it a try. >> >> I am now running FreeBSD 11-RELEASE-p5. On line 462 of file /usr/src/s= ys/netinet/udp_usrreq.c, I replaced: >> >> if (uh->uh_sum) { >> >> with: >> >> if (uh->uh_sum && >> uh->uh_dport !=3D htons(1701) && >> uh->uh_dport !=3D htons(4500)) { >> >> This effectively skips extended UDP checksumming for certain UDP ports= -- here the L2TP and IPsec-NAT-T ports. When I investigated the issue, I= found in one related RFC, that IPsec-NAT-T isn't supposed to do UDP chec= ksumming on the encapsulated packets anyway, and my patch enforces this b= ehaviour. >> >> Best regards >> >> Rolf >> > In this case is that I never get to the use of port 4500 (there are no > packets emitted on that port that I can find); the initial key exchange= > on port 500 is failing, and in-kernel NAT appears to be involved in som= e > fashion because I'm getting inside addresses that are (in some cases) > not being NATted at all despite the fact that as far as I can tell they= > *should* be. > > I'm going to spend some time refactoring the IPFW rule set to > compartmentalize the various paths through it more-fully. Perhaps that= > will shed some more light on the problem, or at least make > more-reasonable an attempt to trace it. > I have completely re-factored the IPFW rule set that I am using here (it was formerly built on top of the "Simple" config in /etc/rc.firewall) to be completely stand-alone and the problem has disappeared. Bottom line -- this appears to have been some sort of problem with the rule set rather than ipfw itself. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090906050805080905000008 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEyMTAwMzIzMDFaME8GCSqGSIb3DQEJBDFCBEDI 1bOF8mB0n7DNPZyg5ujC8xWVTch0GfMWpUzQ/C1gsgFFDQicXtaR/w9VW2Qkb1qJu43Bmz5g GpRC4jEtCjjfMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAAX1uLFpi 2scUfsKL9d3xcwtmbeKb2Nsb4PIKR9tm9HnmvqPG1oZvGkxKXvySqBnMvKxpPJo1oY0Nd7uS s6Gw6GXV57Eczh7TH/GhE1KE85UEugwN7QWXL4bwyKQonJmjKrU62wkpW4y3M9BTu0JQAmEL SIuCokQM86Og2W3L7PZgIk5BHxWk3XMURTdAz4aIkOk9HlTi3C68Wf/DP+qM/enFxmdrZQVj t3eoqS7GmNfzbIMSVDNglvYg1bI/P7oVPHAp41OL82Ch0sGeCrTiU2YSA3wqzpDt9IIb4YuV RK2Z0LWhipBLg15AUiEoTbRFWEtMYxXZ9df+dJ0cA/lOJwuKJ1P1iAqUGTW+xL4fQj7tGi/h DyRlKzlAnJzR6M0HlwLazVELaohLTyHzYMW6t9VkT8YwzHa8I63tJ279wVfVdbm/mZrgo1KV KWe1RbwsR5+jk/4WZaCbCJzD+EkOCyKWmRMqmWAA0LgSyrJpsLBd2M79zjo/AB8ZsQGAijl3 ekYWVUHQbGlvCfcyM/VuSmKg+MfZdkocxKtLWPXnmhvBsIJ2zsgaAWjLLb8GTLMYe0G3BoZZ kB0xdwBilfoJ3sJILCLVoG8ElpwE6hDVaA+68ileZHvMoEa9K3X1jheo+zstZK3QPRjUTi4S cutMRWbp+aSt43iiV5XaHW97JQ0AAAAAAAA= --------------ms090906050805080905000008--