Date: Fri, 13 May 2011 10:32:36 -0300 From: Dr. Rolf Jansen <rj@cyclaero.com> To: VANHULLEBUS Yvan <vanhu@FreeBSD.org> Cc: freebsd-net@freebsd.org Subject: Re: multiple clients behind the same NAT connecting a L2TP/IPsec VPN server behind another NAT Message-ID: <4705CFB3-020C-4D70-B08B-6544FC727E0E@cyclaero.com> In-Reply-To: <20110512090221.GA647@zeninc.net> References: <042051F4-D309-4317-BBE5-5DF9DEEB342C@cyclaero.com> <20110512090221.GA647@zeninc.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello Yvan! Many thanks for your response. Am 12.05.2011 um 06:02 schrieb VANHULLEBUS Yvan: > On Wed, May 11, 2011 at 09:43:35PM -0300, Dr. Rolf Jansen wrote: >=20 >> The only remaining problem is, that from behind the same NAT only >> one client works well. As soon as a connection between a second >> client and the server has been established, the communication of >> both break down. The racoon log shows nothing noticeable here, and >> according to the log both connections are established successfully, >> anyhow, the communication is blocked. >=20 > Sounds like something (racoon ? kernel ? both ?) handles ports in a > bad way in transport mode.... >=20 > Could you send an example of such generated policies/SAs ? Here is the output of /usr/local/sbin/setkey -DP, once 2 clients behind = the same NAT are connected to the L2TP/IPsec-Server in the DMZ of = 70.71.72.73. The IP's are changed, and I re-grouped and tagged the = output. # DEFAULT POLICY 0.0.0.0/0[1701] 0.0.0.0/0[any] udp out ipsec esp/transport//require spid=3D30 seq=3D2 pid=3D3271 refcnt=3D1 # FIRST CONNECTION 192.168.0.100[50300] 70.71.72.73[1701] udp in ipsec esp/transport//unique:1 created: May 12 19:37:20 2011 lastused: May 12 19:37:20 2011 lifetime: 3600(s) validtime: 0(s) spid=3D31 seq=3D4 pid=3D3271 refcnt=3D1 70.71.72.73[1701] 192.168.0.100[50300] udp out ipsec esp/transport//unique:1 created: May 12 19:37:20 2011 lastused: May 12 19:37:20 2011 lifetime: 3600(s) validtime: 0(s) spid=3D32 seq=3D1 pid=3D3271 refcnt=3D1 # SECOND CONNECTION 192.168.0.200[49196] 70.71.72.73[1701] udp in ipsec esp/transport//unique:2 created: May 12 19:37:56 2011 lastused: May 12 19:37:56 2011 lifetime: 3600(s) validtime: 0(s) spid=3D33 seq=3D3 pid=3D3271 refcnt=3D1 70.71.72.73[1701] 192.168.0.200[49196] udp out ipsec esp/transport//unique:2 created: May 12 19:37:56 2011 lastused: May 12 19:37:56 2011 lifetime: 3600(s) validtime: 0(s) spid=3D34 seq=3D0 pid=3D3271 refcnt=3D1 >> racoon is configured to generate unique policies. >=20 > A bit more strange.... SAs should be really hard linked with SPDs, and > even with a confusion with ports, they should be considered as NOT be > for the same tunnel..... Here comes the output of racoonctl show-sa esp. Again, I changed the = IP's and re-grouped and tagged the output. As for the output of setkey = above, according to the time stamps, the first block belongs to the = first established connection, and the second block reflects the status = of the second connection. 192.168.1.1 is the local IP of the VPN-Server = in the DMZ. The address 80.81.82.83 is the public address of the NAT'ed = firewall of from where the two clients 192.168.0.100 and 192.168.0.200 = have established the connection. #FIRST CONNECTION 192.168.1.1[4500] 80.81.82.83[4500]=20 esp-udp mode=3Dtransport spi=3D197527017(0x0bc605e9) = reqid=3D1(0x00000001) E: aes-cbc 596a7dcf 5b25433e 155a33d2 d27e9499 A: hmac-sha1 ca7e8320 a965c807 f7e73238 0c7e2102 3a59c587 seq=3D0x0000001e replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:20 2011 current: May 12 19:41:53 2011 diff: 273(s) hard: 3600(s) soft: 2880(s) last: May 12 19:37:29 2011 hard: 0(s) soft: 0(s) current: 4976(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 30 hard: 0 soft: 0 sadb_seq=3D1 pid=3D3125 refcnt=3D1 80.81.82.83[4500] 192.168.1.1[4500]=20 esp-udp mode=3Dtransport spi=3D73066306(0x045ae742) = reqid=3D1(0x00000001) E: aes-cbc 91c6df1d 604a8eca 2c29b240 517b05c3 A: hmac-sha1 fe368cb6 d31e7b16 6cfb3410 7a8426fe 9246f9b3 seq=3D0x00000058 replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:20 2011 current: May 12 19:41:53 2011 diff: 273(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:43 2011 hard: 0(s) soft: 0(s) current: 11567(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 88 hard: 0 soft: 0 sadb_seq=3D0 pid=3D3125 refcnt=3D1 #SECOND CONNECTION 192.168.1.1[4500] 80.81.82.83[57670]=20 esp-udp mode=3Dtransport spi=3D67190636(0x04013f6c) = reqid=3D2(0x00000002) E: aes-cbc 3ce5335e 94832280 d27f2459 9f4465bf A: hmac-sha1 93b25d41 874432a2 87685009 a95bdf1f 003e8a49 seq=3D0x00000045 replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:56 2011 current: May 12 19:41:53 2011 diff: 237(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:39 2011 hard: 0(s) soft: 0(s) current: 11368(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 69 hard: 0 soft: 0 sadb_seq=3D3 pid=3D3125 refcnt=3D2 80.81.82.83[57670] 192.168.1.1[4500]=20 esp-udp mode=3Dtransport spi=3D95933843(0x05b7d593) = reqid=3D2(0x00000002) E: aes-cbc 83b2e655 e8a416d3 de50f74a 1fbac49b A: hmac-sha1 80481e75 933727f8 b1d21207 b0dd7113 45707403 seq=3D0x0000003a replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:56 2011 current: May 12 19:41:53 2011 diff: 237(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:39 2011 hard: 0(s) soft: 0(s) current: 6497(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 58 hard: 0 soft: 0 sadb_seq=3D2 pid=3D3125 refcnt=3D1 I can see one special thing. The first connection is on both sides = running on port 4500, while the second connection is using a different = port at the peer side. As a matter of fact, during these tests, = connection 2 was still working, but only connection 1 was blocked. >> When a client disconnects from the server, racoon usually purges 2 >> IPsec-SA shortly after. The interesting thing in the case of 2 >> clients from the same NAT is, that it purges one IPsec-SA from the >> client just disconnected, and 1 belonging to the client that is >> still connected. So, it seems that the internal SA house holding of >> racoon got confused. >=20 > See in racoon's debug (-dd to have more verbose) if decision comes > from racoon, from peer (I don't think so) or from the kernel (via a > PFKey message). Yesterday, it turned out to me that this effect shows up only with the = following setkey.conf: flush; spdflush; spdadd 0.0.0.0/0[1701] 0.0.0.0/0[0] udp -P out ipsec = esp/transport//require; spdadd 0.0.0.0/0[0] 0.0.0.0/0[1701] udp -P in ipsec = esp/transport//require; With this setup, the second client (192.168.0.200) could not establish a = connection, and only one half of it was dropped, together with another = half of the first connection (192.168.0.100): 2011-05-12 19:33:11: [80.81.82.83] INFO: DPD: remote (ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06) seems to be dead. 2011-05-12 19:33:11: INFO: purging ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06. 2011-05-12 19:33:11: DEBUG2: getph1: start 2011-05-12 19:33:11: DEBUG2: local: 192.168.1.1[4500] 2011-05-12 19:33:11: DEBUG2: remote: 80.81.82.83[40699] 2011-05-12 19:33:11: DEBUG2: p->local: 192.168.1.1[4500] 2011-05-12 19:33:11: DEBUG2: p->remote: 80.81.82.83[4500] 2011-05-12 19:33:11: DEBUG2: remote identity does match hint 2011-05-12 19:33:11: DEBUG2: no match 2011-05-12 19:33:11: DEBUG: call pfkey_send_dump 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 #### DROPPING ONE HALF OF THE SECOND CONNECTION 2011-05-12 19:33:11: INFO: deleting a generated policy. 2011-05-12 19:33:11: DEBUG: get a src address from ID payload = 192.168.0.200[49189] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: get dst address from ID payload = 70.71.72.73[1701] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(inbound) sent. 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(outbound) sent. 2011-05-12 19:33:11: DEBUG: IV freed 2011-05-12 19:33:11: INFO: purged IPsec-SA spi=3D243828999. #### DROPPING ONE HALF OF THE FIRST CONNECTION 2011-05-12 19:33:11: INFO: deleting a generated policy. 2011-05-12 19:33:11: DEBUG: get a src address from ID payload = 192.168.0.100[50287] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: get dst address from ID payload = 70.71.72.73[1701] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(inbound) sent. 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(outbound) sent. 2011-05-12 19:33:11: DEBUG: IV freed 2011-05-12 19:33:11: INFO: purged IPsec-SA spi=3D6251846. 2011-05-12 19:33:11: INFO: purged ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06. This DOES NOT happen, with my current setkey.conf: flush; spdflush; spdadd 0.0.0.0/0[1701] 0.0.0.0/0[0] udp -P out ipsec = esp/transport//require; With this, a second client can successfully establish a connection. Only = the communication over the first connection is blocked somehow after the = second one has been established. > There is "some chance", but this may involve userland and/or kernel > patching... I am pretty comfortable in programming in C (and others), so I don't = fear patching anything. >> BTW: Using only mpd5, I setup also a PPTP-VPN server running in >> parallel to the L2TP/IPsec one. Multiple PPTP-VPN clients behind the >> same NAT work perfectly well with my server - So, I tend to believe >> that it is really an issue with the IPsec part and not with the L2TP >> (mpd5) part of my setup. >=20 > I don't know MPD so much, ... Yeah, this is OK, since just recently Alexander Motin, who is one of the = programmers of MPD5, wrote to me: "I am not an IPsec expert...". :-) Many thanks for taking your time for looking into my problems! Best regards Rolf
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4705CFB3-020C-4D70-B08B-6544FC727E0E>