From owner-freebsd-net@FreeBSD.ORG Fri May 13 13:32:44 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3EC3106566B; Fri, 13 May 2011 13:32:44 +0000 (UTC) (envelope-from rj@cyclaero.com) Received: from mo-p00-ob6.rzone.de (mo-p00-ob6.rzone.de [IPv6:2a01:238:20a:202:53f0::1]) by mx1.freebsd.org (Postfix) with ESMTP id E43C18FC12; Fri, 13 May 2011 13:32:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1305293562; l=9914; s=domk; d=cyclaero.com; h=To:References:Content-Transfer-Encoding:Cc:Date:In-Reply-To:From: Content-Type:Mime-Version:Subject:X-RZG-CLASS-ID:X-RZG-AUTH; bh=RKltJEwxAboHF4di9HSyfgqL8zo=; b=Y/XZtnvtIsfcaVM91sIznDFdSBU9SU+43etR03QiC4tV8XSTDxVNXnATfG+CA9T/1L6 +f+Ix8++W/Z44eHEMj6xsBwMyFXhb+5lJo9s8OEmkTzK8CQVgG4DpvbNiT8cmZePeh12S CAXS/dcsO03qiWr5Ezg4MAzLe9C7KXpyxgw= X-RZG-AUTH: :O2kGeEG7b/pS1E6gSHOyjPKyNsg/5l1He+DgCy9/8FSej6CwUyslcvR13AejfvgZQ88TEw== X-RZG-CLASS-ID: mo00 Received: from rolf.projectworld.net (189-54-106-151-nd.cpe.vivax.com.br [189.54.106.151]) by post.strato.de (jimi mo12) (RZmta 25.18) with (AES128-SHA encrypted) ESMTPA id N00d48n4DCEchH ; Fri, 13 May 2011 15:32:40 +0200 (MEST) Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Dr. Rolf Jansen In-Reply-To: <20110512090221.GA647@zeninc.net> Date: Fri, 13 May 2011 10:32:36 -0300 Content-Transfer-Encoding: quoted-printable Message-Id: <4705CFB3-020C-4D70-B08B-6544FC727E0E@cyclaero.com> References: <042051F4-D309-4317-BBE5-5DF9DEEB342C@cyclaero.com> <20110512090221.GA647@zeninc.net> To: VANHULLEBUS Yvan X-Mailer: Apple Mail (2.1084) Cc: freebsd-net@freebsd.org Subject: Re: multiple clients behind the same NAT connecting a L2TP/IPsec VPN server behind another NAT X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 May 2011 13:32:45 -0000 Hello Yvan! Many thanks for your response. Am 12.05.2011 um 06:02 schrieb VANHULLEBUS Yvan: > On Wed, May 11, 2011 at 09:43:35PM -0300, Dr. Rolf Jansen wrote: >=20 >> The only remaining problem is, that from behind the same NAT only >> one client works well. As soon as a connection between a second >> client and the server has been established, the communication of >> both break down. The racoon log shows nothing noticeable here, and >> according to the log both connections are established successfully, >> anyhow, the communication is blocked. >=20 > Sounds like something (racoon ? kernel ? both ?) handles ports in a > bad way in transport mode.... >=20 > Could you send an example of such generated policies/SAs ? Here is the output of /usr/local/sbin/setkey -DP, once 2 clients behind = the same NAT are connected to the L2TP/IPsec-Server in the DMZ of = 70.71.72.73. The IP's are changed, and I re-grouped and tagged the = output. # DEFAULT POLICY 0.0.0.0/0[1701] 0.0.0.0/0[any] udp out ipsec esp/transport//require spid=3D30 seq=3D2 pid=3D3271 refcnt=3D1 # FIRST CONNECTION 192.168.0.100[50300] 70.71.72.73[1701] udp in ipsec esp/transport//unique:1 created: May 12 19:37:20 2011 lastused: May 12 19:37:20 2011 lifetime: 3600(s) validtime: 0(s) spid=3D31 seq=3D4 pid=3D3271 refcnt=3D1 70.71.72.73[1701] 192.168.0.100[50300] udp out ipsec esp/transport//unique:1 created: May 12 19:37:20 2011 lastused: May 12 19:37:20 2011 lifetime: 3600(s) validtime: 0(s) spid=3D32 seq=3D1 pid=3D3271 refcnt=3D1 # SECOND CONNECTION 192.168.0.200[49196] 70.71.72.73[1701] udp in ipsec esp/transport//unique:2 created: May 12 19:37:56 2011 lastused: May 12 19:37:56 2011 lifetime: 3600(s) validtime: 0(s) spid=3D33 seq=3D3 pid=3D3271 refcnt=3D1 70.71.72.73[1701] 192.168.0.200[49196] udp out ipsec esp/transport//unique:2 created: May 12 19:37:56 2011 lastused: May 12 19:37:56 2011 lifetime: 3600(s) validtime: 0(s) spid=3D34 seq=3D0 pid=3D3271 refcnt=3D1 >> racoon is configured to generate unique policies. >=20 > A bit more strange.... SAs should be really hard linked with SPDs, and > even with a confusion with ports, they should be considered as NOT be > for the same tunnel..... Here comes the output of racoonctl show-sa esp. Again, I changed the = IP's and re-grouped and tagged the output. As for the output of setkey = above, according to the time stamps, the first block belongs to the = first established connection, and the second block reflects the status = of the second connection. 192.168.1.1 is the local IP of the VPN-Server = in the DMZ. The address 80.81.82.83 is the public address of the NAT'ed = firewall of from where the two clients 192.168.0.100 and 192.168.0.200 = have established the connection. #FIRST CONNECTION 192.168.1.1[4500] 80.81.82.83[4500]=20 esp-udp mode=3Dtransport spi=3D197527017(0x0bc605e9) = reqid=3D1(0x00000001) E: aes-cbc 596a7dcf 5b25433e 155a33d2 d27e9499 A: hmac-sha1 ca7e8320 a965c807 f7e73238 0c7e2102 3a59c587 seq=3D0x0000001e replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:20 2011 current: May 12 19:41:53 2011 diff: 273(s) hard: 3600(s) soft: 2880(s) last: May 12 19:37:29 2011 hard: 0(s) soft: 0(s) current: 4976(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 30 hard: 0 soft: 0 sadb_seq=3D1 pid=3D3125 refcnt=3D1 80.81.82.83[4500] 192.168.1.1[4500]=20 esp-udp mode=3Dtransport spi=3D73066306(0x045ae742) = reqid=3D1(0x00000001) E: aes-cbc 91c6df1d 604a8eca 2c29b240 517b05c3 A: hmac-sha1 fe368cb6 d31e7b16 6cfb3410 7a8426fe 9246f9b3 seq=3D0x00000058 replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:20 2011 current: May 12 19:41:53 2011 diff: 273(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:43 2011 hard: 0(s) soft: 0(s) current: 11567(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 88 hard: 0 soft: 0 sadb_seq=3D0 pid=3D3125 refcnt=3D1 #SECOND CONNECTION 192.168.1.1[4500] 80.81.82.83[57670]=20 esp-udp mode=3Dtransport spi=3D67190636(0x04013f6c) = reqid=3D2(0x00000002) E: aes-cbc 3ce5335e 94832280 d27f2459 9f4465bf A: hmac-sha1 93b25d41 874432a2 87685009 a95bdf1f 003e8a49 seq=3D0x00000045 replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:56 2011 current: May 12 19:41:53 2011 diff: 237(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:39 2011 hard: 0(s) soft: 0(s) current: 11368(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 69 hard: 0 soft: 0 sadb_seq=3D3 pid=3D3125 refcnt=3D2 80.81.82.83[57670] 192.168.1.1[4500]=20 esp-udp mode=3Dtransport spi=3D95933843(0x05b7d593) = reqid=3D2(0x00000002) E: aes-cbc 83b2e655 e8a416d3 de50f74a 1fbac49b A: hmac-sha1 80481e75 933727f8 b1d21207 b0dd7113 45707403 seq=3D0x0000003a replay=3D4 flags=3D0x00000000 state=3Dmature=20 created: May 12 19:37:56 2011 current: May 12 19:41:53 2011 diff: 237(s) hard: 3600(s) soft: 2880(s) last: May 12 19:41:39 2011 hard: 0(s) soft: 0(s) current: 6497(bytes) hard: 0(bytes) soft: 0(bytes) allocated: 58 hard: 0 soft: 0 sadb_seq=3D2 pid=3D3125 refcnt=3D1 I can see one special thing. The first connection is on both sides = running on port 4500, while the second connection is using a different = port at the peer side. As a matter of fact, during these tests, = connection 2 was still working, but only connection 1 was blocked. >> When a client disconnects from the server, racoon usually purges 2 >> IPsec-SA shortly after. The interesting thing in the case of 2 >> clients from the same NAT is, that it purges one IPsec-SA from the >> client just disconnected, and 1 belonging to the client that is >> still connected. So, it seems that the internal SA house holding of >> racoon got confused. >=20 > See in racoon's debug (-dd to have more verbose) if decision comes > from racoon, from peer (I don't think so) or from the kernel (via a > PFKey message). Yesterday, it turned out to me that this effect shows up only with the = following setkey.conf: flush; spdflush; spdadd 0.0.0.0/0[1701] 0.0.0.0/0[0] udp -P out ipsec = esp/transport//require; spdadd 0.0.0.0/0[0] 0.0.0.0/0[1701] udp -P in ipsec = esp/transport//require; With this setup, the second client (192.168.0.200) could not establish a = connection, and only one half of it was dropped, together with another = half of the first connection (192.168.0.100): 2011-05-12 19:33:11: [80.81.82.83] INFO: DPD: remote (ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06) seems to be dead. 2011-05-12 19:33:11: INFO: purging ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06. 2011-05-12 19:33:11: DEBUG2: getph1: start 2011-05-12 19:33:11: DEBUG2: local: 192.168.1.1[4500] 2011-05-12 19:33:11: DEBUG2: remote: 80.81.82.83[40699] 2011-05-12 19:33:11: DEBUG2: p->local: 192.168.1.1[4500] 2011-05-12 19:33:11: DEBUG2: p->remote: 80.81.82.83[4500] 2011-05-12 19:33:11: DEBUG2: remote identity does match hint 2011-05-12 19:33:11: DEBUG2: no match 2011-05-12 19:33:11: DEBUG: call pfkey_send_dump 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 2011-05-12 19:33:11: DEBUG: pk_recv: retry[0] recv()=20 #### DROPPING ONE HALF OF THE SECOND CONNECTION 2011-05-12 19:33:11: INFO: deleting a generated policy. 2011-05-12 19:33:11: DEBUG: get a src address from ID payload = 192.168.0.200[49189] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: get dst address from ID payload = 70.71.72.73[1701] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(inbound) sent. 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(outbound) sent. 2011-05-12 19:33:11: DEBUG: IV freed 2011-05-12 19:33:11: INFO: purged IPsec-SA spi=3D243828999. #### DROPPING ONE HALF OF THE FIRST CONNECTION 2011-05-12 19:33:11: INFO: deleting a generated policy. 2011-05-12 19:33:11: DEBUG: get a src address from ID payload = 192.168.0.100[50287] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: get dst address from ID payload = 70.71.72.73[1701] prefixlen=3D32 ul_proto=3D17 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(inbound) sent. 2011-05-12 19:33:11: DEBUG: call pfkey_send_spddelete 2011-05-12 19:33:11: DEBUG: pfkey spddelete(outbound) sent. 2011-05-12 19:33:11: DEBUG: IV freed 2011-05-12 19:33:11: INFO: purged IPsec-SA spi=3D6251846. 2011-05-12 19:33:11: INFO: purged ISAKMP-SA = spi=3D2b64453a611ace30:dd38274322e05e06. This DOES NOT happen, with my current setkey.conf: flush; spdflush; spdadd 0.0.0.0/0[1701] 0.0.0.0/0[0] udp -P out ipsec = esp/transport//require; With this, a second client can successfully establish a connection. Only = the communication over the first connection is blocked somehow after the = second one has been established. > There is "some chance", but this may involve userland and/or kernel > patching... I am pretty comfortable in programming in C (and others), so I don't = fear patching anything. >> BTW: Using only mpd5, I setup also a PPTP-VPN server running in >> parallel to the L2TP/IPsec one. Multiple PPTP-VPN clients behind the >> same NAT work perfectly well with my server - So, I tend to believe >> that it is really an issue with the IPsec part and not with the L2TP >> (mpd5) part of my setup. >=20 > I don't know MPD so much, ... Yeah, this is OK, since just recently Alexander Motin, who is one of the = programmers of MPD5, wrote to me: "I am not an IPsec expert...". :-) Many thanks for taking your time for looking into my problems! Best regards Rolf