Date: Fri, 23 Nov 2001 20:44:44 +0200 From: Peter Pentchev <roam@ringlet.net> To: security@FreeBSD.org Subject: IPsec tunnel (manual keying) configuration problem Message-ID: <20011123204444.A1304@straylight.oblivion.bg>
next in thread | raw e-mail | index | archive | help
Hi, I'm having a IPsec configuration problem, whereby the two endpoints tunnelling two LAN's fail to see packets to their own "internal" addresses. One of the hosts, the so-called 'portal', is a two-NIC machine with a couple of extras: xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 217.75.128.47 netmask 0xffffff00 broadcast 217.75.128.255 ether 00:50:04:52:62:d2 media: Ethernet 100baseTX status: active xl1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500 inet 217.75.134.1 netmask 0xffffffc0 broadcast 217.75.134.63 inet 217.75.134.11 netmask 0xffffffff broadcast 217.75.134.11 inet6 3ffe:400:10c0::1 prefixlen 64 ether 00:04:76:18:65:aa media: Ethernet autoselect (100baseTX <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 ppp0: flags=8010<POINTOPOINT,MULTICAST> mtu 1500 sl0: flags=c010<POINTOPOINT,LINK2,MULTICAST> mtu 552 faith0: flags=8001<UP,MULTICAST> mtu 1500 stf0: flags=1<UP> mtu 1280 gif1: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280 tunnel inet 217.75.134.1 --> 217.75.128.46 gif2: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1280 tunnel inet 217.75.134.1 --> 128.176.191.66 tun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 1524 inet 172.16.32.5 --> 172.16.32.1 netmask 0xffff0000 Opened by PID 190 tun1: flags=8010<POINTOPOINT,MULTICAST> mtu 1500 -------- end of ifconfig.portal At the time of the problem, its routing table read: Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 217.75.128.1 UGSc 4 14141 xl0 127.0.0.1 127.0.0.1 UH 0 0 lo0 172.16/12 172.16.32.1 UGSc 1 23267 tun0 172.16.32.1 172.16.32.5 UH 2 0 tun0 217.75.128 link#1 UC 5 0 xl0 217.75.128.1 0:1:42:66:cd:0 UHLW 3 0 xl0 1188 217.75.128.2 0:50:4:57:e:c5 UHLW 0 34 xl0 1143 217.75.128.9 0:50:da:51:16:60 UHLW 1 796 xl0 1157 217.75.128.21 0:10:7b:14:4c:74 UHLW 2 0 xl0 962 217.75.128.252 0:60:8c:cb:43:c7 UHLW 0 76 xl0 1002 217.75.134.0 ff:ff:ff:ff:ff:ff UHLWb 0 6 xl1 => 217.75.134/26 link#2 UC 7 0 xl1 217.75.134.1 0:4:76:18:65:aa UHLW 0 3 lo0 217.75.134.9 0:4:76:21:d9:76 UHLW 0 3505 xl1 1151 217.75.134.10 0:1:2:1c:7e:2 UHLW 0 9945 xl1 830 217.75.134.11/32 link#2 UC 0 0 xl1 217.75.134.13 0:1:2:1c:7e:2 UHLW 0 26948 xl1 816 217.75.134.18 0:1:2:1c:7e:2 UHLW 0 88 xl1 18 217.75.134.63 ff:ff:ff:ff:ff:ff UHLWb 0 4 xl1 217.75.134.64/29 link#2 UCSc 1 0 xl1 217.75.134.72/29 217.75.130.66 UGSc 0 280 xl0 217.75.134.96/27 217.75.128.21 UGSc 1 61926 xl0 ------------ end of netstat -rnfinet for portal The other host, called 'vn', has only one network card: xl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500 inet 192.168.9.1 netmask 0xffffff00 broadcast 192.168.9.255 inet 217.75.130.66 netmask 0xfffffffc broadcast 217.75.130.67 inet 192.168.9.2 netmask 0xffffffff broadcast 192.168.9.2 inet 217.75.134.73 netmask 0xfffffff8 broadcast 217.75.134.79 ether 00:04:76:9e:d8:a7 media: Ethernet autoselect (100baseTX <full-duplex>) status: active lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 inet 127.0.0.1 netmask 0xff000000 ----------- end of ifconfig.vn And at the time of the problem, its routing table was: Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire 127.0.0.1 127.0.0.1 UH 0 0 lo0 192.168.9 link#1 UC 2 0 xl0 192.168.9.1 0:4:76:9e:d8:a7 UHLW 0 10 lo0 192.168.9.2 0:4:76:9e:d8:a7 UHLW 1 46 lo0 => 192.168.9.2/32 link#1 UC 1 0 xl0 192.168.9.13 0:e0:18:18:f2:e UHLW 0 5436 xl0 943 217.75.128.47 217.75.130.65 UGHS 7 10463 xl0 217.75.130.64/30 link#1 UC 1 0 xl0 217.75.130.65 0:1:42:3:4e:e4 UHLW 1 294 xl0 176 217.75.134/26 217.75.128.47 UGSc 2 238 xl0 217.75.134.72/29 link#1 UC 2 0 xl0 217.75.134.73 0:4:76:9e:d8:a7 UHLW 1 17 lo0 217.75.134.74 0:e0:18:18:f2:e UHLW 1 148 xl0 294 --------------- end of netstat -rnfinet for vn The IPsec configuration files (fed to setkey -c) are: ---- portal: # Start in the clear: flush all rules flush ; spdflush ; # # Regional offices # # - Varna # spdadd 217.75.134.0/26 217.75.134.72/29 any -P out ipsec ah/tunnel/217.75.128.47-217.75.130.66/require ; spdadd 217.75.134.72/29 217.75.134.0/26 any -P in ipsec ah/tunnel/217.75.130.66-217.75.128.47/require ; add 217.75.128.47 217.75.130.66 ah-old 0x100103 -m any -A keyed-md5 "a 16char pass :P" ; add 217.75.130.66 217.75.128.47 ah-old 0x100104 -m any -A keyed-md5 "another password" ; ---- vn: # Flush all rules flush ; spdflush ; # # The NOC at Bulgaria Online # spdadd 217.75.134.72/29 217.75.134.0/26 any -P out ipsec ah/tunnel/217.75.130.66-217.75.128.47/require ; spdadd 217.75.134.0/26 217.75.134.72/29 any -P in ipsec ah/tunnel/217.75.128.47-217.75.130.66/require ; add 217.75.130.66 217.75.128.47 ah-old 0x100104 -m any -A keyed-md5 "another password" ; add 217.75.128.47 217.75.130.66 ah-old 0x100103 -m any -A keyed-md5 "a 16char pass :P" ; ---- end of IPsec config Now for the problem itself :) After setting up the IPsec connection, the situation is as follows: - 217.75.134.74 (behind vn) to 217.75.134.10 (behind portal) OK - 217.75.134.74 (behind vn) to 217.75.134.1 (portal itself) FAIL - 217.75.134.73 (vn itself) to 218.75.134.10 (behind portal) FAIL - 217.75.134.73 (vn itself) to 217.75.134.1 (portal itself) FAIL Logs from 'tcpdump -nli xl0 -s 1500 host 217.75.128.47' ran on vn: -------- host behind vn to host behind portal (OK) tcpdump: listening on xl0 20:25:35.441768 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x9bc32f2d): 217.75.134.74.1109 > 217.75.134.10.22: S [tcp sum ok] 4036805732:4036805732(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 557883 0> (DF) [tos 0x10] (ttl 63, id 1130, len 60) [tos 0x10] (ttl 64, id 299, len 104, bad cksum 0!) 20:25:35.458566 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0xd8cf8cd6): 217.75.134.10.22 > 217.75.134.74.1109: S [tcp sum ok] 3978490178:3978490178(0) ack 4036805733 win 17376 <mss 1460,nop,wscale 0,nop,nop,timestamp 331880891 557883> (DF) (ttl 63, id 55805, len 60) (ttl 61, id 234, len 104) 20:25:35.458796 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x650382ec): 217.75.134.74.1109 > 217.75.134.10.22: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 557885 331880891> (DF) [tos 0x10] (ttl 63, id 3364, len 52) [tos 0x10] (ttl 64, id 300, len 96, bad cksum 0!) 20:25:35.478764 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0xa51ef937): 217.75.134.10.22 > 217.75.134.74.1109: P [tcp sum ok] 1:53(52) ack 1 win 17376 <nop,nop,timestamp 331880893 557885> (DF) (ttl 63, id 3203, len 104) (ttl 61, id 235, len 148) 20:25:35.577099 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x3b41569): 217.75.134.74.1109 > 217.75.134.10.22: . [tcp sum ok] 1:1(0) ack 53 win 17376 <nop,nop,timestamp 557897 331880893> (DF) [tos 0x10] (ttl 63, id 9477, len 52) [tos 0x10] (ttl 64, id 301, len 96, bad cksum 0!) 20:25:42.099448 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x6641983d): 217.75.134.74.1109 > 217.75.134.10.22: F [tcp sum ok] 1:1(0) ack 53 win 17376 <nop,nop,timestamp 558549 331880893> (DF) [tos 0x10] (ttl 63, id 12887, len 52) [tos 0x10] (ttl 64, id 302, len 96, bad cksum 0!) 20:25:42.113415 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x7086a783): 217.75.134.10.22 > 217.75.134.74.1109: . [tcp sum ok] 53:53(0) ack 2 win 17376 <nop,nop,timestamp 331881556 558549> (DF) (ttl 63, id 17609, len 52) (ttl 61, id 236, len 96) 20:25:42.116880 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x79d0d138): 217.75.134.10.22 > 217.75.134.74.1109: F [tcp sum ok] 53:53(0) ack 2 win 17376 <nop,nop,timestamp 331881557 558549> (DF) (ttl 63, id 8410, len 52) (ttl 61, id 237, len 96) 20:25:42.117077 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x140ec93): 217.75.134.74.1109 > 217.75.134.10.22: . [tcp sum ok] 2:2(0) ack 54 win 17375 <nop,nop,timestamp 558550 331881557> (DF) [tos 0x10] (ttl 63, id 50496, len 52) [tos 0x10] (ttl 64, id 303, len 96, bad cksum 0!) ------------ host behind vn to portal itself (FAIL) tcpdump: listening on xl0 20:24:50.279253 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x3dcec5fd): 217.75.134.74.1107 > 217.75.134.1.22: S [tcp sum ok] 3308045531:3308045531(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 553367 0> (DF) [tos 0x10] (ttl 63, id 47298, len 60) [tos 0x10] (ttl 64, id 291, len 104, bad cksum 0!) 20:24:53.271523 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x1f3e68ca): 217.75.134.74.1107 > 217.75.134.1.22: S [tcp sum ok] 3308045531:3308045531(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 553667 0> (DF) [tos 0x10] (ttl 63, id 41118, len 60) [tos 0x10] (ttl 64, id 292, len 104, bad cksum 0!) 20:24:56.271906 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x35b524de): 217.75.134.74.1107 > 217.75.134.1.22: S [tcp sum ok] 3308045531:3308045531(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 553967 0> (DF) [tos 0x10] (ttl 63, id 52166, len 60) [tos 0x10] (ttl 64, id 293, len 104, bad cksum 0!) 20:24:59.272356 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xdc5787db): 217.75.134.74.1107 > 217.75.134.1.22: S [tcp sum ok] 3308045531:3308045531(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 63, id 42178, len 44) [tos 0x10] (ttl 64, id 294, len 88, bad cksum 0!) ------------- vn itself to portal itself (FAIL) tcpdump: listening on xl0 20:28:40.050942 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xa7127819): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1014198 0> (DF) [tos 0x10] (ttl 64, id 53427, len 60) [tos 0x10] (ttl 64, id 304, len 104, bad cksum 0!) 20:28:43.047830 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xc6e7cae3): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1014498 0> (DF) [tos 0x10] (ttl 64, id 4095, len 60) [tos 0x10] (ttl 64, id 305, len 104, bad cksum 0!) 20:28:46.047863 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x28466906): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1014798 0> (DF) [tos 0x10] (ttl 64, id 52608, len 60) [tos 0x10] (ttl 64, id 306, len 104, bad cksum 0!) 20:28:49.047896 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x5289ac1f): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 24215, len 44) [tos 0x10] (ttl 64, id 307, len 88, bad cksum 0!) 20:28:52.047937 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x371ee2d8): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 35456, len 44) [tos 0x10] (ttl 64, id 308, len 88, bad cksum 0!) 20:28:55.047969 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xf803410b): 217.75.134.73.1046 > 217.75.134.1.22: S [tcp sum ok] 905787053:905787053(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 43261, len 44) [tos 0x10] (ttl 64, id 309, len 88, bad cksum 0!) ------------- vn itself to host behind portal (FAIL) tcpdump: listening on xl0 20:29:09.460730 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x1c53e07): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1017139 0> (DF) [tos 0x10] (ttl 64, id 8588, len 60) [tos 0x10] (ttl 64, id 310, len 104, bad cksum 0!) 20:29:09.478706 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x90226b35): 217.75.134.10.22 > 217.75.134.73.1047: S [tcp sum ok] 956821190:956821190(0) ack 920219842 win 17376 <mss 1460,nop,wscale 0,nop,nop,timestamp 331902291 1017139> (DF) (ttl 63, id 27769, len 60) (ttl 61, id 238, len 104) 20:29:12.458160 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xb0f952e5): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1017439 0> (DF) [tos 0x10] (ttl 64, id 64461, len 60) [tos 0x10] (ttl 64, id 311, len 104, bad cksum 0!) 20:29:12.469876 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x81e6ffb3): 217.75.134.10.22 > 217.75.134.73.1047: S [tcp sum ok] 956821190:956821190(0) ack 920219842 win 17376 <mss 1460,nop,wscale 0,nop,nop,timestamp 331902591 1017139> (DF) (ttl 63, id 13929, len 60) (ttl 61, id 239, len 104) 20:29:12.474621 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x699e1c14): 217.75.134.10.22 > 217.75.134.73.1047: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 331902591 1017439> (DF) (ttl 63, id 44865, len 52) (ttl 61, id 240, len 96) 20:29:15.458207 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xa4ccca90): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460,nop,wscale 0,nop,nop,timestamp 1017739 0> (DF) [tos 0x10] (ttl 64, id 40589, len 60) [tos 0x10] (ttl 64, id 312, len 104, bad cksum 0!) 20:29:15.475532 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x5ce20964): 217.75.134.10.22 > 217.75.134.73.1047: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 331902891 1017739> (DF) (ttl 63, id 56069, len 52) (ttl 61, id 241, len 96) 20:29:18.458225 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0x9afbb58d): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 63933, len 44) [tos 0x10] (ttl 64, id 313, len 88, bad cksum 0!) 20:29:18.477070 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x62f6b0c2): 217.75.134.10.22 > 217.75.134.73.1047: S [tcp sum ok] 956821190:956821190(0) ack 920219842 win 17376 <mss 1460,nop,wscale 0,nop,nop,timestamp 331903191 1017739> (DF) (ttl 63, id 60770, len 60) (ttl 61, id 242, len 104) 20:29:18.480330 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x5d5eca31): 217.75.134.10.22 > 217.75.134.73.1047: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 331903191 1017739> (DF) (ttl 63, id 26186, len 52) (ttl 61, id 243, len 96) 20:29:21.458268 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xb176762f): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 37624, len 44) [tos 0x10] (ttl 64, id 314, len 88, bad cksum 0!) 20:29:21.474610 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x8219bff6): 217.75.134.10.22 > 217.75.134.73.1047: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 331903491 1017739> (DF) (ttl 63, id 46620, len 52) (ttl 61, id 244, len 96) 20:29:24.458301 217.75.130.66 > 217.75.128.47: AH(spi=0x00100104,sumlen=16,seq=0xf3f9d722): 217.75.134.73.1047 > 217.75.134.10.22: S [tcp sum ok] 920219841:920219841(0) win 16384 <mss 1460> (DF) [tos 0x10] (ttl 64, id 1784, len 44) [tos 0x10] (ttl 64, id 315, len 88, bad cksum 0!) 20:29:24.471233 217.75.128.47 > 217.75.130.66: AH(spi=0x00100103,sumlen=16,seq=0x146f4b4c): 217.75.134.10.22 > 217.75.134.73.1047: . [tcp sum ok] 1:1(0) ack 1 win 17376 <nop,nop,timestamp 331903791 1017739> (DF) (ttl 63, id 40803, len 52) (ttl 61, id 245, len 96) The way I read those logs, vn and portal forward packets to other hosts just fine. However, when a packet arrives for the endpoints themselves, it somehow does not reach the TCP stack or something - at least it does not reach the part where the handshake SYN's and ACK's are processed. A connection to portal shows just initial SYN's on the wire, portal does not process them at all. A similar tcpdump ran on portal at the time shows *just the same* - even portal's TCP stack does not receive/process the SYN :( A connection from vn to a host behind portal shows the SYN/ACK arriving back at vn, but then vn keeps retransmitting its SYN - it has neither received the ACK, nor the other side's SYN :( Any help or just ideas would be welcome.. G'luck, Peter -- When you are not looking at it, this sentence is in Spanish. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20011123204444.A1304>