Date: Mon, 24 Jul 2017 17:15:12 +0200 From: "Muenz, Michael" <m.muenz@spam-fetish.org> To: freebsd-net@freebsd.org Subject: Re: NAT before IPSEC - reply packets stuck at enc0 Message-ID: <1e889acf-49d1-b70f-7097-82e6e4dfabb6@spam-fetish.org> In-Reply-To: <c738380c-e0cc-2d32-934e-a05502887b93@yandex.ru> References: <459d59f7-2895-8aed-d547-be46a0fbb918@spam-fetish.org> <a082662c-145e-0132-18ef-083adaa59c33@yandex.ru> <1c0de616-91ff-a6f9-d946-f098bc1a709f@spam-fetish.org> <911903d1-f353-d5d6-d400-d86150f88136@yandex.ru> <2d607e1a-a2c0-0f85-1530-c478962a76cd@spam-fetish.org> <3344e189-cdf0-a2c9-3a2a-645460866f2d@yandex.ru> <1279753e-9ad1-2c02-304e-5001e2bbc82f@spam-fetish.org> <15e6eb38-ef0c-7bfd-5f2c-d2acc8ea1af4@yandex.ru> <cdb7e172-4074-4559-1e91-90c8e9276134@spam-fetish.org> <63e80fcf-915e-2dd5-d8c9-1904c8261c6f@yandex.ru> <1c91cd8f-105d-e886-3126-67505c6c3900@spam-fetish.org> <c738380c-e0cc-2d32-934e-a05502887b93@yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Am 24.07.2017 um 13:18 schrieb Andrey V. Elsukov:
>
> Ok, let's try to debug the problem. Please, use 11.1-RC, it has
> significantly changed IPsec stack.
>
> Apply attached patch to if_enc(4), it makes if_enc a bit useful for
> debugging your problem. You need to rebuild and reinstall
> sys/modules/if_enc.
>
> Now enable verbose BPF logging:
> net.enc.out.ipsec_bpf_mask=3
> net.enc.in.ipsec_bpf_mask=3
>
> According your tcpdump output, you need to set
> net.enc.out.ipsec_filter_mask=2
>
> Show what you will see in the `tcpdump -nvi enc0` with such config
> options. Also, show what you have in the `sysctl net.inet.ip.fw` and
> `ipfw show` output.
>
Great! The guys from OPNsense built me a custom 11.1 kernel with your patch.
Here's one packet on enc0:
root@PB-FW1-FRA:~ # tcpdump -vni enc0
tcpdump: listening on enc0, link-type ENC (OpenBSD encapsulated IP),
capture size 262144 bytes
17:07:41.769313 (authentic,confidential): SPI 0x90b7ba72: IP (tos 0x0,
ttl 63, id 27752, offset 0, flags [none], proto ICMP (1), length 28, bad
cksum b72d (->b82d)!)
10.26.1.1 > 10.24.66.25: ICMP echo request, id 41163, seq 28416,
length 8
17:07:41.777223 (authentic,confidential): SPI 0xcaae18f8: IP (tos 0x0,
ttl 58, id 44180, offset 0, flags [none], proto IPIP (4), length 48)
81.24.74.3 > 213.244.192.191: IP (tos 0x0, ttl 63, id 46327, offset
0, flags [none], proto ICMP (1), length 28)
10.24.66.25 > 10.26.1.1: ICMP echo reply, id 41163, seq 28416, length 8
17:07:41.777240 (authentic,confidential): SPI 0xcaae18f8: IP (tos 0x0,
ttl 63, id 46327, offset 0, flags [none], proto ICMP (1), length 28)
10.26.1.1 > 10.26.1.1: ICMP echo reply, id 33347, seq 28416, length 8
17:07:41.846588 (authentic,confidential): SPI 0x90b7ba72: IP (tos 0x0,
ttl 63, id 61607, offset 0, flags [none], proto ICMP (1), length 28, bad
cksum 32ee (->33ee)!)
10.26.1.1 > 10.24.66.25: ICMP echo request, id 45562, seq 58116,
length 8
17:07:41.854692 (authentic,confidential): SPI 0xcaae18f8: IP (tos 0x0,
ttl 58, id 44196, offset 0, flags [none], proto IPIP (4), length 48)
81.24.74.3 > 213.244.192.191: IP (tos 0x0, ttl 63, id 46335, offset
0, flags [none], proto ICMP (1), length 28)
10.24.66.25 > 10.26.1.1: ICMP echo reply, id 45562, seq 58116, length 8
17:07:41.854706 (authentic,confidential): SPI 0xcaae18f8: IP (tos 0x0,
ttl 63, id 46335, offset 0, flags [none], proto ICMP (1), length 28)
10.26.1.1 > 10.26.1.1: ICMP echo reply, id 40754, seq 58116, length 8
ipfw show:
root@PB-FW1-FRA:~ # ipfw show
00100 0 0 allow pfsync from any to any
00110 0 0 allow carp from any to any
00120 0 0 allow ip from any to any layer2 mac-type 0x0806,0x8035
00130 0 0 allow ip from any to any layer2 mac-type 0x888e,0x88c7
00140 0 0 allow ip from any to any layer2 mac-type 0x8863,0x8864
00150 0 0 deny ip from any to any layer2 not mac-type
0x0800,0x86dd
00179 410 11480 nat 1 log ip from 10.26.2.0/24 to 10.24.66.0/24
00179 414 11816 nat 1 log ip from 10.24.66.0/24 to 10.26.1.1 in recv
enc0
00200 0 0 skipto 60000 ip6 from ::1 to any
00201 44 41006 skipto 60000 ip4 from 127.0.0.0/8 to any
00202 0 0 skipto 60000 ip6 from any to ::1
00203 0 0 skipto 60000 ip4 from any to 127.0.0.0/8
01002 0 0 skipto 60000 udp from any to 10.26.1.1 dst-port 53
keep-state
01002 4 336 skipto 60000 ip from any to { 255.255.255.255 or
10.26.1.1 } in
01002 463 14672 skipto 60000 ip from { 255.255.255.255 or 10.26.1.1
} to any out
01002 0 0 skipto 60000 icmp from { 255.255.255.255 or
10.26.1.1 } to any out icmptypes 0
01002 0 0 skipto 60000 icmp from any to { 255.255.255.255 or
10.26.1.1 } in icmptypes 8
06000 5131 4476281 skipto 60000 tcp from any to any out
06199 10768 1914882 skipto 60000 ip from any to any
30000 0 0 count ip from any to any
60000 0 0 return ip from any to any
60001 0 0 queue 10000 tcp from any to 10.24.66.0/24 via enc0
65533 16410 6447177 allow ip from any to any
65534 0 0 deny ip from any to any
65535 0 0 deny ip from any to any
sysctl:
net.enc.out.ipsec_bpf_mask: 3
net.enc.out.ipsec_filter_mask: 1
net.enc.in.ipsec_bpf_mask: 3
net.enc.in.ipsec_filter_mask: 2
net.enc.out.ipsec_bpf_mask: 3
net.enc.out.ipsec_filter_mask: 1
net.enc.in.ipsec_bpf_mask: 3
net.enc.in.ipsec_filter_mask: 2
root@PB-FW1-FRA:~ # sysctl net.inet.ip.fw
net.inet.ip.fw.dyn_keep_states: 0
net.inet.ip.fw.dyn_keepalive: 1
net.inet.ip.fw.dyn_short_lifetime: 5
net.inet.ip.fw.dyn_udp_lifetime: 10
net.inet.ip.fw.dyn_rst_lifetime: 1
net.inet.ip.fw.dyn_fin_lifetime: 1
net.inet.ip.fw.dyn_syn_lifetime: 20
net.inet.ip.fw.dyn_ack_lifetime: 300
net.inet.ip.fw.dyn_max: 16384
net.inet.ip.fw.dyn_count: 0
net.inet.ip.fw.curr_dyn_buckets: 256
net.inet.ip.fw.dyn_buckets: 256
net.inet.ip.fw.enable: 1
net.inet.ip.fw.static_count: 25
net.inet.ip.fw.default_to_accept: 0
net.inet.ip.fw.tables_sets: 0
net.inet.ip.fw.tables_max: 128
net.inet.ip.fw.default_rule: 65535
net.inet.ip.fw.verbose_limit: 0
net.inet.ip.fw.verbose: 0
net.inet.ip.fw.autoinc_step: 100
net.inet.ip.fw.one_pass: 0
Thanks!
Michael
--
www.muenz-it.de
- Cisco, Linux, Networks
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1e889acf-49d1-b70f-7097-82e6e4dfabb6>