Date: Thu, 07 Jul 2011 17:08:53 +0100 From: Hugo Silva <hugo@barafranca.com> To: freebsd-questions@freebsd.org Subject: Requesting assistance: IPsec-configured FreeBSD system sends unencrypted packets on the wrong interface! Message-ID: <4E15DA15.2090803@barafranca.com>
next in thread | raw e-mail | index | archive | help
Hello,
I'm setting up an IPsec tunnel between a OpenBSD/isakmpd box and a
FreeBSD 8.2-STABLE/ipsec-tools racoon.
It looks something like this:
[lan a(OPENBSD_NET)]-->[openbsd]--i-p-s-e-c-->[freebsd]-->[lan
b(FREEBSD_NET)]
The FreeBSD server also runs OpenVPN, for end-user VPN. The ipv4 range
is FREEBSD_USERVPN/24.
The situation is the following:
- I can ssh in from FREEBSD_USERVPN to OPENBSD_NET (laptop -> (openvpn
udp) -> freebsd-ipsec-machine -> (ipsec-tunnel) -> openbsd-ipsec-machine
-> target-machine).
- I can't ssh/ping from machines in FREEBSD_NET to machines OPENBSD_NET,
even though setkey on FreeBSD reports what appear to be correct settings.
- In fact, tcpdump shows packets destined for OPENBSD_NET (coming from
FREEBSD_NET; from FREEBSD_USERVPN it works, that is part of the mystery)
leaving the external interface (@FreeBSD-ipsec), unencrypted, with
target MAC set to FREEBSD's default gateway.
FreeBSD's setkey -DP with the tunnel up looks something like:
OPENBSD_NET/24[any] FREEBSD_NET/24[any] any
in ipsec
esp/tunnel/OPENBSD_EXTERNAL_IP-FREEBSD_EXTERNAL_IP/require
created: Jul 7 15:20:16 2011 lastused: Jul 7 15:20:16 2011
lifetime: 1200(s) validtime: 0(s)
spid=373 seq=5 pid=87182
refcnt=1
OPENBSD_EXTERNAL_IP[any] FREEBSD_EXTERNAL_IP[any] any
in ipsec
ah/transport//require
created: Jul 7 15:21:24 2011 lastused: Jul 7 15:21:24 2011
lifetime: 1200(s) validtime: 0(s)
spid=375 seq=4 pid=87182
refcnt=1
OPENBSD_NET/24[any] FREEBSD_USERVPN/24[any] any
in ipsec
esp/tunnel/OPENBSD_EXTERNAL_IP-FREEBSD_EXTERNAL_IP/require
created: Jul 7 15:22:15 2011 lastused: Jul 7 15:22:15 2011
lifetime: 1200(s) validtime: 0(s)
spid=377 seq=3 pid=87182
refcnt=1
FREEBSD_NET/24[any] OPENBSD_NET/24[any] any
out ipsec
esp/tunnel/FREEBSD_EXTERNAL_IP-OPENBSD_EXTERNAL_IP/require
created: Jul 7 15:20:16 2011 lastused: Jul 7 15:20:16 2011
lifetime: 1200(s) validtime: 0(s)
spid=374 seq=2 pid=87182
refcnt=1
FREEBSD_EXTERNAL_IP[any] OPENBSD_EXTERNAL_IP[any] any
out ipsec
ah/transport//require
created: Jul 7 15:21:24 2011 lastused: Jul 7 15:21:24 2011
lifetime: 1200(s) validtime: 0(s)
spid=376 seq=1 pid=87182
refcnt=1
FREEBSD_USERVPN/24[any] OPENBSD_NET/24[any] any
out ipsec
esp/tunnel/FREEBSD_EXTERNAL_IP-OPENBSD_EXTERNAL_IP/require
created: Jul 7 15:22:15 2011 lastused: Jul 7 15:22:15 2011
lifetime: 1200(s) validtime: 0(s)
spid=378 seq=0 pid=87182
refcnt=1
OpenBSD's ipsecctl -sa:
FLOWS:
flow ah in from FREEBSD_EXTERNAL_IP to OPENBSD_EXTERNAL_IP peer
FREEBSD_EXTERNAL_IP srcid OPENBSD_EXTERNAL_IP/32 dstid
FREEBSD_EXTERNAL_IP/32 type use
flow ah out from OPENBSD_EXTERNAL_IP to FREEBSD_EXTERNAL_IP peer
FREEBSD_EXTERNAL_IP srcid OPENBSD_EXTERNAL_IP/32 dstid
FREEBSD_EXTERNAL_IP/32 type require
flow esp in from FREEBSD_NET to OPENBSD_NET/24 peer FREEBSD_EXTERNAL_IP
srcid OPENBSD_EXTERNAL_IP/32 dstid FREEBSD_EXTERNAL_IP/32 type use
flow esp out from OPENBSD_NET/24 to FREEBSD_NET peer FREEBSD_EXTERNAL_IP
srcid OPENBSD_EXTERNAL_IP/32 dstid FREEBSD_EXTERNAL_IP/32 type require
flow esp in from FREEBSD_USERVPN/24 to OPENBSD_NET/24 peer
FREEBSD_EXTERNAL_IP srcid OPENBSD_EXTERNAL_IP/32 dstid
FREEBSD_EXTERNAL_IP/32 type use
flow esp out from OPENBSD_NET/24 to FREEBSD_USERVPN/24 peer
FREEBSD_EXTERNAL_IP srcid OPENBSD_EXTERNAL_IP/32 dstid
FREEBSD_EXTERNAL_IP/32 type require
SAD:
esp tunnel from OPENBSD_EXTERNAL_IP to FREEBSD_EXTERNAL_IP spi
0x08ae310d auth hmac-sha2-256 enc aes
ah transport from OPENBSD_EXTERNAL_IP to FREEBSD_EXTERNAL_IP spi
0x08d6347a auth hmac-sha2-256
esp tunnel from OPENBSD_EXTERNAL_IP to FREEBSD_EXTERNAL_IP spi
0x09ec838b auth hmac-sha2-256 enc aes
esp tunnel from FREEBSD_EXTERNAL_IP to OPENBSD_EXTERNAL_IP spi
0x3ab91daa auth hmac-sha2-256 enc aes
ah transport from FREEBSD_EXTERNAL_IP to OPENBSD_EXTERNAL_IP spi
0x44eaed5b auth hmac-sha2-256
esp tunnel from FREEBSD_EXTERNAL_IP to OPENBSD_EXTERNAL_IP spi
0xbfd25ead auth hmac-sha2-256 enc aes
----
racoon is passive, isakmpd is active; OpenBSD's ipsec.conf:
myself = "OPENBSD_EXTERNAL_IP"
mypeer = "FREEBSD_EXTERNAL_IP"
if ="em0"
local_net = "OPENBSD_NET/24"
remote_net = "{ FREEBSD_USERVPN/24, FREEBSD_NET/24 }"
ike esp tunnel from $local_net to $remote_net peer $mypeer main auth
hmac-sha1 enc aes group modp1024 quick auth hmac-sha2-256 enc aes group
modp1024 psk "verysecret"
ike ah transport from $myself to $mypeer main auth hmac-sha1 group
modp1024 quick auth hmac-sha2-256 group modp1024 psk "verysecret"
I have:
- Tried without AH.
- Tried $remote_net = FREEBSD_NET. (the packets will still go out on
the wrong interface)
- Disabling PF. (on one end, on both ends)
When I ping a machine on FREEBSD_NET from a machine on OPENBSD_NET, I
can see the packet arriving on FREEBSD_NET_MACHINE (and enc0 on
OpenBSD-ipsec-router, enc0 on freebsd-ipsec-router), and a icmp echo
reply going out from the target machine.
At the same time, on the FreeBSD IPsec machine, I'll see the echoreq
arriving on enc0, and then the echorep going out on $ext_if .. needless
to say the machine on the other end never sees it.
Trying the opposite (ping a machine on OpenBSD's network from a machine
on the FreeBSD-router side) results in the packet immediately going out
on the wrong interface @ the FreeBSD IPsec machine.. it never reaches
enc0 on the OpenBSD side.
With pf active, FreeBSD shows:
15:36:24.695261 rule 0/0(match): block out on bce1:
SOME_FREEBSD-SIDE_MACHINE > SOME_OPENBSD-SIDE_MACHINE: ICMP echo
request, id 28959, seq 1, length 64
I've tried passing all, thinking it could be some strange pf artifact,
but even then there's nothing on OpenBSD enc0. And as far as one can
trust tcpdump, it really is trying to send the packet unencrypted out on
the external interface; The dst MAC is FreeBSD's default gateway.
I can't for the life of me understand WHY it works from OpenVPN, but not
from the LAN. As shown above, the ipsec.conf file includes both in
$remote_net (I've tried inverting the order of the declaration and
declaring only the freebsd network (then trying to ping from a
freebsd-side machine to a openbsd-side machine), to no avail).
It seems to me that pretty much the same thing is happening: A packet
arrives on an interface (tun0 for openvpn, bce0 for packets from
FreeBSD's internal network) with a destination address that's reachable
over the IPsec tunnel.
The difference is that for some reason packets coming from OpenVPN will
be IPsec processed, and there's connectivity, while packets from the LAN
won't, and they're sent to the default gateway on $ext_if(bce1) -- even
though, to the best of my understanding, both should be matching !
That both should match is defined in the same ipsec.conf rule on
OpenBSD's side (racoon has generate_policy on), as demonstrated earlier.
Furthermore, if setkey -DP's timestamps are to be trusted:
(3:44pm) freebsd/pts/1 root:/root# setkey -DP | grep lastused
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:44:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:44:34 2011
(logged in from my laptop (goes through OpenVPN) to a machine on
OpenBSD's side)
laptop --> (openvpn udp) --> freebsd-ipsec --> (ipsec ah/esp) -->
openbsd-ipsec --> some-openbsd-side-machine
(3:44pm) freebsd/pts/1 root:/root# setkey -DP | grep lastused
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:44:41 2011 <-
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:43:34 2011
created: Jul 7 15:43:34 2011 lastused: Jul 7 15:44:41 2011 <-
Trying to ping from a freebsd-side machine to a openbsd-side machine
doesn't update lastused.
I have been troubleshooting this for some days now, and running out of
ideas fast. Either it's something so simple and obvious that I'm not
seeing it, or something very fishy is going on.
As far as I understand it, packets coming from FREEBSD_NET should match.
They should update the lastused shown in setkey -DP, and be IPsec processed.
The one other thing that occurs to me is trying to get this running over
gif(4) interfaces.. but as far as I understand, this is not required
(and indeed, packets coming from FREEBSD_USERVPN do reach OPENBSD_NET,
which proves that connectivity is working)
Finally, racoon's config is nothing special; The relevant bits are:
remote OPENBSD_EXTERNAL_IP
{
exchange_mode main,aggressive;
passive on;
generate_policy on;
doi ipsec_doi;
situation identity_only;
#my_identifier asn1dn;
#certificate_type x509 "my.cert.pem" "my.key.pem";
nonce_size 16;
initial_contact on;
proposal_check strict; # obey, strict, or claim
proposal {
encryption_algorithm aes;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group modp1024;
}
}
sainfo anonymous
{
pfs_group modp1024;
encryption_algorithm aes;
authentication_algorithm hmac_sha256;
compression_algorithm deflate;
}
Hopefully enough information has been provided to spot an error in the
configuration/logic.
Can you see something obviously wrong with this?
Thanks for reading.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4E15DA15.2090803>