Date: Fri, 13 Apr 2001 18:07:35 -0700 (PDT) From: gunther@aurora.regenstrief.org To: freebsd-gnats-submit@FreeBSD.org Subject: kern/26549: IPsec policies for more than one pair of SA do not work ... Message-ID: <200104140107.f3E17ZH33920@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 26549
>Category: kern
>Synopsis: IPsec policies for more than one pair of SA do not work ...
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Fri Apr 13 18:10:01 PDT 2001
>Closed-Date:
>Last-Modified:
>Originator: Gunther Schadow
>Release:
>Organization:
Regenstrief Institute
>Environment:
FreeBSD prometeus.regenstrief.org 4.2-RELEASE FreeBSD 4.2-RELEASE #0: Thu Apr 12
14:32:03 EST 2001 root@prometeus.regenstrief.org:/usr/NGI/KAME/kame/freebsd
4/sys/compile/PROMETEUS i386
>Description:
I am trying to establish several IPsec ESP tunnels from one system.
Each tunnel by itself works beautifully, but when I try using those
tunnels all together, the kernel suddenly understands no incoming
ESP message. The problem comes and goes with modifications in the
SPD. If I delete all but one pair of SPD rules, the remaining
tunnel works fine. As soon as I add a second pair of SPD rules
the pair just added doesn't works. If I then delete the first
pair, the pair just added suddenly works. I can go round robin with
this and it is always the oldest pair that works and none of the
others.
This problem is critical for me as it puts my entire project
into jeopardy. I will have to work around this in the next
couple of days.
>How-To-Repeat:
You need three machines A and B, and C. We begin with A and B:
On machine A run:
if=ed0
aip=10.10.10.1
bip=10.10.10.2
aipsec=10.99.10
bipsec=10.99.20
ifconfig ${if} inet alias ${aip} netmask 0xffffff00
ifconfig lo0 inet alias ${aipsec}.1 netmask 0xffffff00
route add -net ${bipsec}.0/24 ${aipsec}.1
setkey -c <<END
spdflush;
add ${aip} ${bip} esp 1000 -E simple;
add ${bip} ${aip} esp 1001 -E simple;
spdadd ${aipsec}.0/24 ${bipsec}.0/24 any -P out ipsec
esp/tunnel/${aip}-${bip}/require;
spdadd ${bipsec}.0/24 ${aipsec}.0/24 any -P in ipsec
esp/tunnel/${bip}-${aip}/require;
END
On machine B run
if=sis0
aip=10.10.10.1
bip=10.10.10.2
aipsec=10.99.10
bipsec=10.99.20
ifconfig ${if} inet alias ${bip} netmask 0xffffff00
ifconfig lo0 inet alias ${bipsec}.1 netmask 0xffffff00
route add -net ${aipsec}.0/24 ${bipsec}.1
setkey -c <<END
spdflush;
add ${aip} ${bip} esp 1000 -E simple;
add ${bip} ${aip} esp 1001 -E simple;
spdadd ${aipsec}.0/24 ${bipsec}.0/24 any -P in ipsec
esp/tunnel/${aip}-${bip}/require;
spdadd ${bipsec}.0/24 ${aipsec}.0/24 any -P out ipsec
esp/tunnel/${bip}-${aip}/require;
END
ping ${aip}
ping ${aipsec}.1
should get responses, likewise, one machine A
ping ${bip}
ping ${bipsec}
should get responses just fine.
Now comes the problem, lets add another machine C into
the group. Configure machine C:
if=tl0
aip=10.10.10.1
cip=10.10.10.3
aipsec=10.99.10
cipsec=10.99.30
ifconfig ${if} inet alias ${cip} netmask 0xffffff00
ifconfig lo0 inet alias ${cipsec}.1 netmask 0xffffff00
route add -net ${aipsec}.0/24 ${cipsec}.1
setkey -c <<END
spdflush;
add ${aip} ${cip} esp 2000 -E simple;
add ${cip} ${aip} esp 2001 -E simple;
spdadd ${aipsec}.0/24 ${cipsec}.0/24 any -P in ipsec
esp/tunnel/${aip}-${cip}/require;
spdadd ${cipsec}.0/24 ${aipsec}.0/24 any -P out ipsec
esp/tunnel/${cip}-${aip}/require;
END
As you see, C only has a tunnel to A. Beause A is supposed to
be the logical gateway, the hub, the center of a star topology.
Which means we add this to A:
cip=10.10.10.3
cipsec=10.99.30
route add -net ${cipsec}.0/24 ${aipsec}.1
setkey -c <<END
add ${aip} ${cip} esp 2000 -E simple;
add ${cip} ${aip} esp 2001 -E simple;
spdadd ${aipsec}.0/24 ${cipsec}.0/24 any -P out ipsec
esp/tunnel/${aip}-${cip}/require;
spdadd ${cipsec}.0/24 ${aipsec}.0/24 any -P in ipsec
esp/tunnel/${cip}-${aip}/require;
END
ping ${cip}
# works
ping ${cipsec}.1
# doesn't work
ping ${bipsec}.1
# works
Now delete the SPD entried for B:
setkey -c <<END
spddelete ${aipsec}.0/24 ${bipsec}.0/24 any -P out;
spddelete ${bipsec}.0/24 ${aipsec}.0/24 any -P in;
END
And try again:
ping ${cipsec}.1
This time it works. Now lets add the SPD for the B-tunnel
back in:
setkey -c <<END
spdadd ${aipsec}.0/24 ${bipsec}.0/24 any -P out ipsec
esp/tunnel/${aip}-${bip}/require;
spdadd ${bipsec}.0/24 ${aipsec}.0/24 any -P in ipsec
esp/tunnel/${bip}-${aip}/require;
END
ping ${cipsec}.1
still works! BUT:
ping ${bipsec}.1
doesn't work now.
The problem is entirely on A's side as I can show you with
a tcpdump trace. On C I say:
tcpdump -n -i tl0 host ${aip}
And now I do
ping ${cipsec}.1
tcpdump shows:
19:51:30.945747 10.10.10.1 > 10.10.10.3: ESP(spi=2000,seq=0x14)
19:51:30.945915 10.10.10.3 > 10.10.10.1: ESP(spi=2001,seq=0x24)
19:51:31.953169 10.10.10.1 > 10.10.10.3: ESP(spi=2000,seq=0x15)
19:51:31.953300 10.10.10.3 > 10.10.10.1: ESP(spi=2001,seq=0x25)
as it should, and remember, the C-tunnel works now. Let's go
to B and
ping ${aipsec}.1
tcpdump shows:
19:55:21.963950 10.10.10.2 > 10.10.10.1: ESP(spi=1001,seq=0x42)
19:55:22.975435 10.10.10.2 > 10.10.10.1: ESP(spi=1001,seq=0x43)
see how A never sends an icmp echo reply? It is because it
never gets the icmp messages to the upper layer. Instead
netstat -s -p ip
ip:
2313 total packets received
0 bad header checksums
0 with size smaller than minimum
0 with data size < data length
0 with ip length > max ip packet size
0 with header length < data size
0 with data length < header length
0 with bad options
0 with incorrect version number
0 fragments received
0 fragments dropped (dup or out of space)
0 fragments dropped after timeout
0 packets reassembled ok
2033 packets for this host
>Fix:
Don't use IPsec :-( I will try reverting to gif tunnels with
transport mode, hoping that it will work better.
>Release-Note:
>Audit-Trail:
>Unformatted:
>>>>>> 267 packets for unknown/unsupported protocol
0 packets forwarded (0 packets fast forwarded)
0 packets not forwardable
13 packets received for unknown multicast group
0 redirects sent
1374 packets sent from this host
0 packets sent with fabricated ip header
0 output packets dropped due to no bufs, etc.
0 output packets discarded due to no route
0 output datagrams fragmented
0 fragments created
0 datagrams that can't be fragmented
0 tunneling packets that can't find gif
0 datagrams with bad address in header
As you see, the unknown/unsupported protocol counter goes up with
every icmp request sent. But, the "packets for this host" counter
doen not go up. This means, the packets are not even seen as
belonging to this host!!!
Now let's turn this around once again. Disabling SPD entries for
C and you'll see how B comes back on line:
setkey -c <<END
spddelete ${aipsec}.0/24 ${cipsec}.0/24 any -P out;
spddelete ${cipsec}.0/24 ${aipsec}.0/24 any -P in;
END
ping ${bipsec}.1
works again!
Now to go full circle, let's reestablish C's SPD entries
setkey -c <<END
spdadd ${aipsec}.0/24 ${cipsec}.0/24 any -P out ipsec
esp/tunnel/${aip}-${cip}/require;
spdadd ${cipsec}.0/24 ${aipsec}.0/24 any -P in ipsec
esp/tunnel/${cip}-${aip}/require;
END
ping ${bipsec}.1
still works, BUT
ping ${cipsec}.1
doesn't work. Using tcpdump and netstat shows exactly the
same behavior.
O.K. I hope this is sufficient documentation for the bug
in question. I also hope that this documentation makes
reproduction as easy as possible. Please try it and see
for yourself.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200104140107.f3E17ZH33920>