Date: Sun, 29 Jul 2001 22:47:22 +1000 From: "Chris" <mlnn4@oaks.com.au> To: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: kernel upgrade causes truncated IPSEC packets Message-ID: <200107291247.f6TClRB02477@aussie.org>
next in thread | raw e-mail | index | archive | help
Greets,
I've run up against a wall trying to fix a IPSEC tunnel setup that was
working up until I did a kernel update a few days ago. Certain packets
seem to go out of the system and then vanish into thin air (or thin
electrons, at least:). It's related to the running kernel as if I boot
the old (about three months) kernel, the problem goes away. I have seen
it happen on two machines in different physical locations. I have not
changed the kernel configuration file.
I have tracked the problem as far as the physical PPP interface. The
situation is as follows (you may need to refer to the network description
I have below): if packets enter my IPSEC tunnel from the FreeBSD box,
they are (mostly; see below) encapsulated as AH/ESP as per normal and
make their way to the remote end. This works fine.
If, however, they enter the FreeBSD gateway from the LAN, and use the same
tunnel, they seem to always get lost. They *do* get encapsulated by IPSEC
properly (it's not a forwarding issue), and in fact everything looks fine.
tcpdump shows them going through the tun0 interface and the modem lights
flash happily. Except the AH/ESP packets never turn up at the far end's
router ...
Once I had eliminated other possibilities I was only left with the thought
that it was getting lost inside ppp somewhere, so I turned on ppp async
debugging. This was very helpful, because it showed that the outgoing
packets were being truncated under certain circumstances.
Note that when I say they are truncated, I don't mean at the IP level; the
packet header shows the correct length for the AH and ESP-encapsulated
packet. But the PPP process only sees the first 36 bytes or so of it (for
example, a packet which arrives as 263 bytes becomes 320 bytes after
encapsulation. This is what is shown by both tcpdump and the packet header,
as read from the ppp async stream), but only 36 bytes of it make it through
to the modem (plus the PPP overhead of about 5 bytes).
I have since discovered that this will also happen for some packets that
originate on the local FBSD box. If they are over a certain length (220
bytes prior to encapsulation), they also suffer the 'only send 36 bytes'
problem (I wonder if the fact that 220 + 36 == 256 has something to do
with this?)
FWIW the other machines that we have implementing IPSEC tunnels, and which
are running the same kernel and tunnel config, but which do not use PPP to
connect to the net, do not suffer from this problem. It only seems to hurt
encapsulated packets under the above circumstances when combined with PPP.
I note that some significant changes happened to if_tun.c between the working
and non-working kernels (the old kernel was cvsupped on about April 19) but
I don't know enough about the kernel to work out if this is the cause. I
hope this is enough info for someone who does know it better to figure it
out.
----------------------------
Configuration: we have a number of machines (4.3-STABLE) acting as permanently
dialled-up gateways for an IPSEC-protected VPN connecting their local LAN's
to two central gateways in two cities. This has been in place and working
for a good part of a year.
I recently did a kernel update on the central gateways and that caused no
problems (the previous kernels were a few months old). I then updated two
of the field machines and started experiencing the abovementioned problems.
Topology (IP's are illustrative) -
o Local LAN has machine 'A' at 10.0.58.2/24
o A's default gateway is the FBSD box 'B' at 10.0.58.1/24
o B is dialled up using ppp to routable address 1.2.3.4
o Central gateway 'C' is on the net at 5.6.7.8
o C has an interface hosting a local LAN at 10.0.48.1/24
I use IPSEC transport mode between 'B' and 'C', and have set up a native
IPSEC tunnel (not using gif) between 10.0.58.0/24 and 10.0.48.0/24.
Once key exchange has occurred, I do have a working tunnel in that I can,
from 'A', 'ping -S 10.0.58.1 10.0.48.1' and get replies back. tcpdump
shows ESP encrypted packets as expected. Likewise 10.0.48.1 can ping
10.0.58.1.
However, since the kernel update, if 'C' at 10.0.48.1 pings machine 'A'
at 10.0.58.2, the packets get delivered to 'A' and the replies to 'B',
but the replies go missing after B's tun0. Likewise if 'A' tries to send
traffic to 'C' or any address in 10.0.48.0/24, the packets go missing
due, presumably, to the above PPP issue.
-- Chris
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200107291247.f6TClRB02477>