From owner-freebsd-net@FreeBSD.ORG Thu May 12 12:29:35 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DFCE816A4E5 for ; Thu, 12 May 2005 12:29:34 +0000 (GMT) Received: from espresso2.syncrontech.com (sync-old.syncrontech.com [213.28.98.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id CB5EA43D6A for ; Thu, 12 May 2005 12:29:30 +0000 (GMT) (envelope-from ari@suutari.iki.fi) Received: from guinness.syncrontech.com (guinness.syncrontech.com [62.71.8.57])j4CCTPsc027220; Thu, 12 May 2005 15:29:29 +0300 (EEST) (envelope-from ari@suutari.iki.fi) Received: from [62.71.8.37] (coffee.syncrontech.com [62.71.8.37]) j4CCTJ1D031335; Thu, 12 May 2005 15:29:19 +0300 (EEST) (envelope-from ari@suutari.iki.fi) Message-ID: <42834C11.9000103@suutari.iki.fi> Date: Thu, 12 May 2005 15:29:05 +0300 From: Ari Suutari User-Agent: Mozilla Thunderbird 1.0.2 (Windows/20050317) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Ari Suutari References: <4282F5EC.6060902@suutari.iki.fi> <42833051.10602@suutari.iki.fi> In-Reply-To: <42833051.10602@suutari.iki.fi> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-net@freebsd.org Subject: Re: IPSEC traffic doesn't work realiably after upgrading from 4.11 to 5.4 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 May 2005 12:29:35 -0000 (replying to myself again...) Ari Suutari wrote: > Some more information to this one: This seems to be some kind of > odd routing problem. I just recreated the setup under vmware and noticed > that when the problem occurs the outgoing ESP packets are > flowing on interface that has the default route (em0), not > on tun0. The routing table entry looks correct (ie. points > to tun0), but ESP packets don't seem to obey it > (until setkey -F is issued). There seems to be a field called sa_route, which caches routing information in SA if I understood the code correctly. What happens here is this: - when tun0 goes down, the code in ipsec.c notices that route is not up any more and gets a new route for packet. In this case it gets the default route. - when tun0 goes up again, ESP packets are still sent to default route, because it is still valid. - setkey -F clears this cached information, restoring correct operation. I coudn't find any place where sa_route stuff is invalidated when routing table changes. If so, isn't this kind of a serious problem ? Ari S. > > Ari S. > > Ari Suutari wrote: > >> Hi, >> >> I have upgraded a vpn server from FreeBSD 4.11 to 5.4-RELEASE. >> The box as about 20 vpn connections to other FreeBSD machines, >> the physical connection is via tun0 ... tun20 devices. >> >> Traffic flow is something like this: >> >> my internal net -> >> vpn server em1 -> >> vpn server ipsec -> >> vpn server tun0 -> >> vpn server em0 -> >> internet -> >> remote freebsd fxp0 -> >> remote freebsd tun0 -> >> remote freebsd ipsec -> >> remote net >> >> Remote FreeBSD box is still running 4.11. >> Ipsec is the kame version, not FAST_IPSEC. >> >> (tun0 stuff is created by vtun software, which is used >> to get around various restrictions, like ISP providing >> private addresses only). >> >> This has been working very well for years under FreeBSD 4.x. >> >> After upgrading to 5.4, things seem to work at first. However, >> when physical connection has problems, causing tun0 device to >> go temporarily down on server the vpn never recovers from it. >> When tun0 comes up back again, IPsec SAs seem to be valid >> on both sides. Non-ipsec traffic works without problems >> over tun0 as does *incoming* ipsec traffic from remote >> FreeBSD box. Outgoing ipsec packets seem to vanish completely. >> >> It seems that the problem can also be triggered by running >> ifconfig tun0 down && ifconfig tun0 up. >> >> netstat -s -p ipsec doesn't show any errors. To recover >> from situation, issuing setkey -F to flush all SAs helps. >> Flushing only the SAs related to this connection does not help, >> neither does removing related policies and adding them again. >> >> I would'n like to go back to 4.x series, so I'm looking >> for fix/workaround for this. >> >> Ari S. >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >