From owner-freebsd-net@FreeBSD.ORG Thu Sep 27 15:16:18 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C93316A419 for ; Thu, 27 Sep 2007 15:16:18 +0000 (UTC) (envelope-from vanhu@zeninc.net) Received: from smtp.zeninc.net (reverse-25.fdn.fr [80.67.176.25]) by mx1.freebsd.org (Postfix) with ESMTP id C5AB713C480 for ; Thu, 27 Sep 2007 15:16:17 +0000 (UTC) (envelope-from vanhu@zeninc.net) Received: by smtp.zeninc.net (smtpd, from userid 1000) id 9F56A3F1F; Thu, 27 Sep 2007 16:47:48 +0200 (CEST) Date: Thu, 27 Sep 2007 16:47:48 +0200 From: VANHULLEBUS Yvan To: freebsd-net@freebsd.org Message-ID: <20070927144748.GA26659@zen.inc> References: <46FBB7BC.3010301@xs4all.nl> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <46FBB7BC.3010301@xs4all.nl> User-Agent: All mail clients suck. This one just sucks less. Subject: Re: Racoon 0.7 on FreeBSD 6 with a lot of VPN tunnels X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Sep 2007 15:16:18 -0000 On Thu, Sep 27, 2007 at 04:01:32PM +0200, Seth Mos wrote: > Hello there, Hi. > I have problems with racoon hanging in sbwait state with ipsec-tools 0.6.7 > or getting into a tailspin on ipsec-tools 0.7. > > The problem is that the pfkey interface breaks down with a lot of VPN > tunnels and spd entries. The FreeBSd PR is here. > http://www.freebsd.org/cgi/query-pr.cgi?pr=115651 Yep. Unfortunately, this is a well know issue for a quite long time. > I have 390 discrete IPSEC VPN tunnels and endpoints. I have loaded this > all up into one config. I am using pfSense as the platform of choice. > pfSense 1.2-RC2 specifically which is based on FreeBSD 6.2 Stable p7. Note > that I am also a pfSense developer. > > At this current time I have exactly 112 IPSEC tunnels active. I am using > 3des-sha1 with a 3600 lifetime for phase 1 and aes128-sha1 with a 28800 > lifetime for phase2. > > On ipsec-tools 0.6.7 I can go by for several days before the racoon > process wedges itself into a hanging sbwait state (0% cpu). On ipsec-tools > 0.7 the situation is significantly worse and it starts churning 100% cpu > somewhere every 1-4 hours. > > Basically where 0.6.7 was difficult 0.7 has become unworkable. That's strange, I don't remind any change between 0.6 and 0.7 which may generate such difference. And, to be more exact, I don't remember anything in 0.6 branch (or in any other public branch) which may help things work better in such setup. > The hardware in question is a Dell PE860 with 6 gigabit nics (about 2mbps > ipsec traffic at most) with a DualCore Xeon 3050 2.13Ghz. With 1GB ram. > > In the pfSense kernel we use this patch. > http://cvs.pfsense.com/cgi-bin/cvsweb.cgi/tools/patches/RELENG_6_2/socketvar.h.diff > Which helps significantly. Without this patch the situation is the same as > I have described above but will be reached at about 30-40 active tunnels > instead of the 112 I have now. The first thing you can try is that patch in racoon: --- src/libipsec/pfkey.c 1 Aug 2007 11:52:18 -0000 1.13.4.1 +++ src/libipsec/pfkey.c 27 Sep 2007 14:44:27 -0000 @@ -1801,7 +1801,12 @@ pfkey_open() */ (void)setsockopt(so, SOL_SOCKET, SO_SNDBUF, &bufsiz, sizeof(bufsiz)); (void)setsockopt(so, SOL_SOCKET, SO_RCVBUF, &bufsiz, sizeof(bufsiz)); - + bufsiz = 256 * 1024; + (void)setsockopt(so, SOL_SOCKET, SO_RCVBUF, &bufsiz, sizeof(bufsiz)); + bufsiz = 512 * 1024; + (void)setsockopt(so, SOL_SOCKET, SO_RCVBUF, &bufsiz, sizeof(bufsiz)); + bufsiz = 1024 * 1024; + (void)setsockopt(so, SOL_SOCKET, SO_RCVBUF, &bufsiz, sizeof(bufsiz)); __ipsec_errcode = EIPSEC_NO_ERROR; return so; } (you may have problems to directly apply this patch because it's a copy/paste, but it's quite easy to report manually). And as we have more and more requests about such problems, I'll probably directly commit it for ipsec-tools 0.7.1 / HEAD. But I'm quite sure it will not completly solve your problem, it will just raise the limit. Yvan. -- NETASQ http://www.netasq.com