From owner-freebsd-security Mon Aug 26 10: 1: 5 2002 Delivered-To: freebsd-security@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9368E37B400 for ; Mon, 26 Aug 2002 10:00:58 -0700 (PDT) Received: from dragon.ichi.net (dragon.ichi.net [209.42.196.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 353EC43E6E for ; Mon, 26 Aug 2002 10:00:46 -0700 (PDT) (envelope-from freebsd-security@ichi.net) Received: from coaster (localhost.localdomain [127.0.0.1]) by dragon.ichi.net (8.11.6/8.11.6) with ESMTP id g7QGmA530138; Mon, 26 Aug 2002 12:48:11 -0400 Content-Type: text/plain; charset="iso-8859-1" From: Ju Ichi To: "Sam Leffler (at Usenix)" , Subject: Re: IPSec SPD limit? Date: Mon, 26 Aug 2002 12:59:15 -0400 User-Agent: KMail/1.4.1 References: <200208231624.14487.freebsd-security@ichi.net> <006101c24aff$cce8cd00$52557f42@errno.com> In-Reply-To: <006101c24aff$cce8cd00$52557f42@errno.com> Cc: "Nielsen" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200208261259.15721.freebsd-security@ichi.net> Sender: owner-freebsd-security@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Friday 23 August 2002 7:49 pm, Sam Leffler (at Usenix) wrote: > > We are trying to setup a large IPSec SPD (in excess of 1000 SAs) on the > > following hardware/software config: > > > > Compaq DL360 with dual 1.4GHz processsors > > 2GB RAM > > 4GB swap space > > > > 4.6.1-RELEASE-p11 > > racoon-20020507a > > > > We get a "send: No buffer space available" when trying to read in the > > /etc/ipsec.conf file if it has more than about 1000 entries. Also, if we > do > > a setkey -DP after trying to read in /etc/ipsec.conf we get > > "recv: Resource temporarily unavailable" after it lists some of the SAs. > > > > Several kernel tweaks have been tried. For example, we have tried setting > > MAXUSERS from 0 to 1024 on bit boundaries (0, 128, 256, 512, and 1024). > > FWIW, setting it to 1024 seems to be evil. ;-) We have also tried > various > > settings in the kernel config file on NMBCLUSTERS, NMBUFS, NBUF, MAXDSIZ, > > MAXSSIZ, DFLDSIZ, and MAXFILES. In addition, we have tweaked > > kern.ipc.somaxconn, net.inet.tcp.sendspace, net.inet.tcp.recvspace, > > net.inet.udp.recvspace, and net.inet.udp.maxdgram after reading some > > performance tuning web pages. I can provide additional details as needed, > > but didn't want to make this initial request too long. > > > > Does anyone know of any limits on the number of entries the SPD can hold > and > > if so how to make the limits higher? > > > > setkey -DP returns the SA's via a PF_KEY socket. You're hitting the upper > bound on the amount of data that can be stored in a socket of this type as > all the data is returned en masse (i.e. the process isn't given an > opportunity to read data). PF_KEY sockets inherit the send+recv space > reserves of raw sockets: > > ebb# gdb -k /kernel /dev/mem > ... > (kgdb) p raw_sendspace > $1 = 8192 > (kgdb) p raw_recvspace > $2 = 8192 > > You can either work around this by upping these values or patch the PF_KEY > code to set the reserves on the socket explicitly (and provide sysctl's a la > udp+tcp to control the upper bounds). > > Sam Thanks! I changed /usr/src/sys/net/raw_cb.h as follows: < #define RAWSNDQ 8192 < #define RAWRCVQ 8192 --- > #define RAWSNDQ 65535 > #define RAWRCVQ 65535 So, now we have: (kgdb) p raw_sendspace $1 = 65535 (kgdb) p raw_recvspace $2 = 65535 We are able to get the policy loaded by using "setkey -c" with sleep statements as Nate suggested, but still are getting "recv: Resource temporarily unavailable" when doing a setkey -DP. Anymore ideas on other values to up? Also, Nate, do you know of a way to dump the poicy with setkey so it all shows? In other words, using setkey -c we can slow down the rate of putting entries in, but there doesn't seem to be a way to slow down the rate at which the policy is dumped. Thanks, Ju To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message