From owner-freebsd-security Fri Aug 23 16:50:13 2002 Delivered-To: freebsd-security@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6436C37B400 for ; Fri, 23 Aug 2002 16:50:06 -0700 (PDT) Received: from usenix.org (voyager.usenix.org [131.106.3.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0200F43E65 for ; Fri, 23 Aug 2002 16:50:06 -0700 (PDT) (envelope-from sam@usenix.org) Received: from melange (melange.errno.com [66.127.85.82]) (authenticated (0 bits)) by usenix.org (Switch-2.1.3/Switch-2.1.0) with ESMTP id g7NNntS16799 (using TLSv1/SSLv3 with cipher RC4-MD5 (128 bits) verified NO); Fri, 23 Aug 2002 16:50:03 -0700 (PDT) Message-ID: <006101c24aff$cce8cd00$52557f42@errno.com> From: "Sam Leffler (at Usenix)" To: "Ju Ichi" , References: <200208231624.14487.freebsd-security@ichi.net> Subject: Re: IPSec SPD limit? Date: Fri, 23 Aug 2002 16:49:55 -0700 Organization: Usenix Association MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4807.1700 X-DCC-Usenix-Metrics: voyager 1010; Body=2 Fuz1=2 Fuz2=2 Sender: owner-freebsd-security@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > We are trying to setup a large IPSec SPD (in excess of 1000 SAs) on the > following hardware/software config: > > Compaq DL360 with dual 1.4GHz processsors > 2GB RAM > 4GB swap space > > 4.6.1-RELEASE-p11 > racoon-20020507a > > We get a "send: No buffer space available" when trying to read in the > /etc/ipsec.conf file if it has more than about 1000 entries. Also, if we do > a setkey -DP after trying to read in /etc/ipsec.conf we get > "recv: Resource temporarily unavailable" after it lists some of the SAs. > > Several kernel tweaks have been tried. For example, we have tried setting > MAXUSERS from 0 to 1024 on bit boundaries (0, 128, 256, 512, and 1024). > FWIW, setting it to 1024 seems to be evil. ;-) We have also tried various > settings in the kernel config file on NMBCLUSTERS, NMBUFS, NBUF, MAXDSIZ, > MAXSSIZ, DFLDSIZ, and MAXFILES. In addition, we have tweaked > kern.ipc.somaxconn, net.inet.tcp.sendspace, net.inet.tcp.recvspace, > net.inet.udp.recvspace, and net.inet.udp.maxdgram after reading some > performance tuning web pages. I can provide additional details as needed, > but didn't want to make this initial request too long. > > Does anyone know of any limits on the number of entries the SPD can hold and > if so how to make the limits higher? > setkey -DP returns the SA's via a PF_KEY socket. You're hitting the upper bound on the amount of data that can be stored in a socket of this type as all the data is returned en masse (i.e. the process isn't given an opportunity to read data). PF_KEY sockets inherit the send+recv space reserves of raw sockets: ebb# gdb -k /kernel /dev/mem ... (kgdb) p raw_sendspace $1 = 8192 (kgdb) p raw_recvspace $2 = 8192 You can either work around this by upping these values or patch the PF_KEY code to set the reserves on the socket explicitly (and provide sysctl's a la udp+tcp to control the upper bounds). Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-security" in the body of the message