From owner-freebsd-stable@FreeBSD.ORG Tue Mar 15 22:20:31 2005 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AAFA16A4CE for ; Tue, 15 Mar 2005 22:20:31 +0000 (GMT) Received: from avscan1.sentex.ca (avscan1.sentex.ca [199.212.134.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id F1A0143D49 for ; Tue, 15 Mar 2005 22:20:30 +0000 (GMT) (envelope-from mike@sentex.net) Received: from localhost (localhost.sentex.ca [127.0.0.1]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j2FMKKWj079614; Tue, 15 Mar 2005 17:20:20 -0500 (EST) (envelope-from mike@sentex.net) Received: from avscan1.sentex.ca ([127.0.0.1]) by localhost (avscan1.sentex.ca [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 79398-03; Tue, 15 Mar 2005 17:20:20 -0500 (EST) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by avscan1.sentex.ca (8.12.11/8.12.11) with ESMTP id j2FMKJtT079599; Tue, 15 Mar 2005 17:20:20 -0500 (EST) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.11/8.12.11) with ESMTP id j2FMKEWF069205; Tue, 15 Mar 2005 17:20:14 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <6.2.1.2.0.20050315171421.0525ee30@64.7.153.2> X-Mailer: QUALCOMM Windows Eudora Version 6.2.1.2 Date: Tue, 15 Mar 2005 17:18:43 -0500 To: Sam Leffler From: Mike Tancsa In-Reply-To: <4237523B.7090005@errno.com> References: <6.2.1.2.0.20050315112131.054b56f8@64.7.153.2> <4237523B.7090005@errno.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: by amavisd-new X-Virus-Scanned: by amavisd-new at avscan1b cc: freebsd-stable@freebsd.org Subject: Re: RELENG_5 and FAST_IPSEC limits X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Mar 2005 22:20:31 -0000 At 04:23 PM 15/03/2005, Sam Leffler wrote: >Mike Tancsa wrote: >>Hi, >>We are running into a case where there are too many SAs, and doing a >>setkey -D would fail with a >>"recv: Resource temporarily unavailable" >>after displaying most of the associations. >>Is there a way to get around this, or is there a hard limit ? >># setkey -D | grep ^172 | wc >> 186 372 5096 >>When the remotes are renegotiating, and there are a lot of tunnels in the >>state of mature and dying, this number can go up to 341, but not >>higher. This also seems to send racoon into a hung state that we then >>need to kill off and restart. >>It was suggested in a post that /usr/src/sys/net/raw_cb.h get changed from >> >>#define RAWSNDQ 8192 >>#define RAWRCVQ 8192 >>to something larger like >>#define RAWSNDQ 24576 >>#define RAWRCVQ 24576 >>If this is the underlying issue, will it work on its own, or are there >>other values that need to be tuned ? Will I need to recompile any >>userland apps (e.g. racoon, setkey) and are there any other values I >>would need to adjust > >Looks like you're hitting the limit on returning status information >through a PF_KEY socket. FWIW this is not related to FAST_IPSEC; it's an >issue with PF_KEY and is common to both IPsec implementations. > >Upping the raw socket buffer sizes should permit more information to be >returned but you may always exceed this limit as you can create more SA's >than can be reported in a single msg. Some groups have dealt with this by >changing the PF_KEY api, e.g. to report an incomplete msg so the user-mode >app can retrieve more data with additional reads. If upping the socket >buffer limits doesn't help then you might search for patches. Hi and Thanks for the response! We will start by increasing the 2 values. Are there any potential negative side effects of doing so that we need to be aware of ? i.e. anything else that would need to be changed as a result ? I am pretty sure racoon was getting hung up in pfkey.c in this loop while (1) { if (msg) racoon_free(msg); msg = pk_recv(s, &len); if (msg == NULL) { if (len < 0) goto done; else continue; } if (msg->sadb_msg_type != SADB_DUMP || msg->sadb_msg_pid != pid) continue; ml = msg->sadb_msg_len << 3; bl = buf ? buf->l : 0; buf = vrealloc(buf, bl + ml); if (buf == NULL) { plog(LLV_ERROR, LOCATION, NULL, "failed to reallocate buffer to dump.\n"); goto fail; } memcpy(buf->v + bl, msg, ml); if (msg->sadb_msg_seq == 0) break; } goto done; Should have core dumped him at the time :( Oh well, thats what happens when you do 5am maint! If we plan for the worst case scenario for msg size, I guess this loop should not be an issue. ---Mike