From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 05:15:01 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C4F5621F; Wed, 13 Mar 2013 05:15:01 +0000 (UTC) (envelope-from wollman@hergotha.csail.mit.edu) Received: from hergotha.csail.mit.edu (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2]) by mx1.freebsd.org (Postfix) with ESMTP id 525571E5; Wed, 13 Mar 2013 05:15:00 +0000 (UTC) Received: from hergotha.csail.mit.edu (localhost [127.0.0.1]) by hergotha.csail.mit.edu (8.14.5/8.14.5) with ESMTP id r2D5ExCU037978; Wed, 13 Mar 2013 01:14:59 -0400 (EDT) (envelope-from wollman@hergotha.csail.mit.edu) Received: (from wollman@localhost) by hergotha.csail.mit.edu (8.14.5/8.14.4/Submit) id r2D5EwWC037975; Wed, 13 Mar 2013 01:14:58 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <20800.2898.798984.157469@hergotha.csail.mit.edu> Date: Wed, 13 Mar 2013 01:14:58 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: Limits on jumbo mbuf cluster allocation In-Reply-To: <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> References: <20798.44871.601547.24628@hergotha.csail.mit.edu> <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (hergotha.csail.mit.edu [127.0.0.1]); Wed, 13 Mar 2013 01:14:59 -0400 (EDT) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=disabled version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on hergotha.csail.mit.edu Cc: freebsd-net@freebsd.org, andre@freebsd.org, Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 05:15:01 -0000 < said: > Basically, this patch: > - allows setting of the tcp timeout via vfs.nfsd.tcpcachetimeo > (I'd suggest you go down to a few minutes instead of 12hrs) > - allows TCP caching to be disabled by setting vfs.nfsd.cachetcp=0 > - does the above 2 things you describe to try and avoid the livelock, > although not quite using an lru list > - increases the hash table size to 500 (still a compile time setting) > (feel free to make it even bigger) > - sets nfsrc_floodlevel to at least nfsrc_tcphighwater, so you can > grow vfs.nfsd.tcphighwater as big as you dare Thanks, this looks very good. One quibble with the last bit: I'd do that in a sysctl() handler rather than checking it every time through. If somebody uses a debugger rather than sysctl to change tcphighwater, they deserve what's coming to them. Also, I might suggest adding a counter for how many times we had to go through the "try harder" phase, so that the sysadmin has some indication that the defaults need adjustment. I will test this out later this week and see how it performs. I have a user who has been able to reproducibly clobber servers before, so if he has time and cycles available it should be pretty easy to tell whether it's working or not. -GAWollman