From owner-freebsd-net@FreeBSD.ORG Wed Mar 13 22:27:57 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 56C4CE31; Wed, 13 Mar 2013 22:27:57 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id D93BC328; Wed, 13 Mar 2013 22:27:56 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAH/8QFGDaFvO/2dsb2JhbABDiCG8NYFtdIIqAQEEASNWBRYYAgINGQJZBhOIDgawJ5JbgSONOTQHgi2BEwOWWJECgyYggWw X-IronPort-AV: E=Sophos;i="4.84,840,1355115600"; d="scan'208";a="18875801" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 13 Mar 2013 18:27:49 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C28C7B3F36; Wed, 13 Mar 2013 18:27:49 -0400 (EDT) Date: Wed, 13 Mar 2013 18:27:49 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1814312198.3878858.1363213669787.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20800.2898.798984.157469@hergotha.csail.mit.edu> Subject: Re: Limits on jumbo mbuf cluster allocation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-net@freebsd.org, andre@freebsd.org, Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2013 22:27:57 -0000 Garrett Wollman wrote: > < said: > > > Basically, this patch: > > - allows setting of the tcp timeout via vfs.nfsd.tcpcachetimeo > > (I'd suggest you go down to a few minutes instead of 12hrs) > > - allows TCP caching to be disabled by setting vfs.nfsd.cachetcp=0 > > - does the above 2 things you describe to try and avoid the > > livelock, > > although not quite using an lru list > > - increases the hash table size to 500 (still a compile time > > setting) > > (feel free to make it even bigger) > > - sets nfsrc_floodlevel to at least nfsrc_tcphighwater, so you can > > grow vfs.nfsd.tcphighwater as big as you dare > > Thanks, this looks very good. One quibble with the last bit: I'd do > that in a sysctl() handler rather than checking it every time through. > If somebody uses a debugger rather than sysctl to change tcphighwater, > they deserve what's coming to them. Also, I might suggest adding a > counter for how many times we had to go through the "try harder" > phase, so that the sysadmin has some indication that the defaults need > adjustment. > I agree w.r.t. both comments. I did the one line setting of nfsrc_floodlevel, just because I was getting lazy while doing the patch. And if/when a patch like this (I think if this works, it can easily be added to ivoras@'s patch) goes into head, some way for a sysadmin to monitor how well it's working would be good. A count of the "try harder" attempts seems like a good candidate. Good luck with testing of it, rick > I will test this out later this week and see how it performs. I have > a user who has been able to reproducibly clobber servers before, so if > he has time and cycles available it should be pretty easy to tell > whether it's working or not. > > -GAWollman