From owner-freebsd-net@FreeBSD.ORG Sat Jul 21 19:23:10 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F20A16A418 for ; Sat, 21 Jul 2007 19:23:10 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.freebsd.org (Postfix) with ESMTP id 577F313C442 for ; Sat, 21 Jul 2007 19:23:10 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 061A94798B; Sat, 21 Jul 2007 15:23:10 -0400 (EDT) Date: Sat, 21 Jul 2007 20:23:09 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: "Eric L. Anderson" In-Reply-To: <20070721040009.GB21336@more.net> Message-ID: <20070721202012.C83919@fledge.watson.org> References: <20070720145932.GP6053@more.net> <20070720180546.X39675@fledge.watson.org> <20070721104525.44603382@localhost> <20070721040009.GB21336@more.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: Max NFS mounts for a FreeBSD client? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jul 2007 19:23:10 -0000 On Fri, 20 Jul 2007, Eric L. Anderson wrote: > On Sat, Jul 21, 2007 at 10:45:25AM +1000, Norberto Meijome wrote: >> On Fri, 20 Jul 2007 18:07:37 +0100 (BST) >> Robert Watson wrote: >>> >>> Sounds a bit like something is running out of reserved ports to use -- the >>> credentials error may mean that a port number >1023 was used for an NFS >>> connection. Given that reserved ports start around 600, 420 is about the >>> right number of sockets to reach 1024. >> >> Reserved ports controlled by sysctl : >> >> net.inet.ip.portrange.reservedhigh: 1023 net.inet.ip.portrange.reservedlow: >> 0 >> >> although the 600 rwatson mentions seems to be this one: >> >> net.inet.ip.portrange.lowfirst: 1023 net.inet.ip.portrange.lowlast: 600 >> >> You should be able to tweak these values - as long as you have ephemeral >> ports for the rest of your network activity, you should be ok, right? > > This sounds like we are on the right track. I verified via netstat that all > ports from 600-1023 are being used for NFS after I run my test script. > > I can not change lowfirst to any higher amount. I did change lowlast from > 600 to 1 and now I can mount more than 1000 NFS mounts. This is great but > what kind of side effects am I introducing by making this change? The issue here, presumably, is that each NFS client mountpoint has (and requires) a unique socket, which means a unique TCP/IP or UDP/IP tuple with its respective server endpoint. This is used to demux replies, etc. With TCP/IP NFS mounts, it should be possible, in principle, to be quite a bit more conservative in the use of tuples, as reusing a source IP and port number is only a problem if there's a collision with another mountpoint using identical destination IP and port. It could be that NFS, perhaps with a bit of help from the TCP layer, could be more agressive about reusing existing local port numbers rather than using a new one for every mountpoint. I'm not sure what would be involved infrastructure-wise here, and obviously care would have to be taken not to break UDP/IP mounts. Robert N M Watson Computer Laboratory University of Cambridge