From owner-freebsd-hackers@FreeBSD.ORG Wed Jul 31 20:27:41 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 8DA4C2BC for ; Wed, 31 Jul 2013 20:27:41 +0000 (UTC) (envelope-from db@db.net) Received: from diana.db.net (diana.db.net [66.113.102.10]) by mx1.freebsd.org (Postfix) with ESMTP id 763092640 for ; Wed, 31 Jul 2013 20:27:41 +0000 (UTC) Received: from night.db.net (localhost [127.0.0.1]) by diana.db.net (Postfix) with ESMTP id 357E42AA4BD; Wed, 31 Jul 2013 14:27:32 -0600 (MDT) Received: by night.db.net (Postfix, from userid 1000) id 28C311CC0E; Wed, 31 Jul 2013 15:27:22 -0500 (EST) Date: Wed, 31 Jul 2013 15:27:22 -0500 From: Diane Bruce To: Sergey Kandaurov Subject: Re: Stacking lots of IP's on a single box - any 'gotchas'? Message-ID: <20130731202722.GA11852@night.db.net> References: <34C668004A0D654205D0516B@Mail-PC.tdx.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-hackers@freebsd.org, Karl Pielorz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Jul 2013 20:27:41 -0000 On Wed, Jul 31, 2013 at 11:47:02PM +0400, Sergey Kandaurov wrote: > On 31 July 2013 13:37, Karl Pielorz wrote: > > > > Hi, > > > > We've got a number of boxes we'd like to consolidate - this could mean > > upward of 1,500 IP's on a single box (9.1 amd64). > > > > Last time we did anything like this we hit at issue at around 900 (ntpd > > 'binds' by default to all available IP's - I think we had a workaround for > > that). > > > > This is because select() has a limit on FD_SETSIZE (1024). > If it tries to select > 1024 fds, bad things could happen. > Newer ntpd (not in base) has a feature to bind only to the specific > interface; this was used to run ntpd on boxes with > 1200 IPs on 1 i/face. I can tell you from experience that FreeBSD has no problems at all selecting on more than 1024 fds. In fact, this is what I would suggest you do. Your limit then is going to be the number of anonymous ports on each IP. Figure 32000 or so to be conservative that's more than what you need. > > But is there any hard limit we're likely to encounter putting so many IP's > > on a single machine? - Are there any limits that would likely need tuning to > > support that many IP's? > > > > Unlikely, besides those unrelated things like ntpd+select() et.al. > > -- > wbr, > pluknet - Diane -- - db@FreeBSD.org db@db.net http://www.db.net/~db