Date: Thu, 5 Nov 2009 11:30:01 +0300 From: Igor Sysoev <is@rambler-co.ru> To: freebsd-net@freebsd.org Subject: Re: maximum number of outgoing connections Message-ID: <20091105083001.GC87654@rambler-co.ru> In-Reply-To: <20070820163443.GE20183@rambler-co.ru> References: <20070820151142.GA20183@rambler-co.ru> <46C9BF02.5050007@tomjudge.com> <20070820163443.GE20183@rambler-co.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Aug 20, 2007 at 08:34:43PM +0400, Igor Sysoev wrote:
> On Mon, Aug 20, 2007 at 05:19:14PM +0100, Tom Judge wrote:
>
> > Igor Sysoev wrote:
> > >It seems that FreeBSD can not make more than
> > >
> > >net.inet.ip.portrange.last - net.inet.ip.portrange.first
> > >
> > >simultaneous outgoing connections, i.e., no more than about 64k.
> > >
> > >If I made ~64000 connections 127.0.0.1:XXXX > 127.0.0.1:80, then
> > >connect() to an external address returns EADDRNOTAVAIL.
> > >
> > >net.inet.ip.portrange.randomized is 0.
> > >
> > >sockets, etc. are enough:
> > >
> > >ITEM SIZE LIMIT USED FREE REQUESTS FAILURES
> > >socket: 356, 204809, 13915, 146443, 148189452, 0
> > >inpcb: 180, 204820, 20375, 137277, 147631805, 0
> > >tcpcb: 464, 204800, 13882, 142102, 147631805, 0
> > >tcptw: 48, 41028, 6493, 11213, 29804665, 0
> > >
> > >I saw it on 6.2-STABLE.
> > >
> > >
> >
> > In an ideal world (Not sure if this is quite correct for FreeBSD) TCP
> > connections are tracked with a pair of tupels source-addr:src-port ->
> > dst-addr:dst-port
> >
> > As your always connecting to the same destination service 127.0.0.1:80
> > and always from the same source IP 127.0.0.1 then you only have one
> > variable left to change, the source port. If you where to use the hole
> > of the whole of the port range minus the reserved ports you would only
> > ever be able to make 64512 simultaneous connections. In order to make
> > more connections the first thing that you may want to start changing is
> > the source IP. If you added a second IP to you lo0 interface (say
> > 127.0.0.2) and used a round robin approach to making your out bound
> > connections then you could make around 129k outbound connections.
>
> Connections to 127.0.0.1 were via lo0, external connections are via bge0.
>
> > I am not sure if there are any other constraints that need to be taken
> > into account such as the maximum number of sockets, RAM etc....
>
> No, there are no constraints in memory, sockets, mbufs, clusters, etc.
> If there's contraint in memory, then FreeBSD simply panics.
> If there's contraint in mbuf clusters, then process stucks in zonelimit
> state forever.
>
> I suspect that local address in in_pcbbind_setup() is 0.0.0.0 so there
> is 64K limit.
Recently I looked the issue again and find (with Ruslan Ermilov's help)
that if I set the SO_REUSEADDR option, FreeBSD allows to use the same
local port for different connections:
192.168.1.1:5000 > 10.0.0.1:80
192.168.1.1:5000 > 10.0.0.2:80
Linux allows this by default. I believe FreeBSD should set SO_REUSEADDR
internally while connect().
BTW, bind()ing socket to a local address does not help.
--
Igor Sysoev
http://sysoev.ru/en/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091105083001.GC87654>