From owner-freebsd-hackers@FreeBSD.ORG Mon Sep 19 13:10:12 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2DDF3106566C for ; Mon, 19 Sep 2011 13:10:12 +0000 (UTC) (envelope-from jlpetz@internode.on.net) Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by mx1.freebsd.org (Postfix) with ESMTP id AFC1B8FC1F for ; Mon, 19 Sep 2011 13:10:11 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EANc7d055LEEa/2dsb2JhbABCpx14gVMBAQMCCAIeEhwTARsBAwIGAxEEAQEoBxkgDQkIAgQBEgkCBQuHYLVThngEnBmIbw Received: from ppp121-44-65-26.lns20.syd6.internode.on.net (HELO Minx) ([121.44.65.26]) by ipmail06.adl6.internode.on.net with ESMTP; 19 Sep 2011 22:40:09 +0930 From: "Jarrod Lee Petz" To: "'Daniel Hartmeier'" , References: <007301cc6979$a690f9a0$f3b2ece0$@internode.on.net> <4E616D6E.4030903@FreeBSD.org> <001701cc69d3$aea9a0b0$0bfce210$@internode.on.net> <4E61BA37.2060204@FreeBSD.org> <20110903134634.GA55652@owl.midgard.homeip.net> <4E62B99C.6020707@FreeBSD.org> <001e01cc6a9d$8e62c870$ab285950$@internode.on.net> <20110905151146.GA10185@insomnia.benzedrine.cx> In-Reply-To: <20110905151146.GA10185@insomnia.benzedrine.cx> Date: Mon, 19 Sep 2011 23:10:06 +1000 Message-ID: <000001cc76cd$744d7bb0$5ce87310$@internode.on.net> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQHuPCHqJDmDdIw669xIXvDN9LM1EADkjGxsAa6N1kUCHwUDBgGXq6DUAa0cmgEBYbJsPAKzPzumlLDAQpA= Content-Language: en-au Cc: Subject: RE: TIME_WAIT Assassination in FreeBSD??? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: petz@nisshoko.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Sep 2011 13:10:12 -0000 Hi Daniel, Thank you for the information. I agree with your comment below. " But if you're simply running the ftp client on an otherwise idle host, and two subsequent bind() calls get assigned the same 'random' port, I'd say the port randomization is not working properly :)" Does anyone have any further information or advice they could offer for this issue? I have found a few other references via Google to people experiencing issues with ephemeral port randomisation and FreeBSD. It appears this does not just impact FTP but also WebServers & Proxy software. So as I feared other protocols/software besides active FTP is impacted. To what extent it would seem no one really knows unless things have changed(this stuff is a few years dated). This link in particular http://www.silby.com/eurobsdcon05/eurobsdcon_silbersack.pdf " At present, FreeBSD attempts to avoid this quick port recycling problem by falling back to sequential port allocation whenever the machine is making more than 10 outbound connections per second. This solution is more of a hack than anything, and has been slated to be replaced as soon as a better method can be found." And this http://www.bsdcan.org/2006/papers/ImprovingTCPIP.pdf I retested again, this time on FreeBSD 9.0 Beta 2. I can confirm the behaviour described is still present. If I test with many worker FTP processes or even one process without a delay. Then the ten connections per second limit is reached and port randomisation is disabled making the problem disappear. However when I only have one worker process and a delay(which I usually have to capture a decent tcpdump) then the connection rate is lower and port randomisation stays in effect which produces the issue. So what should would be the best approach for this. I'm still not sure. Does anyone know of work being done on a different port randomisation algorithm? Should I disable port randomisation until/unless a better selection algorithm comes along? This is made out to sound like a fairly big security risk in practice I don't really understand how bad? For secured SSL/TLS traffic would this cause concern? OR Should I just move to passive FTP and try not to worry about other protocols/software encountering sporadic issues? Regards Jarrod -----Original Message----- From: Daniel Hartmeier [mailto:daniel@benzedrine.cx] Sent: Tuesday, 6 September 2011 1:12 AM To: petz@nisshoko.net Cc: freebsd-hackers@freebsd.org Subject: Re: TIME_WAIT Assassination in FreeBSD??? In FreeBSD, the ftp client allocates the port for an active-mode data connection by calling bind(2) with so_port set to 0, which means it lets the kernel pick a port, see http://www.freebsd.org/cgi/cvsweb.cgi/src/contrib/lukemftp/src/Attic/ftp.c?r ev=1.1.1.8;content-type=text%2Fplain;hideattic=0 The kernel code where the port is picked is in function in_pcb_lport(), see http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/netinet/in_pcb.c?rev=1.281;con tent-type=text%2Fplain Basically, there is a range of ports (49152-65535, adjustable with sysctl), and the algorithm picks a random port within that range: if (dorandom) *lastport = first + (arc4random() % (last - first)); It checks whether that port is available. If not, it increments it by one, and tries again, etc. in a loop, until it finds one. So, for your case, it is unlikely that two subsequent bind() calls from the ftp client would result in the same port being picked randomly, unless a large part of the port range is unavailable. You can get port re-use that is quick enough to confuse pf, for instance, by opening new connections (to the same destination address and port) at a high rate, e.g. when running the Apache web server benchmark tool. But if you're simply running the ftp client on an otherwise idle host, and two subsequent bind() calls get assigned the same 'random' port, I'd say the port randomization is not working properly :) HTH, Daniel