From owner-freebsd-pf@FreeBSD.ORG Tue Jun 23 09:23:35 2015 Return-Path: Delivered-To: freebsd-pf@nevdull.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 668CE84A for ; Tue, 23 Jun 2015 09:23:35 +0000 (UTC) (envelope-from freebsd-pf@dino.sk) Received: from mailhost.netlabit.sk (mailhost.netlabit.sk [84.245.65.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EEEB1E4E for ; Tue, 23 Jun 2015 09:23:33 +0000 (UTC) (envelope-from freebsd-pf@dino.sk) Received: from zeta.dino.sk (fw1.dino.sk [84.245.95.252]) (AUTH: LOGIN milan) by mailhost.netlabit.sk with ESMTPA; Tue, 23 Jun 2015 11:23:31 +0200 id 00EB08D9.55892593.00011A2A Date: Tue, 23 Jun 2015 11:23:31 +0200 From: Milan Obuch To: Ian FREISLICH Cc: freebsd-pf@freebsd.org Subject: Re: Large scale NAT with PF - some weird problem Message-ID: <20150623112331.668395d1@zeta.dino.sk> In-Reply-To: References: <20150623101225.4bc7f2d0@zeta.dino.sk> <20150623073856.334ebd61@zeta.dino.sk> <20150621133236.75a4d86d@zeta.dino.sk> <20150620182432.62797ec5@zeta.dino.sk> <20150619091857.304b707b@zeta.dino.sk> <14e119e8fa8.2755.abfb21602af57f30a7457738c46ad3ae@capeaugusta.com> <20150621195753.7b162633@zeta.dino.sk> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.27; i386-portbld-freebsd10.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2015 09:23:35 -0000 On Tue, 23 Jun 2015 10:57:44 +0200 Ian FREISLICH wrote: > Milan Obuch wrote: > > > all tcp a.b.c.d:53802 (10.0.0.220:42808) -> 41.246.55.66:24 > > > ESTABLISHED:ESTABLISHED all tcp a.b.c.e:60794 (10.0.0.38:47825) -> > > > 216.58.223.10:443 ESTABLISHED:FIN_WAIT_2 > > > > > > If all your addresses "a.b.c.X" are the same, it's not round-robin > > > and that's your problem. > > > > > > > Well, this is something I do not fully understand. If my pool were > > a.b.c.0/24, then what you wrote could not be any other way - I think > > this is not what you meant. Or did you think there will be only one > > IP used? That's definitelly not the case, I see many IPs from my /23 > > segment here. > > I just wanted to check that more than 1 address was being used. > OK, it is. If only one IP were used for all traffic, I would run into issues much earlier. > So, I think that the problem is with 9-STABLE. I hate "upgrade to > solve your problems" answers because they may not. I do know that > 10 has seen a lot of work and none of that work will make it back > into 9 because of the PF rewrite. Maybe someone else in this group > will chime in. > That's OK. I am a bit conservative on upgrades here because with hundreds - thousands users you need a bit of stability too, but upgrade to 10-STABLE is currently being prepared. That being written, it will not occur today. > I ran 10-CURRENT in production for as long 10 was CURRENT and then > went to 10-STABLE precisely because I was having state issues > forwarding performance issues with 9. Gleb Smirnof did a significant > rewrite of PF to improve SMP performance. He had access to my > system for debugging on a large installation. > Well, we'll see. I'll let you know how it goes when upgrade will be done. > If you're not already doing so, I'd recomend running CARP + pfsync > so you can test updates while maintaining a known working backup. > If you're running pfsync, I recommend you run it on a different > interface to the one with your traffic and with a cross-over cable > between your machines. The pfsync packet rate caused a small amount > packet loss on other network traffic. > I did not experiment much with CARP and pfsync. I plan to use it, but that means more hardware... and I am not the one who pays for it. Anyway, I would try to redesign the whole thing so it will be easier to maintain and, if necessary, troubleshooting. Regards, Milan