From owner-freebsd-pf@FreeBSD.ORG Tue Jun 23 08:12:29 2015 Return-Path: Delivered-To: freebsd-pf@nevdull.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 688052A2 for ; Tue, 23 Jun 2015 08:12:29 +0000 (UTC) (envelope-from freebsd-pf@dino.sk) Received: from mailhost.netlabit.sk (mailhost.netlabit.sk [84.245.65.72]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F25C99DA for ; Tue, 23 Jun 2015 08:12:28 +0000 (UTC) (envelope-from freebsd-pf@dino.sk) Received: from zeta.dino.sk (fw1.dino.sk [84.245.95.252]) (AUTH: LOGIN milan) by mailhost.netlabit.sk with ESMTPA; Tue, 23 Jun 2015 10:12:26 +0200 id 00EB08DE.558914EA.0001135D Date: Tue, 23 Jun 2015 10:12:25 +0200 From: Milan Obuch To: Ian FREISLICH Cc: freebsd-pf@freebsd.org Subject: Re: Large scale NAT with PF - some weird problem Message-ID: <20150623101225.4bc7f2d0@zeta.dino.sk> In-Reply-To: References: <20150623073856.334ebd61@zeta.dino.sk> <20150621133236.75a4d86d@zeta.dino.sk> <20150620182432.62797ec5@zeta.dino.sk> <20150619091857.304b707b@zeta.dino.sk> <14e119e8fa8.2755.abfb21602af57f30a7457738c46ad3ae@capeaugusta.com> <20150621195753.7b162633@zeta.dino.sk> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.27; i386-portbld-freebsd10.1) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2015 08:12:29 -0000 On Tue, 23 Jun 2015 09:49:57 +0200 Ian FREISLICH wrote: > Milan Obuch wrote: > > As a first step, I did small upgrade, so now I run FreeBSD > > 9.3-STABLE #0 r284695: Mon Jun 22 08:55:29 CEST 2015. > > > > I still see the issue, but I found simpler workaround when bad state > > ocurs - using > > > > pfctl -k > > pfctl -K > > > > in this order seems to remedy the issue for this one affected client > > without affecting other clients. This still does not solve the > > problem, just eases the reaction. > > How is your NAT rule defined? I had a closer look at the way I did > it: > > nat on vlan46 from 10.8.0.0/15 to ! -> xx.xx.xx.xx/24 > round-robin sticky-address > > I think you may be missing the "round-robin" that spreads the mapping > over your pool. The manual says that when more than 1 address is > specified, round-robin is the only pool type allowed, it does not > say that when more than 1 address is specified this is the default > pool option. > Thanks for hint, however, this is not the case I think. My definition is nat on $if_ext from to any -> $pool_ext round-robin sticky-address where contains contains some /24 segments from 10.0.0.0/8 range and one /24 and one /15 segment from 172.16.0.0/12 range, $pool_ext is one /23 public segment. > You can check your state table to see if it is indeed round-robin. > > #pfctl -s sta |grep " (" > ... > all tcp a.b.c.d:53802 (10.0.0.220:42808) -> 41.246.55.66:24 > ESTABLISHED:ESTABLISHED all tcp a.b.c.e:60794 (10.0.0.38:47825) -> > 216.58.223.10:443 ESTABLISHED:FIN_WAIT_2 > > If all your addresses "a.b.c.X" are the same, it's not round-robin > and that's your problem. > Well, this is something I do not fully understand. If my pool were a.b.c.0/24, then what you wrote could not be any other way - I think this is not what you meant. Or did you think there will be only one IP used? That's definitelly not the case, I see many IPs from my /23 segment here. One strange thing occured, however - it looks like if one IP from this /23 range gets used, trouble occurs. I do pfctl -k and pfctl -K for this address and all is well again. As long as this one IP is not used, everything works. When it gets used again, voila, trouble again. As this does not occur that fast, I need to check every now and then, and I am checking the other way too, but it is really annoying if it hits any customer. Regards, Milan