From owner-freebsd-pf@FreeBSD.ORG Tue Jun 23 08:57:17 2015 Return-Path: Delivered-To: freebsd-pf@nevdull.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7BEA5576 for ; Tue, 23 Jun 2015 08:57:17 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-yk0-x22b.google.com (mail-yk0-x22b.google.com [IPv6:2607:f8b0:4002:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 34533E82 for ; Tue, 23 Jun 2015 08:57:17 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: by ykdt186 with SMTP id t186so1737256ykd.0 for ; Tue, 23 Jun 2015 01:57:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=9v7/iFOuwiTGgAFOhcC/a9TyC6f2nKCTxapbQB7vdZM=; b=LY6Cjd9F91bQWa9uhDNStVOnNW1EUzsb3i88Ui+aZae4Y7Xvr8xviKdoRvHYIK6vGI jIodLAMcs6/SvucafupNb5eYTSwFltUPyS7saPtM9H8BGoSotzYhOKIBGqXN2hvvUG9r bLt82llaDwHIpWfx8RfQtyEUxMVuD0fYnO9fFpSxm/0T8fBlYKFP3yKp6GzM3usrhNLj AFTQ7AV5co0mPu9BWX2CCN5jU0EQ0p4w6Rd1LRNRmPAXGocfPW9J4bqwJU+lZso+AzYa Gx0ciPpsb+4ir9KWuPX+dYg4jVPdXysZnZCnf/lplcvGQEeIZATk8qzabi8KmwyhPlsr 3BTw== MIME-Version: 1.0 X-Received: by 10.170.198.142 with SMTP id p136mr29347516yke.70.1435049836193; Tue, 23 Jun 2015 01:57:16 -0700 (PDT) Sender: ermal.luci@gmail.com Received: by 10.129.123.137 with HTTP; Tue, 23 Jun 2015 01:57:16 -0700 (PDT) In-Reply-To: <20150623101225.4bc7f2d0@zeta.dino.sk> References: <20150623073856.334ebd61@zeta.dino.sk> <20150621133236.75a4d86d@zeta.dino.sk> <20150620182432.62797ec5@zeta.dino.sk> <20150619091857.304b707b@zeta.dino.sk> <14e119e8fa8.2755.abfb21602af57f30a7457738c46ad3ae@capeaugusta.com> <20150621195753.7b162633@zeta.dino.sk> <20150623101225.4bc7f2d0@zeta.dino.sk> Date: Tue, 23 Jun 2015 10:57:16 +0200 X-Google-Sender-Auth: 5U445p71fE-A4IdGdcaYxLjfMyg Message-ID: Subject: Re: Large scale NAT with PF - some weird problem From: =?UTF-8?Q?Ermal_Lu=C3=A7i?= To: Milan Obuch Cc: Ian FREISLICH , "freebsd-pf@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Jun 2015 08:57:17 -0000 On Tue, Jun 23, 2015 at 10:12 AM, Milan Obuch wrote: > On Tue, 23 Jun 2015 09:49:57 +0200 > Ian FREISLICH wrote: > > > Milan Obuch wrote: > > > As a first step, I did small upgrade, so now I run FreeBSD > > > 9.3-STABLE #0 r284695: Mon Jun 22 08:55:29 CEST 2015. > > > > > > I still see the issue, but I found simpler workaround when bad state > > > ocurs - using > > > > > > pfctl -k > > > pfctl -K > > > > > > in this order seems to remedy the issue for this one affected client > > > without affecting other clients. This still does not solve the > > > problem, just eases the reaction. > > > > How is your NAT rule defined? I had a closer look at the way I did > > it: > > > > nat on vlan46 from 10.8.0.0/15 to ! -> xx.xx.xx.xx/24 > > round-robin sticky-address > > > > I think you may be missing the "round-robin" that spreads the mapping > > over your pool. The manual says that when more than 1 address is > > specified, round-robin is the only pool type allowed, it does not > > say that when more than 1 address is specified this is the default > > pool option. > > > > Thanks for hint, however, this is not the case I think. My definition is > > nat on $if_ext from to any -> $pool_ext round-robin > sticky-address > > where contains contains some /24 segments from 10.0.0.0/8 > range and one /24 and one /15 segment from 172.16.0.0/12 range, > $pool_ext is one /23 public segment. > > > You can check your state table to see if it is indeed round-robin. > > > > #pfctl -s sta |grep " (" > > ... > > all tcp a.b.c.d:53802 (10.0.0.220:42808) -> 41.246.55.66:24 > > ESTABLISHED:ESTABLISHED all tcp a.b.c.e:60794 (10.0.0.38:47825) -> > > 216.58.223.10:443 ESTABLISHED:FIN_WAIT_2 > > > > If all your addresses "a.b.c.X" are the same, it's not round-robin > > and that's your problem. > > > > Well, this is something I do not fully understand. If my pool were > a.b.c.0/24, then what you wrote could not be any other way - I think > this is not what you meant. Or did you think there will be only one IP > used? That's definitelly not the case, I see many IPs from my /23 > segment here. > > One strange thing occured, however - it looks like if one IP from > this /23 range gets used, trouble occurs. I do pfctl -k and pfctl -K > for this address and all is well again. As long as this one IP is not > used, everything works. When it gets used again, voila, trouble again. > > Can you check if you are reaching the limits on source entries set limit src-nodes 2000 sets the maximum number of entries in the memory pool used for tracking source IP addresses (generated by the sticky-address and src.track options) to 2000. > As this does not occur that fast, I need to check every now and then, > and I am checking the other way too, but it is really annoying if it > hits any customer. > > Regards, > Milan > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > > -- > Ermal >