From owner-freebsd-pf@freebsd.org Mon Jun 29 07:57:12 2015 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44CAF98F81C for ; Mon, 29 Jun 2015 07:57:12 +0000 (UTC) (envelope-from ian.freislich@capeaugusta.com) Received: from mail-wi0-f170.google.com (mail-wi0-f170.google.com [209.85.212.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA35D1D6D for ; Mon, 29 Jun 2015 07:57:11 +0000 (UTC) (envelope-from ian.freislich@capeaugusta.com) Received: by wiwl6 with SMTP id l6so91364896wiw.0 for ; Mon, 29 Jun 2015 00:57:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references :mime-version:content-type:content-id:content-transfer-encoding:date :message-id; bh=EUO3mJedPBLIDrjDdg1rHCN9ql2k3pwiwDWxQS42tkM=; b=f4xGAxd1ozGGFDC7pgUPdHS3Q3Nre/CrupIdbtB9KJsVP09lkQ6/f5mia7tPa43e4I nhIRvQEX0TcIpfnMI982wJoh/uWLrbP7/wWJpwLhbvyd6SVDcjHS5pGK+z2pEOWGZwbo O8UUh/nxczZZHMS9YN17QWzBak/uWjAgZQQsgIUG6S2PRkRCe78DOnA7ADeL5Zwpq5/+ kHvjikP4Z1HiZVjFTROrRzackxUPA4q9rxlB5bBloH/423uHkXttD5xtIeXSOKsEW1uc ayBngYuplkvkY3msv7LPCzoh2e/+GfWYihB9+0CpYopI6sH94bclrjrnUJ/1ZB+fq/RI cKSA== X-Gm-Message-State: ALoCoQnyuYoyMtwjXd2XwXalyyP5kfSAOrl9LDJcOGLOGLOrAW7XmLtHr7ItJrzF4Q/QYawEMRaN X-Received: by 10.180.36.4 with SMTP id m4mr19852888wij.34.1435564154267; Mon, 29 Jun 2015 00:49:14 -0700 (PDT) Received: from clue.co.za ([197.89.34.55]) by mx.google.com with ESMTPSA id q4sm62644571wju.14.2015.06.29.00.49.13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 29 Jun 2015 00:49:13 -0700 (PDT) From: Ian FREISLICH X-Google-Original-From: Ian FREISLICH Received: from localhost ([127.0.0.1] helo=zen) by clue.co.za with esmtp (Exim 4.85 (FreeBSD)) (envelope-from ) id 1Z9ToU-000Ph3-KT; Mon, 29 Jun 2015 09:49:10 +0200 To: Milan Obuch cc: Daniel Hartmeier , freebsd-pf@freebsd.org Subject: Re: Large scale NAT with PF - some weird problem In-Reply-To: <20150629094317.5a0cd61a@zeta.dino.sk> References: <20150629094317.5a0cd61a@zeta.dino.sk> <20150620182432.62797ec5@zeta.dino.sk> <20150619091857.304b707b@zeta.dino.sk> <14e119e8fa8.2755.abfb21602af57f30a7457738c46ad3ae@capeaugusta.com> <20150621195753.7b162633@zeta.dino.sk> <20150623112331.668395d1@zeta.dino.sk> <20150628100609.635544e0@zeta.dino.sk> <20150629065838.GA13722@insomnia.benzedrine.ch> X-Attribution: BOFH MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <98767.1435564150.1@zen> Content-Transfer-Encoding: quoted-printable Date: Mon, 29 Jun 2015 09:49:10 +0200 Message-Id: X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 07:57:12 -0000 Milan Obuch wrote: > On Mon, 29 Jun 2015 08:58:38 +0200 > Daniel Hartmeier wrote: > = > > On Sun, Jun 28, 2015 at 10:06:09AM +0200, Milan Obuch wrote: > > = > > > So, now I am at 10.2-PRERELEASE, r284884, and the issue is still > > > here. It is totally weird, just change of IP the device is being > > > natted to makes the issue disappear for this particular customer, > > > but as soon as this exact IP is used again, the issue is here again. > > = > > I'd go over the entire network config (pf.conf, pfctl -sa, rc.conf, > > netstat -anr, ifconfig, arp -an) and look for any mistake, like a > > typo or a netmask which isn't what you thought it is (like on an > > alias), or for any weirdness related to that IP address. > > = > > Daniel > = > Thanks for hint, there is some logic in there, however > = > grep /etc/* > = > yields nothing, it is never mentioned in any config, just as part of > pool in pf.conf statement > = > nat on $if_ext from to any -> $pool_ext round-robin sticky-add= ress > = > It is not mentioned in 'pfctl -sa' output, 'arp -an' output, > 'netstat -anr' output... nowhere. > = > I did not mention this box runs quagga for configuring network, mainly > routing via OSPF, but I do not think it is relevant to the problem I > see as this is basically userland process communicating with forwarding > path in kernel to configure routing, nothing else, and, naturally, it > does not work with this particular IP either. I should have seen it > otherwise in some of above mentioned commands output, I think. > = > Just to repeat myself a bit, when this problematic state occurs, some > intenal IP is translated to this one offending public IP, and > communication is broken in such a way I see no returning packets from > outside world on uplink interface in tcpdump even if I know they are > there because I can ping some other box outside where I can verify that > and they are there... > = > I just found some other strange, to me, thing - in 'pfctl -sa' output, > section SOURCE TRACKING NODES, almost all entries are in form > = > -> ( st= ates = ..., connections ..., rate ... ) > = > but there are some of them with first IP being public, second one > 0.0.0.0 - where they could come from? Also, there are only couple of > them, but in one is something even a bit more weird - in parens is > 'states 4294967295', which seems a bit absurd to me, also, worth to > mention, it is 0xffffffff in hexadecimal, and this looks like some > underflow issue in the code. Try making your pool smaller. With the number of NAT states you mentioned earlier, you should easily manage with a /24. Ian -- = Ian Freislich