From owner-freebsd-ipfw@freebsd.org Sun May 21 22:14:40 2017 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06B91D78BD4 for ; Sun, 21 May 2017 22:14:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E01081371 for ; Sun, 21 May 2017 22:14:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v4LMEdwu037524 for ; Sun, 21 May 2017 22:14:39 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-ipfw@FreeBSD.org Subject: [Bug 219316] Wildcard matching of ipfw flow tables Date: Sun, 21 May 2017 22:14:40 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: lutz@donnerhacke.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-ipfw@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 May 2017 22:14:40 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219316 --- Comment #11 from lutz@donnerhacke.de --- For flows the extend to ip:mask (per entry) does not really help: - ports and protocol numbers are not covered - hashs are not radix trees, they can handle only an uniform mask And there is already a mask in the hash. I do only modify it. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ipfw@freebsd.org Sun May 21 22:41:30 2017 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 229F5D772E2 for ; Sun, 21 May 2017 22:41:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1248C1F7A for ; Sun, 21 May 2017 22:41:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v4LMfT15099950 for ; Sun, 21 May 2017 22:41:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-ipfw@FreeBSD.org Subject: [Bug 219316] Wildcard matching of ipfw flow tables Date: Sun, 21 May 2017 22:41:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: lutz@donnerhacke.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-ipfw@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 May 2017 22:41:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D219316 --- Comment #12 from lutz@donnerhacke.de --- Ah, I missed the previous comment. >>1) Large Scale NAT violates the happy eyeball requirement, that a given c= lient >> should always use the same external IP while communicating to a given se= rvice. > On what timescale? Forever? As long as the client has the same (CGN) IP (from 10.64.0.0/10). > If a client is idle for 5 minutes (no sessions) can > it start using a new IP? No. That violates the happy-eyeball contraint. Several web services bind the session to the external visible IP. If this IP does change, the customer ha= s to login again and again. We already made this mistake (using LSN). >>2) Mapping all customers to a single IP does not work either, because the= re >> are too much connections originating by those customers. > How may remote addresses are you talking too? > You can reuse the same address and port to may different remote addresses= .. That would surprise me. Such an implementation would require dynamic memory= for the NAT tables. I do not see such a memory usage on my FreeBSD machines. I = did see such an effect on a CISCO ASA. See: https://lutz.donnerhacke.de/Blog/High-memory-with-extended-PAT-on-ASA >> Consequently a deterministically selected group of clients has to share = the >> same NAT table using a single external IP. A typical approach is to use= =20 >> wildcards to match the right NAT instance: > you just said that "Mapping all customers to a single IP does not work .." > and yet that is what you show here.. Am I misreading it? The classical NAT setting does not distinguish between the client IPs and therefore does either have a single IP or LSN. My setup partitions the clients by their IPs and then I use a "single IP per partition" NAT. > How many clients are we talking about here? 10? 100? 1000? 10K? 100K? 1M? > and are these clients all on separate hardware? or are they coming from a > small number of session aggregator machines? Currently I have ~10k clients per hardware, the setup scales horizontally. = If I get more clients, I add additional machines and tell them in DHCP to use a different gateway (next machine). >> add 2100 nat 100 ipv4 from 100.64.0.0:255.192.0.63 to any xmit ext out >> add 2101 nat 101 ipv4 from 100.64.0.1:255.192.0.63 to any xmit ext out >> add 2102 nat 102 ipv4 from 100.64.0.2:255.192.0.63 to any xmit ext out >> >> This approach is inefficient, tables could help. But tables does not sup= port >> wildcard masking of lookup data. With such an wildcard mask, especially = the >> flow tables could greatly improve performance. > I don't quite understand this bit > my memory is that you can have a table > 100.64.0.0:255.192.0.63 0 > 100.64.0.1:255.192.0.63 1 ... > nat tablearg ip from table (x) to any out xmit XX0 You are right. That's the setup I'm used before switching to this flow based NAT. I only used the very early setup to demonstrate the problem. My fault. > what am I missing? You are missing the privacy expectations and the Law Enforcement Agencies. = For privacy, we like to use different external IPs for the same client reaching different services. That's why flows. For LEAs we need to tell exactly which user war involved in a specific sess= ion, so we need to log some data about NAT. This is an overwhelming large amount= of data, so we like to push down the necessary logs. This can be done by allocation blocks of ports to a customer instead of individual ports. In order to carefully assign such port ranges, they need to be large (at le= ast 300 per customer in order to access Google Maps without errors). That's why= we need to heavily reuse port (ranges) and this requires multiple NAT tables p= er customer. The only separation method left is to include the destination address, port and protocol. That's why we switched to flows. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ipfw@freebsd.org Wed May 24 09:02:43 2017 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01806D7B715 for ; Wed, 24 May 2017 09:02:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E593B1125 for ; Wed, 24 May 2017 09:02:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v4O92gJH051575 for ; Wed, 24 May 2017 09:02:42 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-ipfw@FreeBSD.org Subject: [Bug 217620] ipfw flow specification parsing Date: Wed, 24 May 2017 09:02:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-ipfw@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 May 2017 09:02:43 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217620 --- Comment #2 from commit-hook@freebsd.org --- A commit references this bug: Author: ae Date: Wed May 24 09:01:55 UTC 2017 New revision: 318777 URL: https://svnweb.freebsd.org/changeset/base/318777 Log: MFC r318400: Allow zero port specification in table entries with type flow. PR: 217620 Changes: _U stable/11/ stable/11/sbin/ipfw/tables.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-ipfw@freebsd.org Wed May 24 09:05:34 2017 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B4CBD7B835 for ; Wed, 24 May 2017 09:05:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AB281363 for ; Wed, 24 May 2017 09:05:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v4O95YGk068898 for ; Wed, 24 May 2017 09:05:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-ipfw@FreeBSD.org Subject: [Bug 217620] ipfw flow specification parsing Date: Wed, 24 May 2017 09:05:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ae@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ae@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution assigned_to bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 May 2017 09:05:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D217620 Andrey V. Elsukov changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |FIXED Assignee|freebsd-ipfw@FreeBSD.org |ae@FreeBSD.org Status|New |Closed CC| |ae@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.=