From owner-freebsd-questions@freebsd.org Thu Aug 26 21:12:05 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B187A67119A for ; Thu, 26 Aug 2021 21:12:05 +0000 (UTC) (envelope-from 93ab.82.c37900021b83ee.9aadb35bdef40199041493f615702ded@email-od.com) Received: from s1-b515.socketlabs.email-od.com (s1-b515.socketlabs.email-od.com [142.0.181.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GwbBr2rDqz4ph0 for ; Thu, 26 Aug 2021 21:12:04 +0000 (UTC) (envelope-from 93ab.82.c37900021b83ee.9aadb35bdef40199041493f615702ded@email-od.com) X-Thread-Info: OTNhYi4xMi5jMzc5MDAwMjFiODNlZS5mcmVlYnNkLXF1ZXN0aW9ucz1mcmVlYnNkLm9yZw== Received: from r3.us-east-1.aws.in.socketlabs.com (r3.us-east-1.aws.in.socketlabs.com [142.0.191.3]) by mxh4.email-od.com with ESMTP(version=Tls12 cipher=Aes256 bits=256); Thu, 26 Aug 2021 17:11:50 -0400 Received: from oceanview.tundraware.com (oceanview.tundraware.com [45.55.60.57]) by r3.us-east-1.aws.in.socketlabs.com with ESMTP(version=Tls12 cipher=Aes256 bits=256); Thu, 26 Aug 2021 17:11:50 -0400 Received: from [192.168.0.2] (ozzie.tundraware.com [75.145.138.73]) (authenticated bits=0) by oceanview.tundraware.com (8.16.1/8.16.1) with ESMTPSA id 17QLBVZh026598 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Thu, 26 Aug 2021 16:11:31 -0500 (CDT) (envelope-from tundra@tundraware.com) Subject: Re: ipfw Table Organization To: FreeBSD Mailing List References: <9e6cd8e2-a06e-468b-7245-d5ff13309763@tundraware.com> From: Tim Daneliuk Message-ID: <25ed1e6f-fe69-5b3b-c459-00a115cfbb5e@tundraware.com> Date: Thu, 26 Aug 2021 16:11:26 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.4 (oceanview.tundraware.com [45.55.60.57]); Thu, 26 Aug 2021 16:11:31 -0500 (CDT) X-TundraWare-MailScanner-Information: Please contact the ISP for more information X-TundraWare-MailScanner-ID: 17QLBVZh026598 X-TundraWare-MailScanner: Found to be clean X-TundraWare-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (not cached, score=-4.824, required 6, autolearn=not spam, ALL_TRUSTED -1.00, BAYES_00 -1.90, NICE_REPLY_A -2.00, TW_PF 0.08) X-TundraWare-MailScanner-From: tundra@tundraware.com X-Spam-Status: No X-Rspamd-Queue-Id: 4GwbBr2rDqz4ph0 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[tundraware.com:s=slkey,email-od.com:s=dkim]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:142.0.176.0/20]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[tundraware.com:+,email-od.com:+]; DMARC_POLICY_ALLOW(-0.50)[tundraware.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[142.0.181.21:from,142.0.191.3:received]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[tundra@tundraware.com,93ab.82.c37900021b83ee.9aadb35bdef40199041493f615702ded@email-od.com]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:53658, ipnet:142.0.180.0/22, country:US]; FROM_NEQ_ENVFROM(0.00)[tundra@tundraware.com,93ab.82.c37900021b83ee.9aadb35bdef40199041493f615702ded@email-od.com]; MAILMAN_DEST(0.00)[freebsd-questions]; DWL_DNSWL_NONE(0.00)[email-od.com:dkim] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Aug 2021 21:12:05 -0000 On 8/24/21 5:30 PM, Michael Sierchio wrote: > On Tue, Aug 24, 2021 at 2:47 PM Tim Daneliuk via freebsd-questions < > freebsd-questions@freebsd.org> wrote: > >> Is there any particular advantage - performance or otherwise - to breaking >> up >> a large ipfw table into smaller tables? >> >> We have a few firewalls approaching 100,000 rules for blocking addresses >> and CIDR blocks. > > > Do you really mean 100,000 firewall rules? 100,000 CIDR blocks is not > a problem. You should probably consolidate CIDR blocks before adding them > to a > table, because it's a longest-prefix-match. > > >> The IPS are read from separate text files in a loop >> in the firewall init code, but are all written to a single table. > > > I have a framework that collects IPs and CIDR blocks from various sources > (for blocking). > Two tables are used for this – so I can atomically replace the table > contents via table swap. > None of this is done in the firewall init code, it's all done via a > cronjob. I use the table arg to > store an integer that says what the source was. The firewall init script > only gets invoked at > startup, or when rules change. > > This >> is easy to maintain, but the concern is that we may be clobbering runtime >> performance. >> > > Did you know you can add an entire file to a table, if the lines consist of > > > > ? > > Empirically, this works for up to 8192 entries, so I split the file into > files of that size, > add them, then delete the splits. My pcengines box has > > CPU: AMD GX-412TC SOC (998.15-MHz K8-class > CPU) > > > *root@hearst:/usr/src 210#* ipfw table reject list | wc -l > > 99787 > > Something with decent power could easily filter 250,000 CIDR blocks. As I thought about this, it led to a followup question. Imagine I have populated a table and then run this command: ipfw add deny all from table\(10\) to any via em0 If I then later update the contents of table 10, will those changes go live on the firewall, or is the binding of table content to firewall rules only relevant at the time the "add deny" is invoked? To be more specific, I want to be able to build the table from a set of flat files that list all the naughty IPs and CIDR blocks at boot time. But I'd like to add/delete from that list as the system is running without having to reload the firewall every time. tl;dr I don't understand ipfw table semantics very well ...