From owner-freebsd-questions@freebsd.org Tue Aug 24 22:31:16 2021 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 94FE86681E8 for ; Tue, 24 Aug 2021 22:31:16 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GvP374CmQz3vJ1 for ; Tue, 24 Aug 2021 22:31:15 +0000 (UTC) (envelope-from kudzu@tenebras.com) Received: by mail-lf1-x12b.google.com with SMTP id k5so48697404lfu.4 for ; Tue, 24 Aug 2021 15:31:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tenebras-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=aLeG1Xh4G48OothaUE3ILMFHDi66Gsm+3/Y+OyG6xMs=; b=A1bhWSgKj9Pzk7wGktjgPj+yuagEq/Q+II9TD87fhEIi9EJ6ht91xq7fO1zi/tGDJn MYr0IjwsWUWoLcuMW6FrZXjhbMdSxw9Jy2gC33Q86w5IcNCEXWDB96YTedY3L6EHkSIq glK3QdJMO8IICZc6mtubJuW/vGhZPg8/ooZbGwAIxCgKxUMqCzDKrSNSfwzzxkjSWHfK uo0reOq79nf7NPYqe2mWJK3O2nEvIw/i0g8DoOF48po2bNHEdRnJEPM6pUPa0aZmCUDt uj37blMF/Sl9CP2hcf/8VGvFWgRqpjU+gKcjbuRzlvwVaY/zCQGovKm1hj2k2vu07rid Spvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=aLeG1Xh4G48OothaUE3ILMFHDi66Gsm+3/Y+OyG6xMs=; b=NPgzaZrzNn8UJPrIsd2WDvOHUJvLUGu/tqZJTslE0NWJAvVeQSx6sbDZLcYiYFlm9U A2hI9a09iWsXp4mlFyKvgTWFdcb/aEFJEYrnnnbMih5otcMEV7uNgaduDmh3lWqjifxo 7DbR+ZsMuGX7GlZHJ1/ix2OYeBJmeOyr6Wfo8sPYtLqsWG+0GYDzi0ZPHUwfDGpRxKE+ tjsbIEveIGPSXv6FfHDXlCUsLcbo5QPKJSJw9uEToh4Oa8zihZb2V/SRPD5SfRBHCBlO yiXdf+9NyBJQ6sj23fhajLdIlSGgkIpFlMPR2A9AYHXtpYasFCfH7W06hbEKei0WgRZP /U8A== X-Gm-Message-State: AOAM533NWmoiJ6L48MnT63LzkFJEMESgLvmy0z0+OewKDRJedKx7d+ZC FFI1L12dWMXhG9nVvystCrg+SFuxf6l4rZ9v1QCVLFcJMYc8oyam X-Google-Smtp-Source: ABdhPJwbI7I/NBMd38Zx5vUEmDSQ3QEoYlXgfltewYq1qbtbiIvYomymYnl0HrqJTZTTq8fP4MC+81UQyBNyVFrlHyM= X-Received: by 2002:a05:6512:21cf:: with SMTP id d15mr30467659lft.548.1629844273951; Tue, 24 Aug 2021 15:31:13 -0700 (PDT) MIME-Version: 1.0 References: <9e6cd8e2-a06e-468b-7245-d5ff13309763@tundraware.com> In-Reply-To: <9e6cd8e2-a06e-468b-7245-d5ff13309763@tundraware.com> From: Michael Sierchio Date: Tue, 24 Aug 2021 15:30:38 -0700 Message-ID: Subject: Re: ipfw Table Organization To: FreeBSD Mailing List X-Rspamd-Queue-Id: 4GvP374CmQz3vJ1 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tenebras-com.20150623.gappssmtp.com header.s=20150623 header.b=A1bhWSgK; dmarc=none; spf=none (mx1.freebsd.org: domain of kudzu@tenebras.com has no SPF policy when checking 2a00:1450:4864:20::12b) smtp.mailfrom=kudzu@tenebras.com X-Spamd-Result: default: False [0.04 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[tenebras-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.96)[-0.960]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-questions@freebsd.org]; DMARC_NA(0.00)[tenebras.com]; NEURAL_SPAM_MEDIUM(0.98)[0.978]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[tenebras-com.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::12b:from]; NEURAL_SPAM_SHORT(0.33)[0.326]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-questions]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Aug 2021 22:31:16 -0000 On Tue, Aug 24, 2021 at 2:47 PM Tim Daneliuk via freebsd-questions < freebsd-questions@freebsd.org> wrote: > Is there any particular advantage - performance or otherwise - to breakin= g > up > a large ipfw table into smaller tables? > > We have a few firewalls approaching 100,000 rules for blocking addresses > and CIDR blocks. Do you really mean 100,000 firewall rules? 100,000 CIDR blocks is not a problem. You should probably consolidate CIDR blocks before adding them to a table, because it's a longest-prefix-match. > The IPS are read from separate text files in a loop > in the firewall init code, but are all written to a single table. I have a framework that collects IPs and CIDR blocks from various sources (for blocking). Two tables are used for this =E2=80=93 so I can atomically replace the tabl= e contents via table swap. None of this is done in the firewall init code, it's all done via a cronjob. I use the table arg to store an integer that says what the source was. The firewall init script only gets invoked at startup, or when rules change. This > is easy to maintain, but the concern is that we may be clobbering runtime > performance. > Did you know you can add an entire file to a table, if the lines consist of ? Empirically, this works for up to 8192 entries, so I split the file into files of that size, add them, then delete the splits. My pcengines box has CPU: AMD GX-412TC SOC (998.15-MHz K8-class CPU) *root@hearst:/usr/src 210#* ipfw table reject list | wc -l 99787 Something with decent power could easily filter 250,000 CIDR blocks.