From eugen@grosbein.net Tue Aug 10 05:08:15 2021 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 91724175436B; Tue, 10 Aug 2021 05:08:49 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [IPv6:2a01:4f8:c2c:26d8::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GkLZm49SSz3CdL; Tue, 10 Aug 2021 05:08:48 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id 17A58YFA086207 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 10 Aug 2021 05:08:34 GMT (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: alfadev@protonmail.com Received: from [10.58.0.10] (dadvw [10.58.0.10]) by eg.sd.rdtc.ru (8.16.1/8.16.1) with ESMTPS id 17A58Xgs057667 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT); Tue, 10 Aug 2021 12:08:33 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: Throughput extremely decreases when IPFW 7000 mac based rules activated To: alfadev , "freebsd-hackers@FreeBSD.org" , "freebsd-ipfw@FreeBSD.org" , "Alexander V. Chernikov" , "Andrey V. Elsukov" References: From: Eugene Grosbein Message-ID: <7a737f3d-e291-e8a1-b629-09365a99c937@grosbein.net> Date: Tue, 10 Aug 2021 12:08:15 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=4.0 required=5.0 tests=BAYES_05,LOCAL_FROM, NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.2 X-Spam-Report: * -0.5 BAYES_05 BODY: Bayes spam probability is 1 to 5% * [score: 0.0257] * 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record * -0.0 SPF_PASS SPF: sender matches SPF record * 2.6 LOCAL_FROM From my domains * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * -0.0 NICE_REPLY_A Looks like a legit reply (A) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on hz.grosbein.net X-Spam-Level: *** X-Rspamd-Queue-Id: 4GkLZm49SSz3CdL X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=fail (mx1.freebsd.org: domain of eugen@grosbein.net does not designate 2a01:4f8:c2c:26d8::2 as permitted sender) smtp.mailfrom=eugen@grosbein.net X-Spamd-Result: default: False [-1.65 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; NEURAL_HAM_SHORT(-0.55)[-0.549]; FREEMAIL_TO(0.00)[protonmail.com,FreeBSD.org,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/32, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_SPF_FAIL(1.00)[-all]; FREEFALL_USER(0.00)[eugen]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[grosbein.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_AMI_RCVD_FAIL(0.00)[62.231.161.221:server fail]; RCVD_TLS_ALL(0.00)[] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N CC'ing more knowledgeable eyes that may have something to add. 09.08.2021 21:58, alfadev via freebsd-hackers wrote: > Hi, I have freebsd 11.2 server with IPFW firewall > > 870Mbits Fiber Net exist in my data center > > There are 7000 defined mac based rules on IPFW and 3000 of them active client . There is no problem before IPFW rules loading but when i load IPFW rules, > > throughput extremely decreases up to 80Mbits. There are not any error logs. I could not find what is the problem. > > Any help would be appreciated at this point. The search over ipfw rules is linear, so no wonder it decreases drastically when the list grows so big. Also, layer-2 frames and then layer-3 packets may pass over ipfw matching process upto four times\ unless you carefully create your ruleset like this: ipfw add 10 skipto 1000 not layer2 # do not MAC-filter layer3 packets that have L2 header stripped already anyway ipfw add 20 allow out # do not MAC-filter packets leaving the firewall, MAC-filter incoming only ipfw add 30 ... # start MAC-filtering here ... ipfw add 1000 ... # firewall part for layer3 packets Also, if you do filtering bridge, you should carefully read if_bridge(4) manual page, section PACKET FILTERING and disable extra passes over packet filters such as: sysctl net.link.bridge.pfil_member=0 # disable extra passes over ipfw ruleset for bridge members, filter the bridge itself only Such ruleset could decrease filtering overhead several times but I'm afraid that ipfw is not right tool for this task at the moment. ipfw has "tables" to optimize large list matching and they perform great but for layer3 IP matching, not for layer2 MAC matching. Also, your 11.2 version is quite old and you may need to upgrade to 11.4-STABLE at least to catch up with bugfixes and/or optimizations.