From owner-freebsd-net@freebsd.org Tue Aug 27 18:06:30 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 978F7DB9FD for ; Tue, 27 Aug 2019 18:06:30 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward104j.mail.yandex.net (forward104j.mail.yandex.net [5.45.198.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46Hxdd1sPPz49Jb for ; Tue, 27 Aug 2019 18:06:28 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback29g.mail.yandex.net (mxback29g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:329]) by forward104j.mail.yandex.net (Yandex) with ESMTP id CD60B4A01B7; Tue, 27 Aug 2019 21:06:25 +0300 (MSK) Received: from smtp3o.mail.yandex.net (smtp3o.mail.yandex.net [2a02:6b8:0:1a2d::27]) by mxback29g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id mgTQ0uvA3K-6PAKvaPS; Tue, 27 Aug 2019 21:06:25 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1566929185; bh=ZAP4MojBQDbq6iBPdWnO71KRdPj8pmXjXdB0G6eVjDw=; h=In-Reply-To:From:Date:References:To:Subject:Message-ID; b=AKg4V13bX/dSEGBbL2Kt/UgSh/La6n9ALA4drAUo8OnKj9ax/RUhS3igKfa8prkao 1rfpnphFUywTdFULBPq2UkK4Ek7Kk0ioqPahT8LQlFARMUhtsS8chly5V7/0n1fAk2 bXQwVDBBkMJDAjmY9a1PULqW7NkQmmUtg2rLWblE= Received: by smtp3o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id G63pF4Vsbn-6PGO7vxW; Tue, 27 Aug 2019 21:06:25 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Subject: Re: finding optimal ipfw strategy To: Victor Gamov , Eugene Grosbein , freebsd-net@freebsd.org References: <4ff39c8f-341c-5d72-1b26-6558c57bff8d@grosbein.net> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= mQENBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAG0JUFuZHJleSBWLiBFbHN1a292IDxidTdjaGVyQHlhbmRleC5ydT6JATgEEwECACIFAkwB F1kCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEAHF6gQQyKF6qmYIAI6ekfm1VA4T vqankI1ISE6ku4jV7UlpIQlEbE7/8n3Zd6teJ+pGOQhN5qk8QE7utdPdbktAzi+x7LIJVzUw 4TywZLXGrkP7VKYkfg6oyCGyzITghefQeJtr2TN4hYCkzPWpylkue8MtmqfZv/6royqwTbN+ +E09FQNvTgRUYJYTeQ1qOsxNRycwvw3dr2rOfuxShbzaHBB1pBIjGrMg8fC5pd65ACH5zuFV A0CoTNGMDrEZSfBkTW604UUHFFXeCoC3dwDZRKOWJ3GmMXns65Ai5YkA63BSHEE1Qle3VBhd cG1w0CB5FBV3pB27UVnf0jEbysrDqW4qN7XMRFSWNAy5AQ0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAYkBHwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: Date: Tue, 27 Aug 2019 21:03:19 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="t4aIzluZvDmDHeKPHFtgv72lbWVPGkAP8" X-Rspamd-Queue-Id: 46Hxdd1sPPz49Jb X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=AKg4V13b; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of bu7cher@yandex.ru designates 5.45.198.247 as permitted sender) smtp.mailfrom=bu7cher@yandex.ru X-Spamd-Result: default: False [-6.19 / 15.00]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip4:5.45.192.0/19]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[yandex.ru:+]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[247.198.45.5.list.dnswl.org : 127.0.5.1]; ASN(0.00)[asn:13238, ipnet:5.45.192.0/18, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; IP_SCORE(0.00)[ipnet: 5.45.192.0/18(-4.81), asn: 13238(-3.71), country: RU(0.01)]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; IP_SCORE_FREEMAIL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Aug 2019 18:06:30 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --t4aIzluZvDmDHeKPHFtgv72lbWVPGkAP8 Content-Type: multipart/mixed; boundary="eLoN6jvRA70jdKvRzdV2b1T5l5r22FWpQ"; protected-headers="v1" From: "Andrey V. Elsukov" To: Victor Gamov , Eugene Grosbein , freebsd-net@freebsd.org Message-ID: Subject: Re: finding optimal ipfw strategy References: <4ff39c8f-341c-5d72-1b26-6558c57bff8d@grosbein.net> In-Reply-To: --eLoN6jvRA70jdKvRzdV2b1T5l5r22FWpQ Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 26.08.2019 19:25, Victor Gamov wrote: > More general question about my current config.=C2=A0 I have about 200Mb= it > input multicasts which bridged and filtered later (about 380 Mbit > bridged if trafshow does not lie me :-) ) >=20 > My FreeBSD box (12.0-STABLE r348449 GENERIC=C2=A0 amd64)=C2=A0 has one = "Intel(R) > Xeon(R) CPU E31270 @ 3.40GHz"=C2=A0 and 4-ports=C2=A0 "Intel(R) PRO/100= 0 > PCI-Express Network Driver".=C2=A0 HT disabled and traffic mainly incom= e via > igb0 and out both via igb0 and igb2.=C2=A0 About 30 VLANs now active so= me at > igb0 and some at igb2. >=20 >=20 > And I have following `top` stat: > =3D=3D=3D=3D=3D > CPU 0:=C2=A0 0.0% user,=C2=A0 0.0% nice, 80.5% system,=C2=A0 0.0% inter= rupt, 19.5% idle > CPU 1:=C2=A0 0.0% user,=C2=A0 0.0% nice, 34.1% system,=C2=A0 0.0% inter= rupt, 65.9% idle > CPU 2:=C2=A0 0.0% user,=C2=A0 0.0% nice, 17.1% system,=C2=A0 0.0% inter= rupt, 82.9% idle > CPU 3:=C2=A0 0.0% user,=C2=A0 0.0% nice, 46.3% system,=C2=A0 0.0% inter= rupt, 53.7% idle > =3D=3D=3D=3D=3D This doesn't look like heavy ipfw load. E.g. this is top output from slightly loaded firewall (300Mbytes/s ~500kpps): last pid: 58184; load averages: 9.07, 8.98, 8.83 up 72+07:45:55 21:01:36 821 processes: 36 running, 680 sleeping, 105 waiting CPU 0: 0.0% user, 0.0% nice, 0.0% system, 28.1% interrupt, 71.9% idle= CPU 1: 0.0% user, 0.0% nice, 0.0% system, 33.6% interrupt, 66.4% idle= CPU 2: 0.0% user, 0.0% nice, 0.0% system, 28.9% interrupt, 71.1% idle= CPU 3: 0.0% user, 0.0% nice, 0.0% system, 27.3% interrupt, 72.7% idle= CPU 4: 0.0% user, 0.0% nice, 0.0% system, 21.1% interrupt, 78.9% idle= CPU 5: 0.0% user, 0.0% nice, 0.0% system, 26.6% interrupt, 73.4% idle= CPU 6: 0.0% user, 0.0% nice, 0.0% system, 28.1% interrupt, 71.9% idle= CPU 7: 0.0% user, 0.0% nice, 0.0% system, 21.1% interrupt, 78.9% idle= CPU 8: 0.0% user, 0.0% nice, 0.0% system, 35.2% interrupt, 64.8% idle= CPU 9: 0.0% user, 0.0% nice, 0.0% system, 29.7% interrupt, 70.3% idle= CPU 10: 0.0% user, 0.0% nice, 0.0% system, 27.3% interrupt, 72.7% idle= CPU 11: 0.0% user, 0.0% nice, 0.0% system, 19.5% interrupt, 80.5% idle= CPU 12: 0.0% user, 0.0% nice, 0.0% system, 32.8% interrupt, 67.2% idle= CPU 13: 0.0% user, 0.0% nice, 0.0% system, 34.4% interrupt, 65.6% idle= CPU 14: 0.0% user, 0.0% nice, 0.0% system, 29.7% interrupt, 70.3% idle= CPU 15: 0.0% user, 0.0% nice, 0.0% system, 26.6% interrupt, 73.4% idle= CPU 16: 0.0% user, 0.0% nice, 0.0% system, 28.9% interrupt, 71.1% idle= CPU 17: 0.0% user, 0.0% nice, 0.0% system, 34.4% interrupt, 65.6% idle= CPU 18: 0.0% user, 0.0% nice, 0.0% system, 36.7% interrupt, 63.3% idle= CPU 19: 0.0% user, 0.0% nice, 0.0% system, 21.9% interrupt, 78.1% idle= CPU 20: 0.0% user, 0.0% nice, 0.0% system, 21.1% interrupt, 78.9% idle= CPU 21: 0.0% user, 0.0% nice, 0.0% system, 32.0% interrupt, 68.0% idle= CPU 22: 0.0% user, 0.0% nice, 0.0% system, 33.6% interrupt, 66.4% idle= CPU 23: 0.0% user, 0.0% nice, 0.0% system, 26.6% interrupt, 73.4% idle= CPU 24: 0.0% user, 0.0% nice, 0.0% system, 21.9% interrupt, 78.1% idle= CPU 25: 0.0% user, 0.0% nice, 0.0% system, 21.1% interrupt, 78.9% idle= CPU 26: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle= CPU 27: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle= # pmcstat -S instructions -Tw1 PMC: [INSTR_RETIRED_ANY] Samples: 443074 (100.0%) , 0 unresolved Key: q =3D> exiting... %SAMP IMAGE FUNCTION CALLERS 39.2 kernel sched_idletd fork_exit 10.9 ipfw.ko ipfw_chk ipfw_check_packet 3.6 kernel cpu_search_lowest cpu_search_lowest 2.8 kernel lock_delay _mtx_lock_spin_cookie 2.5 kernel _rm_rlock in6_localip:1.3 pfil_run_hooks:0.6 2.2 kernel rn_match ta_lookup_radix:1.5 fib6_lookup_nh_basic:0.6 As you can see, when ipfw produces high load, interrupt column is more than system. --=20 WBR, Andrey V. Elsukov --eLoN6jvRA70jdKvRzdV2b1T5l5r22FWpQ-- --t4aIzluZvDmDHeKPHFtgv72lbWVPGkAP8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAl1lcGcACgkQAcXqBBDI oXqHjAf/WcebHfqWh98HZQGOWchcfjl321Gvvw9+kJ9HCIB201ZQ7xzEXIDl8IST 5W8lMAHh4UkYqyd/Om9thL4T5lQCDO9jAw/B8A4VmQuheMY5dVvO2mGd2mk6+gH/ 7QHVVK/BU9r5tiX8oCXFEGBs6T3cZPT7HaWF4BLuFV0B0VRF7V9fH/XxgkLRzsLm nTVvAXjRekHuK5CXOI8kJBgwV0hA83GwnRGdwZaL33q/vQpWJRK6D1xHZ3477lg3 hykidpvwoQw1bJhR0LWtAAZ0CaDjv2SLCplDsG89ZmREsImWZmgU9DC1RiPdSgKb Bnv4joEkFEYXz8k87einnE2xHCsdbg== =GzyU -----END PGP SIGNATURE----- --t4aIzluZvDmDHeKPHFtgv72lbWVPGkAP8--