From owner-freebsd-net@freebsd.org Tue Aug 27 20:20:21 2019 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E3D42DE984 for ; Tue, 27 Aug 2019 20:20:21 +0000 (UTC) (envelope-from vit@otcnet.ru) Received: from mail.otcnet.ru (mail.otcnet.ru [194.190.78.3]) by mx1.freebsd.org (Postfix) with ESMTP id 46J0c50RYYz4Hsr for ; Tue, 27 Aug 2019 20:20:20 +0000 (UTC) (envelope-from vit@otcnet.ru) Received: from Victors-MacBook-Air-2.local (unknown [195.91.148.145]) by mail.otcnet.ru (Postfix) with ESMTPSA id 6A4CD89DE4; Tue, 27 Aug 2019 23:20:18 +0300 (MSK) Subject: Re: finding optimal ipfw strategy To: Eugene Grosbein , "Andrey V. Elsukov" , freebsd-net@freebsd.org References: <4ff39c8f-341c-5d72-1b26-6558c57bff8d@grosbein.net> <50b0748d-bf8a-4de9-58bf-800ddd4f9c27@grosbein.net> <60d075ee-0d39-6606-ea5a-35e27818162a@otcnet.ru> <032b67e1-5359-4f00-a449-26c43b70a467@grosbein.net> From: Victor Gamov Organization: OTCnet Message-ID: <328ff261-f247-976f-ccba-e4b4e9098d8c@otcnet.ru> Date: Tue, 27 Aug 2019 23:20:16 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <032b67e1-5359-4f00-a449-26c43b70a467@grosbein.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 46J0c50RYYz4Hsr X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of vit@otcnet.ru designates 194.190.78.3 as permitted sender) smtp.mailfrom=vit@otcnet.ru X-Spamd-Result: default: False [-6.51 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+a:mail.otcnet.ru]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[otcnet.ru]; TO_DN_SOME(0.00)[]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; IP_SCORE(-3.33)[ip: (-8.76), ipnet: 194.190.78.0/24(-4.38), asn: 50822(-3.50), country: RU(0.01)]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:50822, ipnet:194.190.78.0/24, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Aug 2019 20:20:21 -0000 On 27/08/2019 22:59, Eugene Grosbein wrote: > 28.08.2019 2:22, Victor Gamov wrote: > >>> Also, you should monitor interrupt numbers shown by "systat -vm 3" for igb* devices >>> at hours of most load. If they approach 8000 limit but not exceed it, >>> you may be suffering from this and should raise the limit with /boot/loader.conf: >>> >>> hw.igb.max_interrupt_rate=32000 >> >> It's about 5000-7000 per rxq > > 7000 is quite close considering it is average for quite long period (a second or seconds). > It can hit 8000 for some ticks easily in your case. Eugene, can you explain me what max_interrupt_rate means? Calomel explain it as "# maximum number of interrupts per second generated by single igb". So if I increase it then more irq per sec will be generated and more context switch produced. Why I need to increase it? > You should raise the limit as soon as possible. I'd advise to double it upto 16000 first > but if you cannot afford extra reboot better use 32000 as it's just fine for 1Gbps link. # sysctl hw.igb.max_interrupt_rate sysctl: unknown oid 'hw.igb.max_interrupt_rate' # Is it possible to set it at boot time only? -- CU, Victor Gamov