From owner-freebsd-stable@freebsd.org Fri Apr 5 15:08:27 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51B8D1579C13 for ; Fri, 5 Apr 2019 15:08:27 +0000 (UTC) (envelope-from mach@swishmail.com) Received: from vorlon.swishmail.com (vorlon.swishmail.com [208.72.56.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7ED5F959B5 for ; Fri, 5 Apr 2019 15:08:26 +0000 (UTC) (envelope-from mach@swishmail.com) Received: (qmail 68322 invoked by uid 89); 5 Apr 2019 15:01:43 -0000 Received: from unknown (HELO ?IPv6:2001:b030:14e:100:adf0:7e78:dd58:9ed?) (mach@swishmail.com@2001:b030:14e:100:adf0:7e78:dd58:9ed) by 2602:ffb8::208:72:56:19 with ESMTPSA (ECDHE-RSA-AES128-GCM-SHA256 encrypted, authenticated); 5 Apr 2019 15:01:43 -0000 To: freebsd-stable@freebsd.org From: Kris von Mach Subject: em performs worse than igb (latency wise) in 12? Message-ID: Date: Fri, 5 Apr 2019 23:01:42 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Rspamd-Queue-Id: 7ED5F959B5 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of mach@swishmail.com designates 208.72.56.19 as permitted sender) smtp.mailfrom=mach@swishmail.com X-Spamd-Result: default: False [0.11 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.58)[-0.580,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:208.72.56.0/22]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.74)[-0.745,0]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[swishmail.com]; NEURAL_SPAM_SHORT(0.86)[0.862,0]; MX_GOOD(-0.01)[mxfilter1.nyc.swishmail.com,mxfilter2.nyc.swishmail.com]; IP_SCORE(-0.01)[country: US(-0.06)]; RCVD_IN_DNSWL_LOW(-0.10)[19.56.72.208.list.dnswl.org : 127.0.5.1]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:14469, ipnet:208.72.56.0/22, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 15:08:27 -0000 Hello, I've upgraded from Stable 11 to Stable 12, and noticed that igb has been = removed and is now part of em driver. However, the performance seems a=20 lot worse. This is using HP 366FLR which is just HP's version of Intel=20 i350 I believe. I've tried incorporating the sysctl settings I used in 11 into 12: dev.igb.0.fc=3D0 dev.igb.0.iflib.rx_budget=3D65535 hw.igb.rx_process_limit=3D"-1" hw.igb.num_queues=3D"2" hw.igb.rxd=3D"2048"=C2=A0 # (default 1024) hw.igb.txd=3D"2048"=C2=A0 # (default 1024) hw.igb.max_interrupt_rate=3D"32000" # (default 8000) hw.em.rx_process_limit=3D"-1" hw.em.num_queues=3D"2" hw.em.rxd=3D"2048"=C2=A0 # (default 1024) hw.em.txd=3D"2048"=C2=A0 # (default 1024) hw.em.max_interrupt_rate=3D"32000" But that didn't seem to help. Switching back to 11, of course restored the performance. The performance hit I'm talking about seems to be with latency, not=20 throughput. To give you an example, this server is serving a website that is=20 monitored from different locations and all locations the latency has=20 increased by about 30-50% using stable 12. Any ideas as to how to improve latency in 12? Can you compile igb driver = in 12? Has something else other than the network driver changed in 12=20 that could cause an increase in latency?