Date: Wed, 7 Aug 2019 18:14:01 +0300 From: Sergey Akhmatov <sergey@akhmatov.ru> To: Greg Rivers <gcr+freebsd-stable@tharned.org>, freebsd-stable@freebsd.org Subject: Re: Regression in IXGBE(4) on 12.0-RELEASE? Message-ID: <e6940108-630e-e4cf-02a9-2fe35071c219@akhmatov.ru> In-Reply-To: <15504838.vxEf1mJfsr@no.place.like.home> References: <15504838.vxEf1mJfsr@no.place.like.home>
next in thread | previous in thread | raw e-mail | index | archive | help
On 07/08/2019 07:15, Greg Rivers wrote: > These Intel NICs have been performing very well for me: > > ix0@pci0:7:0:0: class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00 > vendor = 'Intel Corporation' > device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' > class = network > subclass = ethernet > > On 11.2-RELEASE, the cards probe like this: > > ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x7000-0x701f mem 0xf7f00000-0xf7ffffff,0xf7ef0000-0xf7ef3fff irq 28 at device 0.0 numa-domain 0 on pci1 > ix0: Using MSI-X interrupts with 9 vectors > ix0: Ethernet address: 8c:dc:d4:af:8a:e0 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix0: netmap queues/slots: TX 8/2048, RX 8/2048 > > As expected, all 8 queues are enabled on the NIC, with 8 corresponding interrupt threads pinned to 8 different CPUs > > But on 12.0-RELEASE, the cards probe like this: > > ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver> port 0x4020-0x403f mem 0xe2900000-0xe29fffff,0xe2a04000-0xe2a07fff at device 0.0 numa-domain 0 on pci4 > ix0: using 2048 tx descriptors and 2048 rx descriptors > ix0: msix_init qsets capped at 16 > ix0: Unable to map MSIX table > ix0: Using an MSI interrupt > ix0: allocated for 1 queues > ix0: allocated for 1 rx queues > ix0: Ethernet address: 48:df:37:62:be:38 > ix0: PCI Express Bus: Speed 5.0GT/s Width x8 > ix0: netmap queues/slots: TX 1/2048, RX 1/2048 > ix0: link state changed to UP > > MSI-X is disabled, and only one queue is enabled. I'm sure this is going to perform poorly under high network loads. Does anyone know why this is? Don't know if it could help somehow, but I've got two identical boxes with similar NICs side by side, one running 11.2 and the other one 12.0 and I don't have such problems. If someone points me where to look, I could help collecting more data to know how my system is different from Greg's. $ freebsd-version 12.0-RELEASE-p7 $ pciconf -lvc ix0 ix0@pci0:17:0:0: class=0x020000 card=0x7a128086 chip=0x10fb8086 rev=0x01 hdr=0x00 vendor = 'Intel Corporation' device = '82599ES 10-Gigabit SFI/SFP+ Network Connection' class = network subclass = ethernet cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit, vector masks cap 11[70] = MSI-X supports 64 messages, enabled Table in map 0x20[0x0], PBA in map 0x20[0x2000] cap 10[a0] = PCI-Express 2 endpoint max data 256(512) FLR RO NS link x8(x8) speed 5.0(5.0) ASPM disabled(L0s) ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected ecap 0003[140] = Serial 1 90e2baffff809b08 ecap 000e[150] = ARI 1 ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled 0 VFs configured out of 64 supported First VF RID Offset 0x0180, VF RID Stride 0x0002 VF Device ID 0x10ed Page Sizes: 4096 (enabled), 8192, 65536, 262144, 1048576, 4194304 $ dmesg ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver> port 0x2fc0-0x2fdf mem 0x91200000-0x912fffff,0x910fc000-0x910fffff irq 32 at device 0.0 on pci2 ix0: using 2048 tx descriptors and 2048 rx descriptors ix0: msix_init qsets capped at 16 ix0: pxm cpus: 8 queue msgs: 63 admincnt: 1 ix0: using 8 rx queues 8 tx queues ix0: Using MSIX interrupts with 9 vectors ix0: allocated for 8 queues ix0: allocated for 8 rx queues ix0: Ethernet address: 90:e2:ba:80:9b:08 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix0: netmap queues/slots: TX 8/2048, RX 8/2048 ========== $ freebsd-version 11.2-RELEASE-p3 $ dmesg ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 3.2.12-k> port 0x2fc0-0x2fdf mem 0x91200000-0x912fffff,0x910fc000-0x910fffff irq 32 at device 0.0 numa-domain 0 on pci2 ix0: Using MSI-X interrupts with 9 vectors ix0: Ethernet address: 90:e2:ba:80:99:d0 ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix0: netmap queues/slots: TX 8/2048, RX 8/2048
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e6940108-630e-e4cf-02a9-2fe35071c219>