Date: Wed, 21 Oct 2020 20:53:53 +0000 From: bugzilla-noreply@freebsd.org To: net@FreeBSD.org Subject: [Bug 248652] iflib: netmap pkt-gen large TX performance difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC Message-ID: <bug-248652-7501-4pLmbB8AfB@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-248652-7501@https.bugs.freebsd.org/bugzilla/> References: <bug-248652-7501@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248652 --- Comment #31 from Vincenzo Maffione <vmaffione@FreeBSD.org> --- (In reply to Krzysztof Galazka from comment #30) Hi Krzysztof, I agree, and created a separate review for this possible change: https://reviews.freebsd.org/D26896 It would be nice if you guys could run some tests to validate the change against normal TCP/IP stack usage (e.g. non-netmap). Speaking about the non-iflib driver, I guess it is acceptable for the ixl_x= mit routine to always set the report flag on the EOP packet. However, this is not acceptable for netmap, and indeed the non-iflib netmap= ixl driver is only setting it twice per ring (see https://github.com/freebsd/freebsd/blob/stable/11/sys/dev/netmap/if_ixl_net= map.h#L221-L223). This is, in my opinion, what explains the huge difference in performance between non-iflib and iflib for ixl. If my understanding is correct, and according to our past experience with netmap, the report flag will cause the NIC to initiate a DMA transaction to either set the DD bit in the descriptor, or perform a memory write to update the shadow TDH. This is particularly expensive in netmap when done for each descriptor, specially because netmap uses single-descriptor packets. Interrupt moderation can also help a lot to mitigate the CPU overhead, but = as far as I see it does not limit the writeback DMA transactions, and therefor= e it does not help in the netmap use-case. Moreover, my understanding is that if= lib is not using hardware interrupts on ixl (nor ix), but rather is using "softirqs", so I guess that interrupt moderation does not play a role here.= I may be wrong on this last point. I suspect the 1-queue pkt-gen hang problem may be due to something else, ra= ther than the report flag change. As a matter of facts, the same logic was worki= ng fine on the non-iflib driver. --=20 You are receiving this mail because: You are on the CC list for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-248652-7501-4pLmbB8AfB>