Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 21 Oct 2020 20:53:53 +0000
From:      bugzilla-noreply@freebsd.org
To:        net@FreeBSD.org
Subject:   [Bug 248652] iflib: netmap pkt-gen large TX performance difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC
Message-ID:  <bug-248652-7501-4pLmbB8AfB@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-248652-7501@https.bugs.freebsd.org/bugzilla/>
References:  <bug-248652-7501@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248652

--- Comment #31 from Vincenzo Maffione <vmaffione@FreeBSD.org> ---
(In reply to Krzysztof Galazka from comment #30)
Hi Krzysztof,
  I agree, and created a separate review for this possible change:
https://reviews.freebsd.org/D26896
It would be nice if you guys could run some tests to validate the change
against normal TCP/IP stack usage (e.g. non-netmap).

Speaking about the non-iflib driver, I guess it is acceptable for the ixl_x=
mit
routine to always set the report flag on the EOP packet.
However, this is not acceptable for netmap, and indeed the non-iflib netmap=
 ixl
driver is only setting it twice per ring (see
https://github.com/freebsd/freebsd/blob/stable/11/sys/dev/netmap/if_ixl_net=
map.h#L221-L223).
This is, in my opinion, what explains the huge difference in performance
between non-iflib and iflib for ixl.

If my understanding is correct, and according to our past experience with
netmap, the report flag will cause the NIC to initiate a DMA transaction to
either set the DD bit in the descriptor, or perform a memory write to update
the shadow TDH.
This is particularly expensive in netmap when done for each descriptor,
specially because netmap uses single-descriptor packets.
Interrupt moderation can also help a lot to mitigate the CPU overhead, but =
as
far as I see it does not limit the writeback DMA transactions, and therefor=
e it
does not help in the netmap use-case. Moreover, my understanding is that if=
lib
is not using hardware interrupts on ixl (nor ix), but rather is using
"softirqs", so I guess that interrupt moderation does not play a role here.=
 I
may be wrong on this last point.

I suspect the 1-queue pkt-gen hang problem may be due to something else, ra=
ther
than the report flag change. As a matter of facts, the same logic was worki=
ng
fine on the non-iflib driver.

--=20
You are receiving this mail because:
You are on the CC list for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-248652-7501-4pLmbB8AfB>