Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 14 Oct 2020 12:08:36 +0000
From:      bugzilla-noreply@freebsd.org
To:        net@FreeBSD.org
Subject:   [Bug 248652] iflib: netmap pkt-gen large TX performance difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC
Message-ID:  <bug-248652-7501-3QXOe8SXpW@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-248652-7501@https.bugs.freebsd.org/bugzilla/>
References:  <bug-248652-7501@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D248652

--- Comment #16 from Sylvain Galliano <sg@efficientip.com> ---
(In reply to Vincenzo Maffione from comment #14)

Here are the results:

X520 with 1 queue
ix0: PCI Express Bus: Speed 5.0GT/s Width x8
ix0: netmap queues/slots: TX 1/2048, RX 1/2048

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=3D0  (default value)

pkt-gen:
683.502433 main_thread [2639] 4.215 Mpps (4.227 Mpkts 2.023 Gbps in 1002819
usec) 465.43 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=3D300

pkt-gen:
750.688608 main_thread [2639] 6.496 Mpps (6.646 Mpkts 3.118 Gbps in 1023000
usec) 465.45 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=3D200

pkt-gen:
771.736855 main_thread [2639] 8.907 Mpps (9.112 Mpkts 4.275 Gbps in 1022999
usec) 465.45 avg_batch 99999 min_space

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=3D100

pkt-gen:
804.554603 main_thread [2639] 14.136 Mpps (14.147 Mpkts 6.785 Gbps in 10007=
48
usec) 465.45 avg_batch 99999 min_space
-> close to 10G line rate

*******************************************

sysctl dev.ix.0.iflib.nm_tx_tmr_us=3D90

pkt-gen:
872.156329 main_thread [2639] 14.880 Mpps (15.054 Mpkts 7.142 Gbps in 10117=
21
usec) 466.96 avg_batch 99999 min_space



Now using same X520 NIC using 4 queues.

ix1: PCI Express Bus: Speed 5.0GT/s Width x8
ix1: netmap queues/slots: TX 4/2048, RX 4/2048

*******************************************

sysctl dev.ix.1.iflib.nm_tx_tmr_us=3D0 (default)

pkt-gen:
047.988586 main_thread [2639] 13.596 Mpps (13.623 Mpkts 6.526 Gbps in 10020=
02
usec) 443.03 avg_batch 99999 min_space
-> close to max speed (thanks to 4 queue)

*******************************************

sysctl dev.ix.1.iflib.nm_tx_tmr_us=3D400

pkt-gen:
094.224581 main_thread [2639] 14.887 Mpps (14.904 Mpkts 7.146 Gbps in 10011=
73
usec) 440.75 avg_batch 99999 min_space


Looks really good for X520 NIC whatever the number of queue I used.



Now same tests using XL710 NIC (40G) using 1 queue:

ixl1: PCI Express Bus: Speed 8.0GT/s Width x8
ixl1: netmap queues/slots: TX 1/1024, RX 1/1024

*******************************************

sysctl dev.ixl.1.iflib.nm_tx_tmr_us=3D0 (default)

pkt-gen:
324.883066 main_thread [2639] 12.270 Mpps (13.044 Mpkts 5.890 Gbps in 10630=
00
usec) 16.53 avg_batch 99999 min_space

*******************************************

sysctl dev.ixl.1.iflib.nm_tx_tmr_us=3D100

pkt-gen:
350.497566 main_thread [2639] 12.246 Mpps (12.258 Mpkts 5.878 Gbps in 10010=
03
usec) 16.48 avg_batch 99999 min_space

no changes.


Now testing XL710 with 4 queues:

ixl0: PCI Express Bus: Speed 8.0GT/s Width x8
ixl0: netmap queues/slots: TX 4/1024, RX 4/1024

*******************************************

sysctl dev.ixl.0.iflib.nm_tx_tmr_us=3D0 (default)

pkt-gen:
614.766048 main_thread [2639] 13.671 Mpps (14.539 Mpkts 6.562 Gbps in 10634=
94
usec) 15.75 avg_batch 99999 min_space

*******************************************

sysctl dev.ixl.0.iflib.nm_tx_tmr_us=3D100

pkt-gen:
640.652549 main_thread [2639] 13.672 Mpps (13.795 Mpkts 6.562 Gbps in 10090=
01
usec) 15.79 avg_batch 99999 min_space


No changes using XL710 NIC (as a reminder, using FreeBSD 11 without iflib, I
can reach +40Mpps on XL710 using pkt-gen)

--=20
You are receiving this mail because:
You are on the CC list for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-248652-7501-3QXOe8SXpW>