Date: Tue, 3 Feb 2015 19:16:44 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: freebsd-net <freebsd-net@freebsd.org> Cc: Hans Petter Selasky <hselasky@freebsd.org> Subject: net device drivers need to set if_hw_tsomaxsegcount and friends Message-ID: <830919216.3191973.1423009004865.JavaMail.root@uoguelph.ca>
next in thread | raw e-mail | index | archive | help
Hi,
If you are an author/maintainer of a network device driver that
supports TSO, please add a few lines to your driver to set these
TSO related parameters in "struct ifnet", before you call
ether_ifattach(). Thanks go to hselasky@ for committing the patch
for these.
if_hw_tsomax - Maximum size of a TSO segment in bytes, excluding
any link level header(s).
Many devices expect this value to be in the 16bit
length in the IP header part of the TSO segment,
but not all.
--> If it must fit in the 16bit IP length field,
then I think IP_MAXPACKET (65535) would be
the correct value.
*** See below w.r.t. weird default for this, which
is probably incorrect/suboptimal for your hardware.
if_hw_tsomaxsegcount - The maximum number of discontiguous transmit
buffers that a TSO segment can live in, typically referred
to as transmit segments.
(This is essentially the upper bound on the number of data
mbufs that tcp_output() can use for the TSO segment.)
--> If your hardware allows it, please make this at least
35. (35 is sufficient for a 64K data payload + assorted
headers for NFS/iSCSI, TCP/IP etc.)
if_hw_tsomaxsegsize - The maximum size in bytes of each of these
transmit segments.
--> If this is less than MCLBYTES, you may need to
reduce if_hw_tsomaxsegcount by one, if the link
level header needs to be in one of these transmit
segments. (Again, see below for this case.)
*** The weird default value for if_hw_tsomax is a workaround for a
problem where devices with a transmit segment limit of 32 would
fail for a TSO segment of just under 64K.
- When a TSO segment was less than 64K by less than the size of
the link level header, the driver couldn't fit the TSO segment
into 32 * MCLBYTES when the size of the link level (MAC) header
was included.
--> Since the device only supported 32 transmit segments, even
m_defrag() couldn't get a > than 64K TSO segment
(including link header) to work.
Therefore, the default for if_hw_tsomax was set to:
32 * MCLBYTES - (max size of ethernet link level header)
as a hack/workaround for the problem. (It is only slightly smaller
than IP_MAXPACKET, which was the previous default.)
--> If the driver sets if_hw_tsomaxsegcount correctly, this hack should
not be needed.
For the weird case of if_hw_tsomaxsegsize < MCLBYTES, you might need to
set if_hw_tsomaxsegcount to 1 less than the hardware limit so that you
have a transmit segment available for the link level (ethernet) header.
If you don't set these values in your driver, the default value of
if_hw_tsomax will probably make things work, but your driver will
end up calling m_defrag() over and over and over again for TSO
segments, if your hardware supports less than 35 transmit segments.
If your hardware supports >= 35 transmit segments, then you might
still be using TSO suboptimally.
So, please set these...Thanks, rick
ps: Here's the comment from if_var.h, in case it does a better
job of describing these.
/*
237 * Network adapter TSO limits:
238 * ===========================
239 *
240 * If the "if_hw_tsomax" field is zero the maximum segment
241 * length limit does not apply. If the "if_hw_tsomaxsegcount"
242 * or the "if_hw_tsomaxsegsize" field is zero the TSO segment
243 * count limit does not apply. If all three fields are zero,
244 * there is no TSO limit.
245 *
246 * NOTE: The TSO limits only apply to the data payload part of
247 * a TCP/IP packet. That means there is no need to subtract
248 * space for ethernet-, vlan-, IP- or TCP- headers from the
249 * TSO limits unless the hardware driver in question requires
250 * so.
251 */
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?830919216.3191973.1423009004865.JavaMail.root>