Date: Wed, 30 May 2012 11:30:47 -0400 From: Andrew Gallatin <gallatin@cs.duke.edu> To: freebsd-net@freebsd.org Cc: Colin Percival <cperciva@freebsd.org> Subject: Re: [please review] TSO mbuf chain length limiting patch Message-ID: <4FC63D27.70807@cs.duke.edu> In-Reply-To: <4FC635CC.5030608@freebsd.org> References: <4FC635CC.5030608@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 05/30/12 10:59, Colin Percival wrote: > Hi all, > > The Xen virtual network interface has an issue (ok, really the issue is with > the linux back-end, but that's what most people are using) where it can't > handle scatter-gather writes with lots of pieces, aka. long mbuf chains. > This currently bites us hard with TSO enabled, since it produces said long > mbuf chains. Colin, Thanks for pointing me at this. I've been talking about this with bz@ a little. I've never been clear about what the max TSO size supported by FreeBSD is. The NIC I maintain (mxge) is limited to 64K - epsilon for both IPv4 *AND* IPv6. Up until now, this has been enforced by the 16-bit ip length limit of IPv4 and we have not had IPv6 TSO until this week. With IPv6, I'm worried that FreeBSD may now send packets down larger than I could handle. In my case, however, the problem is not s/g list length, but rather it is internal limits in the NIC which limit us to 64K - epsilon for IPv6 as well. I think there may be other NICs in the same boat for IPv6 (and maybe even some which cannot handle the full 64K for IPv4). Your approach would not work well for my size limit. For example, I'd have to set the limit to 4 mbufs to stay under 64KB. This would be assuming the worst case of 16KB jumbo mbufs, so that would limit me to ~8KB per TSO if 2KB mbufs were used. I think a better approach would be to have a limit on the size of the pre-segmented TCP payload size sent to the driver. I tend to think that this would be more generically useful, and it is a better match for the NDIS APIs, where a driver must specify the max TSO size. I think the changes to the TCP stack might be simpler (eg, they would seem to jive better with the existing "maxmtu" approach). I think this could work for you as well. You could set the Xen max tso size to be 32K (derived from 18 pages/skb, multiplied by a typical 2KB mbuf size, with some slack built in). If the chain was too large, you could m_defrag it down to size. Drew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4FC63D27.70807>