Date: Thu, 1 Mar 2007 14:45:49 +0000 (GMT) From: Robert Watson <rwatson@FreeBSD.org> To: Andre Oppermann <andre@freebsd.org> Cc: freebsd-net@freebsd.org, gallatin@freebsd.org, freebsd-current@freebsd.org, kmacy@freebsd.org Subject: Re: Large TCP send socket buffer optimizations Message-ID: <20070301144321.O98688@fledge.watson.org> In-Reply-To: <45E6E2E8.5060408@freebsd.org> References: <45E6E2E8.5060408@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 1 Mar 2007, Andre Oppermann wrote: > With the TCP socket buffer autosizing and generally larger socket buffers > for high bandwidth and high delay connections tcp_output() has become > increasingly inefficient for sending segments. For every segment sent it is > traversing the entire socket buffer mbuf chain until it finds the offset to > continue from. Usually this is close to the end of the chain. Once it got > past a few dozen mbufs it starts to bust the CPU caches and performance > starts to fall off. > > This patch solves the problem by maintaining an offset pointer in the socket > buffer to give tcp_output() the closest mbuf right away avoiding the > traversal from the beginning. > > With this patch we should be able to compete nicely for the Internet land > speed record again. > > The patch is here: > > http://people.freebsd.org/~andre/sockbuf_sndptr-20070301.diff > > Any testing, especially on 10Gig cards, and feedback appreciated. I'm going to be on- and off-line from now through about the last week of March on travel to LinuxForum.dk, AsiaBSDCon, and UKUUG. However, once I get back online from that in late March through the end of April, I will be spending quite a bit of time looking at high performance packet capture issues, so can certainly spend some time looking at this. Per our earlier discussions and my previous exploration of this issue, I see precisely this sort of problem with TCP as socket buffer size grows, so should be set up to fairly easily test for an improvement once I have my feet on the ground. If I'm lucky I may get a breather for a few days from about 15 March - 18 March and could take an early look at it then. Thanks, Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070301144321.O98688>