From owner-freebsd-net@FreeBSD.ORG Tue Mar 19 07:06:32 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3F40CFEC for ; Tue, 19 Mar 2013 07:06:32 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8A16A803 for ; Tue, 19 Mar 2013 07:06:31 +0000 (UTC) Received: (qmail 6982 invoked from network); 19 Mar 2013 08:17:50 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 19 Mar 2013 08:17:50 -0000 Message-ID: <51480E6D.9050708@freebsd.org> Date: Tue, 19 Mar 2013 08:06:21 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: Garrett Wollman Subject: Re: Limits on jumbo mbuf cluster allocation References: <20798.44871.601547.24628@hergotha.csail.mit.edu> <75232221.3844453.1363146480616.JavaMail.root@erie.cs.uoguelph.ca> <20807.59845.764047.618551@hergotha.csail.mit.edu> In-Reply-To: <20807.59845.764047.618551@hergotha.csail.mit.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-net@freebsd.org, Rick Macklem , Ivan Voras X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Mar 2013 07:06:32 -0000 On 19.03.2013 05:29, Garrett Wollman wrote: > < said: > >> I've attached a patch that has assorted changes. > > So I've done some preliminary testing on a slightly modified form of > this patch, and it appears to have no major issues. However, I'm > still waiting for my user with 500 VMs to have enough free to be able > to run some real stress tests for me. > > I was able to get about 2.5 Gbit/s throughput for a single streaming > client over local 10G interfaces with jumbo frames (through a single > switch and with LACP on both sides -- how well does lagg(4) interact > with TSO and checksum offload?) This is a little bit disappointing > (considering that the filesystem can do 14 Gbit/s locally) but still > pretty decent for one single-threaded client. This obviously does not > implicate the DRC changes at all, but does suggest that there is room > for more performance improvement. (In previous tests last year, I > was able to get a sustained 8 Gbit/s when using multiple clients.) I > also found that one of our 10G switches is reordering TCP segments in > a way that causes poor performance. Of course testing a single vs. multiple clients has a different dynamic on the control loop between TCP, RPC, NFS, VFS and disk. The aggregate multiple is likely to be much higher. There may be optimization potential in fine-tuning these control loops. Also NFS (old and new) do not do direct dispatch of the VM pages as in sendfile but copy the data into mbufs. The performance improvement on highly loaded systems may be non-trivial, however it is not so easy to implement. -- Andre > I'll hopefully have some proper testing results later in the week. > > -GAWollman > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > >