From owner-freebsd-net@FreeBSD.ORG Tue Sep 10 23:04:10 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id AFBB3C57 for ; Tue, 10 Sep 2013 23:04:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 741692524 for ; Tue, 10 Sep 2013 23:04:10 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aj4MAAWkL1KDaFve/2dsb2JhbABYA4FxBoFIUYMqvzaBO3SCJQEBAQQBAQEgJgUYCAsbBwoEAQEBAgINFgMCKQEJFQkIBggHBAEcBIdhDLEUkgeBKYx5gQUkEAcRgliBNAOVLYN4kDyDPCAygQM5 X-IronPort-AV: E=Sophos;i="4.90,881,1371096000"; d="scan'208";a="51854391" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 10 Sep 2013 19:04:03 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 4096EB3F14; Tue, 10 Sep 2013 19:04:03 -0400 (EDT) Date: Tue, 10 Sep 2013 19:04:03 -0400 (EDT) From: Rick Macklem To: Mike Tancsa Message-ID: <332980519.21486297.1378854243253.JavaMail.root@uoguelph.ca> In-Reply-To: <522FA2C3.7010109@sentex.net> Subject: Re: TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: Barney Cordoba , David Wolfskill , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Sep 2013 23:04:10 -0000 Mike Tancsa wrote: > On 9/10/2013 6:42 PM, Barney Cordoba wrote: > > NFS has been broken since Day 1, so lets not come to conclusions > > about > > anything > > as it relates to NFS. > > iSCSI is NFS ? > It would be really nice if you could try trasz`s new iSCSI stack and see how well it works. (I, for one, am hoping it makes it into 10.0, but it may be too late.) rick > ---Mike > > > > > BC > > > > ------------------------------------------------------------------------ > > *From:* Mike Tancsa > > *To:* Rick Macklem > > *Cc:* FreeBSD Net ; David Wolfskill > > > > *Sent:* Wednesday, September 4, 2013 11:26 AM > > *Subject:* TSO help or hindrance ? (was Re: TSO and FreeBSD vs > > Linux) > > > > On 9/4/2013 8:50 AM, Rick Macklem wrote: > >> David Wolfskill wrote: > >>> > >>> > >>> I noticed that when I tried to write files to NFS, I could write > >>> small > >>> files OK, but larger ones seemed to ... hang. > >>> * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. > >>> * "sysctl net.inet.tcp.tso" showed "1" -- enabled. > >>> > >>> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked > >>> without > >>> a hitch or a whine. And I was able to copy all 117709618 bytes, > >>> not > >>> just > >>> 2097152 (2^21). > >>> > >>> Is the above expected? It came rather as a surprise to me. > >>> > >> Not surprising to me, I'm afraid. When there are serious NFS > >> problems > >> like this, it is often caused by a network fabric issue and broken > >> TSO is at the top of the list w.r.t. cause. > > > > > > I was just experimenting a bit with iSCSI via FreeNAS and was a > > little > > disappointed at the speeds I was getting. So, I tried disabling tso > > on > > both boxes and it did seem to speed things up a bit. Data and > > testing > > methods attached in a txt file. > > > > I did 3 cases. > > > > Just boot up FreeNAS and the initiator without tweaks. That had > > the > > worst performance. > > disable tso on the nic as well as via sysctl on both boxes. That > > had the > > best performance. > > re-enable tso on both boxes. That had better performance than the > > first > > case, but still not as good as totally disabling it. I am guessing > > something is not quite being re-enabled properly ? But its > > different > > than the other two cases ?!? > > > > tgt is FreeNAS-9.1.1-RELEASE-x64 (a752d35) and initiator is r254328 > > 9.2 > > AMD64 > > > > The FreeNAS box has 16G of RAM, so the file is being served out of > > cache > > as gstat shows no activity when sending out the file > > > > > > > > ---Mike > > > > > > -- > > ------------------- > > Mike Tancsa, tel +1 519 651 3400 > > Sentex Communications, mike@sentex.net > > Providing Internet services since 1994 www.sentex.net > > Cambridge, Ontario Canada http://www.tancsa.com/ > > > > _______________________________________________ > > freebsd-net@freebsd.org mailing > > list > > http://lists.freebsd.org/mailman/listinfo/freebsd-net > > To unsubscribe, send any mail to > > "freebsd-net-unsubscribe@freebsd.org > > " > > > > > -- > ------------------- > Mike Tancsa, tel +1 519 651 3400 > Sentex Communications, mike@sentex.net > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada http://www.tancsa.com/ >