Date: Tue, 10 Sep 2013 15:42:16 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Mike Tancsa <mike@sentex.net>, Rick Macklem <rmacklem@uoguelph.ca> Cc: David Wolfskill <david@catwhisker.org>, FreeBSD Net <net@freebsd.org> Subject: Re: TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) Message-ID: <1378852936.95477.YahooMailNeo@web121601.mail.ne1.yahoo.com> In-Reply-To: <52275136.1010105@sentex.net> References: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> <52275136.1010105@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
NFS has been broken since Day 1, so lets not come to conclusions about anything as it relates to NFS. BC ________________________________ From: Mike Tancsa <mike@sentex.net> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: FreeBSD Net <net@freebsd.org>; David Wolfskill <david@catwhisker.org> Sent: Wednesday, September 4, 2013 11:26 AM Subject: TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) On 9/4/2013 8:50 AM, Rick Macklem wrote: > David Wolfskill wrote: >> >> >> I noticed that when I tried to write files to NFS, I could write >> small >> files OK, but larger ones seemed to ... hang. >> * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. >> * "sysctl net.inet.tcp.tso" showed "1" -- enabled. >> >> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked >> without >> a hitch or a whine. And I was able to copy all 117709618 bytes, not >> just >> 2097152 (2^21). >> >> Is the above expected? It came rather as a surprise to me. >> > Not surprising to me, I'm afraid. When there are serious NFS problems > like this, it is often caused by a network fabric issue and broken > TSO is at the top of the list w.r.t. cause. I was just experimenting a bit with iSCSI via FreeNAS and was a little disappointed at the speeds I was getting. So, I tried disabling tso on both boxes and it did seem to speed things up a bit. Data and testing methods attached in a txt file. I did 3 cases. Just boot up FreeNAS and the initiator without tweaks. That had the worst performance. disable tso on the nic as well as via sysctl on both boxes. That had the best performance. re-enable tso on both boxes. That had better performance than the first case, but still not as good as totally disabling it. I am guessing something is not quite being re-enabled properly ? But its different than the other two cases ?!? tgt is FreeNAS-9.1.1-RELEASE-x64 (a752d35) and initiator is r254328 9.2 AMD64 The FreeNAS box has 16G of RAM, so the file is being served out of cache as gstat shows no activity when sending out the file ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@FreeBSD.ORG Tue Sep 10 22:54:34 2013 Return-Path: <owner-freebsd-net@FreeBSD.ORG> Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 6C09DABB for <net@freebsd.org>; Tue, 10 Sep 2013 22:54:34 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 2A593248F for <net@freebsd.org>; Tue, 10 Sep 2013 22:54:34 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.7/8.14.7) with ESMTP id r8AMqvPS032514; Tue, 10 Sep 2013 18:52:57 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <522FA2C3.7010109@sentex.net> Date: Tue, 10 Sep 2013 18:52:51 -0400 From: Mike Tancsa <mike@sentex.net> Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Barney Cordoba <barney_cordoba@yahoo.com> Subject: Re: TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) References: <979862494.17918795.1378299005617.JavaMail.root@uoguelph.ca> <52275136.1010105@sentex.net> <1378852936.95477.YahooMailNeo@web121601.mail.ne1.yahoo.com> In-Reply-To: <1378852936.95477.YahooMailNeo@web121601.mail.ne1.yahoo.com> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.74 on 64.7.153.18 Cc: David Wolfskill <david@catwhisker.org>, Rick Macklem <rmacklem@uoguelph.ca>, FreeBSD Net <net@freebsd.org> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>; List-Post: <mailto:freebsd-net@freebsd.org> List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>, <mailto:freebsd-net-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 10 Sep 2013 22:54:34 -0000 On 9/10/2013 6:42 PM, Barney Cordoba wrote: > NFS has been broken since Day 1, so lets not come to conclusions about > anything > as it relates to NFS. iSCSI is NFS ? ---Mike > > BC > > ------------------------------------------------------------------------ > *From:* Mike Tancsa <mike@sentex.net> > *To:* Rick Macklem <rmacklem@uoguelph.ca> > *Cc:* FreeBSD Net <net@freebsd.org>; David Wolfskill <david@catwhisker.org> > *Sent:* Wednesday, September 4, 2013 11:26 AM > *Subject:* TSO help or hindrance ? (was Re: TSO and FreeBSD vs Linux) > > On 9/4/2013 8:50 AM, Rick Macklem wrote: >> David Wolfskill wrote: >>> >>> >>> I noticed that when I tried to write files to NFS, I could write >>> small >>> files OK, but larger ones seemed to ... hang. >>> * "ifconfig -v em0" showed flags TSO4 & VLAN_HWTSO turned on. >>> * "sysctl net.inet.tcp.tso" showed "1" -- enabled. >>> >>> As soon as I issued "sudo net.inet.tcp.tso=0" ... the copy worked >>> without >>> a hitch or a whine. And I was able to copy all 117709618 bytes, not >>> just >>> 2097152 (2^21). >>> >>> Is the above expected? It came rather as a surprise to me. >>> >> Not surprising to me, I'm afraid. When there are serious NFS problems >> like this, it is often caused by a network fabric issue and broken >> TSO is at the top of the list w.r.t. cause. > > > I was just experimenting a bit with iSCSI via FreeNAS and was a little > disappointed at the speeds I was getting. So, I tried disabling tso on > both boxes and it did seem to speed things up a bit. Data and testing > methods attached in a txt file. > > I did 3 cases. > > Just boot up FreeNAS and the initiator without tweaks. That had the > worst performance. > disable tso on the nic as well as via sysctl on both boxes. That had the > best performance. > re-enable tso on both boxes. That had better performance than the first > case, but still not as good as totally disabling it. I am guessing > something is not quite being re-enabled properly ? But its different > than the other two cases ?!? > > tgt is FreeNAS-9.1.1-RELEASE-x64 (a752d35) and initiator is r254328 9.2 > AMD64 > > The FreeNAS box has 16G of RAM, so the file is being served out of cache > as gstat shows no activity when sending out the file > > > > ---Mike > > > -- > ------------------- > Mike Tancsa, tel +1 519 651 3400 > Sentex Communications, mike@sentex.net <mailto:mike@sentex.net> > Providing Internet services since 1994 www.sentex.net > Cambridge, Ontario Canada http://www.tancsa.com/ > > _______________________________________________ > freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org > <mailto:freebsd-net-unsubscribe@freebsd.org>" > -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1378852936.95477.YahooMailNeo>