From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 12:42:33 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49A78477; Thu, 27 Mar 2014 12:42:33 +0000 (UTC) Received: from mail-qg0-x22b.google.com (mail-qg0-x22b.google.com [IPv6:2607:f8b0:400d:c04::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8A9CBEC; Thu, 27 Mar 2014 12:42:32 +0000 (UTC) Received: by mail-qg0-f43.google.com with SMTP id f51so2699579qge.30 for ; Thu, 27 Mar 2014 05:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=gfDL+vv4UPGWHYXuoDe+0WPwvQKOv8dbqRm4AT0K/mo=; b=tpKPaHW+XoyUIemJukrXVnlZSVqJAi3elCEz9L8X+klwfaQCpu/qQLG4a/TvSsGjhK D+yhqJWM+8T5PJESmBgHrlibRBI8/WU4v17qibm1f3vyo64vYA6sRQDxCqVNoYPO/LgN J1hzPbFOvt/deh69wNeXZfWpnxenZV1TSVWVIFuXGaJuPjHcbYCrKEfTEYwB9RiT/Ak8 NcTOCC6HfhKQUowM+jEw4HW/2GFuFnZSDM8bBcQmxTxvBx2QykuVkXrjHynKmecFDy4/ zhWld4SV1KFpVyol4H1Fcw7Rdv5qXBXTsaxrpyLgcrPudQOsVq3mL0zJjjFplVEE6/T6 g5Tw== MIME-Version: 1.0 X-Received: by 10.140.80.209 with SMTP id c75mr1538712qgd.79.1395924151988; Thu, 27 Mar 2014 05:42:31 -0700 (PDT) Received: by 10.96.79.97 with HTTP; Thu, 27 Mar 2014 05:42:31 -0700 (PDT) In-Reply-To: References: <20140326023334.GB2973@michelle.cdnetworks.com> <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 09:42:31 -0300 Message-ID: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem From: Christopher Forgeron To: araujo@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , Rick Macklem , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 12:42:33 -0000 I'm quite sure the problem is on 9.2-RELEASE, not 9.1-RELEASE or earlier, as a 9.2-STABLE from last year I have doesn't exhibit the problem. New code in if.c at line 660 looks to be what is starting this, which makes me wonder how TSO was being handled before 9.2. I also like Rick's NFS patch for cluster size. I notice an improvement, but don't have solid numbers yet. I'm still stress testing it as we speak. On Wed, Mar 26, 2014 at 11:44 PM, Marcelo Araujo wrote: > Hello All, > > > 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > > > Well, bumping it from 32->35 is all it would take for NFS (can't comment > > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 > > (just so others aren't confused by the above comment). I understand > > your point was w.r.t. using 100 without blowing the kernel stack, but > > since the testers have been using "ix" with the 82599 chip which is > > limited to 32 transmit segments... > > > > However, please increase any you know can be safely done from 32->35, > rick > > > > > I have plenty of machines using Intel X540 that is based on 82599 chipset. > I have applied Rick's patch on ixgbe to check if the packet size is bigger > than 65535 or cluster is bigger than 32. So far till now, on FreeBSD > 9.1-RELEASE this problem does not happens. > > Unfortunately all my environment here is based on 9.1-RELEASE, with some > merges from 10-RELEASE such like: NFS and IXGBE. > > Also I have applied the patch that Rick sent in another email with the > subject 'NFS patch to use pagesize mbuf clusters'. And we can see some > performance boost over 10Gbps Intel. However here at the company, we are > still doing benchmarks. If someone wants to have my benchmark result, I can > send it later. > > I'm wondering, if this update on ixgbe from 32->35 could be applied also > for versions < 9.2. I'm thinking, that this problem arise only on 9-STABLE > and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE doesn't > share it. > > Best Regards, > -- > Marcelo Araujo > araujo@FreeBSD.org > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >