From owner-freebsd-net@FreeBSD.ORG Thu Mar 27 02:45:01 2014 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 54089194; Thu, 27 Mar 2014 02:45:01 +0000 (UTC) Received: from mail-wg0-x234.google.com (mail-wg0-x234.google.com [IPv6:2a00:1450:400c:c00::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9909CF16; Thu, 27 Mar 2014 02:45:00 +0000 (UTC) Received: by mail-wg0-f52.google.com with SMTP id k14so1957774wgh.23 for ; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:reply-to:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=WxTzjm+ofTFvUdW3JPNglL8ykzwepfyCuwfZNI/YPEg=; b=EKPr2w+hTR6vs88hZhC6PFPf6I6eM/01VOGK3B14YBHAo2lb+XI+7xXczRU6pf+S7N dTD2nFOv5Y3jelyLhogcN+/u2I1KBDrp/SsrYJYkkhSDXzc/v16wNeAJfvfERYDZYG2u mtnzLsudIFbzRFFeS4p/OX2chVU1JICd7j80pqPuo6zhvZ2JRTBs57OtklKMMCm+5hF9 7O7VzmDZiZg44g3QIrlrwR8/f+yIHPYVai3RfIQpEdtrBVvLy8cT40HTb2WMggzgYuwj LpsdjqS7sm6vKkeaCyf/SDmXMT6aZ7xES+CT6ejjyEg6Z1/fPYD6EDEau8P3pGUetls+ jfqA== MIME-Version: 1.0 X-Received: by 10.180.89.102 with SMTP id bn6mr35999708wib.28.1395888298103; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) Received: by 10.216.190.199 with HTTP; Wed, 26 Mar 2014 19:44:58 -0700 (PDT) In-Reply-To: <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> References: <20140326023334.GB2973@michelle.cdnetworks.com> <1903781266.1237680.1395880068597.JavaMail.root@uoguelph.ca> Date: Thu, 27 Mar 2014 10:44:58 +0800 Message-ID: Subject: Re: RFC: How to fix the NFS/iSCSI vs TSO problem From: Marcelo Araujo To: Rick Macklem Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: pyunyh@gmail.com, FreeBSD Filesystems , Alexander Motin , FreeBSD Net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list Reply-To: araujo@FreeBSD.org List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 02:45:01 -0000 Hello All, 2014-03-27 8:27 GMT+08:00 Rick Macklem : > > Well, bumping it from 32->35 is all it would take for NFS (can't comment > w.r.t. iSCSI). ixgbe uses 100 for the 82598 chip and 32 for the 82599 > (just so others aren't confused by the above comment). I understand > your point was w.r.t. using 100 without blowing the kernel stack, but > since the testers have been using "ix" with the 82599 chip which is > limited to 32 transmit segments... > > However, please increase any you know can be safely done from 32->35, rick > > I have plenty of machines using Intel X540 that is based on 82599 chipset. I have applied Rick's patch on ixgbe to check if the packet size is bigger than 65535 or cluster is bigger than 32. So far till now, on FreeBSD 9.1-RELEASE this problem does not happens. Unfortunately all my environment here is based on 9.1-RELEASE, with some merges from 10-RELEASE such like: NFS and IXGBE. Also I have applied the patch that Rick sent in another email with the subject 'NFS patch to use pagesize mbuf clusters'. And we can see some performance boost over 10Gbps Intel. However here at the company, we are still doing benchmarks. If someone wants to have my benchmark result, I can send it later. I'm wondering, if this update on ixgbe from 32->35 could be applied also for versions < 9.2. I'm thinking, that this problem arise only on 9-STABLE and consequently on 9.2-RELEASE. And fortunately or not 9.1-RELEASE doesn't share it. Best Regards, -- Marcelo Araujo araujo@FreeBSD.org