From owner-svn-src-head@freebsd.org Tue Sep 15 08:32:23 2015 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07610A03437; Tue, 15 Sep 2015 08:32:23 +0000 (UTC) (envelope-from royger@gmail.com) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 91D8C1B0D; Tue, 15 Sep 2015 08:32:22 +0000 (UTC) (envelope-from royger@gmail.com) Received: by wiclk2 with SMTP id lk2so17587347wic.0; Tue, 15 Sep 2015 01:32:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=HRmqDr61snTkstS9DwRJr/XOzUP7mwrL26rHYkIaJw8=; b=SkOtSKIQc+u4fxIsH+0+QIu3zYqw84Riz3SeG90emMV316fcXid4FwhJvEtsWsiJ7r O+LMeOMRjxHMtZHen1VaTY2an/JjrT6b16JmJ8AQKe1ic93n0vz+JYk7R+MkrvheAxri +eZLp10Z/T9fid6iWJ4DpkMn/9ipSyx3h961JF9A7h8qfaG4hE9OhOZUfe4J9KJlSdnk TntB3ocDFbmHZ+m+tgQKAhtoxOp8GPhtuwK57nsytPg0ptw7rF9Hxcx6uSKa6/pt1mR1 9sUEVSQ2XewCMl0nnB6hn/9U3q/Qv7x4ZbDPTanWsxpq0eIhrSdfEkc2te0k0on5Er8F NBTQ== X-Received: by 10.180.103.72 with SMTP id fu8mr4901235wib.7.1442305940100; Tue, 15 Sep 2015 01:32:20 -0700 (PDT) Received: from [172.16.1.30] (195.Red-83-39-7.dynamicIP.rima-tde.net. [83.39.7.195]) by smtp.gmail.com with ESMTPSA id lu5sm19794536wjb.9.2015.09.15.01.32.19 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Sep 2015 01:32:19 -0700 (PDT) Sender: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: Re: svn commit: r271946 - in head/sys: dev/oce dev/vmware/vmxnet3 dev/xen/netfront kern net netinet ofed/drivers/net/mlx4 sys To: Hans Petter Selasky , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201409220827.s8M8RRHB031526@svn.freebsd.org> <55F69093.5050807@FreeBSD.org> <55F6935C.9000000@selasky.org> From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= Message-ID: <55F7D783.1080406@FreeBSD.org> Date: Tue, 15 Sep 2015 10:32:03 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <55F6935C.9000000@selasky.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Sep 2015 08:32:23 -0000 El 14/09/15 a les 11.29, Hans Petter Selasky ha escrit: > On 09/14/15 11:17, Roger Pau Monné wrote: >> El 22/09/14 a les 10.27, Hans Petter Selasky ha escrit: >>> Author: hselasky >>> Date: Mon Sep 22 08:27:27 2014 >>> New Revision: 271946 >>> URL: http://svnweb.freebsd.org/changeset/base/271946 >>> >>> Log: >>> Improve transmit sending offload, TSO, algorithm in general. >>> >>> The current TSO limitation feature only takes the total number of >>> bytes in an mbuf chain into account and does not limit by the number >>> of mbufs in a chain. Some kinds of hardware is limited by two >>> factors. One is the fragment length and the second is the fragment >>> count. Both of these limits need to be taken into account when doing >>> TSO. Else some kinds of hardware might have to drop completely valid >>> mbuf chains because they cannot loaded into the given hardware's DMA >>> engine. The new way of doing TSO limitation has been made backwards >>> compatible as input from other FreeBSD developers and will use >>> defaults for values not set. >>> >>> Reviewed by: adrian, rmacklem >>> Sponsored by: Mellanox Technologies >> >> This commit makes xen-netfront tx performance drop from ~5Gbits/sec >> (with debug options enabled) to 446 Mbits/sec. I'm currently looking, >> but if anyone has ideas they are welcome. >> > > Hi Roger, > > Looking at the netfront code you should subtract 1 from tsomaxsegcount > prior to r287775. The reason might simply be that 2K clusters are used > instead of 4K clusters, causing m_defrag() to be called. > >> ifp->if_hw_tsomax = 65536 - (ETHER_HDR_LEN + >> ETHER_VLAN_ENCAP_LEN); >> ifp->if_hw_tsomaxsegcount = MAX_TX_REQ_FRAGS; >> ifp->if_hw_tsomaxsegsize = PAGE_SIZE; > > After r287775 can you try these settings: > > ifp->if_hw_tsomax = 65536; > ifp->if_hw_tsomaxsegcount = MAX_TX_REQ_FRAGS; > ifp->if_hw_tsomaxsegsize = PAGE_SIZE; > > And see if the performance is the same like before? FWIW, just using r287775 seems to solve the problem, even if I leave if_hw_tsomax with it's current value. Roger.