From owner-svn-src-all@freebsd.org Mon Sep 14 10:51:05 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A9D1A030CB; Mon, 14 Sep 2015 10:51:05 +0000 (UTC) (envelope-from royger@gmail.com) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 402121875; Mon, 14 Sep 2015 10:51:05 +0000 (UTC) (envelope-from royger@gmail.com) Received: by wiclk2 with SMTP id lk2so135075854wic.0; Mon, 14 Sep 2015 03:51:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-type:content-transfer-encoding; bh=07+HMYWW8b2ZLxd0S9JCO3JAa4RjFwLqC5mMnnofCFQ=; b=nQDKhPewkjXks21j51MACkz/X1t3KEpQnfwoy6av9v6Ngf0rw8WzGmYSe3RAeyEsgg j6aee64FWl318mxzGrhfHi/umUT6f2yIOb8P1oxZvCQLziRVHpxVA3ySJSxUDCNroeuM 7BRAQBBaTwDtSTa+1APZvLLg3qIVzJPtY/3W1uWI3VT1z+TLP43mAVKrFV9cTP1sC979 NvsEMvitVVRCWVwIl98U3rEELUalOUyWiIFraSTkurvsFf0n3w+nrK/gCWEf0TUKpaUj EzaBW7WAVeP9fr2LoX6CykRtJLFQYhqNUT0iv2wQ/zs1mY1JQVcHYQKaibQLvTatksn5 9EKw== X-Received: by 10.194.7.197 with SMTP id l5mr29112949wja.153.1442227863721; Mon, 14 Sep 2015 03:51:03 -0700 (PDT) Received: from [172.16.1.30] (195.Red-83-39-7.dynamicIP.rima-tde.net. [83.39.7.195]) by smtp.gmail.com with ESMTPSA id ub7sm13671467wib.17.2015.09.14.03.51.02 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 Sep 2015 03:51:03 -0700 (PDT) Sender: =?UTF-8?Q?Roger_Pau_Monn=C3=A9?= Subject: Re: svn commit: r271946 - in head/sys: dev/oce dev/vmware/vmxnet3 dev/xen/netfront kern net netinet ofed/drivers/net/mlx4 sys To: Hans Petter Selasky , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201409220827.s8M8RRHB031526@svn.freebsd.org> <55F69093.5050807@FreeBSD.org> <55F6935C.9000000@selasky.org> From: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= X-Enigmail-Draft-Status: N1110 Message-ID: <55F6A694.7020404@FreeBSD.org> Date: Mon, 14 Sep 2015 12:51:00 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <55F6935C.9000000@selasky.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Sep 2015 10:51:05 -0000 El 14/09/15 a les 11.29, Hans Petter Selasky ha escrit: > On 09/14/15 11:17, Roger Pau Monné wrote: >> El 22/09/14 a les 10.27, Hans Petter Selasky ha escrit: >>> Author: hselasky >>> Date: Mon Sep 22 08:27:27 2014 >>> New Revision: 271946 >>> URL: http://svnweb.freebsd.org/changeset/base/271946 >>> >>> Log: >>> Improve transmit sending offload, TSO, algorithm in general. >>> >>> The current TSO limitation feature only takes the total number of >>> bytes in an mbuf chain into account and does not limit by the number >>> of mbufs in a chain. Some kinds of hardware is limited by two >>> factors. One is the fragment length and the second is the fragment >>> count. Both of these limits need to be taken into account when doing >>> TSO. Else some kinds of hardware might have to drop completely valid >>> mbuf chains because they cannot loaded into the given hardware's DMA >>> engine. The new way of doing TSO limitation has been made backwards >>> compatible as input from other FreeBSD developers and will use >>> defaults for values not set. >>> >>> Reviewed by: adrian, rmacklem >>> Sponsored by: Mellanox Technologies >> >> This commit makes xen-netfront tx performance drop from ~5Gbits/sec >> (with debug options enabled) to 446 Mbits/sec. I'm currently looking, >> but if anyone has ideas they are welcome. >> > > Hi Roger, > > Looking at the netfront code you should subtract 1 from tsomaxsegcount > prior to r287775. The reason might simply be that 2K clusters are used > instead of 4K clusters, causing m_defrag() to be called. > >> ifp->if_hw_tsomax = 65536 - (ETHER_HDR_LEN + >> ETHER_VLAN_ENCAP_LEN); >> ifp->if_hw_tsomaxsegcount = MAX_TX_REQ_FRAGS; >> ifp->if_hw_tsomaxsegsize = PAGE_SIZE; > > After r287775 can you try these settings: > > ifp->if_hw_tsomax = 65536; > ifp->if_hw_tsomaxsegcount = MAX_TX_REQ_FRAGS; > ifp->if_hw_tsomaxsegsize = PAGE_SIZE; > > And see if the performance is the same like before? Yes, performance seems to be fine after setting if_hw_tsomax to 65536. Is there some documentation about the usage of if_hw_tsomax? Does the network subsystem already takes care of subtracting the space for ether header and the vlan encapsulation, so it's no longer needed to specify them in if_hw_tsomax? Also, this commit was MFC'ed to stable/10 and 10.2 suffers from the same problem. Can we issue and EN to get this fixed in 10.2? Roger.