From owner-freebsd-net@freebsd.org Wed Jul 11 16:53:30 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 199EB1028D36 for ; Wed, 11 Jul 2018 16:53:30 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AC677BD39 for ; Wed, 11 Jul 2018 16:53:29 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg1-x544.google.com with SMTP id y5-v6so3070655pgv.1 for ; Wed, 11 Jul 2018 09:53:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:cc:references:from:openpgp:autocrypt:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=7/0AkUyiCyYgZ6woF/zdkBFMtfgXCnXuce2CVYwlGck=; b=OODjvX/Jx3B9ePfBDNl8E195ED8SPikWxioCVa68NZg7wCQ2OtgJodKR3dznx3hDqO Dv0SNFErg/eRBfTQm8Ls1Pq3CCyFVr6Khp7KvieHoDdizNnLokh+HJkXHAxZMq4AFX7+ bMEKtkVk04Muechb/putAHg1JM0ouaRXrFLscMGzDWBbYNGa7DwgE7iFez3Df8Er1kqK r0b1uDZFbYaecF0MD/QyzwARoT3osiX62T1UR8Auv/yFSOHCvHWpP90YLt5TljG6gkVe rl2eb6T5+O6SEGe0ynFA2+u4r0uS6AC/i0W6I2wF4S0RrJePaVE3HnvY9nfvol0rB6Gd vXpw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:cc:references:from:openpgp :autocrypt:message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=7/0AkUyiCyYgZ6woF/zdkBFMtfgXCnXuce2CVYwlGck=; b=ozJ8rSvjcwLKTHMbaLwIN5linFkp5iqRc7ptZtPwE3VgnwTmD2zEdZuLVWVYle6NY3 Im6nFf9S7kFxjOvFaMYl8LcyXvunMZSmXIBfqOpUrc0/zVJbKJtsoL8sb0hEcm/aDq+u ZXWajeXjlVG/4AiBWyla+bywjcSVGCGaukTkO3YwxW2qvoAazVg6l5rfUbHbvqwxXTX4 D7P/Nq9BHtBMEYVACda7p2X/7HO9mYKQ811eD6s8sh8J/SSc0vKrcc08DHjkGCfohL+P pT07iqmG60xu1NjG22RcWIYAVX+SMAfZoaTueswOWk+LuiFcFcLOmkVaagLnzbBsklY7 vR7w== X-Gm-Message-State: APt69E0WHogJ+viOycWKsozqppbZ+y+dVXdWW4uNLH1Tsy25La2+hlGQ G/ArK5u6a55V4K3i+CpKXR7KcA== X-Google-Smtp-Source: AAOMgpf8i8jYZm0U8j0vhMoDf+qrbRAGhQ3zl1hNaktf1qR8ICQSdgMw0O7V0jzmm6s/Ok9BXxMt8g== X-Received: by 2002:a63:4857:: with SMTP id x23-v6mr21845738pgk.30.1531328008493; Wed, 11 Jul 2018 09:53:28 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [12.32.117.8]) by smtp.googlemail.com with ESMTPSA id g15-v6sm26169072pfg.98.2018.07.11.09.53.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 11 Jul 2018 09:53:27 -0700 (PDT) Sender: Navdeep Parhar Subject: Re: netmap & chelsio To: "Eggert, Lars" Cc: "freebsd-net@freebsd.org" , Giuseppe Lettieri References: <94019926-B14F-47AB-8AD9-C01B4A0E199F@netapp.com> <20180705154710.GA3353@ox> <0199F257-F9D2-41D0-B4F9-26FBA1607F61@netapp.com> <20180706071739.GA13401@ox> <30D2A259-6B2E-44CA-969D-F51F4D929DDE@netapp.com> From: Navdeep Parhar Openpgp: preference=signencrypt Autocrypt: addr=np@FreeBSD.org; prefer-encrypt=mutual; keydata= xsDiBEosaGcRBACOXnXquGEW53BjpMt2jViod/TUf1xgjMekcbDxqOODPeX7eYfrwJ8G6BCN OpGjBmWDu/JcNj4Z+gmTilJ6WLZQ7ecFZfEeO91pt6ys0cyWh0xfO+/mT83D7W81S/kqrJBk QbBIdV6LumevdErHo272r8RcMELC4Ru87eRtX3hmEwCgnnGNJMpQFUfYTt5XE7nY0yQoeV8D /0OcWmJbEZWxX9O7AuliCe3zd2Dw0B4LB9SZ2Dis7+gpVd3xVgYnt5wRE9kM+ThgrMA/wqr8 07qmEG6bcfUsfwwGN9YUtNF3xAN07cXTs026sCIFNZK816PrThBzCgkwR7pDpkMzGWIBr8Wi XXy0eB+JlQ6UV4PEiXuZ5ulzP0b1A/9CZm3wJfrNC0r1gMyrfVedg4zwKU997bmPLGcYs+rW XDTI9CvMseOUYn4CoDZQCp/9zxuHK+VU7Y/w0c/hVE5ERACSn4SjN2unEDstK9njZBMHEPVk Ae/YvSG5cmc97SHlVE+eu/bbLKcvFb6rRLPOaVFQJMJA2VJEGWtYhvP7Zc0fTmF2ZGVlcCBQ YXJoYXIgPG5wQEZyZWVCU0Qub3JnPsJgBBMRAgAgBQJKLGhnAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQyrIrk6yriBL0MQCfUJOiS2PbJFDeiav1ylcXXwfpggAAoJRoS7GDENGy M4BzjJ4b0ptZqTLRzsFNBEosaGcQCACFCWs47SL4DQA6bNDlVJu4w8wLf8uVOyatuGmdXX8Y /OTVQJgA3vS+ODNVJCxhKVlvhcn7bhBdGdWKS9K+lr8+eEvr4hf2bQpesoHC+uFgKyILkCBN L8raixbhysyq0pfZWWDJMyn+G42BG1yJJi+bykygdpYnbIVA8dYHmBibI8mkPKOHSohjXT1S RfGGn+l1w54OO4NlJhCXMkjTA/Z9Bt4XeaiR85uJi0UUfV8FGZHhgSvT+/P1xIvz+nytuehS P/QLXl13CtAG/nKVkAcZnsT/3NrJ4Z2r45k+c50Wrf210scAaBogrrV5eIHfNGgOANApN8+8 vj+aXO4pXRuXAAMFB/44ea8rd+P5N3OMrfuM8i91Qe1bJ+BIoroKPOr8jvCry0h3QpdfLKUN IgaqbS3JZeBJ8HHnWSGCF+o6H5gzRe1hvylPEclLPDCuPe7T746h9Mzejf2hNDJvOg+BuweD ZW4KhovVbdS+syJEvpGF4bO8qgHT2CKgruXSHbFetdQWbkM0rfMmTuo0GcR2BEVrPb/SPFv6 4ZZyAZzmnGO4vT1bzClnTzJixrDpH74M3vSEYegMB4KdbLYBi8Jx4QUKgVEhJHjJubKWX4et yU/uuehOC3xYrmr1UXvsom3U8r36Dvdo77Yr3dgDVXa7bolNx0TIhdWxZI+R4z9E75QY+/wg wkkEGBECAAkFAkosaGcCGwwACgkQyrIrk6yriBI+JQCfUxgyqGtzZvLh5Al7gsTmRc11PLwA niD3NfWGRcO2+9uxSSQqRH1ywC4n Message-ID: Date: Wed, 11 Jul 2018 09:53:26 -0700 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <30D2A259-6B2E-44CA-969D-F51F4D929DDE@netapp.com> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jul 2018 16:53:30 -0000 On 07/11/18 07:58, Eggert, Lars wrote: > Hi, > > I have netmap working with the T6 cards now. > > However, performance is very poor. It seems to take several milliseconds after a NIOCTXSYNC ioctl before the tail is updated? Try changing lazy_tx_credit_flush to 0 on the running kernel with a debugger, or compile the driver with it set to 0 -- it's in t4_netmap.c: int lazy_tx_credit_flush = 1; I'm surprised I don't have a tunable/sysctl for it. I'll add one really soon. Regards, Navdeep > > In case it matters, here is what is in loader.conf: > > hw.cxgbe.num_vis=2 > hw.cxgbe.fl_pktshift=0 > hw.cxgbe.ntxq=1 > hw.cxgbe.nrxq=1 > hw.cxgbe.qsize_txq=512 > hw.cxgbe.qsize_rxq=512 > hw.cxgbe.cong_drop=1 > hw.cxgbe.pause_settings=1 > hw.cxgbe.autoneg=0 > hw.cxgbe.nm_rx_nframes=1 > hw.cxgbe.nm_rx_ndesc=1 > > Lars >