From owner-svn-src-all@FreeBSD.ORG Wed May 28 00:42:11 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B81828B3; Wed, 28 May 2014 00:42:11 +0000 (UTC) Received: from mail-pa0-x235.google.com (mail-pa0-x235.google.com [IPv6:2607:f8b0:400e:c03::235]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F9A827B2; Wed, 28 May 2014 00:42:11 +0000 (UTC) Received: by mail-pa0-f53.google.com with SMTP id kp14so9948201pab.26 for ; Tue, 27 May 2014 17:42:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=YcGYU8s/61iCfYYEv35u54Jk6cSckf6TByH0PRAs8KM=; b=GWywMUep5bjjOLQT7Ldfg/UO2+/8TxAlJQZpjUjQLmBEpef7ByDvmDk5KuoNp58VFw OKOKSU2A604jbIy/VD8/xWoK3VNs1z0ePPuOWNzajUJaXwv3IE+v9gLxWgDHCLQcR9Ad 8DCexNnLw1e38xZY2m9nKGIooGE7Jt31OGQRa/qsvKUTzrHzy33KqviQmdAoJSO/TsiB 84mbgLbA5M5XjGdP7OZ6CAVpZRagoQrq4Iy6IiP0oUQhXFW5ywH5aywojO61O/gpKKI+ vEzfULX1qQSlPSv8P1UsPn3ssmwqHh9TZi1ylUmMH4BNmfsyuMjRhJH4QQHfDbkTOkpO BjxA== X-Received: by 10.66.244.140 with SMTP id xg12mr35369659pac.23.1401237730662; Tue, 27 May 2014 17:42:10 -0700 (PDT) Received: from [10.192.166.0] (stargate.chelsio.com. [67.207.112.58]) by mx.google.com with ESMTPSA id oa3sm25319955pbb.15.2014.05.27.17.42.09 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 27 May 2014 17:42:09 -0700 (PDT) Sender: Navdeep Parhar Message-ID: <538530E0.9090309@FreeBSD.org> Date: Tue, 27 May 2014 17:42:08 -0700 From: Navdeep Parhar User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.5.0 MIME-Version: 1.0 To: Luigi Rizzo Subject: Re: svn commit: r266757 - in head/sys: conf dev/cxgbe dev/cxgbe/common modules/cxgbe/if_cxgbe References: <201405271818.s4RIIff9045225@svn.freebsd.org> In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 May 2014 00:42:11 -0000 On 05/27/14 17:36, Luigi Rizzo wrote: >=20 >=20 >=20 > On Tue, May 27, 2014 at 8:18 PM, Navdeep Parhar > wrote: >=20 > Author: np > Date: Tue May 27 18:18:41 2014 > New Revision: 266757 > URL: http://svnweb.freebsd.org/changeset/base/266757 >=20 > Log: > cxgbe(4): netmap support for Terminator 5 (T5) based 10G/40G card= s. > Netmap gets its own hardware-assisted virtual interface and won't= take > over or disrupt the "normal" interface in any way. You can use b= oth > simultaneously. >=20 > For kernels with DEV_NETMAP, cxgbe(4) carves out an ncxl inter= face > (note the 'n' prefix) in the hardware to accompany each cxl > interface. These two ifnet's per port share the same wire but re= ally > are separate interfaces in the hardware and software. Each gets > its own > L2 MAC addresses (unicast and multicast), MTU, checksum caps, etc= =2E > You > should run netmap on the 'n' interfaces only, that's what they ar= e > for. >=20 > With this, pkt-gen is able to transmit > 45Mpps out of a single > 40G port >=20 >=20 > =E2=80=8Band just for the records, this is with 1 core in userland. > Very cool. >=20 > Do you have an easy way to check what kind of throughput > you get when the netmap interface has one hw queue ? Both tx and rx are at around ~33Mpps right now with one hardware queue. That's more than enough to saturate a 40G link with 128B sized frames. Regards, Navdeep >=20 > cheers > luigi >=20 > of a T580 card. 2 port tx is at ~56Mpps total (28M + 28M) as of = now. > Single port receive is at 33Mpps but this is very much a work in > progress. I expect it to be closer to 40Mpps once done. In any = case > the current effort can already saturate multiple 10G ports of a T= 5 > card > at the smallest legal packet size. T4 gear is totally untested. >=20 > trantor:~# ./pkt-gen -i ncxl0 -f tx -D 00:07:43:ab:cd:ef > 881.952141 main [1621] interface is ncxl0 > 881.952250 extract_ip_range [275] range is 10.0.0.1:0 > to 10.0.0.1:0 > 881.952253 extract_ip_range [275] range is 10.1.0.1:0 > to 10.1.0.1:0 > 881.962540 main [1804] mapped 334980KB at 0x801dff000 > Sending on netmap:ncxl0: 4 queues, 1 threads and 1 cpus. > 10.0.0.1 -> 10.1.0.1 (00:00:00:00:00:00 -> 00:07:43:ab:cd:ef) > 881.962562 main [1882] Sending 512 packets every 0.000000000 s > 881.962563 main [1884] Wait 2 secs for phy reset > 884.088516 main [1886] Ready... > 884.088535 nm_open [457] overriding ifname ncxl0 ringid 0x0 flags= 0x1 > 884.088607 sender_body [996] start > 884.093246 sender_body [1064] drop copy > 885.090435 main_thread [1418] 45206353 pps (45289533 pkts in > 1001840 usec) > 886.091600 main_thread [1418] 45322792 pps (45375593 pkts in > 1001165 usec) > 887.092435 main_thread [1418] 45313992 pps (45351784 pkts in > 1000834 usec) > 888.094434 main_thread [1418] 45315765 pps (45406397 pkts in > 1002000 usec) > 889.095434 main_thread [1418] 45333218 pps (45378551 pkts in > 1001000 usec) > 890.097434 main_thread [1418] 45315247 pps (45405877 pkts in > 1002000 usec) > 891.099434 main_thread [1418] 45326515 pps (45417168 pkts in > 1002000 usec) > 892.101434 main_thread [1418] 45333039 pps (45423705 pkts in > 1002000 usec) > 893.103434 main_thread [1418] 45324105 pps (45414708 pkts in > 1001999 usec) > 894.105434 main_thread [1418] 45318042 pps (45408723 pkts in > 1002001 usec) > 895.106434 main_thread [1418] 45332430 pps (45377762 pkts in > 1001000 usec) > 896.107434 main_thread [1418] 45338072 pps (45383410 pkts in > 1001000 usec) > ... >=20 > Relnotes: Yes > Sponsored by: Chelsio Communications. >=20 > Added: > head/sys/dev/cxgbe/t4_netmap.c (contents, props changed) > Modified: > head/sys/conf/files > head/sys/dev/cxgbe/adapter.h > head/sys/dev/cxgbe/common/common.h > head/sys/dev/cxgbe/common/t4_hw.c > head/sys/dev/cxgbe/t4_main.c > head/sys/dev/cxgbe/t4_sge.c > head/sys/modules/cxgbe/if_cxgbe/Makefile >=20