From owner-freebsd-net@freebsd.org Wed Mar 8 14:28:40 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33796D03A6A for ; Wed, 8 Mar 2017 14:28:40 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DA8F81C88 for ; Wed, 8 Mar 2017 14:28:39 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id v28ESc1Y054558 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 8 Mar 2017 09:28:38 -0500 (EST) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c] ([IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id v28ESbvY025702; Wed, 8 Mar 2017 09:28:37 -0500 (EST) (envelope-from mike@sentex.net) Subject: Re: Chelsio netmap support ? (RELENG_11) To: Navdeep Parhar References: <93e10699-ecc7-c062-bf85-a26de0de9633@sentex.net> <901224a7-f3ed-84a4-5402-3542cd0e865b@sentex.net> Cc: "freebsd-net@freebsd.org" From: Mike Tancsa Organization: Sentex Communications Message-ID: Date: Wed, 8 Mar 2017 09:28:37 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Mar 2017 14:28:40 -0000 On 3/7/2017 9:08 PM, Navdeep Parhar wrote: > On Tue, Mar 7, 2017 at 5:46 PM, Mike Tancsa wrote: > >> >> # dmesg | grep netm >> netmap: loaded module >> vcxl0: netmap queues/slots: TX 2/1023, RX 2/1024 >> vcxl0: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) >> vcxl1: netmap queues/slots: TX 2/1023, RX 2/1024 >> vcxl1: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) >> igb0: netmap queues/slots: TX 4/1024, RX 4/1024 >> igb1: netmap queues/slots: TX 4/1024, RX 4/1024 >> >> It maxes out at about 800Kpps with and without netmap. Is there a way > > Are you actually using a netmap based application that acts as a > packet router or is this just the vcxl interface running as a normal > ifnet? the later, vcxl running normal ifnet. I thought there would be a benefit to utilizing netmap ? Sorry, this is not clear to me. > >> to increase the queues for the Chelsio nic, like the onboard igb ? > > If you're not running a netmap based router get rid of the num_vis=2 > and simply try with the cxl0/cxl1 interfaces. They should each have 4 > rxq/4 txq on your system. In case you want to increase the number of > queues, use this: The tests with the regular cxl also show the box topping out at 0.8Mpps for forwarding. > > The "NIC" queues are the normal tx/rx queues, the "netmap" queues are > active when the interface is in netmap mode. > > Does netsend generate a single flow or multiple flows? If it's a > single flow it will use a single queue only. I think its as a single flow. However, I was using a separate box to generate a second flow as well. It still topped out at about 800Kpps before dropping packets. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/