From owner-freebsd-net@freebsd.org Wed Mar 8 02:08:50 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADEAACFB063 for ; Wed, 8 Mar 2017 02:08:50 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg0-x22a.google.com (mail-pg0-x22a.google.com [IPv6:2607:f8b0:400e:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7931C1527 for ; Wed, 8 Mar 2017 02:08:50 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg0-x22a.google.com with SMTP id 187so7206392pgb.3 for ; Tue, 07 Mar 2017 18:08:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ig4ff7scxFgzvHoAv95Fj24G5cmSbMbC1vWWPwkDV/4=; b=UUR8ZNu0al/jLBDizQx1vUqkN37rGDzhJgOcC4X5GXgI7gMgNjIm04T9ObDhipBACe SJar6islOeeDfWPtTZGTXpN+UWIcVZWD7jPyG/xd39QLLPM9Vkt5n1e/4SA/qfDnMHyU MTdLeWfd/GpPP43PwpipVeC5Ck1qDO7xd1IU60m2Rx547t984oHK6PnPlwh3yk9oFxNW J8r7u+EcoWwTFOgsoQo963evPLGLLvfGl3iDYmn+FoSFoCPtiSwNk9nKEK5lELGka/qJ d+9LcOQrVmQe16y9hl6SVvCgisbKMwV7ogPwP9XpQdzar2bXc1PDGZeNY4ibYiG+rXpE 7qow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ig4ff7scxFgzvHoAv95Fj24G5cmSbMbC1vWWPwkDV/4=; b=QhLF8jhA17Cxs6TE7wqJSNQcALCgT/NaM+GimHMq8v0xVfCoZtCbO23r4iaepYkzZO HN+iZSsx6h0U9yX44EjQLb0HS7UKjXrFwxZaJLxU1MAkbMnPDs9VtxLhVlJXsiZ5+4bE FJM1N77R23EODKCHKvT7TVzeHa0JqFydJaiW1vTt0V7OSnS1dZhU2nZXTZYphXy+uPXJ R6vsaQv2Xnz3HpM5aITgfGKgCoyNM0R4e4tegXw17LQlMKEzG+vHG6xdSWyO8bsNFNCK nTugxk04E2fiFkSMQYx5M35Jemlf44zFJZ7GHqZrtcf0ya1u9NQkx7CkT4Gmnp9jOFci q4Qw== X-Gm-Message-State: AMke39nJ27NcvGm3ckDd04YxdNrZs+XYe8MaTyJYEzaGcpcp+ElszlmJqpmG6SmryEvzgBe7VlyQq4cKE84U0w== X-Received: by 10.84.176.131 with SMTP id v3mr4878508plb.7.1488938929811; Tue, 07 Mar 2017 18:08:49 -0800 (PST) MIME-Version: 1.0 Received: by 10.100.182.172 with HTTP; Tue, 7 Mar 2017 18:08:48 -0800 (PST) In-Reply-To: <901224a7-f3ed-84a4-5402-3542cd0e865b@sentex.net> References: <93e10699-ecc7-c062-bf85-a26de0de9633@sentex.net> <901224a7-f3ed-84a4-5402-3542cd0e865b@sentex.net> From: Navdeep Parhar Date: Tue, 7 Mar 2017 18:08:48 -0800 Message-ID: Subject: Re: Chelsio netmap support ? (RELENG_11) To: Mike Tancsa Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Mar 2017 02:08:50 -0000 On Tue, Mar 7, 2017 at 5:46 PM, Mike Tancsa wrote: > > # dmesg | grep netm > netmap: loaded module > vcxl0: netmap queues/slots: TX 2/1023, RX 2/1024 > vcxl0: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) > vcxl1: netmap queues/slots: TX 2/1023, RX 2/1024 > vcxl1: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) > igb0: netmap queues/slots: TX 4/1024, RX 4/1024 > igb1: netmap queues/slots: TX 4/1024, RX 4/1024 > > It maxes out at about 800Kpps with and without netmap. Is there a way Are you actually using a netmap based application that acts as a packet router or is this just the vcxl interface running as a normal ifnet? > to increase the queues for the Chelsio nic, like the onboard igb ? If you're not running a netmap based router get rid of the num_vis=2 and simply try with the cxl0/cxl1 interfaces. They should each have 4 rxq/4 txq on your system. In case you want to increase the number of queues, use this: # cxl0/cxl1 etc. hw.cxgbe.nrxq10g=8 hw.cxgbe.ntxq10g=8 # vcxl0/vcxl1's "normal" queues hw.cxgbe.nrxq_vi=8 hw.cxgbe.ntxq_vi=8 # vcxl0/vcxl1's netmap queues hw.cxgbe.nnmrxq_vi=8 hw.cxgbe.nnmtxq_vi=8 Check in dmesg after you reboot with your new settings > vcxl0: 1 txq, 1 rxq (NIC); 1 txq, 1 rxq (TOE); 2 txq, 2 rxq (netmap) The "NIC" queues are the normal tx/rx queues, the "netmap" queues are active when the interface is in netmap mode. Does netsend generate a single flow or multiple flows? If it's a single flow it will use a single queue only. Regards, Navdeep