From owner-freebsd-net@freebsd.org Sat Jan 23 18:56:27 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30B9CA8E07B for ; Sat, 23 Jan 2016 18:56:27 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02BE51C7E for ; Sat, 23 Jan 2016 18:56:27 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pa0-x234.google.com with SMTP id uo6so59938590pac.1 for ; Sat, 23 Jan 2016 10:56:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent; bh=B0suCoHijZWCF07iMB+xFzlSPJIywOn/nPzFHqJBPT4=; b=o04PFlu8tkQb65+LEzLqk55CpZj5t9bZutfwoSR8h92esSK+PAwjFqy4NrmtRtZm+v hpn3Bg+Ias2Sz4psRsj4FdDVrHAiRmNlA7oC/CB8Et6njgKFcfDrIgDpgbVNYDL+OvTR VHFio2tBhZfCBLbQkWHcd8+iIWuf+eEYLjZTA0z+DnToSBV1zhkKtEmJbCcQfxCcpD3O hB5ArBl6NI3PrGUr46GcWCqtaSpdW85W1nhnWKbki+Eu4idORBIOzVxt7o56yNNlkgjo 5tSKW5rdHmYqKo+ujQoQFrIySCY8E4g9RIIefZ5u8Kp+d5QwnI6QA7mmDaqD2zJ0yfG6 F+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=B0suCoHijZWCF07iMB+xFzlSPJIywOn/nPzFHqJBPT4=; b=CCE479GkFNCaVVC1MbBOdam7ANbrBQVoP6Tk04/QXc53ORPrpV8uI1s11s2oQ76BAu Z7Ip5KA3JJ1ZI7f+S5OpBYFWDDkjCoIbbWYFypEvyHhpyQVsnT7yxw7e/477rr6DVFxP Ld27Mat2m9pMsbE3oRg2xtk8nE3uRApm4FsYDEGLJSaNclcb99Y97WFO0YiZyXH8PH5R pVkSj9DhBh0Sj3o+eew1qczWi3hvIrDjrX6Jc9vyHk7y9kt5JfM+2DV9UYMzJf9Dt2qG GnmBuEI7C13217EhuIofgZLr0suDsID494cLU2CYrXkyTkF8ForvVM+sg8AEI7IZ4D1z nlog== X-Gm-Message-State: AG10YOQOdnpzqDXdBcJhJcj/zGo6N9gnrU+1o7Mu1u1lIZ3CikOlQSx4wWoLYxee7sK2dA== X-Received: by 10.66.101.3 with SMTP id fc3mr13885877pab.2.1453575386727; Sat, 23 Jan 2016 10:56:26 -0800 (PST) Received: from ox ([2601:641:c001:8a00:591d:d471:ff4a:b8bf]) by smtp.gmail.com with ESMTPSA id 2sm17982640pfr.22.2016.01.23.10.56.24 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Sat, 23 Jan 2016 10:56:25 -0800 (PST) Date: Sat, 23 Jan 2016 10:56:23 -0800 From: Navdeep Parhar To: Marcus Cenzatti Cc: freebsd-net@freebsd.org Subject: Re: Chelsio T520-SO-CR low performance (netmap tested) for RX Message-ID: <20160123185623.GC4574@ox> Mail-Followup-To: Marcus Cenzatti , freebsd-net@freebsd.org References: <20160123053428.2091EA0121@smtp.hushmail.com> <20160123154052.GA4574@ox> <20160123171300.0F448A0121@smtp.hushmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160123171300.0F448A0121@smtp.hushmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Jan 2016 18:56:27 -0000 On Sat, Jan 23, 2016 at 03:12:59PM -0200, Marcus Cenzatti wrote: ... > intel# ./pkt-gen -i ix0 -f tx -d 00:07:43:33:8d:c1 -s 00:07:e9:44:d2:ba > 267.767848 main [1715] interface is ix0 > 267.767990 extract_ip_range [291] range is 0.0.0.0:90 to 0.0.0.0:90 > 267.768006 extract_ip_range [291] range is 0.0.0.0:7 to 0.0.0.0:7 Does this mean the packets are being transmitted with source and destination IP all 0? Try to provide a more reasonable IP address range. The T5 receiver might be throwing away "obviously" bad frames. (See my other email for how to check if it's dropping frames) > 267.872796 main [1910] mapped 334980KB at 0x801c00000 > Sending on netmap:ix0: 8 queues, 1 threads and 1 cpus. > 00 -> 00 (00:00:00:00:00:00 -> ff:ff:ff:ff:ff:ff) This you've fixed already. L2 broadcasts will get replicated by the receiver and will be delivered to both the cxl and the ncxl interface. The ncxl interface is set to drop on congestion but the cxl interface is set to emit PAUSE on congestion. cxl plugs into the stack, which is slow at pps workloads, and so L2 broadcasts will result in PAUSE out of the port and will slow down the transmitter. Regards, Navdeep