From owner-freebsd-net@freebsd.org Sat Jan 23 18:54:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70ABDA8FF99 for ; Sat, 23 Jan 2016 18:54:55 +0000 (UTC) (envelope-from cenzatti@hush.com) Received: from smtp3.hushmail.com (smtp3a.hushmail.com [65.39.178.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.hushmail.com", Issuer "smtp.hushmail.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3DCA91AA8 for ; Sat, 23 Jan 2016 18:54:54 +0000 (UTC) (envelope-from cenzatti@hush.com) Received: from smtp3.hushmail.com (smtp3a.hushmail.com [65.39.178.201]) by smtp3.hushmail.com (Postfix) with SMTP id 57600E046A for ; Sat, 23 Jan 2016 18:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=hush.com; h=date:to:subject:from; s=hush; bh=OnP01EcKQ96h0GVj83+XkXDkr4RWi+QQ5wqDZIKMS9U=; b=YPfNOcqefU1dAZ1DUbW+ktREi/Wu2Ed9QFjsTlIsOAKlWTPVUmadSTuhXClaCPLXTqzkATAsTumy/jHmHmXZp13m3RJ5H50/8uQEYphC0WMZE+DkmiyP8sBdCR2VjPpjdjTNCilxkJsCgbdvXg6fCGMwjBhXmDRkR4zb3C/qO6X9d2QsPKumZeHjp8P8xwmXg2MB1JxkoAIXTsvuuUJHKIfh4tphI+uhibJz2/PTaHnCaf0Fjd6aGFVw5cFd149YWB0Esev7YyNXQcAN5ntGT8OUsxm29sUMwo7IqRBpfsaHfOzrbKYFxZRuFSnirS3IUDGhaB3EOf4bgxPh78NKSQ== Received: from smtp.hushmail.com (w5.hushmail.com [65.39.178.80]) by smtp3.hushmail.com (Postfix) with ESMTP; Sat, 23 Jan 2016 18:54:53 +0000 (UTC) Received: by smtp.hushmail.com (Postfix, from userid 99) id 0FA7CA0128; Sat, 23 Jan 2016 18:54:53 +0000 (UTC) MIME-Version: 1.0 Date: Sat, 23 Jan 2016 16:54:52 -0200 To: "Navdeep Parhar" Cc: "Luigi Rizzo" , freebsd-net@freebsd.org Subject: Re: Chelsio T520-SO-CR low performance (netmap tested) for RX From: "Marcus Cenzatti" In-Reply-To: <20160123183836.GB4574@ox> References: <20160123053428.2091EA0121@smtp.hushmail.com> <20160123154052.GA4574@ox> <20160123171300.0F448A0121@smtp.hushmail.com> <20160123174840.32B1DA0121@smtp.hushmail.com> <20160123183836.GB4574@ox> Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="UTF-8" Message-Id: <20160123185453.0FA7CA0128@smtp.hushmail.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Jan 2016 18:54:55 -0000 On 1/23/2016 at 4:38 PM, "Navdeep Parhar" wrote: > >On Sat, Jan 23, 2016 at 03:48:39PM -0200, Marcus Cenzatti wrote: >... >> >> woops, my bad, yes probably we had some drop, with -S and -D now >I get 1.2Mpps. > >Run "netstat -hdw1 -i cxl" on the receiver during your test. >Do you >see errs and/or idrops incrementing? The input "packets" counter >should >match what the transmitter is claiming to transmit at. > >Also check the output of this: ># sysctl -n dev.t5nex.0.misc.tp_err_stats >It is ok if you see tnlCongDrops, but any of the Errs counter >going up >is not good -- it means the incoming frames had errors. > >Do you know if the transmitter will pad up so as not to put runts >on the >wire? If not then you might want to bump up the size of the frame >explicitly (there's some pkt-gen knob for this). > >Regards, >Navdeep > the size frames I sent based on luigi's suggestions show some particular behaviour as you see it? here is the output for netstat when I pkt-gen -f tx un-throttled (14Mpps): input (Total) output packets errs idrops bytes packets errs bytes colls drops 900k 0 0 55M 3 0 550 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 9 0 2.4K 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 900k 0 0 55M 3 0 422 0 0 as for the error stats: # sysctl -n dev.t5nex.0.misc.tp_err_stats channel 0 channel 1 channel 2 channel 3 macInErrs: 292496 0 0 0 hdrInErrs: 0 0 0 0 tcpInErrs: 0 0 0 0 tcp6InErrs: 0 0 0 0 tnlCongDrops: 30750953 0 0 0 tnlTxDrops: 0 0 0 0 ofldVlanDrops: 0 0 0 0 ofldChanDrops: 0 0 0 0 ofldNoNeigh: 0 ofldCongDefer: 0