From owner-freebsd-net@freebsd.org Fri Sep 15 18:26:04 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FCB4E010C4 for ; Fri, 15 Sep 2017 18:26:04 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: from mail-pg0-x244.google.com (mail-pg0-x244.google.com [IPv6:2607:f8b0:400e:c05::244]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 304B1726C9 for ; Fri, 15 Sep 2017 18:26:04 +0000 (UTC) (envelope-from nparhar@gmail.com) Received: by mail-pg0-x244.google.com with SMTP id d8so1705402pgt.3 for ; Fri, 15 Sep 2017 11:26:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=WcXYENPxnnsElZpe1gt3DA7GSeKrHXh6wzJ2zBWk6iA=; b=SXaivq5zS+mMPmN8f6dVT278BhIHKF4DN2NfTeHG695IadeqLgCSviTJ3bo2QeLn0C V7IG4GIfkcFoY7+qvBEDSuUtUou50oTyWaRPci2ubUMlM1uOzuUVaz4is9NCwC5QhdpL 65jILtbKUS7LAN7DYqCA33iomE9PgKLZhNJI1K5topDf8K8PMau4Eky5Ff6vpLljnW/Y YiunFTCmkCkLDN4iORGNr4Fgw41yHXLVNDggr3mAs8uMIC53rME4CvvDuU335Li5vTnb +YLYvezJ/4nwGJIQdM+o7NdAj4SmsfLKXLbAFDEBlNnGI4l8FPfr1WJtsrFRMjJs2sdI lAFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=WcXYENPxnnsElZpe1gt3DA7GSeKrHXh6wzJ2zBWk6iA=; b=RVDC9kPtZrAZUnfex2o2ZpGBguP5iT7WKbCpzS4ol4kwsk9fLW+Goqli1DdEoZPmsE qaJQl1lV/PnEGGMHoTXr+WHSwOnp4rJBCJsdgw51uvSCM5CBrEfvl2mLybRDh71kZYCB vwBfYXcTr1WQKjUqcUyeiQI+Kkp99DemfTZzp3mw354mdxWvrl2tRgMSfr24CUPdQCax zzRNlwaE3BENT+W5MuafwIPm8FsRcsuOfeTrT4VKG6TzFEFfrQmrqbjYSXLaRJGQ6nUG wLJR/NQF1GmzyjC3nTK23m9utVRL9/aRaRDaHnsQCwzFpN26HeYu+w7HvIHxnjMMN7eX ZzMA== X-Gm-Message-State: AHPjjUiB7FGsbymh18yU0aiICWRHvI7cCh5HOpwFHPDP3D6cO2WkfaMY ZSmgDEzXMyblcbmOi14DW6/aqJEFgbFELb3ovn4= X-Google-Smtp-Source: AOwi7QAP5h2WTEJEG9k3KlGCeVzOkC7Rr9bbtjMF4pokXi3xfUzJuA1qkk8yyXls6N8kiEs/nq96JhhYDyd7D7wtL1U= X-Received: by 10.84.253.145 with SMTP id a17mr6779899plm.116.1505499963639; Fri, 15 Sep 2017 11:26:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.100.139.136 with HTTP; Fri, 15 Sep 2017 11:26:03 -0700 (PDT) In-Reply-To: <8bc1eb83-b56e-69e4-544f-1d64e9c44cb7@gmail.com> References: <8bc1eb83-b56e-69e4-544f-1d64e9c44cb7@gmail.com> From: Navdeep Parhar Date: Fri, 15 Sep 2017 11:26:03 -0700 Message-ID: Subject: Re: packets per second routing tests: mismatch between switch (~4M pps) and netstat (~10M pps) scores To: "Caraballo-Vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP]" Cc: "freebsd-net@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Sep 2017 18:26:04 -0000 The tx errors/drops are almost always due to software queue overflow. There isn't much that can go wrong once the frame has successfully been submitted to the hw for tx. Do you have a lot of PAUSE coming in during the tests where you see just 31Kpps tx? You can monitor the incoming PAUSE frames with this: # sysctl dev.cc | grep rx_pause # sysctl dev.cc..stats has a lot of stats straight from the hardware. The hardware's tx_ stats should be clean even though netstat reports a lot of errors/drops as long as the drops are in the software. Regards, Navdeep On Fri, Sep 15, 2017 at 8:18 AM, Caraballo-Vega, Jordan A. (GSFC-6062)[COMPUTER SCIENCE CORP] wrote: > Hi all, > > We are currently running udp 1500 bytes pps tests through a Dell PE R530 > and two Chelsio T62100-LP-CR NICs acting as a router. At the switch we > are seeing ~4M pps, while with "netstat -w1 -d -h" we are seeing ~10M > pps or ~31k in the same test. Any idea of why scores are so different? > It could be a misconfiguration of my network interfaces? Where should I > look at? > > Scores from the switch are as follow: > > 2017-09-15 10:57:57: duration=360, nodes=24, streams=85ea/2040agg, > frame_size=1500, pps=1499977, Mbps=17999: Zone-1(in:57%, out:0%) > Zone-2(in:0%, out:14%) > > Scores from netstat are as follow: > > input (Total) output > packets errs idrops bytes packets errs bytes colls drops > 17M 0 0 23G 10M 0 14G 0 0 > 17M 0 0 23G 10M 0 14G 0 0 > 17M 0 0 23G 10M 0 14G 0 0 > 17M 0 0 23G 10M 0 14G 0 0 > 17M 0 0 24G 31k 7.8M 1.9M 0 7.8M > 17M 0 0 24G 31k 7.8M 1.9M 0 7.8M > 17M 0 0 24G 31k 7.8M 1.9M 0 7.8M > 17M 0 0 24G 31k 7.8M 1.9M 0 7.8M > > Interfaces are bonded as follow (/etc/rc.conf): > > ifconfig_cc0="-tso -lro mtu 9000 up" > ifconfig_cc1="-tso -lro mtu 9000 up" > ifconfig_cc2="-tso -lro mtu 9000 up" > ifconfig_cc3="-tso -lro mtu 9000 up" > cloned_interfaces="lagg0 lagg1 vlan0 vlan1" > ifconfig_lagg0="laggproto lacp laggport cc0 laggport cc1" > ifconfig_lagg1="laggproto lacp laggport cc2 laggport cc3" > ifconfig_vlan0="inet 172.16.2.1/24 vlan 20 vlandev lagg0" > ifconfig_vlan1="inet 172.16.1.1/24 vlan 10 vlandev lagg1" > > Thanks in advance, > > - Jordan > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"