From owner-freebsd-net@freebsd.org Fri Sep 23 17:59:13 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0FE8BE6011 for ; Fri, 23 Sep 2016 17:59:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8747175C for ; Fri, 23 Sep 2016 17:59:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x241.google.com with SMTP id b184so3934274wma.3 for ; Fri, 23 Sep 2016 10:59:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:content-transfer-encoding:subject:message-id:date:to :mime-version; bh=FJne38Bhq32l4emy71aPCw6XDHykPa5QdXcMDr2HFh0=; b=owh4a0tPr9z4YogEhj8oMnW2lao59bQGv0IjiOp9G0WKtMCLo6sZHzzq4oEQenHZAK DbNK/BjWun262rsQO7Lr/jKxk/uIvL048ci7Deu98XkFKs2c6s15WwG4wX8EwgtZ3b9i +BHUNkt3NJ8oBcYOjBqx5YCwEjjz+H90VHegn35EbWXs9HLPJsQcT1yPgtGCrO2BwPvU WFbMy6Hl1fz3Z2dk/OTtezZrBOteZJD/73xaVymCZkYfpcXZ8zZdfqzmovcCzWiYBUC5 iOFXXpYDQ7/wiEhT32F/hF9F+0ooOeEzfvUslzuhZsuORYhOB0+tGX6ctX7vrbTimPzu nI7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-transfer-encoding:subject :message-id:date:to:mime-version; bh=FJne38Bhq32l4emy71aPCw6XDHykPa5QdXcMDr2HFh0=; b=JwR4OcJDbnTB8Xi+EjDyl0Rxs7fiyV7zqQ0ABvsRvf37w6aKGYZFbwtXJ+CrYbeqTs j09xoO8iOuCf6OcUAz0DfZ/upsEsOPNd3PZnJ12en4ZpG2l0TtKjzgIyfB9aEmkfwROi mWJDWQpwx97m5dd8hd8yQ3JC9yNzyJPKPwwLvaJCO0ne/Ak8K3R1b17jh+aizNIyrCyy A1Rz3PAwoK33/Wasgahz1zB3N3HyycJSEU7K44D3Br/37K+vrvYmE+2k7JeGjYMK6ZNG LAw4O8HtAF2RNrCDXlVWCZ0/l+vINQynT21gfWvRmDoeyM4mem1ESWAahValnK8eN9X/ 9yag== X-Gm-Message-State: AA6/9RmFozU8+AhUlUQ2BD/tPNLMfYkt4r7pG465zYE4fpzvNSEYZv3xhjFN64KXjDvo+w== X-Received: by 10.194.13.4 with SMTP id d4mr8253200wjc.218.1474653551746; Fri, 23 Sep 2016 10:59:11 -0700 (PDT) Received: from macbook-air-de-benjamin-1.home (LFbn-1-7159-4.w90-116.abo.wanadoo.fr. [90.116.90.4]) by smtp.gmail.com with ESMTPSA id bw9sm8313112wjc.33.2016.09.23.10.59.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 23 Sep 2016 10:59:11 -0700 (PDT) From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: netstat counts input traffic twice Message-Id: <94953762-4397-476D-AD61-A39914F27938@gmail.com> Date: Fri, 23 Sep 2016 19:59:09 +0200 To: FreeBSD Net Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Sep 2016 17:59:14 -0000 Hello, I found a strange issue where input traffic is counted twice, sometimes more. How to reproduce : dst# netstat -b -I mlxen1 Name Mtu Network Ipkts Ibytes Opkts Obytes mlxen 9000 223135371 2297323715986 242534891 1594979072449 mlxen - 192.168.0.0/1 140330886 1143518393245 239792545 1583789250111 dst# sysctl hw.mlxen1.stat.rx_bytes hw.mlxen1.stat.rx_bytes: 1151840666259 dst# iperf -s src# iperf -c dst -i 1 -t 10 -n 1G [ ID] Interval Transfer Bandwidth [ 3] 0.0- 0.3 sec 1.00 GBytes 33.0 Gbits/sec dst# netstat -b -I mlxen1 Name Mtu Network Ipkts Ibytes Opkts Obytes mlxen 9000 223257341 2299482904332 242578378 1594982116541 mlxen - 192.168.0.0/1 140378866 1144594630061 239836031 1583791511391 dst# sysctl hw.mlxen1.stat.rx_bytes hw.mlxen1.stat.rx_bytes: 1152922946009 Results : iperf on dst gives good results : dst# iperf -s [ ID] Interval Transfer Bandwidth [ 3] 0.0- 0.3 sec 1.00 GBytes 33.0 Gbits/sec hw.mlxen1.stat.rx_bytes also gives good results : 1152922946009 - 1151840666259 = 1GG However, netstat : 2299482904332 - 2297323715986 = 2GB (!) 1144594630061 - 1143518393245 = 1GB Strangely enough input packets are counted twice for the "link" interface. Why ? Is there a bug somewhere ? Of course looking at the real time results with netstat -w 1 gives the same uncorrect results. As well as with systat -ifstat. iftop (which I do not like so much because it sometimes hangs the traffic) seems to give good results however. I tested this on 3 similar servers, in all possible ways. Every time I get these strange results. This behaviour of course totally brakes reporting tools based on netstat. My 2 systems : # uname -r 11.0-RC3 # uptime 7:40p.m. up 47 mins, 1 users, load averages: 0.00, 0.00, 0.00 # mst status MST devices: ------------ pci0:133:0:0 - MT27500 Family [ConnectX-3] # flint -d pci0:133:0:0 q Image type: FS2 FW Version: 2.36.5000 FW Release Date: 26.1.2016 Product Version: 02.36.50.00 Rom Info: type=PXE version=3.4.718 devid=4099 Device ID: 4099 PSID: MT_1090110023 Thank you very much ! Ben