Date: Mon, 26 Mar 2012 10:42:15 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-net@freebsd.org Cc: Alex Tutubalin <lexa@lexa.ru>, Jeff Roberson <jeff@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: 9-STABLE + Infiniband - incorrect interface counters Message-ID: <201203261042.15948.jhb@freebsd.org> In-Reply-To: <4F6DB568.1060604@lexa.ru> References: <4F6DB568.1060604@lexa.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Saturday, March 24, 2012 7:52:08 am Alex Tutubalin wrote: > Hi, > > I'm playing with two FreeBSD 9-STABLE boxes connected via 10Gbps > Infiniband (more details below) in Infiniband connected mode. > > I see incorrect interface statistics (e.g. in netstat output), output > counters are 2x more than expected. > > EXAMPLE, ftp transfer of 1 GiB file: > > ftp> put file /dev/null > local: file remote: /dev/null > 229 Entering Extended Passive Mode (|||57978|) > 150 Opening BINARY mode data connection for '/dev/null'. > 100% |***********************************| 953 MiB 390.43 MiB/s > 00:00 ETA > 226 Transfer complete. > 1000000000 bytes sent in 00:02 (390.13 MiB/s) > > Netstat on receiving side, counters are correct (for input): > > lexa@home-gw:/home/lexa# netstat -I ib1 5 > input (ib1) output > packets errs idrops bytes packets errs bytes colls > 0 0 0 0 0 0 0 0 > 13955 0 0 222688126 9027 0 1192796 0 > 48921 0 0 780832960 32129 0 4240596 0 > 0 0 0 0 0 0 80 0 > > Sum of bytes (input) is 1003521086, as expected. > > Netstat on sending size, output is 2x more: > > lexa@new-gw:/home/lexa# netstat -I ib0 5 > input (ib0) output > packets errs idrops bytes packets errs bytes colls > 1 0 0 100 0 0 0 0 > 41162 0 0 2305210 62878 0 2008325984 0 > 1 0 0 100 0 0 0 0 > > It looks like packet count is correct (13955+48921=62876, two packets > missed somewhere), while byte count is exact 2x more. Yes, this is a bug. if_obytes already gets incremented in IFQ_HANDOFF(), so the IB code doesn't need to do it again. Try the patch at www.freebsd.org/~jhb/patches/ipoib_obytes.patch I can't speak to the MTU issue though. -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201203261042.15948.jhb>