Date: Thu, 2 Jun 2016 21:11:32 +0000 From: Sreekanth Rupavatharam <rupavath@juniper.net> To: hiren panchasara <hiren@strugglingcoder.info> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "jfvogel@gmail.com" <jfvogel@gmail.com> Subject: Re: Possible transmit/stats problem in igb driver. Message-ID: <9A903EE5-3F2C-46C0-B563-1150F81E3507@juniper.net> In-Reply-To: <20160602202015.GG8994@strugglingcoder.info> References: <D7944476-98AD-4548-99E3-6E88648E2B06@juniper.net> <20160602202015.GG8994@strugglingcoder.info>
next in thread | previous in thread | raw e-mail | index | archive | help
Inline >Apart from stats, do you see anything else going wrong? i.e. do you >actually see less packets (arp replies??) than expected? [SR] The packets are not going out on the wire. The tool doesn’t receive the packets. That’s how I started noticing the issue. >Taking your example, tx_packets is something we count in the drivers and >total_pkts_txd is calculated in the card and we just read it off of it >to report (E1000_TPT). [SR] Correct. My main question would be under what circumstance would the packet handed off to hardware will *not* be transmitted?. Especially considering there are no transmit errors or pause frames received. There are no dma tx failures either. That’s the baffling part. I tried another exercise where I used ping of various sizes going out, but that doesn’t seem to trigger the problem. >To understand your setup better, ixia is the sender and your box with >igb(4) is the receiver and your are sending arp requests to it. Yes, correct. >Can you post following for working (size <= 64bytes) and non-working >(size > 64bytes) cases for before/after? > >sysctl dev.igb | grep tx_packets >sysctl dev.igb | grep total_pkts_txd >sysctl dev.igb | grep rx_packets >sysctl dev.igb | grep total_pkts_recvd Before(not working): dev.igb.1.queue0.tx_packets: 24907933 dev.igb.1.queue0.rx_packets: 18086575 dev.igb.1.mac_stats.total_pkts_recvd: 25057359 dev.igb.1.mac_stats.total_pkts_txd: 16647169 After(not working): dev.igb.1.queue0.tx_packets: 24913324 dev.igb.1.queue0.rx_packets: 18091832 dev.igb.1.mac_stats.total_pkts_recvd: 25062618 dev.igb.1.mac_stats.total_pkts_txd: 16647545 >netstat -sp arp The difference is 5391 for queue0.tx_packets but for mac_stats.total_pkts_txd is 376 Everything else is matching up. Before (working) dev.igb.1.queue0.tx_packets: 25359165 dev.igb.1.queue0.rx_packets: 18526094 dev.igb.1.mac_stats.total_pkts_recvd: 25508763 dev.igb.1.mac_stats.total_pkts_txd: 16831587 After(working) dev.igb.1.queue0.tx_packets: 25364597 dev.igb.1.queue0.rx_packets: 18531398 dev.igb.1.mac_stats.total_pkts_recvd: 25514009 dev.igb.1.mac_stats.total_pkts_txd: 16836833 Another interesting stat is before_notworking:dev.igb.1.interrupts.tx_queue_empty: 16646890 after_notworking:dev.igb.1.interrupts.tx_queue_empty: 16647266 The difference here is exactly 376 which is the number of packets that the device actually claims to have transmitted. It’s as though it didn’t see the other packets en-queued in the ring descriptor. I can’t do netstat just for arp as these are coming in a tunnel(Packets don’t’ show up as arp on the interface). However, I did see the packet rate was about 500 packets/sec >(You can netstat -z to clear the counters between runs.) > >Cheers, >Hiren
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9A903EE5-3F2C-46C0-B563-1150F81E3507>