Date: Tue, 31 Mar 2015 19:01:52 -0700 From: hiren panchasara <hiren@strugglingcoder.info> To: freebsd-net@freebsd.org Cc: jfv@freebsd.org, erj@freebsd.org, nitroboost@gmail.com Subject: Re: em(4): difference between missed_packets and rx_overrun Message-ID: <20150401020152.GM10892@strugglingcoder.info> In-Reply-To: <20150327173334.GB39674@strugglingcoder.info> References: <20150326200853.GA19536@strugglingcoder.info> <20150327173334.GB39674@strugglingcoder.info>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On 03/27/15 at 10:33P, hiren panchasara wrote: > + jfv, erj from Intel. > > On 03/26/15 at 01:08P, hiren panchasara wrote: > > This is what we are seeing on em(4) 82574L chipset running stable/10: > > > > dev.em.0.mac_stats.missed_packets: 1441927 > > dev.em.0.interrupts.rx_overrun: 153 > > > > From the datasheet: > > http://www.intel.com/content/www/us/en/ethernet-controllers/82574l-gbe-controller-datasheet.html > > > > 10.2.7.4 Missed Packets Count - MPC (0x04010; R) > > Counts the number of missed packets. Packets are missed when the receive > > FIFO has insufficient space to store the incoming packet. This could be > > caused because of too few buffers allocated, or because there is > > insufficient bandwidth on the IO bus. Events setting this counter > > cause RXO, the receiver overrun interrupt, to be set. This register > > does not increment if receives are not enabled. > > > > 10.2.4.1 Interrupt Cause Read Register - ICR (0x000C0; RC/WC) > > RXO Receiver Overrun > > Set on receive data FIFO overrun. Could be caused either because > > there are no available buffers or because PCIe receive bandwidth is > > inadequate. > > > > So, first one is a count and another one is an interrupt. Are these 2 > > related? Both seem to be happen when on card FIFO gets full. We see no > > evidence of RX queue on the host being full based on > > dev.em.0.mac_stats.recv_no_buff. > > > > Many a times we see missed_packets increasing without rx_overrun > > changing. > > > > The spec says there is a 40KB buffer on card which seems to be used by > > both RX and TX? Is is split between them for 20KB each? OR is it > > possible that when we are doing high rate TX, we use up that buffer and > > RX suffers from that? So, I found some seemingly useful things in the spec: 10.2.12 Diagnostic Register Descriptions I tried to read on-chip FIFO head/tail to make a sense of what we have there when we see mpc (missed packet count) increase. In em_update_stats_counters() whenever I see mpc increasing, I go print following registers. fhead = E1000_READ_REG(&adapter->hw, E1000_RDFH); ftail = E1000_READ_REG(&adapter->hw, E1000_RDFT); fheads = E1000_READ_REG(&adapter->hw, E1000_RDFHS); ftails = E1000_READ_REG(&adapter->hw, E1000_RDFTS); fcount = E1000_READ_REG(&adapter->hw, E1000_RDFPC); But I do not see any pattern here. Some times I see a large differenec between head and tail and still no mpc increase and sometimes mpc++ when head/tail are following eachother closely. Also, fcount E1000_RDFPC which is supposed to be "The number of received packets currently in the RX FIFO." is always 0 OR a moderately low number. That doesn't make sense to me. If FIFO is not full why do we get so many missed packets? cheers, Hiren [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQF8BAEBCgBmBQJVG1GPXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lLSYH/27M2X+5Kj3YCgPH5kEWMyts +kO7dOrPPLa1VOR1qC++M1SE3+0N9m7TSSAXe0NLretU1ln+v+2G94xAOoRh7IXl p+07fO/zMgqwsKVjzsahkSC7c8L/EdXgWd5kT5XKtuV6PyONgXUVTcaFFwDgJ74v g/w6CgsqVURKgH+n/uteG77VCZkrKzQ/HIvUMKOnjXw53HmbwdeAF3SmH7W2Hslq EeyYdH3lmxxWjiQW3Yb/JpXP6Ig2JiElsvjrsPLopKkVk9TKhigWQT5RDjOHtTmc HVz0kCRVhXUQbG5DDGlShnfig+TVJPmAmpmGxgkCgHdR38BEY+KX1peGICISZoU= =2bW7 -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150401020152.GM10892>