From owner-freebsd-net@FreeBSD.ORG Thu Mar 26 20:08:54 2015 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 95F0EEAD; Thu, 26 Mar 2015 20:08:54 +0000 (UTC) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EED4129; Thu, 26 Mar 2015 20:08:54 +0000 (UTC) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPSA id B87EECF218; Thu, 26 Mar 2015 13:08:53 -0700 (PDT) Date: Thu, 26 Mar 2015 13:08:53 -0700 From: hiren panchasara To: freebsd-net@freebsd.org Subject: em(4): difference between missed_packets and rx_overrun Message-ID: <20150326200853.GA19536@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) Cc: nitroboost@gmail.com X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Mar 2015 20:08:54 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable This is what we are seeing on em(4) 82574L chipset running stable/10: dev.em.0.mac_stats.missed_packets: 1441927 dev.em.0.interrupts.rx_overrun: 153 =46rom the datasheet: http://www.intel.com/content/www/us/en/ethernet-controllers/82574l-gbe-cont= roller-datasheet.html 10.2.7.4 Missed Packets Count - MPC (0x04010; R) Counts the number of missed packets. Packets are missed when the receive FIFO has insufficient space to store the incoming packet. This could be caused because of too few buffers allocated, or because there is insufficient bandwidth on the IO bus. Events setting this counter cause RXO, the receiver overrun interrupt, to be set. This register does not increment if receives are not enabled. 10.2.4.1 Interrupt Cause Read Register - ICR (0x000C0; RC/WC) RXO Receiver Overrun Set on receive data FIFO overrun. Could be caused either because there are no available buffers or because PCIe receive bandwidth is inadequate. So, first one is a count and another one is an interrupt. Are these 2 related? Both seem to be happen when on card FIFO gets full. We see no evidence of RX queue on the host being full based on dev.em.0.mac_stats.recv_no_buff. Many a times we see missed_packets increasing without rx_overrun changing. The spec says there is a 40KB buffer on card which seems to be used by both RX and TX? Is is split between them for 20KB each? OR is it possible that when we are doing high rate TX, we use up that buffer and RX suffers from that? Any insights would be helpful to understand the problem. Cheers, Hiren --u3/rZRmxL6MmkK24 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (FreeBSD) iQF8BAEBCgBmBQJVFGdUXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/l8QoH/3xKvDx9inKcwiPW1authYpw P/o7TCALanXNp2RyRjSdLnKr1EU4Kv6Twh1qlSun3N9JuxQbVdRCJiF6bAKsdeMm uvWXFOIOCy1rBbctiVvXUXgPMIEOhywNr7nbdEILV/dFpBMkhGxr9bZPtE7j88cK 0sX6sO8HLE1b94s/SufMMr/cvJr4m3GbNlSxcq2NjUUKafXJohmVaXfJcp9nXRPz 148FUCvLL5/DbatzOyg1UQOXItOk2QghIouNcRhd0ls7yTU4BjGDL2z/c/dFOvfM OV+7jl398uy1k6XnsGDX+TmGunajtIHCPQz4gwV5CJQ0Qq/UqrPs0neBPebUGXo= =09Jg -----END PGP SIGNATURE----- --u3/rZRmxL6MmkK24--