From owner-freebsd-net@FreeBSD.ORG Tue Aug 18 22:16:07 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76F9D106568B for ; Tue, 18 Aug 2009 22:16:07 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63907.mail.re1.yahoo.com (web63907.mail.re1.yahoo.com [69.147.97.122]) by mx1.freebsd.org (Postfix) with SMTP id 30D6D8FC6B for ; Tue, 18 Aug 2009 22:16:06 +0000 (UTC) Received: (qmail 52563 invoked by uid 60001); 18 Aug 2009 22:16:06 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1250633766; bh=JIhdzu6PSvZHVTDDszqd1jHyLPaIIM6s9MUXLm3h0vQ=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=cmeVSnKVFqYOde5wfu+iEoksTZXy3/MyLNjKvh3rZywmKDoMsZzr2EtOOMvrHmmjIaC0vUcYBpPtpq5KFvfBMTygn9Gk+wf8Vtz/xH1viDLh799rRl8c2YqydI4X/Pmxo7xaP3E9leuywRvgYumzV8hl74umnAqC542EJAPRxLQ= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=DqaibM4/mlFo0f69sxX9uYv05w/HECrRuhzM9JqJdzzWdsw5GIpWFJL0BKisGMMXOQk7o1KTKZeIIDSk87QWp7rsWMfS96/YWXrPZDbd0XjHRo+bANEO8PZ2KZLW03vtKGNbo6jZT7PakFa6oPsPCQSBFKTpXSb38/fuLHgcGKw=; Message-ID: <373149.52091.qm@web63907.mail.re1.yahoo.com> X-YMail-OSG: JwOWzIAVM1nL_S3QvTKG5grU2hdWiMAGXiW9rcxg_k5bm4PckhBfvCgckyWz8hNiyDzCCno2Tpq.WZ2qSPUUEp4jDjlRw4sLdRCE8KxX9DhLRY35aVJd.MOg7onGahvOdyJfDD7RW_GyiVoU578K2_mX50W07ZzJ0kKbvtKDpUPEWwSfYxSCyl7I7WnIzym1hlanhe3ObKVjV4h6ucZAA3BG9rED9cjOaRGlhfWqQZbrFZDsrip1igWx72eNW0Ha4kuoKp04njmkQsxNhKqFYDUgG6k- Received: from [66.176.162.245] by web63907.mail.re1.yahoo.com via HTTP; Tue, 18 Aug 2009 15:16:06 PDT X-Mailer: YahooMailClassic/6.1.2 YahooMailWebService/0.7.338.2 Date: Tue, 18 Aug 2009 15:16:06 -0700 (PDT) From: Barney Cordoba To: freebsd-net@freebsd.org, Manish Vachharajani In-Reply-To: <5bc218350908171524m5a46c3dbm3e6af625c51370d0@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Subject: Re: Dropped vs. missed packets in the ixgbe driver X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2009 22:16:07 -0000 =0A=0A--- On Mon, 8/17/09, Manish Vachharajani wrote:=0A=0A> From: Manish Vachharajani =0A>= Subject: Dropped vs. missed packets in the ixgbe driver=0A> To: freebsd-ne= t@freebsd.org=0A> Date: Monday, August 17, 2009, 6:24 PM=0A> I've been doin= g some performance=0A> testing on freebsd 7.2 and noticed=0A> that the ixgb= e driver does not report missed packets as=0A> dropped when=0A> queried via= netstat -id (the ixgbe driver in the 8.0=A0=0A> beta has a=0A> similar iss= ue).=A0 A missed packet is a packet that was=0A> correctly=0A> received by = the NIC but because it was out of descriptors=0A> and internal=0A> memory, = the packet had to be dropped by the NIC itself -- a=0A> hardware=0A> regist= er counts such events.=A0 Instead the driver only=0A> reports drops=0A> tha= t are due to the driver itself, i.e., the NIC DMAed the=0A> packet to=0A> m= emory but the driver had to drop something because it was=0A> out of=0A> mb= ufs.=A0 Is the miss count reported elsewhere (besides=0A> via a kernel=0A> = printf from the driver)?=A0 At the end of the email I=0A> give a stats dump= =0A> from the driver and from netstat, note the number of=0A> packets dropp= ed=0A> is 0 as reported by netstat but the Missed field in the=0A> dmesg ou= tput=0A> shows many missed packets.=0A> =0A> >From my perspective, it is di= sconcerting to see=0A> performance=0A> degradation on the link, along with = TCP ack retransmits,=0A> packet=0A> reordering, etc. (on a point-to-point l= ink with no switch=0A> in between)=0A> but then see no drops reported by ne= tstat because the=0A> driver didn't=0A> drop the packet, the NIC did.=A0 Th= e fix should be=0A> straight-forward and=0A> I'll gladly make a patch assum= ing that it is indeed a bug=0A> and not a=0A> conscious design choice.=0A> = =0A> Here is the relevant netstat output=0A> =0A> Name=A0 =A0 Mtu Network= =A0 =A0=0A> =A0=A0=A0Address=A0 =A0 =A0 =A0 =A0=0A> =A0 =A0 Ipkts Ierrs=A0 = =A0 Opkts=0A> Oerrs=A0 Coll Drop=0A> ix0=A0 =A0 1500 =A0 =A0 =A0=0A= > 00:30:48:94:60:ec=A0 =A0 =A0 =A0 0=A0=0A> =A0=A0=A00=A0 =A0 =A0 =A0 1=0A>= 0=A0 =A0=A0=A00=A0 =A0 0=0A> ix0=A0 =A0 1500 192.168.105.0 192.168.105.2= =A0=0A> =A0 =A0 =A0 =A0 =A0 0=A0=0A> =A0=A0=A0-=A0 =A0 =A0 =A0 0=0A> -=A0 = =A0=A0=A0-=A0 =A0 -=0A> ix1=A0 =A0 1500 =A0 =A0 =A0=0A> 00:30:48:94= :60:ed=A0 =A0 =A0 11M=A0=0A> =A0=A0=A00=A0 =A0=A0=A06.1M=0A> 0=A0 =A0=A0= =A00=A0 =A0 0=0A> ix1=A0 =A0 1500=0A> 192.168.5.0=A0=A0=A0192.168.5.2=A0 = =A0 =A0=0A> =A0 =A0 =A0 10M=A0 =A0=A0=A0-=A0=0A> =A0=A0=A06.1M=0A> -=A0 = =A0=A0=A0-=A0 =A0 -=0A> =0A> And here is the dmesg output after doing a sys= ctl=0A> dev.ix.1.stats=3D1=0A> =0A> ix1: Std Mbuf Failed =3D 0=0A> ix1: Mis= sed Packets =3D 413872=0A> ix1: Receive length errors =3D 0=0A> ix1: Crc er= rors =3D 0=0A> ix1: Driver dropped packets =3D 0=0A> ix1: watchdog timeouts= =3D 0=0A> ix1: XON Rcvd =3D 616428212235=0A> ix1: XON Xmtd =3D 0=0A> ix1: = XOFF Rcvd =3D 616428212235=0A> ix1: XOFF Xmtd =3D 0=0A> ix1: Total Packets = Rcvd =3D 12424533=0A> ix1: Good Packets Rcvd =3D 12010661=0A> ix1: Good Pac= kets Xmtd =3D 6419128=0A> ix1: TSO Transmissions =3D 0=0A> =0A> Manish=0A= =0Athe debug sysctl show more interesting info. Don't get too excited=0Aabo= ut doing performance testing with that driver. Its not designed=0Ato be any= higher in performance than any of the other intel drivers.=0A=0ABarney=0A= =0A=0A