From owner-freebsd-net@FreeBSD.ORG Mon Jan 4 16:02:20 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9081A1065696 for ; Mon, 4 Jan 2010 16:02:20 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63906.mail.re1.yahoo.com (web63906.mail.re1.yahoo.com [69.147.97.121]) by mx1.freebsd.org (Postfix) with SMTP id 48E688FC1D for ; Mon, 4 Jan 2010 16:02:19 +0000 (UTC) Received: (qmail 55121 invoked by uid 60001); 4 Jan 2010 16:02:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1262620933; bh=fKWE5aqtKvKV+fBj6vsHzmG7CGODWDglZaL1v+fo/RU=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=TMX9ZxGR3+Am7+zg7wBogZ7WA5yiC7Mq+yB4Y3601aj2/sOppxTcFT+25HJAEZ81ywQKmedrhYFzbtdAaPzp72d3s7punc+PCY0GBFJDMDeNQkRxFBvCMXypdoUrdp9pcIUsSQ/3jDUtaKp7jogEkxvIk0tbvgv3jV91R0IYKCM= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=gS+xKbkXHfX6x4dBUdNxst7Jq0KaDQKDH20UMwwxUABpw3oC9i7EONpmFC1FtKdPc6guBOWDM+HCdRvFMMIgDfb9+0GPm0Um8XudDV1TLJosZ9j1lzxJZbYMhH68o3bwhUyqMtpBSIaJivGp9m6+Uj6OxHe/ofZgJcf/3UgQpwM=; Message-ID: <935818.54864.qm@web63906.mail.re1.yahoo.com> X-YMail-OSG: vsMt6KcVM1nsThm3p1d9ZSidZQ.B4qFov1pt2c4VyUiJ7QasezrGl3vnhQFnDlmCYdBUJAkFMQmToybBciTTVs_puH7lXsOa8klWyyOh5HoiKZNMXDQH0XKkWfJdWtRZHiUmbYOKKYTZpjxPiHfpWkwVs_KL4_eOZRDNZW_._stVF4tnGvkUbRBER3OyQhY2R5oUM7WmF7xQFoeykCEM99RGDRyLXWcerpqS2NaQzUewsCD772sXuc8YO1DVWRsTMvLoMhOaYMMSzwdm1WNioC1hue5hiWaSmJoopbpDntg9kaXyMqw- Received: from [98.203.21.152] by web63906.mail.re1.yahoo.com via HTTP; Mon, 04 Jan 2010 08:02:13 PST X-Mailer: YahooMailClassic/9.0.20 YahooMailWebService/0.8.100.260964 Date: Mon, 4 Jan 2010 08:02:13 -0800 (PST) From: Barney Cordoba To: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <133FB4C8-AF96-4CBC-8291-2325592AB628@lurchi.franken.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, jfvogel@gmail.com Subject: Re: igb interrupt moderation X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Jan 2010 16:02:20 -0000 =0A=0A--- On Sun, 1/3/10, Michael T=FCxen wrote:=0A=0A> From: Michael T=FCxen =0A= > Subject: Re: igb interrupt moderation=0A> To: "Michael T=FCxen" =0A> Cc: "Barney Cordoba" , freebsd-net@freebsd.org, jfvogel@gmail.com=0A> Date: Sunday, January 3,= 2010, 9:54 AM=0A> Dear all,=0A> =0A> I just figured out that there is a ne= wer version of the=0A> spec=0A> available: 2.45. Some of the issues as indi= cated inline=0A> are already resolved. =0A> =0A> Best regards=0A> Michael= =0A> =0A> On Jan 3, 2010, at 2:55 PM, Michael T=FCxen wrote:=0A> =0A> > Hi = Barney, Hi Jack,=0A> > =0A> > some comments and some more questions inside.= ..=0A> > =0A> > Best regards=0A> > Michael=0A> > =0A> > On Jan 2, 2010, at = 8:42 PM, Barney Cordoba wrote:=0A> > =0A> >> Jack,=0A> >> =0A> >> I'm tryin= g to get some clarification on=0A> differences I'm finding between=0A> >> t= he 82575 and 82576 parts with respect to=0A> interrupt moderation. The spec= =0A> >> I have for the 82576 (82576_Datasheet_v2p1.pdf)=0A> indicates that = the =0A> > I'm only commenting 82576. You can get rev 2.41 from=0A> intels = website...=0A> It is really 2.45 now=0A> >> =0A> >> ITR algorithm is differ= ent than the one used (I=0A> don't have one of the=0A> >> secret copies of = the 82575 spec). The algorithm=0A> shown is=0A> >> =0A> >> interrupts/sec = =3D 1/(2 * 10-6sec x interval) (page=0A> 295, Section 7.3.4)=0A> >> =0A> >>= which is clearly wrong from practice. I have an=0A> 82576 (device id 10C9)= =0A> > If you look at section 8.8.12, you find other=0A> formulas...=0A> > = Jack: Which ones are correct?=0A> The formulas is 8.8.12 are gone. Issue re= solved.=0A> >> if I use the 125d setting in the example get just=0A> under = 32000 interrupts=0A> >> per second. Clearly your code doesnt implement=0A> = this, nor do you have=0A> >> different settings for the 82575 and 82576 par= ts.=0A> So I assume that the =0A> >> same formula for the em parts hold for= the igb=0A> parts, and that the =0A> >> datasheet is wrong?=0A> >> =0A> >>= There does seem to be a slight difference. The=0A> setting that gets 1000= =0A> >> ints/second on the 82575 generates about 1020 on=0A> the 82576. Not= a big=0A> >> deal but I wonder why there's a difference? Is the=0A> refere= nce clock for=0A> >> these something that may not be fixed and could=0A> va= ry from board to =0A> >> board? Note that both devices are on the same MB.= =0A> >> =0A> >> Also, it seems that settings to EITR over 32767=0A> wrap on= the 82576 (for=0A> >> example writing 32768 to EITR is the same as=0A> wri= ting a 1). So the=A0 minimum setting on the 82576 is=0A> around 125 ints/se= cond. The 82575 can accept =0A> >> values up the 65535 before wrapping. =0A= > > Hmm, looking at the table in 8.8.12 would suggest:=0A> > Setting it to = one sets a reserved bit, but does not=0A> change the interval.=0A> > Settin= g it to 2^15 should set the LLI_EN bit, but does=0A> not change in interval= .=0A> > =0A> > Jack is setting the register to=0A> > igb_low_latency: 128= =0A> > igb_ave_latency: 450=0A> > igb_bulk_latency: 1200=0A> > =0A> > This = would result in intervals of:=0A> > igb_low_latency: 32=0A> > igb_ave_laten= cy: 112=0A> > igb_bulk_latency: 300=0A> > Jack: What are the corresponding = interrupt rates? The=0A> spec provides different=0A> >=A0 =A0 =A0 formulas = and talks about a 1us,=0A> 2us or 8us counter. Not sure what is right...=0A= > The interrupt rates are according to the formula in the=0A> spec=0A> igb_= low_latency: 31250=0A> igb_ave_latency: 8929=0A> igb_bulk_latency: 3333=0A>= Jack: Is this right?=0A> > Jack: Why are you setting bit1 (which is reserv= ed) in=0A> the case igb_ave_latency?=0A> Still valid.=0A> > =0A> > And anot= her question for Jack:=0A> > In igb_update_aim() you do=0A> > =A0=A0=A0 if = (olditr !=3D newitr) {=0A> > =A0=A0=A0 =A0=A0=A0 /* Change=0A> interrupt ra= te */=0A> > =A0=A0=A0 =A0=A0=A0=0A> rxr->eitr_setting =3D newitr;=0A> > =A0= =A0=A0 =A0=A0=A0=0A> E1000_WRITE_REG(&adapter->hw,=0A> E1000_EITR(rxr->me),= =0A> > =A0=A0=A0 =A0=A0=A0 =A0 =A0=0A> newitr | (newitr << 16));=0A> > =A0= =A0=A0 }=0A> > So why are setting the higher bits of the EITR? You=0A> are = setting=0A> > igb_low_latency: the LL Counter becomes 0, the=0A> moderation= counter becomes 16=0A> > igb_ave_latency: the LL Counter becomes 2, the=0A= > moderation counter becomes 56=0A> > igb_bulk_latency: the LL Counter beco= mes 16, the=0A> moderation counter becomes 148=0A> Still valid.=0A> > =0A> = > I really do not understand these settings. Maybe the=0A> spec is wrong? O= r you do mean=0A> > =A0=A0=A0 if (olditr !=3D newitr) {=0A> > =A0=A0=A0 =A0= =A0=A0 /* Change=0A> interrupt rate */=0A> > =A0=A0=A0 =A0=A0=A0=0A> rxr->e= itr_setting =3D newitr;=0A> > =A0=A0=A0 =A0=A0=A0=0A> E1000_WRITE_REG(&adap= ter->hw, E1000_EITR(rxr->me),=0A> newitr);=0A> > =A0=A0=A0 }=0A> > Or do yo= u want to preserve the counters, set the=0A> CNT_INGR bit and mean=0A> > = =A0=A0=A0 if (olditr !=3D newitr) {=0A> > =A0=A0=A0 =A0=A0=A0 /* Change=0A>= interrupt rate */=0A> > =A0=A0=A0 =A0=A0=A0=0A> rxr->eitr_setting =3D newi= tr;=0A> > =A0=A0=A0 =A0=A0=A0=0A> E1000_WRITE_REG(&adapter->hw, E1000_EITR(= rxr->me),=0A> 0x80000000 | newitr);=0A> > =A0=A0=A0 }=0A> > =0A> > Could yo= u clarify that?=0A> Still valid.=0A> >> =0A> >> The 82576 document doesn't = have a map of the=0A> register that I can find, so=0A> >> Im curious as to = whether these observations are=0A> something I can assume is=0A> >> true ac= ross all parts and motherboards/cards, or=0A> is there some=0A> >> implemen= tation variance that will cause these to=0A> only apply to the ones=0A> >> = I happen to be testing?=0A> >> =0A> >> Thanks,=0A> >> =0A> >> Barney=0A> >>= =0A> >> =0A> >> =0A> >> =0A=0AJust to add a question to this for Jack. Can= you force a close to =0Aimmediate interrupt by writing a low value to the = counter in the =0AEITR register? =0A=0AWhat happens when you write a zero? = Does it just restart, or does=0Ait gen an interrupt?=0A=0ABarney=0A=0A=0A =