From owner-freebsd-net@FreeBSD.ORG Sun Jan 3 17:52:02 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 01891106566B for ; Sun, 3 Jan 2010 17:52:02 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63902.mail.re1.yahoo.com (web63902.mail.re1.yahoo.com [69.147.97.117]) by mx1.freebsd.org (Postfix) with SMTP id A0DB38FC12 for ; Sun, 3 Jan 2010 17:52:00 +0000 (UTC) Received: (qmail 31450 invoked by uid 60001); 3 Jan 2010 17:45:17 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1262540717; bh=uyW8xGexs6lkREbp7q+XA6NO5F5SPDtxCHZ8Vuo71hU=; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=XmtORA8ZXruWXc+vrKYtf39dYUD386++oFel+rG8VVvpxfcv3i7pqNTm6sfCqsDJGAccrnyVyNBmyf/ShxMdOtO1zMlSGbThoW2Jsgq8hf9zm1n0QZjUKytLri5MeOJJqkaKwhIYG+UHXfsfeSC63Vd4OZdMVwHNoGn2Pcu1Bkc= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=2VeIhlyLgsyx+2fP8pWc5b6SYZqyMIItWZb2N/fGsxiRcn+AKBqWwHhKxbauhsRz6Fo4GnZnFLzddP4CRSLetKmz+q1KvLAEaO1ZtdqyTyAcsLGti57IO2CDFS+4Sna8eJoCrh6QsPcnVE1kGd+v3WzhK90xBOzLFnID5SOuRo8=; Message-ID: <256746.31214.qm@web63902.mail.re1.yahoo.com> X-YMail-OSG: xgV41CQVM1kh0ZBk2yDWMukX1umdbMmCDWJzujXZekQ5ookTnafVxJU3.KtvsYft441xs77SERgqLJ2oNbvYYFnuB3OaYmhXBYR0ig2sPVXhrxggdCO0JKnOdQcvC1H9pabRlsojxhjX8HUvBbG1_OKWmGZ6O6oM7pk5c1miymNtO5nK.hGFSCqTxRsJ9waz32cQeyolFNArySsBSC7GjM3eL9sSuIPmunwiNWszlZPtowJlAI6xIduWn2i4ZccsI_soKJr7Zx2HganMCx9pFfdYfES65zBFK5U6j3CNNc0Vr8doJ6lOAJx_6yg- Received: from [98.203.21.152] by web63902.mail.re1.yahoo.com via HTTP; Sun, 03 Jan 2010 09:45:17 PST X-Mailer: YahooMailClassic/9.0.20 YahooMailWebService/0.8.100.260964 Date: Sun, 3 Jan 2010 09:45:17 -0800 (PST) From: Barney Cordoba To: =?iso-8859-1?Q?Michael_T=FCxen?= In-Reply-To: <269879E8-0B26-4D9E-82B3-809401AA7F6E@lurchi.franken.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, Mike Tancsa Subject: Re: igb interrupt moderation X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Jan 2010 17:52:02 -0000 =0A=0A--- On Sun, 1/3/10, Michael T=FCxen wrote:=0A=0A> From: Michael T=FCxen =0A= > Subject: Re: igb interrupt moderation=0A> To: "Barney Cordoba" =0A> Cc: "Mike Tancsa" , freebsd-net@freeb= sd.org=0A> Date: Sunday, January 3, 2010, 12:14 PM=0A> On Jan 3, 2010, at 6= :00 PM, Barney=0A> Cordoba wrote:=0A> =0A> > =0A> > =0A> > --- On Sun, 1/3/= 10, Michael T=FCxen =0A> wrote:=0A> > =0A= > >> From: Michael T=FCxen =0A> >> Subjec= t: Re: igb interrupt moderation=0A> >> To: "Mike Tancsa" = =0A> >> Cc: "Barney Cordoba" ,=0A> jfvogel@gmail.= com,=0A> freebsd-net@freebsd.org=0A> >> Date: Sunday, January 3, 2010, 11:3= 8 AM=0A> >> On Jan 3, 2010, at 5:23 PM, Mike=0A> >> Tancsa wrote:=0A> >> = =0A> >>> At 11:13 AM 1/3/2010, Michael T=FCxen wrote:=0A> >>>>> =0A> >>>>> = Just a separate datapoint about this=0A> driver,=0A> >> unless I apply=0A> = >>>>> =0A> >>>>> http://people.freebsd.org/~yongari/igb/igb.buf.patch6=0A> = >>>>> =0A> >>>>> the driver is not really usable for me=0A> in=0A> >> RELEN= G_8 on the dual port version of the card=0A> >>>> Could you elaborate on wh= at you mean by=0A> "not=0A> >> really usable"?=0A> >>> =0A> >>> =0A> >>> Hi= ,=0A> >>>=A0 =A0 =A0 =A0=A0=A0Some=0A> link state issues=0A> >> (getting co= nfused about what port is up), problems=0A> at high=0A> >> packet rates.=A0= I dont have this card in=0A> production, but=0A> >> in my test environment= it was much more stable on=0A> RELENG_8=0A> >> with the above patch in tha= t I was not able to=0A> wedge the=0A> >> box.=A0 pps rates were pretty ok o= n a low end=0A> i7 as=0A> >> well.=0A> >> Thanks for the information. I'll = give it a try. I=0A> have a=0A> >> problem when I flood=0A> >> a system wit= h SCTP INITs. The system under attack=0A> becomes=0A> >> completely unrespo= nsive=0A> >> on the console. However, it continues to send=0A> INIT-ACKs=0A= > >> back. After the last=0A> >> commit from Jack it recovers after the att= ack. Not=0A> yet sure=0A> >> what is going on.=0A> >> Using the em driver d= oes not have the problem.=0A> However,=0A> >> when using the em=0A> >> driv= er only one core is fully used, when using the=0A> igb=0A> >> driver both c= ores are fully=0A> >> used. Unfortunately I do not have a more than dual=0A= > core=0A> >> machine available for=0A> >> this testing...=0A> > =0A> > Try= em and lower the interrupt moderation to something=0A> like 500 (about=0A>= > 100 packets per int is good). The latency isn't going=0A> to be noticabl= e and=0A> > you'll see your cpu burden reduced quite a bit. =0A> I'll try. = Thanks.=0A> > =0A> > Are you using a single NIC on a server, or do you have= =0A> a firewall or=0A> > bridge?=0A> The system is a sender/receiver for SC= TP. I'm interested in=0A> the 82576=0A> since it provides checksum offloadi= ng for it. I use one or=0A> two ports=0A> for simultaneous data transfer. T= he cards using the em=0A> driver do=0A> not support this feature. So I'm tr= ying to verify that the=0A> performance=0A> goes up when using hardware che= cksum. But under attack,=0A> this is currently=0A> not the case... =0A> > = =0A> > Barney=0A=0AAre you using just 1 queue? Just because you're using bo= th cpus=0Adoesn't mean its efficient. The 8257x has separate interrupts fo= r =0Atransmit and receive, so 1 queue will be a closer match to the em=0Adr= iver so you can gauge if the offload is effective. I don't know how=0Afar j= ack has gotten in addressing the lock contention issue in igb.=0AObviously,= try all scenarios. What seems obvious rarely plays out in=0Apractice.=0A= =0ABarney=0A=0A=0A