From owner-freebsd-net@FreeBSD.ORG Sun May 12 00:15:43 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7F70315D for ; Sun, 12 May 2013 00:15:43 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm33-vm2.bullet.mail.ne1.yahoo.com (nm33-vm2.bullet.mail.ne1.yahoo.com [98.138.229.66]) by mx1.freebsd.org (Postfix) with ESMTP id 32AD8E1 for ; Sun, 12 May 2013 00:15:42 +0000 (UTC) Received: from [98.138.226.178] by nm33.bullet.mail.ne1.yahoo.com with NNFMP; 12 May 2013 00:15:35 -0000 Received: from [98.138.89.171] by tm13.bullet.mail.ne1.yahoo.com with NNFMP; 12 May 2013 00:15:35 -0000 Received: from [127.0.0.1] by omp1027.mail.ne1.yahoo.com with NNFMP; 12 May 2013 00:15:35 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 950744.29547.bm@omp1027.mail.ne1.yahoo.com Received: (qmail 50045 invoked by uid 60001); 12 May 2013 00:15:35 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1368317735; bh=NmLpg1FuCFPQBUqo2WX/m6cDJdsoxjsYOornAiJmEXs=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=a0+kZ89acSsbyI2bXxIGdxHVwMDurpXp09Xct4PI2VtIj3Y03N3zBrbPuhctKaDdVhFgz78V7uV0tzhrX8ThMVq6pL97K8Djy3ITsi7z2qdIF5CxLPL79+0vUw/ffS2ZjeQ20MWMtUFycgK9wADWy/UTtBJj7kJ5QcUoed5QvXI= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=m6WzRkwBsz4Wr9QpfUACzNh218UFdt8KbHD5w52ozkCNhIANFn4EupDuQGlx6WLZmzWbzYTFkleoe9LydN8A/IyhvfCzrIcjzPJ2Iv2zP+lnHPUaVWweqhVL+SX7oUXcYz5KlHPeySggwJ7f2Dm2km2NlbKfPfNfRgykbUF0/xM=; X-YMail-OSG: bBh59x4VM1k5jgzl6USm5zbnL1l_6tCpcY3VfyR8P2rZLJW 7UOZQa_.MYOR0jWVMrL6HVApyQjxZLDmKi2LtZ5UrA0w.AJSC2gLLM1yDK73 jZjusVCXMLLEwnVxNAd2oJ6RI3OtA2gKt8oOQRCoCYZmCXgMuhRHzzSuvVXk nVuUFH168Ey3peZS_ga_J6.If8ykF1geITeNWLjB.t62GaBAii1kNcSEnQsW M4hNQ66cldiKZOrl1ckWIB4ZjulMt0vFR.e7MXphOztK5NRwJpp2zA4hYDUm IWVthCAVYESoXgR0uKvi49CW6_mC95QXijzkD_Vvalav9SXa00YMy0hALTkI PMLyTm4eEnbZFOOmUQW7PTg9v8TzRM21sCu_imE0Y8TJqNKhJCmndDC6j9zq XAcXVHAfBl3KzSjoUUDl5W_5KBelmt499dIrDcGHYfEe.ycmIJs0XpHx4TmT 5s0LfJ783MuREjQN.gKRR4P3pbeQwukdx9KF2RtUTzNp4iqEIXjEJGP5Se9_ tUln8pihfQt9HRn23uwQqTi7X3MKozUeX6_uA83SgUw-- Received: from [98.203.118.124] by web121603.mail.ne1.yahoo.com via HTTP; Sat, 11 May 2013 17:15:35 PDT X-Rocket-MIMEInfo: 002.001, CgotLS0gT24gU2F0LCA1LzExLzEzLCBIb29tYW4gRmF6YWVsaSA8aG9vbWFuZmF6YWVsaUBnbWFpbC5jb20.IHdyb3RlOgoKPiBGcm9tOiBIb29tYW4gRmF6YWVsaSA8aG9vbWFuZmF6YWVsaUBnbWFpbC5jb20.Cj4gU3ViamVjdDogUmU6IEhpZ2ggQ1BVIGludGVycnVwdCBsb2FkIG9uIGludGVsIEkzNTBUNCB3aXRoIGlnYiBvbiA4LjMKPiBUbzogIkJhcm5leSBDb3Jkb2JhIiA8YmFybmV5X2NvcmRvYmFAeWFob28uY29tPgo.IENjOiAiRXVnZW5lIEdyb3NiZWluIiA8ZWdyb3NiZWluQHJkdGMucnU.LCBmcmUBMAEBAQE- X-Mailer: YahooMailClassic/15.1.8 YahooMailWebService/0.8.141.536 Message-ID: <1368317735.40898.YahooMailClassic@web121603.mail.ne1.yahoo.com> Date: Sat, 11 May 2013 17:15:35 -0700 (PDT) From: Barney Cordoba Subject: Re: High CPU interrupt load on intel I350T4 with igb on 8.3 To: Hooman Fazaeli In-Reply-To: <518EA643.5010505@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-net@freebsd.org, =?iso-8859-1?Q?Cl=E9ment_Hermann_=28nodens=29?= , Eugene Grosbein X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 May 2013 00:15:43 -0000 =0A=0A--- On Sat, 5/11/13, Hooman Fazaeli wrote:= =0A=0A> From: Hooman Fazaeli =0A> Subject: Re: Hig= h CPU interrupt load on intel I350T4 with igb on 8.3=0A> To: "Barney Cordob= a" =0A> Cc: "Eugene Grosbein" = , freebsd-net@freebsd.org, ""Cl=E9ment Hermann (nodens)"" =0A> Date: Saturday, May 11, 2013, 4:12 PM=0A> On 5/11/2013 8:26 PM, B= arney Cordoba=0A> wrote:=0A> > Clearly you don't understand the problem. Yo= ur logic is=0A> that because other drivers are defective also; therefore it= s=0A> not a driver problem? The problem is caused by a=0A> multi-threaded d= river that=0A> > haphazardly launches tasks and that doesn't manage the=0A>= case that the rest of the system can't handle the load. It's=0A> no differ= ent than a driver that barfs when mbuf clusters are=0A> exhausted. The answ= er=0A> > isn't to increase memory or mbufs, even though that may=0A> allevi= ate the problem. The answer is to fix the driver, so=0A> that it doesn't cr= ash the system for an event that is wholly=0A> predictable. igb has=0A> > 1= ) too many locks and 2) exasperates the problem by=0A> binding to cpus, whi= ch causes it to not only have to wait=0A> for the lock to free, but also fo= r a specific cpu to become=0A> free. So it chugs along=0A> > happily until = it encounters a bottleneck, at which=0A> point it quickly blows up the enti= re system in a domino=0A> effect. It needs to manage locks more efficiently= , and also=0A> to detect when the backup is=0A> > unmanageable. Ever since = FreeBSD 5 the answer has been=0A> "it's fixed in 7, or its fixed in 9, or i= t's fixed in 10".=0A> There will always be bottlenecks, and no driver shoul= d blow=0A> up the system no matter=0A> > what intermediate code may present= a problem. Its the=0A> driver's responsibility to behave and to drop packe= ts if=0A> necessary. BC=0A> =0A> And how the driver should behave? You sugg= est dropping the=0A> packets. Even if we accept=0A> that dropping packets i= s a good strategy in all=0A> configurations (which I doubt), the driver is= =0A> definitely not the best place to implement it, since that=0A> involves= duplication of similar=0A> code between drivers. Somewhere like the Ethern= et layer is a=0A> much better choice to watch=0A> load of packets and drop = them to prevent them to eat all the=0A> cores. Furthermore, ignoring=0A> th= e fact that pf is not optimized for multi-processors and=0A> blaming driver= s for not adjusting=0A> themselves with the this pf's fault, is a bit unfai= r, I=0A> believe.=0A> =0A=0AIt's easier to make excuses than to write a rea= lly good driver. I'll=0Agrant you that.=0ABC