From owner-freebsd-net@FreeBSD.ORG Fri Mar 29 16:27:54 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DD9DDFF2 for ; Fri, 29 Mar 2013 16:27:54 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from nm26-vm1.bullet.mail.ne1.yahoo.com (nm26-vm1.bullet.mail.ne1.yahoo.com [98.138.91.61]) by mx1.freebsd.org (Postfix) with ESMTP id A9C03E6F for ; Fri, 29 Mar 2013 16:27:54 +0000 (UTC) Received: from [98.138.226.180] by nm26.bullet.mail.ne1.yahoo.com with NNFMP; 29 Mar 2013 16:25:09 -0000 Received: from [98.138.89.246] by tm15.bullet.mail.ne1.yahoo.com with NNFMP; 29 Mar 2013 16:25:09 -0000 Received: from [127.0.0.1] by omp1060.mail.ne1.yahoo.com with NNFMP; 29 Mar 2013 16:25:09 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 639782.61289.bm@omp1060.mail.ne1.yahoo.com Received: (qmail 80213 invoked by uid 60001); 29 Mar 2013 16:25:09 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1364574309; bh=ImxqLC5IhVgH/+8U/jz8FlGWLkSjo6fsSPGoJUE1uwI=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=5UwvG41h0wKNzN+3ABM6CIvA3+2PApxwHJ4vJ4Cbs5tYpJ2VMxnqCj32UEm1MQQiL/m4FUYfZPPo42zgg5CLA0p/DEJCvKAJ6b/uc1uwUFI2B9eDVlLgvuE3QTH3oK0ndBUv9TEfZfzWoYTq86pA9pY+hleeJDP3IP6EL7pV6Do= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type; b=mcI3OUaO2I1QSES9cpNFqy6rv4k8++zvXEXPj8RBWpx3gzeE5UZdw8BeMZHWjLCXiTHkybf7a4ZfggudaUIbp767b1fUttpRH1I4X3th0wlrWshJygQin0SBkB39w0m2VLQmjYnXyp1lKcGqWjD3XbXtM+8gaR4Do73PjWwkZBI=; X-YMail-OSG: c7Az.DoVM1n.AHAf810eVHs9eSjnYyTYv8plQkb4tVZBDh9 jprumm5.pSPpU3WmmA6_.PQgdarpChTzSvfXUgyNDXZSJRfzoTe73yvQR4mi UHs7B7Nnkc.h2FlhLpZ.TCz3UEbtiIw7xVNmK1bpSGgVRc2rsalEJH6IHpCf ui6Rgp8jJF09VKvcoVNp7dxUDDyvIm_qS2hA5wKWb6aJ4TMGCorl8odTaZeg dfGieCrnQkQAh_oPbyDT.jAzc22k.yCUun3KPXNDoOUfHg4TK4zJaxRD6NO0 I2dwH5OvKsNG728QYD9lRH3.ioVOUFoKje8mx3u_7R1ctOa_9TSq6mQCrPY4 op5rzYca5a0_h0X10oplxUP0X.B.zuVof0BPEzeZoYWsB1P1_BuwcINea21A YiY2gGxFJ2uU5p8q4hKh4WCrhijfSl7MP8fbWy8Fispxhw6UDVhyp4E5jYtZ FFfSUTCKt4_9PdqM47jjadJ50iN5w1oc3E8gUGQ4sjRHSGpxYdeA4R8CHxb0 T31qrbvUIBvV8cNPo7XVaizhjZXRPKQ-- Received: from [98.203.118.124] by web121605.mail.ne1.yahoo.com via HTTP; Fri, 29 Mar 2013 09:25:09 PDT X-Rocket-MIMEInfo: 002.001, CgotLS0gT24gRnJpLCAzLzI5LzEzLCBQaWVwZXIsIEplZmZyZXkgRSA8amVmZnJleS5lLnBpZXBlckBpbnRlbC5jb20.IHdyb3RlOgoKPiBGcm9tOiBQaWVwZXIsIEplZmZyZXkgRSA8amVmZnJleS5lLnBpZXBlckBpbnRlbC5jb20.Cj4gU3ViamVjdDogUkU6IGlnYiBhbmQgQUxUUSBpbiA5LjEtcmMzCj4gVG86ICJCYXJuZXkgQ29yZG9iYSIgPGJhcm5leV9jb3Jkb2JhQHlhaG9vLmNvbT4sICJKYWNrIFZvZ2VsIiA8amZ2b2dlbEBnbWFpbC5jb20.LCAiTmljayBSb2dlcnMiIDxuY3JvZ2Vyc0BnbWFpbC5jb20BMAEBAQE- X-Mailer: YahooMailClassic/15.1.7 YahooMailWebService/0.8.139.530 Message-ID: <1364574309.74010.YahooMailClassic@web121605.mail.ne1.yahoo.com> Date: Fri, 29 Mar 2013 09:25:09 -0700 (PDT) From: Barney Cordoba Subject: RE: igb and ALTQ in 9.1-rc3 To: Jack Vogel , Nick Rogers , Jeffrey EPieper In-Reply-To: <2A35EA60C3C77D438915767F458D65687D46B19E@ORSMSX101.amr.corp.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: "freebsd-net@freebsd.org" , "Clement Hermann \(nodens\)" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Mar 2013 16:27:54 -0000 --- On Fri, 3/29/13, Pieper, Jeffrey E wrote: > From: Pieper, Jeffrey E > Subject: RE: igb and ALTQ in 9.1-rc3 > To: "Barney Cordoba" , "Jack Vogel" , "Nick Rogers" > Cc: "freebsd-net@freebsd.org" , "Clement Hermann (nodens)" > Date: Friday, March 29, 2013, 11:45 AM > > > -----Original Message----- > From: owner-freebsd-net@freebsd.org > [mailto:owner-freebsd-net@freebsd.org] > On Behalf Of Barney Cordoba > Sent: Friday, March 29, 2013 5:51 AM > To: Jack Vogel; Nick Rogers > Cc: freebsd-net@freebsd.org; > Clement Hermann (nodens) > Subject: Re: igb and ALTQ in 9.1-rc3 > > > > --- On Thu, 3/28/13, Nick Rogers > wrote: > > > From: Nick Rogers > > Subject: Re: igb and ALTQ in 9.1-rc3 > > To: "Jack Vogel" > > Cc: "Barney Cordoba" , > "Clement Hermann (nodens)" , > "freebsd-net@freebsd.org" > > > Date: Thursday, March 28, 2013, 9:29 PM > > On Thu, Mar 28, 2013 at 4:16 PM, Jack > > Vogel > > wrote: > > > Have been kept fairly busy with other matters, > one > > thing I could do short > > > term is > > > change the defines in igb the way I did in the em > > driver so you could still > > > define > > > the older if_start entry. Right now those are > based on > > OS version and so you > > > will > > > automatically get if_transmit, but I could change > it to > > be IGB_LEGACY_TX or > > > so, > > > and that could be defined in the Makefile. > > > > > > Would this help? > > > > I'm currently using ALTQ successfully with the em > driver, so > > if igb > > behaved the same with respect to using if_start instead > of > > if_transmit > > when ALTQ is in play, that would be great. I do not > > completely > > understand the change you propose as I am not very > familiar > > with the > > driver internals. Any kind of patch or extra > > Makefile/make.conf > > definition that would allow me to build a 9-STABLE > kernel > > with an igb > > driver that works again with ALTQ, ASAP, would be much > > appreciated. > > > > > > > > Jack > > > > > > > > > > > > On Thu, Mar 28, 2013 at 2:31 PM, Nick Rogers > > > wrote: > > >> > > >> On Tue, Dec 11, 2012 at 1:09 AM, Jack Vogel > > > wrote: > > >> > On Mon, Dec 10, 2012 at 11:58 PM, Gleb > > Smirnoff > > >> > wrote: > > >> > > > >> >> On Mon, Dec 10, 2012 at 03:31:19PM > -0800, > > Jack Vogel wrote: > > >> >> J> UH, maybe asking the owner of > the > > driver would help :) > > >> >> J> > > >> >> J> ... and no, I've never been > aware of > > doing anything to stop > > >> >> supporting > > >> >> altq > > >> >> J> so you wouldn't see any > commits. If > > there's something in the altq > > >> >> code > > >> >> or > > >> >> J> support (which I have nothing > to do > > with) that caused this no-one > > >> >> informed > > >> >> J> me. > > >> >> > > >> >> Switching from if_start to > if_transmit > > effectively disables ALTQ > > >> >> support. > > >> >> > > >> >> AFAIR, there is some magic > implemented in > > other drivers that makes them > > >> >> modern (that means using > if_transmit), but > > still capable to switch to > > >> >> queueing > > >> >> mode if SIOCADDALTQ was casted upon > them. > > >> >> > > >> >> > > >> > Oh, hmmm, I'll look into the matter after > my > > vacation. > > >> > > > >> > Jack > > >> > > >> Has there been any progress on resolving this > > issue? I recently ran > > >> into this problem upgrading my servers from > 8.3 to > > 9.1-RELEASE and am > > >> wondering what the latest recommendation is. > I've > > used ALTQ and igb > > >> successfully for years and it is unfortunate > it no > > longer works. > > >> Appreciate any advice. > > >> > > > >Do yourself a favor and either get a cheap dual port > 82571 card or > >2 cards and disable the IGB ports. The igb driver is > defective, and until > >they back out the new, untested multi-queue stuff you're > just neutering > >your system trying to use it. > > > >Frankly this project made a huge mistake by moving > forward with multi > >queue just for the sake of saying that you support it; > without having > >any credible plan for implementing it. That nonsense > that Bill Macy did > >should have been tarballed up and deposited in the trash > folder. The > >biggest mess in programming history. > > > >That being said, the solution is not to hack the igb > driver; its to make > >ALTQ if_transmit compatible, which shouldn't be all that > difficult. > > > >BC > > I may be misunderstanding what you are saying, but if the > solution is, as you say "not to hack the igb driver", then > how is it defective in this case? Or are you just directing > vitriol toward Intel? Multi-queue is working fine in igb. > > Jeff It works like crap. Your definition of "works" is that it doesnt crash. Mine is that it works better with multiple queues than with 1, which it doesn't. And if you load a system up, it will blow up with multiqueue before it will with 1 queue. The point of using Multiqueue isn't to exhaust all of the cpus instead of just 2. It's to get past the wall of using only 2 cpus when they're exhausted. The goal is to increase the capacity of the system; not to make it look like you're using more cpus without any analysis of overall system efficiency. That's my definition of defective. The default tuning is wrong also, because is ASSumes that you only have 1 card in the box. So if you have 2 cards, it has 2 rx queues contending for the same cpus, which is exactly the opposite of what you want. There's no vitriol towards Intel. That's what Jack said; that Intel drivers are just samples and they're not optimized. Obviously some drivers are better than others because he's spent more time on them. What I don't like is that nobody ever tells anyone which drivers are good and which suck. If you think that a driver is a driver then you should be doing something else with your life other than networking. The truth is that there is no plan for multiqueue; the idea that spreading things across cpus randomly is better is just plain wrong; in fact there's a strong case to be made to keep things on the same cpu (and in the same cache). Igb is a college project. Using it will make your system slower than using good single queue cards. BC