Date: Fri, 29 Mar 2013 09:25:09 -0700 (PDT) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Jack Vogel <jfvogel@gmail.com>, Nick Rogers <ncrogers@gmail.com>, Jeffrey EPieper <jeffrey.e.pieper@intel.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "Clement Hermann \(nodens\)" <nodens2099@gmail.com> Subject: RE: igb and ALTQ in 9.1-rc3 Message-ID: <1364574309.74010.YahooMailClassic@web121605.mail.ne1.yahoo.com> In-Reply-To: <2A35EA60C3C77D438915767F458D65687D46B19E@ORSMSX101.amr.corp.intel.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--- On Fri, 3/29/13, Pieper, Jeffrey E <jeffrey.e.pieper@intel.com> wrote: > From: Pieper, Jeffrey E <jeffrey.e.pieper@intel.com> > Subject: RE: igb and ALTQ in 9.1-rc3 > To: "Barney Cordoba" <barney_cordoba@yahoo.com>, "Jack Vogel" <jfvogel@gmail.com>, "Nick Rogers" <ncrogers@gmail.com> > Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org>, "Clement Hermann (nodens)" <nodens2099@gmail.com> > Date: Friday, March 29, 2013, 11:45 AM > > > -----Original Message----- > From: owner-freebsd-net@freebsd.org > [mailto:owner-freebsd-net@freebsd.org] > On Behalf Of Barney Cordoba > Sent: Friday, March 29, 2013 5:51 AM > To: Jack Vogel; Nick Rogers > Cc: freebsd-net@freebsd.org; > Clement Hermann (nodens) > Subject: Re: igb and ALTQ in 9.1-rc3 > > > > --- On Thu, 3/28/13, Nick Rogers <ncrogers@gmail.com> > wrote: > > > From: Nick Rogers <ncrogers@gmail.com> > > Subject: Re: igb and ALTQ in 9.1-rc3 > > To: "Jack Vogel" <jfvogel@gmail.com> > > Cc: "Barney Cordoba" <barney_cordoba@yahoo.com>, > "Clement Hermann (nodens)" <nodens2099@gmail.com>, > "freebsd-net@freebsd.org" > <freebsd-net@freebsd.org> > > Date: Thursday, March 28, 2013, 9:29 PM > > On Thu, Mar 28, 2013 at 4:16 PM, Jack > > Vogel <jfvogel@gmail.com> > > wrote: > > > Have been kept fairly busy with other matters, > one > > thing I could do short > > > term is > > > change the defines in igb the way I did in the em > > driver so you could still > > > define > > > the older if_start entry. Right now those are > based on > > OS version and so you > > > will > > > automatically get if_transmit, but I could change > it to > > be IGB_LEGACY_TX or > > > so, > > > and that could be defined in the Makefile. > > > > > > Would this help? > > > > I'm currently using ALTQ successfully with the em > driver, so > > if igb > > behaved the same with respect to using if_start instead > of > > if_transmit > > when ALTQ is in play, that would be great. I do not > > completely > > understand the change you propose as I am not very > familiar > > with the > > driver internals. Any kind of patch or extra > > Makefile/make.conf > > definition that would allow me to build a 9-STABLE > kernel > > with an igb > > driver that works again with ALTQ, ASAP, would be much > > appreciated. > > > > > > > > Jack > > > > > > > > > > > > On Thu, Mar 28, 2013 at 2:31 PM, Nick Rogers > <ncrogers@gmail.com> > > wrote: > > >> > > >> On Tue, Dec 11, 2012 at 1:09 AM, Jack Vogel > <jfvogel@gmail.com> > > wrote: > > >> > On Mon, Dec 10, 2012 at 11:58 PM, Gleb > > Smirnoff <glebius@freebsd.org> > > >> > wrote: > > >> > > > >> >> On Mon, Dec 10, 2012 at 03:31:19PM > -0800, > > Jack Vogel wrote: > > >> >> J> UH, maybe asking the owner of > the > > driver would help :) > > >> >> J> > > >> >> J> ... and no, I've never been > aware of > > doing anything to stop > > >> >> supporting > > >> >> altq > > >> >> J> so you wouldn't see any > commits. If > > there's something in the altq > > >> >> code > > >> >> or > > >> >> J> support (which I have nothing > to do > > with) that caused this no-one > > >> >> informed > > >> >> J> me. > > >> >> > > >> >> Switching from if_start to > if_transmit > > effectively disables ALTQ > > >> >> support. > > >> >> > > >> >> AFAIR, there is some magic > implemented in > > other drivers that makes them > > >> >> modern (that means using > if_transmit), but > > still capable to switch to > > >> >> queueing > > >> >> mode if SIOCADDALTQ was casted upon > them. > > >> >> > > >> >> > > >> > Oh, hmmm, I'll look into the matter after > my > > vacation. > > >> > > > >> > Jack > > >> > > >> Has there been any progress on resolving this > > issue? I recently ran > > >> into this problem upgrading my servers from > 8.3 to > > 9.1-RELEASE and am > > >> wondering what the latest recommendation is. > I've > > used ALTQ and igb > > >> successfully for years and it is unfortunate > it no > > longer works. > > >> Appreciate any advice. > > >> > > > >Do yourself a favor and either get a cheap dual port > 82571 card or > >2 cards and disable the IGB ports. The igb driver is > defective, and until > >they back out the new, untested multi-queue stuff you're > just neutering > >your system trying to use it. > > > >Frankly this project made a huge mistake by moving > forward with multi > >queue just for the sake of saying that you support it; > without having > >any credible plan for implementing it. That nonsense > that Bill Macy did > >should have been tarballed up and deposited in the trash > folder. The > >biggest mess in programming history. > > > >That being said, the solution is not to hack the igb > driver; its to make > >ALTQ if_transmit compatible, which shouldn't be all that > difficult. > > > >BC > > I may be misunderstanding what you are saying, but if the > solution is, as you say "not to hack the igb driver", then > how is it defective in this case? Or are you just directing > vitriol toward Intel? Multi-queue is working fine in igb. > > Jeff It works like crap. Your definition of "works" is that it doesnt crash. Mine is that it works better with multiple queues than with 1, which it doesn't. And if you load a system up, it will blow up with multiqueue before it will with 1 queue. The point of using Multiqueue isn't to exhaust all of the cpus instead of just 2. It's to get past the wall of using only 2 cpus when they're exhausted. The goal is to increase the capacity of the system; not to make it look like you're using more cpus without any analysis of overall system efficiency. That's my definition of defective. The default tuning is wrong also, because is ASSumes that you only have 1 card in the box. So if you have 2 cards, it has 2 rx queues contending for the same cpus, which is exactly the opposite of what you want. There's no vitriol towards Intel. That's what Jack said; that Intel drivers are just samples and they're not optimized. Obviously some drivers are better than others because he's spent more time on them. What I don't like is that nobody ever tells anyone which drivers are good and which suck. If you think that a driver is a driver then you should be doing something else with your life other than networking. The truth is that there is no plan for multiqueue; the idea that spreading things across cpus randomly is better is just plain wrong; in fact there's a strong case to be made to keep things on the same cpu (and in the same cache). Igb is a college project. Using it will make your system slower than using good single queue cards. BC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1364574309.74010.YahooMailClassic>