Date: Wed, 12 Dec 2012 08:49:31 +0100 From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= <eri@freebsd.org> To: Karim Fodil-Lemelin <fodillemlinkarim@gmail.com> Cc: freebsd-net <freebsd-net@freebsd.org>, "Clement Hermann \(nodens\)" <nodens2099@gmail.com> Subject: Re: igb and ALTQ in 9.1-rc3 Message-ID: <CAPBZQG1HEvQ%2Bb7Ruq=262FafBGM-2FoMtZ4d0%2Bs8Z0%2B8WKoxJw@mail.gmail.com> In-Reply-To: <50C79252.5000509@gmail.com> References: <20121211075853.GU48639@FreeBSD.org> <1355231140.51621.YahooMailClassic@web121603.mail.ne1.yahoo.com> <CAPBZQG030sSLT__b4cYA2e07t-77pvv9S3f8HnZfYpsXKEqTkQ@mail.gmail.com> <50C74990.2090803@gmail.com> <CAPBZQG3d1uzyEgWK3SALiVwkaEXdCZUmkM3VZA-9QB1EsL-=JQ@mail.gmail.com> <50C79252.5000509@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Dec 11, 2012 at 9:06 PM, Karim Fodil-Lemelin < fodillemlinkarim@gmail.com> wrote: > On 11/12/2012 11:27 AM, Ermal Lu=E7i wrote: > >> On Tue, Dec 11, 2012 at 3:56 PM, Karim Fodil-Lemelin < >> fodillemlinkarim@gmail.com> wrote: >> >> On 11/12/2012 9:15 AM, Ermal Lu=E7i wrote: >>> >>> On Tue, Dec 11, 2012 at 2:05 PM, Barney Cordoba < >>>> barney_cordoba@yahoo.com >>>> >>>>> **wrote: >>>>> >>>> >>>> --- On Tue, 12/11/12, Gleb Smirnoff <glebius@FreeBSD.org> wrote: >>>>> >>>>> From: Gleb Smirnoff <glebius@FreeBSD.org> >>>>> >>>>>> Subject: Re: igb and ALTQ in 9.1-rc3 >>>>>> To: "Jack Vogel" <jfvogel@gmail.com> >>>>>> Cc: "Clement Hermann (nodens)" <nodens2099@gmail.com>, "Barney >>>>>> Cordoba" >>>>>> >>>>>> <barney_cordoba@yahoo.com>, freebsd-net@FreeBSD.org >>>>> >>>>> Date: Tuesday, December 11, 2012, 2:58 AM >>>>>> On Mon, Dec 10, 2012 at 03:31:19PM >>>>>> -0800, Jack Vogel wrote: >>>>>> J> UH, maybe asking the owner of the driver would help >>>>>> :) >>>>>> J> >>>>>> J> ... and no, I've never been aware of doing anything to >>>>>> stop supporting altq >>>>>> J> so you wouldn't see any commits. If there's something >>>>>> in the altq code or >>>>>> J> support (which I have nothing to do with) that caused >>>>>> this no-one informed >>>>>> J> me. >>>>>> >>>>>> Switching from if_start to if_transmit effectively disables >>>>>> ALTQ support. >>>>>> >>>>>> AFAIR, there is some magic implemented in other drivers that >>>>>> makes them >>>>>> modern (that means using if_transmit), but still capable to >>>>>> switch to queueing >>>>>> mode if SIOCADDALTQ was casted upon them. >>>>>> >>>>>> It seems pretty difficult to say that something is compatible with >>>>>> >>>>> something else if it hasn't been tested in a few years. >>>>> >>>>> It seems to me that ATLQ is the one that should handle if_transmit. >>>>> although it's a good argument for having a raw "send" function in >>>>> drivers. Ethernet drivers don't need more than a send() routing that >>>>> loads a packet into the ring. The decision on what to do if you can't >>>>> queue a packet should be in the network layer, if we must still call >>>>> things layers. >>>>> >>>>> "start" is a leftover from a day when you stuffed a buffer and waited >>>>> for an interrupt to stuff in another. The whole idea is antiquated. >>>>> >>>>> Imagine drivers that pull packets off of a card and simply queue it; >>>>> and that you simply submit a packet to be queued for transmit. Instea= d >>>>> of trying to find 35 programmers that understand all of the lock BS, >>>>> you only need to have a couple. >>>>> >>>>> I always disable all of the gobbledegook like checksum offloading. Th= ey >>>>> just muddy the water and have very little effect on performance. A >>>>> modern >>>>> cpu can do a checksum as fast as you can manage the "capabilities" >>>>> without >>>>> disrupting the processing path. >>>>> >>>>> With FreeBSD, every driver is an experience. Some suck so bad that th= ey >>>>> should come with a warning. The MSK driver is completely useless, as >>>>> an example. >>>>> >>>>> >>>>> BC >>>>> ______________________________****_________________ >>>>> freebsd-net@freebsd.org mailing list >>>>> http://lists.freebsd.org/****mailman/listinfo/freebsd-net<http://list= s.freebsd.org/**mailman/listinfo/freebsd-net> >>>>> <h**ttp://lists.freebsd.org/**mailman/listinfo/freebsd-net<http://lis= ts.freebsd.org/mailman/listinfo/freebsd-net> >>>>> > >>>>> To unsubscribe, send any mail to "freebsd-net-unsubscribe@**fre** >>>>> ebsd.org <http://freebsd.org><freebsd-net-**unsubscribe@freebsd.org<f= reebsd-net-unsubscribe@freebsd.org> >>>>> > >>>>> >>>>> " >>>>> >>>>> During implementation of if_transmit altq was not considered at all= . >>>>> >>>> The default if_transmit provides some compatibility but that is void >>>> since >>>> altq has not been converted to call if_transmit after processing the >>>> mbuf. >>>> >>>> ALTQ can be adapted quite easily to if_transmit model it just wasn't >>>> done >>>> at the time. >>>> With if_transmit model it can even be modularized and not be a compile >>>> kernel option since the queue of the iface is abstracted now. >>>> >>>> I have always wanted to do a diff but have not yet got to it. >>>> The change is quite simple just provide an altq_transmit default metho= d >>>> and >>>> just hook into if_transmit model on the fly. >>>> You surely need to handle some iface events and enable altq based on >>>> request but its is not a hard to implement. >>>> >>>> I will always have this in my TODO but not sure when i can get to it. >>>> >>>> The issue is not only that igb doesn't support if_transmit or if_sta= rt >>>> >>> method but that ALTQ isn't multiqueue ready and still uses the IFQ_LOCK >>> for >>> all of its enqueue/dequeue operations. A simple drop in of if_transmit = is >>> bound to cause race conditions on any multiqueue driver with ALTQ. >>> >>> I do have a patch set for this on igb but its ugly and needs more work >>> although it should get you going. Let me know if your interested I will >>> clean it up and send it over. For more information on ALTQ discussion a= nd >>> igb please read this thread: http://freebsd.1045724.n5.** >>> nabble.com/em-igb-if-transmit-****drbr-and-ALTQ-td5760338.html<http://n= abble.com/em-igb-if-transmit-**drbr-and-ALTQ-td5760338.html> >>> **<http://freebsd.1045724.n5.**nabble.com/em-igb-if-transmit-** >>> drbr-and-ALTQ-td5760338.html<http://freebsd.1045724.n5.nabble.com/em-ig= b-if-transmit-drbr-and-ALTQ-td5760338.html> >>> > >>> >>> Best regards, >>> >>> Karim. >>> >>> >>> ALTQ needs to do serialization of packets to apply its policy. >> While it can be extended to map driver queues to ALTQ queues that is not >> easy done without getting some interface to the driver to communicate. >> The poor man implementation would be to pass an index argument when >> calling >> driver transmit routine. >> >> I doubt it is realistic to map ALTQ queues to driver queues since ALTQ > can have many more queues then what any driver would need and those queue= s > are controlled by users in so many ways it is almost impossible to cater > for all cases in all device drivers. This could lead to TX rings being > quickly filled up and packets being dropped. A good example of this would > be to have two ALTQ queues. One very fast 1Gbps and one very slow 512Kbps= . > Having to reference packets through indexes inside the driver queues for > each of the corresponding packets in ALTQ's fast queue would result in to= o > much overhead IMO and a potential ring exhaustion by the slow queue. Both > queues having different queue backlog criteria for achieving their rate > they must be maintained somewhat separately (which is what ALTQ does > through its class_queue). > > That is a matter of policy rather than implementation. If the implementation is there the policy maker can decide to do 1:1 mapping with hw queues since it provides quite a bit of flexibility and avoids buffering to much. > ALTQ needs to be both a consumer and a producer of a device driver (or > ideally to some interface to it). It needs to be able to 'consume' packet= s > from an outgoing path to enqueue them and some way (historically if_start= ) > to tell a driver to send a given packet. While the enqueue operation is > usually through a direct dispatch the dequeuing can be done asynchronousl= y > by a timer through ALTQ's token bucket regulator. Not unlike buf ring ALT= Q > can, in the same if_transmit works, enqueue a packet and dequeue another > packet to be sent. > > Dummynet does the same and ALTQ would be quite similar to it in that regard when it gets moved to if_transmit. They just hook in different level and ALTQ does not need a looping by the place it chose to hook in. > What is needed is a clear API for sending packets and a comprehensive set > of rules for locking TX queues. Right now FreeBSD has ifqueue (if_start) > with its well known IFQ_LOCK and if_transmit (drbr) with driver managed T= X > locks and while it seems the trend is to move towards if_transmit, the > transition is not completed and the locking too much dependent on the > driver implementation to be currently useable by ALTQ. > > Karim. > --=20 Ermal
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAPBZQG1HEvQ%2Bb7Ruq=262FafBGM-2FoMtZ4d0%2Bs8Z0%2B8WKoxJw>