Date: Thu, 6 Dec 2012 10:02:03 -0800 (PST) From: Barney Cordoba <barney_cordoba@yahoo.com> To: Robert Watson <rwatson@FreeBSD.org> Cc: freebsd-net@freebsd.org Subject: Re: Latency issues with buf_ring Message-ID: <1354816923.71234.YahooMailClassic@web121601.mail.ne1.yahoo.com> In-Reply-To: <alpine.BSF.2.00.1212060936010.78351@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--- On Thu, 12/6/12, Robert Watson <rwatson@FreeBSD.org> wrote: > From: Robert Watson <rwatson@FreeBSD.org> > Subject: Re: Latency issues with buf_ring > To: "Andre Oppermann" <oppermann@networx.ch> > Cc: "Barney Cordoba" <barney_cordoba@yahoo.com>, "Adrian Chadd" <adrian@freebsd.org>, "John Baldwin" <jhb@freebsd.org>, freebsd-net@freebsd.org > Date: Thursday, December 6, 2012, 4:39 AM > On Tue, 4 Dec 2012, Andre Oppermann > wrote: > > > For most if not all ethernet drivers from 100Mbit/s the > TX DMA rings are so large that buffering at the IFQ level > doesn't make sense anymore and only adds latency. So > it could simply directly put everything into the TX DMA and > not even try to soft-queue. If the TX DMA ring is full > ENOBUFS is returned instead of filling yet another > queue. However there are ALTQ interactions and other > mechanisms which have to be considered too making it a bit > more involved. > > I asserted for many years that software-side queueing would > be subsumed by increasingly large DMA descriptor rings for > the majority of devices and configurations. However, > this turns out not to have happened in a number of > scenarios, and so I've revised my conclusions there. I > think we will continue to need to support transmit-side > buffering, ideally in the form of a set of "libraries" that > device drivers can use to avoid code replication and > integrate queue management features fairly transparently. > > I'm a bit worried by the level of copy-and-paste between > 10gbps device drivers right now -- for 10/100/1000 drivers, > the network stack contains the majority of the code, and the > responsibility of the device driver is to advertise hardware > features and manage interactions with rings, interrupts, > etc. On the 10gbps side, we see lots of code > replication, especially in queue management, and it suggests > to me (as discussed for several years in a row at BSDCan and > elsehwere) that it's time to do a bit of revisiting of > ifnet, pull more code back into the central stack and out of > device drivers, etc. That doesn't necessarily mean > changing notions of ownership of event models, rather, > centralising code in libraries rather than all over the > place. This is something to do with some care, of > course. > > Robert More troubling than that is the notion that the same code that's suitable for 10/100Gb/s should be used in a 10Gb/s environment. 10Gb/s requires a completely different way of thinking. BC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1354816923.71234.YahooMailClassic>