Date: Wed, 11 Mar 2015 10:08:30 -0400 From: John Baldwin <jhb@freebsd.org> To: Gleb Smirnoff <glebius@freebsd.org> Cc: svn-src-projects@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r277122 - projects/ifnet/sys/dev/msk Message-ID: <44632162.sBlScnaX0r@ralph.baldwin.cx> In-Reply-To: <20150306024101.GT17947@FreeBSD.org> References: <201501130902.t0D927NE077024@svn.freebsd.org> <20150114144358.GD15484@FreeBSD.org> <20150306024101.GT17947@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Friday, March 06, 2015 05:41:01 AM Gleb Smirnoff wrote: > On Wed, Jan 14, 2015 at 05:43:58PM +0300, Gleb Smirnoff wrote: > T> On Wed, Jan 14, 2015 at 09:33:04AM -0500, John Baldwin wrote: > T> J> I posted some ideas about how to handle this in a thread several years > T> J> ago on net@ with various alternatives. In that case I was focused on > T> J> buf_ring and I settled on an approach where a draining thread marked the > T> J> queue as "busy" while it was draining it and cleared that flag before > T> J> checking the head of the queue. The enqueue process returned a > T> J> different errno value (EINPROGRESS or some such) if it queued a packet > T> J> into a "busy" queue and the transmit routines were changed to 1) always > T> J> enqueue the packet, and 2) if EINPROGRESS wasn't returned, use a > T> J> blocking mtx_lock and start transmitting. > T> J> > T> J> However, even this model has some downsides in that one thread might be > T> J> stuck transmitting packets queued by other threads and never pop back > T> J> out to userland to service its associated application (a kind of > T> J> starvation of the user side of the thread). Note that the mtx_trylock > T> J> approach has the same problem. It might be nice to have a sort of limit > T> J> on the number of packets a thread is willing to enqueue, but then you > T> J> have the problem of ensuring any packets still on the queue when it hits > T> J> its limit aren't also delayed indefinitely. > T> > T> Thanks, I will try to code that. > > John, can you please look at this patch? It is against projects/ifnet. > > The idea is that if_snd_enqueue() tells us whether we grabbed to queue and > own it or not. If we grabbed it, we go processing it to the end. However, > we keep accounting on how many packets we processed there. If other > producer notices that we processed too much, it will preempt the queue. > > Looks like a design that matches your demands. However, extra code needs > to be put into drivers foo_start() functions, since now we need to disown > the queue if we stop processing it for some reason different to queue getting > empty. I think this patch is not a bad approach. It resembles the last thing I posted to net@ except that you have added the burst length cap which is a nice addition. Of course, this uses a lock to do so which buf_ring tries to avoid. (I also eventually would love to have a way to move the enqueue out of drivers entirely still where there is a callback for "drain this queue" that only gets called in the !EBUSY case. I can't recall if that is compatible with your stacking approach, but it would make it harder for drivers to get this wrong if we can avoid duplicating it N times.) -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?44632162.sBlScnaX0r>