Date: Sun, 30 Jul 2006 16:25:47 +0100 (BST) From: Robert Watson <rwatson@FreeBSD.org> To: Max Laier <max@love2party.net> Cc: freeebsd-net@freebsd.org, freebsd-arch@freebsd.org Subject: Re: Changes in the network interface queueing handoff model Message-ID: <20060730160933.D16341@fledge.watson.org> In-Reply-To: <200607301659.16323.max@love2party.net> References: <20060730141642.D16341@fledge.watson.org> <200607301659.16323.max@love2party.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 30 Jul 2006, Max Laier wrote: >> Conceptual review as well as banchmarking, etc, would be most welcome. > > This begs the question: What about ALTQ? > > If we maintain the fallback mechanism in _handoff, we can just add > ALTQ_IS_ENABLED() to the test. Otherwise every driver's startmbuf function > would have to take care of ALTQ itself, which is not preferable. Maxime just asked me the same question, and I realized that I had, of course, forgotten to mention ALTQ. A few observations/questions: - An underlying assumption of ALTQ is that queueing occurs in software. This turns out to be decreasingly true with modern network hardware, where the queueing occurs in a combination of software and hardware thanks to the descriptor ring. Has anyone compared the effectiveness of ALTQ on gig-e systems with the effectiveness on 10/100mbps systems? I would anticipate that it would have an effect on the latter, but little or no effect on the former? - ALTQ actually also already does a bit of what I describe -- IFQ_DRV_DEQUEUE() and friends actually manage two ifnet queues, if_snd (the public queue), and if_drv_head, a second queue protected by the device driver lock. Because it bulk dequeues from one queue to the other, it already works to amortize if_snd locking, at the cost of maintaining two queues and substantially more complicated logic. - One of the side effects of this change is that it does complicate device drivers, especially if they are going to rely on software queueing. In the example changes to if_em in my patch, the start path goes from a simple loop around the send queue to considering three cases: needing to queue, handling the optimized (empty) queue case, and needing to dequeue. Quite a bit of this logic will be common across device drivers and might be something we can abstract out some. There are two things going on in my proposed change: ownership (and hence locking and interface) with a queue moves, and code moves. It could be we could transfer the locking (etc) and move less code. Thoughts on this would be welcome. Notice that in the patch I do leave backwards compatible support for if_start, if only because rewriting all network device drivers is error prone and arduous. I assume this will be a temporary condition, but it could be that we could leave it as a permanent one to support older devices that won't be updated (ISA ethernet cards, etc), where the existing queueing model works reasonably well. In an earlier iteration of this patch, I had em_startmbuf() call a utility routine in if.c to handle enqueueing followed by calling em_start(). This meant no optimized fast path, but less code change. > I strongly agree with you comment about how messed up ifq_*/if_* in > if_var.h are - and I'm afraid that's partly me fault for bringing in ALTQ. Heh. I actually meant to remove that comment before posting the patch, so it's a bit more blunt than perhaps entirely wise. :-) A couple of times in the past I've attempted to work on a rename to basically swap the two naming schemes and clean things up, but usually I've stalled when it occurs to me just how big the task is. At this point, I think the right thing to do is make a decision about the semantics of the interface, and then walk the tree and clean up after we've simplified things. Robert N M Watson Computer Laboratory University of Cambridge
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060730160933.D16341>