Date: Wed, 6 Oct 2010 18:42:56 +0100 From: "Robert N. M. Watson" <rwatson@freebsd.org> To: Karim Fodil-Lemelin <kfl@xiplink.com> Cc: Juli Mallett <jmallett@freebsd.org>, Rui Paulo <rpaulo@freebsd.org>, Ryan Stone <rysto32@gmail.com>, Luigi Rizzo <rizzo@iet.unipi.it>, FreeBSD Net <net@freebsd.org> Subject: Re: mbuf changes Message-ID: <3C5CDDB3-73BF-4551-8F42-1FBCDB757AB6@freebsd.org> In-Reply-To: <4CAB4BE6.3070307@xiplink.com> References: <4C9DA26D.7000309@freebsd.org> <AANLkTim7oRyVYY3frn7=cn4Et8Acbcq9cXja_bR6YWvP@mail.gmail.com> <4CA51024.8020307@freebsd.org> <alpine.BSF.2.00.1010021627230.49031@fledge.watson.org> <9AD4923A-72AE-4FE3-A869-3AF8ECBF17E2@FreeBSD.org> <AANLkTi=uxARo5O9MASrx9mg-39E2=x05RXxcKUB62JrB@mail.gmail.com> <0DB8120D-C02A-49A1-8013-1ED818EDE7E6@freebsd.org> <20101003131330.GA85551@onelab2.iet.unipi.it> <4CAB4BE6.3070307@xiplink.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 5 Oct 2010, at 17:01, Karim Fodil-Lemelin wrote: > I will share some of the experience I had doing embed mtags. Hopefully = its relevant :) >=20 > The idea of carrying a certain amount of mbuf tags within the mbuf = structure is somewhat similar but much cleaner, imo, then Linux's skbuff = char cb[40 - 48] (it was 40bytes in 2.4.x ...). Now this idea is not new = although as you know the devil is in the details... Hi Karim: This sounds like very interesting work, and something we should figure = out how to generalize for FreeBSD. I had also been pondering something = along these lines in order to improve MAC label performance when using = ubiquitously labeled policies. Since MAC often stores references to = externally managed data from mbuf labels (i.e., deep structures, rather = than just bytes of data) it's especially important to make sure we get = the tear-down stuff right... Robert >=20 > What we did for BSD is create a container in the mbuf and extend the = API with functions we (pompously) called m_tag_fast_alloc() and = m_tag_fast_free(). This means the standard m_tag_alloc() is still = supported across the system and the old behavior is unchanged (list of = allocated struct attached to the packet header). Whats different is the = availability of a 'fast' call that directly uses the container within = the mbuf, effectively avoiding those malloc and cache misses. I'll = explain later how we effectively support calling m_tag_delete on a = 'fast' tag. >=20 > The trick to save CPU cycles was also to quickly revert back to the = standard tag mechanism if some component in the system is manipulating = the tag list by deleting elements. Effectively, the m_tag_fast_free is a = NOP and fast tags are not deleted once allocated (unless m_free is = called on the mbuf of course). When m_tag_delete is called the container = simply becomes 'fast tag' invalid for further additions. This is not = flexible but has the merit of reducing the overall number of operations = given that almost no components are deleting tags without deleting the = mbuf (loopback does but its a special case). >=20 > One last thing we did is perform various operational tests to come up = with the most statistically optimized container size. Now this is much = easier to do on a proprietary system then for a general purpose OS but = its certainly possible. >=20 > Finally, we did see speed increase for our application and if someone = is interested I could provide a patch although I would have to rewrite = it without the proprietary bits in it. >=20 > Best regards, >=20 > Karim.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3C5CDDB3-73BF-4551-8F42-1FBCDB757AB6>