Date: Fri, 21 Feb 2003 23:33:31 -0500 From: Bosko Milekic <bmilekic@unixdaemons.com> To: Terry Lambert <tlambert2@mindspring.com> Cc: Hiten Pandya <hiten@unixdaemons.com>, FreeBSD-arch@FreeBSD.ORG Subject: Re: Mbuf flags cleanup proposal Message-ID: <20030221233331.A81541@unixdaemons.com> In-Reply-To: <3E56F8DE.5453DB88@mindspring.com>; from tlambert2@mindspring.com on Fri, Feb 21, 2003 at 08:13:18PM -0800 References: <20030221151007.GA60348@unixdaemons.com> <3E5673E7.F3F1FA4F@mindspring.com> <20030221150743.A79345@unixdaemons.com> <3E56B3F5.9EF3F9FE@mindspring.com> <20030221201728.A80661@unixdaemons.com> <3E56F8DE.5453DB88@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Now that we're getting back into technical discussion mode...
On Fri, Feb 21, 2003 at 08:13:18PM -0800, Terry Lambert wrote:
> Bosko Milekic wrote:
> > In FreeBSD (notice I said "FreeBSD," not TerryBSD, SunOS, Linux, or
> > whatever else), network buffer allocations have for the longest time
> > been done seperately and for good enough reason; as a result, FreeBSD
> > has adopted some fairly serious optimizations for what concerns the
> > way they (and supporting structures) are allocated.
>
> The mbufs, historically, were allocates out of a zalloci zone,
> which was the only allocator type (again, historically) to support
> allocation at interrupt time. The argument that this allocation
> was seperate has more to do with all interrupt allocation being
> seperate, than it does with a special case for mbufs alone.
Actually, they weren't really. I'm fairly confident that they never
were (at least not from 2.2.2 and on). I believe that this was a
local change you had, because the mbufs and clusters were always
really allocated their own way, from their own map, as long as I've
been working on the code (and probably even before that).
> Nevertheless, it's generally true that the first thing I do when
> converting a FreeBSD kernel for embedded processing in network
> equipment is "replace the mbuf allocator". Generally, I use a
> machdep.c KVA preallocated freelist (FWIW), which is a heck of a
> lot faster than anything that's ever been committed to the FreeBSD
> CVS repository. As such, there are some good arguments for seperation,
> or at least layered abstraction, of the mbuf allocator.
[...]
> > Similarily, UMA is a great
> > allocator, and it does similar things for general-purpose allocations
> > in the system. If you ask me whether or not mbuf allocations can be
> > made to use UMA? The answer is yes. If you ask me whether
> > performance is going to be better? I don't know for sure, but I can
> > tell you that in order to solve the issues I bring up it's going to be
> > difficult, and I _do_ know that if you don't solve them, performance
> > is going to suck, comparatively speaking.
>
> I understand this, as well. To my mind, it is a matter of will, on
> the part of you and Jeff, where some of the internals of Jeff's code
> need to have hooks made available for the additional processing the
> mbuf code needs to do.
>
> Let me say that I believe that this *will* happen, sooner or later,
> and that any change that would make this more difficult later is
> going to have to be backed out. Better that it never went in, as
> arguing to change something recently committed is very difficult,
> for social, rather than technical reasons.
The mbuf allocator currently is not that complicated, really. My
argument is that I really don't think that the effort to hack those
hooks into UMA only to tinker with them so that performance is as good
for things like generating refcounts for clusters on-the-fly and
struggling to keep the common allocation case (including
initialization of fields) down to a single function is worth it. I
think that in the end it's all going to boil down to ~same performance
at the expense of UMA being suddenly more complicated to maintain and
understand, which was what you initially wanted to avoid for current
and future maintainers of the code. And, granted, you would have a
valid basis to argue the opposite if the APIs really were that
different and if each allocator was really that entrenched in its own
"philosophy," but the fact is that the APIs have always been similar
with the exception of ONE behavior - that is that the mbuf code
doesn't indefinitely block with M_{,TRY}WAIT... but you've already
agreed (I think) that indefinite blocking is not good anyway, right?
:-) In any case, that behavior is easily modifiable without trashing
the allocator. Seriously, just s/M_TRYWAIT/M_WAIT/ and change the
cv_timedwait() call in subr_mbuf.c to an indefinite block on the cv
and, poof, you're done (again, not that I think that would be the
correct thing to do).
> > Solving them would require what I think is relatively serious
> > modification to UMA which, in my opinion anyway, would uglify [sic]
> > it.
>
> I understand this as well; what I don't understand is the unwillingness
> to discuss it, or to do the "uglification" anyway.
Dude, seriously, try it!
> > I *have* looked at it, and I think that the fact that UMA does
> > allocations for all objects using the same techniques is great and - I
> > can't speak for Jeff - but *I* wouldn't want to hack at it just so
> > that we can get the optimizations/solutions we currently have for mbuf
> > allocations. And, you know what? If *you* think it's worth it, why
> > don't *YOU* do it and waste hours on end to, finally, get something
> > that _maybe_ performs as well at the expense of an uglier
> > (not-so-general-and-simple-anymore) allocator.
>
> I think it's possible to do -- as long as there are not changes
> that preclude it, such as renaming manifest constants to be use
> specific, etc.. Changes such as you were proposing.
Actually, I stick to what I said initially which is that one has
nothing to do with the other... whether or not the allocations
suddenly start coming from A or B in the background isn't going to
change the API. That is, to allocate an mbuf you'll keep doing
m_get(), etc. All it may change is the behavior, and this only
slightly.
[...]
> Regards,
> -- Terry
Regards,
--
Bosko Milekic * bmilekic@unixdaemons.com * bmilekic@FreeBSD.org
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030221233331.A81541>