Date: Fri, 24 Apr 1998 19:16:36 -0500 From: Chris Csanady <ccsanady@friley585.res.iastate.edu> To: Garrett Wollman <wollman@khavrinen.lcs.mit.edu> Cc: Luigi Rizzo <luigi@labinfo.iet.unipi.it>, current@FreeBSD.ORG Subject: Re: Bandwidth throttling etc. Message-ID: <199804250016.TAA01428@friley585.res.iastate.edu> In-Reply-To: Your message of "Fri, 24 Apr 1998 17:26:48 EDT." <199804242126.RAA10941@khavrinen.lcs.mit.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
><<On Fri, 24 Apr 1998 21:32:19 +0200 (MET DST), Luigi Rizzo <luigi@labinfo.iet .unipi.it> said: > >>> What are the relative merits of increasing the size of mbufs vs. going >>> right to clusters? > >> a cluster is 2 KB, an mbuf is 8-16 times smaller. >> Moreover, a cluster also requires an associated mbuf, so you lose in >> locality of references, etc. Whether we use mbufs clusters or whatever, the packets should be contiguous. I think that this in itself causes somewhat of a problem with regards to the current socket buffer handling. Currently, the fact that the advertised window is calculated based on the total memory consumed by the mbufs--and not only the data, is causing serious performance problems. This interaction with delayed acks is the only Really Bad thing wrong with our stack that I have noticed. I believe that BSDI has done some work to deal with this by creating an m_compress function that collapses wasted space. Unfortunately, this is copy takes time.. perhaps a more relaxed policy should be used since memory is so cheap. (or at least have the option) Any one else looked into this? Someone mentioned the linux stack earlier, and they do indeed use contiguous packets. However, I think that they also use the generic kernel allocator to create these. This seems like an easy way to have per-interface buffers, although I have to wonder how efficient this would be. Would this be acceptable? I am not sure how they dealt with the above delayed ack problem, but it seems to be gone in the current kernels. (There are other nasty things in it's place however--we still have not found an acceptable kernel ;) >What you Really Want is for each interface to manage its own >allocations. When you want to send a packet, you ask the interface >for a buffer, and it gives you one of an appropriate size and shape >that it knows how to transmit efficiently. I've been trying to move >us in the direction where this is at least conceivable, but since I'm >no longer working directly for researchers who would like to see such >a thing, I have not had the time to work on continuing this work as I >would have liked. This is what I Really Want. :) I have been wanting to do work on this for quite some time, although i can't seem to find enough of it. There seem to be a few things that I am not really sure how to deal with anymore. The first is, how exactly can true card->userspace and userspace->card transfers be fit into the new architecture. Aside from this, it seems that the pbuf architecture is more or less a an excellent single copy architecture. Since the buffers are per device, this is definately a step in the right direction. What do you do though when the card hands it's buffers up the stack though? Perhaps it would just be best to copy if the buffer can not actually be completely serviced, although I am not certain of the entire effect of this sort of decision.. For zero copy, things would get interesting. You would need to at least feed headers to the protocol layer. I really have little idea here.. although, it is not an issue as long as the only api supported is sockets. (which requires a copy, since it does not allow you to wire your buffers down..) >A lot of the work is not actually difficult, just tedious; if there >are people out there who are willing to work on parts of this cleanup, >I'd be happy to describe in more detail what needs to be done. I would be interested an a description of what you had in mind. Although I have a general idea of what the end goal is, it is somewhat difficult to look at in pieces. I unfortunately don't have enough time to do what I'd like, and I still fear the integration problems. I may have some time this summer though, at last. :) Chris To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199804250016.TAA01428>