Date: Mon, 19 Jun 2000 16:11:59 -0700 From: Mike Smith <msmith@freebsd.org> To: Jonathan Lemon <jlemon@flugsvamp.com> Cc: arch@FreeBSD.ORG Subject: Re: kblob discussion. Message-ID: <200006192311.QAA10064@mass.osd.bsdi.com> In-Reply-To: Your message of "Mon, 19 Jun 2000 17:20:41 CDT." <20000619172041.G37084@prism.flugsvamp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > *cough*. Are you sure about that? An output accelerator is a new I/O > > > method regardless of what you want to call it. > > > > *sigh* > > > > Kblob is not a new output method. It's a bolt-on to the 'write' > > interface, as opposed to, say, zero-copy I/O. > > Yes, it is a "bolt-on" thing, not a well designed item. That's debatable. > And as far as "zero-copy" goes, I would consider that a fairly overloaded > term. Here, I'm applying it to anything that saves copies, not just > a magical method of getting data between user and kernel spaces. Correct. Note that kblob doesn't involve any more copies than absolutely necessary. > > > And for a lot of cases > > > you don't need stuff to actually go to user space. In many situations, > > > you will receive data, perform operations on a very small part of it, > > > and then send the data elsewhere, whether to disk or network. > > > > However, for a user-space application to do this, you want it to come out > > to user-space, right? > > No. You cut out what I wrote - you may only need a very small part > of the data to actually go to user space. However, it still has to go there, right? That being the actual point. > > Irregardless, this is irrelevant in this context. > > Not irrelevant; you're not looking at the big picture here. Actually, I think I'm looking at a *bigger* picture - what can we do to make FreeBSD a better platform for real-world applications? The ability to send static content in the most efficient manner possible is a key "big picture" item. You're so tied up in what would be the most wonderful engineering solution that you're completely ignoring this. > > > Kblob only addresses part of the problem. > > > > Kblob addresses exactly one problem, and does it in an expedient fashion. > > Exactly, so my statement stands: it only addresses a small part of > the problem. No, it addresses _one_ problem, comfortably, in its entirety, in a fashion which can be (and has been) rapidly implemented and deployed. > > > Also, if I want to send data from disk -> network, with the kblob > > > API, I essentially have the following sequence of calls: > > > > > > read(fd, address, length) > > > kblob(address, length) > > > kblobsend(...) > > > > > > which is to say, the data is copied from > > > disk -> userspace (assuming direct I/O) > > > userspace -> kernel, > > > kernel -> network. > > > > > > Why shouldn't I be able to bypass second copy entirely and do: > > > disk -> kernel (data now in kernel 'blob') > > > kernel -> network > > > > Because this isn't what you use kblob for. That's what you use sendfile > > for. > > But sendfile doesn't leave the data cached in kernel space for reuse, > so it isn't suitable for this. It does. > > The entire use of the kblob API is: > > > > user -> kernel > > > > kernel -> network > > kernel -> network > > kernel -> network > > kernel -> network > > kernel -> network > > kernel -> network > > kernel -> network > > etc., > > And you have to get the data into user space first, when the data > may be coming from disk. Thus you incur one copy into userspace > first, when a better API might avoid it altogether. I *know* what > kblob is designed for, and I'm telling you that the API is limiting, > and will not be suitable for what I have. I don't think that anyone *cares* about what you have at this juncture. What you're threatening will have so many complex options and implementation issues that it will entirely fail to address the actual, real-world problems that the big-picture people are looking at. > > > Again, this is not addressed in the kblob API. Don't throw sendfile() > > > up as an example, because I want the data to be kept in the kernel > > > after being read from disk. > > > > It is. You remember that thing called the buffer cache? > > Not applicable here. If it was, why the heck aren't you using sendfile > in the first place and relying on the kernel to keep the file cached? > Hmm? If that was the way things worked, why isn't sendfile() good enough? Because it's _still_too_expensive_. Kblob is about _performance_, not "technical elegance". > > No. Since the data in a kblob came from userspace, and can't be modified > > in the kernel, this would be pointless. > > Exactly. You're looking at kblob as a "static send-only buffer which > comes from user space". I'm looking at kblob as an "in-kernel object > cache". The two views are _NOT_ that far apart, and with a little API > help, the current code can be made much more flexible. Why are you > opposed to that? Why would anyone be opposed to putting a Perl interpreter in the kernel? > > Again, folks; kblob is an optimisation for a very common performance > > case, not a network engineer's wet dream. It's meant to address a > > real-world problem in an efficient fashion. > > It is a narrow optimization for a specific problem. Correct. It just happens to be a specific problem that comes up _very_ often. As I've already proposed, kblob should be implemented as a loadable system call. It'll do what it's meant to do, for the people that want it to do it, and the rest of you can ignore it in happiness. You're welcome to your API, but you need to face the fact that it's just as irrelevant to kblob's target audience as kblob is to you. -- \\ Give a man a fish, and you feed him for a day. \\ Mike Smith \\ Tell him he should learn how to fish himself, \\ msmith@freebsd.org \\ and he'll hate you for a lifetime. \\ msmith@cdrom.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200006192311.QAA10064>