From owner-freebsd-hackers Fri May 21 13:46:16 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from mailer.syr.edu (mailer.syr.edu [128.230.18.29]) by hub.freebsd.org (Postfix) with ESMTP id 8655A14EF0 for ; Fri, 21 May 1999 13:46:11 -0700 (PDT) (envelope-from cmsedore@mailbox.syr.edu) Received: from rodan.syr.edu by mailer.syr.edu (LSMTP for Windows NT v1.1a) with SMTP id <0.6FCB0850@mailer.syr.edu>; Fri, 21 May 1999 16:46:13 -0400 Received: from localhost (cmsedore@localhost) by rodan.syr.edu (8.8.7/8.8.7) with SMTP id QAA03661; Fri, 21 May 1999 16:46:10 -0400 (EDT) X-Authentication-Warning: rodan.syr.edu: cmsedore owned process doing -bs Date: Fri, 21 May 1999 16:46:10 -0400 (EDT) From: Christopher Sedore X-Sender: cmsedore@rodan.syr.edu To: Mike Smith Cc: Zhihui Zhang , freebsd-hackers@FreeBSD.ORG Subject: Re: mmap of a network buffer In-Reply-To: <199905211920.MAA01832@dingo.cdrom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 21 May 1999, Mike Smith wrote: > > > > I really do not know how to describe the problem. But a friend here asks > > me how to mmap a network buffer so that there is no need to copy the data > > from user space to kernel space. We are not sure whether FreeBSD can > > create a device file (mknod) for a network card, and if so, we can use the > > mmap() call to do so because mmap() requires a file descriptor. We assume > > that the file descriptor can be acquired by opening the network device. > > If this is infeasible, is there another way to accomplish the same goal? > > Use sendfile() for zero-copy file transmission; in all other cases it's > necessary to copy data into the kernel. Memory-mapping a network > buffer makes no sense if you just think about it for a moment... > > There's also very little need for this under "real" circumstances; some > simple tests have demonstrated we can sustain about 800Mbps throughput > (UDP), and the bottleneck here seems to be checksum calculations, not > copyin/out. > Oddly enough, I was just getting ready to implement something like this. Not because of copyin performance issues, but because async io for sockets could be done better if I didn't have to do a copyin. copyin has to have curproc==(proc with the buffer from which to copy) which means that I have to do a context switch for every socket buffer sized chunk (best case) or every io op (worst case). In my testing, I've been doing 3000 context switches/second just in verification of a simple configuration--real load will be 20-100x what current load is, and I'm not sure that 60,000-300,000 context switches/sec is desirable. My hope was to map the user's buffer into kernel space so that I could do event driven io on the socket without having to context switch to an aiod for every io operation. Is this really a bad idea? I am a little concerned about running out of kernel address space, but I don't think that's an immediate problem. Such an implementation would lend itself to doing zero-copy writes async writes with some frobbing of the send routines. It would also bypass some of the messing around done to do socket buffers--that is, there would not be a limit per se on socket buffering for writes since they would be mapped user space. One might want to put arbitrary limits in place to ensure that an unreasonable amount of memory isn't locked. Thoughts? -Chris To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message