From owner-freebsd-net@FreeBSD.ORG Thu Jul 20 18:21:58 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2293616A4DA; Thu, 20 Jul 2006 18:21:58 +0000 (UTC) (envelope-from julian@elischer.org) Received: from a50.ironport.com (a50.ironport.com [63.251.108.112]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33D8743D49; Thu, 20 Jul 2006 18:21:57 +0000 (GMT) (envelope-from julian@elischer.org) Received: from unknown (HELO [192.168.2.4]) ([10.251.60.66]) by a50.ironport.com with ESMTP; 20 Jul 2006 11:21:53 -0700 Message-ID: <44BFC9C0.6030606@elischer.org> Date: Thu, 20 Jul 2006 11:21:52 -0700 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.13) Gecko/20060414 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Jack Vogel References: <200607192230.14939.mi+mx@aldan.algebra.com> <20060720025003.GF924@tnn.dglawrence.com> <44BFB667.60106@elischer.org> <2a41acea0607201111x84c4ef8jf8cdb50d3ffa28e0@mail.gmail.com> In-Reply-To: <2a41acea0607201111x84c4ef8jf8cdb50d3ffa28e0@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Mikhail Teterin , "David G. Lawrence" , net@freebsd.org Subject: Re: complement to sendfile()? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jul 2006 18:21:58 -0000 Jack Vogel wrote: > We, myself and Prafulla Deuskar at Intel LAD, have a driver and stack > changes that support Intel's new I/OAT DMA hardware. This is a > DMA engine in the chipset. shades of the old PC with the built in DMA on hte motherboard.... :-) > There is potential to use the hardware > in a number of ways, what we have right now is a receive-side > async dma assist from tcp_input() to soreceive(). The engine > copies from the mbufs direct to user buffers asynchronously, > and only requires the client process to be woken when its > buffers are full. The stack changes to support this are relatively > small, mainly we need to pin user buffers and provide a pointer > down to the stack in order to process them, and of course the > core driver for the engine. > > Similar work was done for Linux and is in the 2.6.18 kernel. > > We see as much as a 20% improvement in CPU utilization > when doing a sustained iperf test. > > We are hoping to get this code into CURRENT soon if there > is interest. > > Cheers, > > Jack > > > > On 7/20/06, Julian Elischer wrote: > >> David G. Lawrence wrote: >> >> >>Hello! >> >> >> >>My program receives data from the socket and writes it to a file -- >> with the >> >>usual read()/write() tedium. >> >> >> >>Is there anything zero-copying like sendfile() for the socket->file >> direction? >> >> >> >>In fact, sendfile's API may allow to use it in any direction, but >> the manual >> >>is quite explicit, that the second (destination) argument must be >> socket. >> >> >> >>recvfile()? Thanks! >> >> >> >> >> > >> > sendfile() could be extended to allow arbitrary file descriptor >> types as >> >the source and destination, but the zero-copy nature of it can only >> work >> >in the file to socket direction. This is because network buffers can >> be made >> >out of filesystem buffers (file->network direction), but for the >> network to >> >file direction network packets arrive non-deterministically. With >> the right >> >network hardware it would in theory be possible to have the TCP code >> run >> >on the network card and it could DMA the TCP stream directly into file >> >buffers. If pigs had wings, they could fly. :-) >> > >> > >> >> >> We used to do something like this in BSD4.3 (not FreeBSD 4.3) in 1991, >> but we used >> a propriatary filesystem and a proprietary protocol that knew each >> other's data types. >> >> >-DG >> > >> >David G. Lawrence >> >President >> >Download Technologies, Inc. - http://www.downloadtech.com - (866) >> 399 8500 >> >The FreeBSD Project - http://www.freebsd.org >> >Pave the road of life with opportunities. >