From owner-freebsd-net@FreeBSD.ORG Thu Jul 20 18:11:46 2006 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9CA0B16A4DA for ; Thu, 20 Jul 2006 18:11:46 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.181]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1796F43D53 for ; Thu, 20 Jul 2006 18:11:45 +0000 (GMT) (envelope-from jfvogel@gmail.com) Received: by py-out-1112.google.com with SMTP id c63so880767pyc for ; Thu, 20 Jul 2006 11:11:45 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=R4reoWroUOvLgFaxm/REdfYa6zvOAybozS2hnAG8UqM4P5n2Oofqow8TTM84o58dtxZfSPj26543+8YSdkPQlHmouSu7QxLBzhYqRgU6r1ugiRA9D42VVJUcwhu7XNMkecCCD6mOAFU3RAQdntrunHpEpCfgVeWSr7CwrUAXyFY= Received: by 10.35.41.12 with SMTP id t12mr1493022pyj; Thu, 20 Jul 2006 11:11:45 -0700 (PDT) Received: by 10.35.119.14 with HTTP; Thu, 20 Jul 2006 11:11:44 -0700 (PDT) Message-ID: <2a41acea0607201111x84c4ef8jf8cdb50d3ffa28e0@mail.gmail.com> Date: Thu, 20 Jul 2006 11:11:44 -0700 From: "Jack Vogel" To: "Julian Elischer" In-Reply-To: <44BFB667.60106@elischer.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200607192230.14939.mi+mx@aldan.algebra.com> <20060720025003.GF924@tnn.dglawrence.com> <44BFB667.60106@elischer.org> Cc: Mikhail Teterin , jfvogel@gmail.com, "David G. Lawrence" , net@freebsd.org Subject: Re: complement to sendfile()? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jul 2006 18:11:46 -0000 We, myself and Prafulla Deuskar at Intel LAD, have a driver and stack changes that support Intel's new I/OAT DMA hardware. This is a DMA engine in the chipset. There is potential to use the hardware in a number of ways, what we have right now is a receive-side async dma assist from tcp_input() to soreceive(). The engine copies from the mbufs direct to user buffers asynchronously, and only requires the client process to be woken when its buffers are full. The stack changes to support this are relatively small, mainly we need to pin user buffers and provide a pointer down to the stack in order to process them, and of course the core driver for the engine. Similar work was done for Linux and is in the 2.6.18 kernel. We see as much as a 20% improvement in CPU utilization when doing a sustained iperf test. We are hoping to get this code into CURRENT soon if there is interest. Cheers, Jack On 7/20/06, Julian Elischer wrote: > David G. Lawrence wrote: > > >>Hello! > >> > >>My program receives data from the socket and writes it to a file -- with the > >>usual read()/write() tedium. > >> > >>Is there anything zero-copying like sendfile() for the socket->file direction? > >> > >>In fact, sendfile's API may allow to use it in any direction, but the manual > >>is quite explicit, that the second (destination) argument must be socket. > >> > >>recvfile()? Thanks! > >> > >> > > > > sendfile() could be extended to allow arbitrary file descriptor types as > >the source and destination, but the zero-copy nature of it can only work > >in the file to socket direction. This is because network buffers can be made > >out of filesystem buffers (file->network direction), but for the network to > >file direction network packets arrive non-deterministically. With the right > >network hardware it would in theory be possible to have the TCP code run > >on the network card and it could DMA the TCP stream directly into file > >buffers. If pigs had wings, they could fly. :-) > > > > > > > We used to do something like this in BSD4.3 (not FreeBSD 4.3) in 1991, > but we used > a propriatary filesystem and a proprietary protocol that knew each > other's data types. > > >-DG > > > >David G. Lawrence > >President > >Download Technologies, Inc. - http://www.downloadtech.com - (866) 399 8500 > >The FreeBSD Project - http://www.freebsd.org > >Pave the road of life with opportunities.