From owner-freebsd-hackers@FreeBSD.ORG Tue Apr 2 18:03:31 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CC91528E for ; Tue, 2 Apr 2013 18:03:31 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) by mx1.freebsd.org (Postfix) with ESMTP id 8B7C45FA for ; Tue, 2 Apr 2013 18:03:30 +0000 (UTC) Received: from jre-mbp.elischer.org (ppp121-45-245-181.lns20.per2.internode.on.net [121.45.245.181]) (authenticated bits=0) by vps1.elischer.org (8.14.5/8.14.5) with ESMTP id r32HwDx4061065 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 2 Apr 2013 10:58:16 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <515B1C30.5080703@freebsd.org> Date: Wed, 03 Apr 2013 01:58:08 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: gary mazzaferro Subject: Re: Need advice on sys5 shm and zero copy sockets References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Apr 2013 18:03:31 -0000 On 2/8/13 4:22 AM, gary mazzaferro wrote: > Hi, > > I was told to post this question here (Ken Merry), it would be a good > place to get some help. I'm not sure this is doable without a kernel > module, which I don't want to add. > > I'll explain what I'm attempting.. > > I'm designing a high speed rest motor for cloud execution environment. > > 1) I'd like to eliminate copy from the tcp stack to the application(s). > > 2) I'm also sharing the buffers across processes and jails. So I'd > like to preserve the zero-copy in a msg pipe/unix socket > > 3) Some buffers will go to disk file systems. > > > Wish list: > 4) I'd like it to work with sctp because I like it for local networking :) > > 5) I'd like to provision memory pools on a per > application/connection/ip port basis. > > Ultimate Goal: > 6) Additionally, I'm injecting "code" from a foreign process into the > workflow of another process (state machine). The connection between > them will be a signal and shared state information. > > I'm assuming item (6) is a separate issue, but it may impact the direction.. > > I've tried shm with zero copy sockets with linux and it just will not work !! > > BTW, I'm returning to freebsd after far too many years > > cheers, > gary > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > this sound somewhat like what I did back in the 90s with BSD4.3 unfortunately it was not done with TCP (or sctp of course) what we did was to create a special shared memeory device driver. Then we added ioctls to the disk driver layer to write named blocks of memory from that device to the raw device (we didin't use a filesystem). We also changed the network drivers to use named blocks of memory in that device for packet reception. We added a special protocol which used recvmsg() and and sendmesg() to pass ownership of those named blocks between the process that had mmapped them and the protocol. The protocol had an IP header but also a small protocol specific header of our own.. we sent packets that were larger than a page. zero copy from disk->process, process->network and network->disk (and reverse of course) of course it was all on proprietary protocols so not of use to you. julian