Date: Wed, 03 Apr 2013 01:58:08 +0800 From: Julian Elischer <julian@freebsd.org> To: gary mazzaferro <garym@oedata.com> Cc: freebsd-hackers@freebsd.org Subject: Re: Need advice on sys5 shm and zero copy sockets Message-ID: <515B1C30.5080703@freebsd.org> In-Reply-To: <CAGwOJnybaQ7%2BAgnoiFZEBruvFyZOKjW3S8tAYzkTx7pX9K%2B_Vw@mail.gmail.com> References: <CAGwOJnybaQ7%2BAgnoiFZEBruvFyZOKjW3S8tAYzkTx7pX9K%2B_Vw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/8/13 4:22 AM, gary mazzaferro wrote: > Hi, > > I was told to post this question here (Ken Merry), it would be a good > place to get some help. I'm not sure this is doable without a kernel > module, which I don't want to add. > > I'll explain what I'm attempting.. > > I'm designing a high speed rest motor for cloud execution environment. > > 1) I'd like to eliminate copy from the tcp stack to the application(s). > > 2) I'm also sharing the buffers across processes and jails. So I'd > like to preserve the zero-copy in a msg pipe/unix socket > > 3) Some buffers will go to disk file systems. > > > Wish list: > 4) I'd like it to work with sctp because I like it for local networking :) > > 5) I'd like to provision memory pools on a per > application/connection/ip port basis. > > Ultimate Goal: > 6) Additionally, I'm injecting "code" from a foreign process into the > workflow of another process (state machine). The connection between > them will be a signal and shared state information. > > I'm assuming item (6) is a separate issue, but it may impact the direction.. > > I've tried shm with zero copy sockets with linux and it just will not work !! > > BTW, I'm returning to freebsd after far too many years > > cheers, > gary > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > > this sound somewhat like what I did back in the 90s with BSD4.3 unfortunately it was not done with TCP (or sctp of course) what we did was to create a special shared memeory device driver. Then we added ioctls to the disk driver layer to write named blocks of memory from that device to the raw device (we didin't use a filesystem). We also changed the network drivers to use named blocks of memory in that device for packet reception. We added a special protocol which used recvmsg() and and sendmesg() to pass ownership of those named blocks between the process that had mmapped them and the protocol. The protocol had an IP header but also a small protocol specific header of our own.. we sent packets that were larger than a page. zero copy from disk->process, process->network and network->disk (and reverse of course) of course it was all on proprietary protocols so not of use to you. julian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?515B1C30.5080703>