Date: Thu, 12 Apr 2007 20:08:49 +0100 (BST) From: Daniel Taylor <daniel_h_taylor@yahoo.co.uk> To: freebsd-hackers@freebsd.org Subject: tcp connection splitter Message-ID: <20070412190849.63355.qmail@web27705.mail.ukl.yahoo.com>
next in thread | raw e-mail | index | archive | help
Not a FreeBSD question per se but I thought I'd ask anyway, as there are many knowledgeable people here. I am writing a server (which btw is going to run on FreeBSD), acting as a streaming gateway for multiple TCP clients. The server connects to a source of data (which only accepts a single connection) and needs to distribute this data to multiple TCP clients. With the exception of the initial login sequence, communication is mostly one-way (data source to server, and server to clients). What is the most efficient data structure for this scenario? I need something that would scale to many client connections. The simplest solution is to maintain a separate packet queue for each client, and copy each packet received from the source to all N client queues. Of course, this solution involves a lot of memory allocations (the server receives 1-10MB of data/second), a lot of memcpy()s, and doesn't scale very well. Also, adding a packet to N queues is expensive because it needs to acquire and release N mutex locks (one for each client queue.) At the moment, I have a solution which allocates a reference-counted packet, and adds a *pointer* to it to each of the N client queues. This saves on memcpy because copying a pointer is very fast. Each enqueue bumps the refcount, each dequeue decreases it; when the refcount drops to 0, the packet is free()'d (by whomever happened to dequeue it last). If the clients are "slow" (can't keep up with the source), it's ok to drop them; hence I'm toying with the idea of having a single circular queue accessed by N clients. It would certainly save on memory allocation, but at the same time increase lock contention, because the server and N clients will all be locking the same queue mutex to access it. I could use read-write locks here (single writer, multiple readers) but it doesn't help with write lock contention. There must be some clever lockless algorithm to achieve the same effect. Any suggestions? Thanks, Dan ___________________________________________________________ To help you stay safe and secure online, we've developed the all new Yahoo! Security Centre. http://uk.security.yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070412190849.63355.qmail>