From owner-freebsd-hackers@FreeBSD.ORG Wed Aug 30 13:44:50 2006 Return-Path: X-Original-To: freebsd-hackers@FreeBSD.org Delivered-To: freebsd-hackers@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9C49D16A4DF; Wed, 30 Aug 2006 13:44:50 +0000 (UTC) (envelope-from babkin@verizon.net) Received: from vms040pub.verizon.net (vms040pub.verizon.net [206.46.252.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2052143D6D; Wed, 30 Aug 2006 13:44:46 +0000 (GMT) (envelope-from babkin@verizon.net) Received: from vms074.mailsrvcs.net ([192.168.1.3]) by vms040.mailsrvcs.net (Sun Java System Messaging Server 6.2-4.02 (built Sep 9 2005)) with ESMTPA id <0J4T004W3CU0EMJD@vms040.mailsrvcs.net>; Wed, 30 Aug 2006 08:44:24 -0500 (CDT) Received: from 198.190.8.100 ([198.190.8.100]) by vms074.mailsrvcs.net (Verizon Webmail) with HTTP; Wed, 30 Aug 2006 08:44:23 -0500 (CDT) Date: Wed, 30 Aug 2006 08:44:23 -0500 (CDT) From: Sergey Babkin X-Originating-IP: [198.190.8.100] To: Gleb Smirnoff , Martin Eugen Message-id: <18678818.48281156945464270.JavaMail.root@vms074.mailsrvcs.net> MIME-version: 1.0 Content-type: text/plain; charset=ISO-8859-1 Content-transfer-encoding: 7bit X-Mailman-Approved-At: Fri, 01 Sep 2006 13:08:10 +0000 Cc: freebsd-hackers@FreeBSD.org Subject: Re: Re: SOCK_DGRAM optimization needed... X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: babkin@users.sf.net List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2006 13:44:50 -0000 >From: Gleb Smirnoff >On Fri, Aug 18, 2006 at 10:41:36AM +0300, Martin Eugen wrote: >M> I have a simple application, that deals with lots of dgram sockets (UDP). >M> Thousands of them. Basically, its purpose is to >M> maintain pairs of sockets and when data is received on one of the sockets it >M> peeks through it (doing some simple >M> statistic calculations) and then forwards it to the other socket. >M> Because of the hudge number of reads and writes (probably about a 10 packets >M> per second per socket pair) it generates a significant load >M> on the system, that I would like to minimize. I'm currently evaluating if it >M> would be possible to add simple 'routing' functionality in the socket layer >M> in the kernel, because frankly I'm not able to think of anything else. > >As Robert said you can try to put this into kernel. That is, you can >write down a netgraph node, that does the routing. Then connect thousands >of ng_ksocket(4) to it. > >If netgraph(4) survives such a big graph (I hope so), you will get quite >fast forwarding. You should also implement a fast ng_findhook method I think it doesn't have to be a big graph. Just make your module sit under IP and look inside all the packets coming up. If the packet matches one it's looking for, update the stats, change the port number and send it up to the IP node. Otherwise just send it to the IP node. With this approach you won't even really need the changing of port numbers if all you do is the stats, just make the "real" app listen on the same ports as where the packets are sent to. BTW, some many years ago I wrote a STREAMS generic IP traffic accounting module that worked like this. -SB