From owner-freebsd-hackers@FreeBSD.ORG Wed Jun 20 19:10:25 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E4D9316A46C for ; Wed, 20 Jun 2007 19:10:25 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from rock.polstra.com (rock.polstra.com [64.119.0.113]) by mx1.freebsd.org (Postfix) with ESMTP id ACFA513C43E for ; Wed, 20 Jun 2007 19:10:25 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from [10.0.0.64] (adsl-sj-11-62.rockisland.net [64.119.11.62]) (authenticated bits=0) by rock.polstra.com (8.13.8/8.13.8) with ESMTP id l5KItXX2077058 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 20 Jun 2007 11:55:33 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: <46797825.10900@polstra.com> Date: Wed, 20 Jun 2007 11:55:33 -0700 From: John Polstra User-Agent: Thunderbird 1.5.0.12 (Macintosh/20070509) MIME-Version: 1.0 To: Julian Elischer References: <467787EF.9060009@elischer.org> In-Reply-To: <467787EF.9060009@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (rock.polstra.com [64.119.0.113]); Wed, 20 Jun 2007 11:55:34 -0700 (PDT) Cc: freebsd-hackers@freebsd.org, Nicolas Cormier Subject: Re: in-kernel tcp server X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 20 Jun 2007 19:10:26 -0000 Julian Elischer wrote: > Nicolas Cormier wrote: >> Hi, >> >> I'm trying to write a little tcp-server kernel module (like tftp). >> I didn't find a lot of documents about the kernel network programming, >> just one thread which talks about netgraph. >> In the freebsd includes I found /usr/include/sys/socketvar.h (so*). >> >> What's the easy way to create a basic tcp server >> (create/bind/listen/accept/send/recv) : use netgraph's ksocket or so* >> ? >> >> Thanks in advance ! >> PS: the whole job must be done in the kernel. > > > > yes it can (and has been) done.. > John Polstra did it many years ago.. using netgraph ksockets. > He had an in-kernel web server. > At least I THINK it was him :-) Yes, that's right. I started out using netgraph ksockets, but later on it evolved, mainly for performance reasons. (I needed it to be really, really fast.) The first change was that I eliminated the ksockets and worked directly at the link layer, using ng_ether nodes. I implemented a small, stripped down TCP stack and bypassed the FreeBSD native TCP/IP/socket layers. This was still done with netgraph, using just the ng_ether nodes talking to my own ng_webclient / ng_webserver nodes. It improved the performance immensely. More recently I restructured it quite a bit to get better MP performance using FreeBSD 7.x. (The original version was based on 4.x). I found that the netgraph locking and internode communication mechanism impacted performance too much under 7.x. So I eliminated the ng_ether nodes and made the webserver / webclient nodes talk directly to the interfaces via the if_input / if_output hooks. It still uses netgraph, but really only as a configuration and management mechanism. No actual network traffic flows between netgraph nodes. This change also resulted in a big performance improvement. Unfortunately, my contract forbids me to release the source code publicly. John