From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 21 01:12:40 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id CB10616A400 for ; Thu, 21 Jun 2007 01:12:40 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from rock.polstra.com (rock.polstra.com [64.119.0.113]) by mx1.freebsd.org (Postfix) with ESMTP id A9B3E13C448 for ; Thu, 21 Jun 2007 01:12:35 +0000 (UTC) (envelope-from jdp@polstra.com) Received: from [10.0.0.64] (adsl-sj-11-62.rockisland.net [64.119.11.62]) (authenticated bits=0) by rock.polstra.com (8.13.8/8.13.8) with ESMTP id l5L1CXDC083330 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 20 Jun 2007 18:12:34 -0700 (PDT) (envelope-from jdp@polstra.com) Message-ID: <4679D081.7070600@polstra.com> Date: Wed, 20 Jun 2007 18:12:33 -0700 From: John Polstra User-Agent: Thunderbird 1.5.0.12 (Macintosh/20070509) MIME-Version: 1.0 To: Julian Elischer References: <467787EF.9060009@elischer.org> <46797825.10900@polstra.com> <46799032.5060009@polstra.com> <467999C9.9000402@elischer.org> In-Reply-To: <467999C9.9000402@elischer.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (rock.polstra.com [64.119.0.113]); Wed, 20 Jun 2007 18:12:34 -0700 (PDT) Cc: freebsd-hackers@freebsd.org Subject: Re: in-kernel tcp server X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jun 2007 01:12:40 -0000 Julian Elischer wrote: > I would actually like to address the performance issues. > > is there any chance the oldest version (4.x based) might be released, > or at least it would be nice to get the code snippet that attaches to eh > ng_ksocket and > reads and writes the stream.. > > I could make a TCP ECHO node that way and use it for tracking down the > bottlenecks > I'm not too interested in the actual webserver itself. I don't have the ksocket version any more. It was an early experiment (in 2001) that I discarded pretty quickly. The later 4.x-based version that bypassed the TCP stack and socket layer performed well on uniprocessor systems. I didn't feel netgraph was a performance problem at all on that version. But as multiprocessor systems became more mainstream, the 4.x version wasn't able to take advantage of the added CPUs. Also, it didn't support ACPI and had trouble booting on some of the newer hardware. For those reasons, I updated to a 7.x-based system. At that point, the newer SMP-friendly netgraph started to impact performance pretty seriously. The allocation/deallocation of netgraph's queue items seemed to be a big part of the problem. In 4.x we just passed mbufs around, without any other allocations or deallocations. In 7.x, the mbufs are wrapped up in queue items that have to be allocated and freed, and that added a lot of overhead. I think also that the reader-writer locking in netgraph was impacting performance. It's a really elegant locking scheme, but my node graphs were so simple that I didn't really need it. I don't view netgraph as having serious performance problems. It's just that I was aiming for maximal performance (in terms of HTTP sessions per second), and was willing to do otherwise unreasonable things to get it. John