From owner-freebsd-hackers@FreeBSD.ORG Thu Jun 21 01:42:56 2007 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0072B16A468 for ; Thu, 21 Jun 2007 01:42:56 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outS.internet-mail-service.net (outS.internet-mail-service.net [216.240.47.242]) by mx1.freebsd.org (Postfix) with ESMTP id DE09813C447 for ; Thu, 21 Jun 2007 01:42:55 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Wed, 20 Jun 2007 18:42:55 -0700 Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id D9FFA125B4B; Wed, 20 Jun 2007 18:42:54 -0700 (PDT) Message-ID: <4679D79E.5030200@elischer.org> Date: Wed, 20 Jun 2007 18:42:54 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: John Polstra References: <467787EF.9060009@elischer.org> <46797825.10900@polstra.com> <46799032.5060009@polstra.com> <467999C9.9000402@elischer.org> <4679D081.7070600@polstra.com> In-Reply-To: <4679D081.7070600@polstra.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org Subject: Re: in-kernel tcp server X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jun 2007 01:42:56 -0000 John Polstra wrote: > Julian Elischer wrote: > >> I would actually like to address the performance issues. >> >> is there any chance the oldest version (4.x based) might be released, >> or at least it would be nice to get the code snippet that attaches to >> eh ng_ksocket and >> reads and writes the stream.. >> >> I could make a TCP ECHO node that way and use it for tracking down the >> bottlenecks >> I'm not too interested in the actual webserver itself. > > I don't have the ksocket version any more. It was an early experiment > (in 2001) that I discarded pretty quickly. > > The later 4.x-based version that bypassed the TCP stack and socket layer > performed well on uniprocessor systems. I didn't feel netgraph was a > performance problem at all on that version. But as multiprocessor > systems became more mainstream, the 4.x version wasn't able to take > advantage of the added CPUs. Also, it didn't support ACPI and had > trouble booting on some of the newer hardware. > > For those reasons, I updated to a 7.x-based system. At that point, the > newer SMP-friendly netgraph started to impact performance pretty > seriously. The allocation/deallocation of netgraph's queue items seemed > to be a big part of the problem. In 4.x we just passed mbufs around, > without any other allocations or deallocations. In 7.x, the mbufs are > wrapped up in queue items that have to be allocated and freed, and that > added a lot of overhead. I think also that the reader-writer locking in > netgraph was impacting performance. It's a really elegant locking > scheme, but my node graphs were so simple that I didn't really need it. Hmm ok, but if you did find some old ksocket based code sitting around, i'd love to try it in -current and work on the bottlenecks.. I do need to try fix netgraph bottlenecks... I'll certainly look at what I can do about the queue items. I may make a per-cpu cache of them. > > I don't view netgraph as having serious performance problems. It's just > that I was aiming for maximal performance (in terms of HTTP sessions per > second), and was willing to do otherwise unreasonable things to get it. > > John