From owner-freebsd-stable@FreeBSD.ORG Tue Dec 25 02:20:16 2007 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 472C516A419 for ; Tue, 25 Dec 2007 02:20:16 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.183]) by mx1.freebsd.org (Postfix) with ESMTP id 21A2013C4EC for ; Tue, 25 Dec 2007 02:20:16 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so3189530waf.3 for ; Mon, 24 Dec 2007 18:20:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=Mh51avYwYsEa6TkYXEh7IJTBzK86tIqnnQeL517VG+w=; b=KciAnbyyUALTIJHPXjsIsoQNc5aSJoa8mVOIvLqGMHs2HOvoQ2mo2SEZ4nTB1kdtJlk8KIc2WbIAVWtk7GTY3y1V3WlK6YlsLKzaFAt4j+Fxc/OoHootyjyOPVwJutY84zlN/Wnabg2CnkDM5oHNuPGjOlu4GyxHwfILR/kX610= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=HT9En6zeQDf4Ai1zvAZqhJNarEN05agy5XJGNaZR88se49kMq1XFGxM4OE1wfXhFIX03BJv5VgtCjfrTABWvyCnJxhx5Fmj876cGyKbli/HWSP+HWJr9fVWzQxQgkLkmEOE4DqclWzM2AdFfv73XoHBx8b4fZwf89Pju7teR4Y8= Received: by 10.142.242.8 with SMTP id p8mr1612624wfh.232.1198549215784; Mon, 24 Dec 2007 18:20:15 -0800 (PST) Received: by 10.143.155.13 with HTTP; Mon, 24 Dec 2007 18:20:15 -0800 (PST) Message-ID: Date: Tue, 25 Dec 2007 11:20:15 +0900 From: "Adrian Chadd" Sender: adrian.chadd@gmail.com To: "Brett Glass" In-Reply-To: <200712241756.KAA21950@lariat.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <200712220531.WAA09277@lariat.net> <476FBED0.2080400@samsco.org> <200712241549.IAA19650@lariat.net> <476FDA10.4060107@samsco.org> <200712241653.JAA20845@lariat.net> <476FE868.8080704@samsco.org> <200712241756.KAA21950@lariat.net> X-Google-Sender-Auth: 8f3294b8dd25f63d Cc: stable@freebsd.org Subject: Re: SMP on FreeBSD 6.x and 7.0: Worth doing? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Dec 2007 02:20:16 -0000 On 25/12/2007, Brett Glass wrote: > >It sounds like you're pretty convinced you know what the problem is. > > Again, I'd have to instrument either the FreeBSD kernel or Squid to > be 100% sure. But it APPEARS that it's a problem with large > numbers of threads trying to do file I/O simultaneously and blocking. I'd still check the namei cache; Squid can and does chew through stupid amounts of pathnames. Its why I hacked up that "ifs" thing a few years ago but was suddenly (contractually) required to not hack on caching for a while.. > (Aside: I wish that Squid used FreeBSD's sendfile(2) API once it got past > the headers and was just streaming out a cached page. I suspect that > this would make delivery of cache hits a lot more efficient, because > sendfile(2) does "zero copy" transfers and works entirely within > the kernel.) There's plenty more work to do in Squid before sendfile() would actually matter. I'm slowly working it all out in my spare time and as support contract funding permits. In fact, I think the Linux evolution of sendfile lets you write out a userspace buffer before you glue it to something else like a socket or file fd. That'd be more helpful here. (No idea on the directory sizes these days - perhaps larger files with dir hashing would help; but I certainly haven't benchmarked it. I just use COSS for small objects. :) Adrian -- Adrian Chadd - adrian@freebsd.org