From owner-freebsd-stable@FreeBSD.ORG Mon Dec 24 17:57:02 2007 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E118116A417 for ; Mon, 24 Dec 2007 17:57:02 +0000 (UTC) (envelope-from brett@lariat.net) Received: from lariat.net (lariat.net [66.119.58.2]) by mx1.freebsd.org (Postfix) with ESMTP id 6A3DF13C461 for ; Mon, 24 Dec 2007 17:57:02 +0000 (UTC) (envelope-from brett@lariat.net) Received: from anne-o1dpaayth1.lariat.org (IDENT:ppp1000.lariat.net@lariat.net [66.119.58.2]) by lariat.net (8.9.3/8.9.3) with ESMTP id KAA21950; Mon, 24 Dec 2007 10:56:49 -0700 (MST) Message-Id: <200712241756.KAA21950@lariat.net> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 24 Dec 2007 10:56:45 -0700 To: Scott Long From: Brett Glass In-Reply-To: <476FE868.8080704@samsco.org> References: <200712220531.WAA09277@lariat.net> <476FBED0.2080400@samsco.org> <200712241549.IAA19650@lariat.net> <476FDA10.4060107@samsco.org> <200712241653.JAA20845@lariat.net> <476FE868.8080704@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Cc: stable@freebsd.org Subject: Re: SMP on FreeBSD 6.x and 7.0: Worth doing? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Dec 2007 17:57:03 -0000 At 10:12 AM 12/24/2007, Scott Long wrote: >It's not the same kind of hashing. The kind of "hashing" that squid >does on the filesystem is sub optimal for UFS performance. Squid doesn't do any "hashing" on the file system, as far as I know. It does, of course, have a hashed directory of cached Web objects. What it does, however, is lay out files and subdirectories so that there are no more than 256 entries in each directory. (The top level directory usually has only 16 subdirectories in it.) What I am saying is that by keeping directory size down and using hexadecimal numbers as the file names, the Squid storage conventions will likely allow the UFS hashing scheme to work very well -- possibly optimally. Which isn't a surprise, because they were designed with UFS in mind. >It sounds like you're pretty convinced you know what the problem is. Again, I'd have to instrument either the FreeBSD kernel or Squid to be 100% sure. But it APPEARS that it's a problem with large numbers of threads trying to do file I/O simultaneously and blocking. (Aside: I wish that Squid used FreeBSD's sendfile(2) API once it got past the headers and was just streaming out a cached page. I suspect that this would make delivery of cache hits a lot more efficient, because sendfile(2) does "zero copy" transfers and works entirely within the kernel.) >For others who might want help with this, tweaking >vfs.ufs.dirhash_maxmem is what is needed. A bit of a balancing act is >needed if you're on i386 since you'll risk exhausting KVA unless you >also tweak KVA_PAGES. If you'd like, I can reassemble the system and try this. What would your suggested values for these tunables be? --Brett