From owner-freebsd-current@FreeBSD.ORG Sat Jan 31 20:46:47 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3040C16A4CE; Sat, 31 Jan 2004 20:46:47 -0800 (PST) Received: from smtp.des.no (flood.des.no [217.116.83.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id C697843D45; Sat, 31 Jan 2004 20:46:44 -0800 (PST) (envelope-from des@des.no) Received: by smtp.des.no (Pony Express, from userid 666) id 3C7945311; Sun, 1 Feb 2004 05:46:43 +0100 (CET) Received: from dwp.des.no (des.no [80.203.228.37]) by smtp.des.no (Pony Express) with ESMTP id 4562F5308; Sun, 1 Feb 2004 05:46:21 +0100 (CET) Received: by dwp.des.no (Postfix, from userid 2602) id BF2D033C6A; Sun, 1 Feb 2004 05:46:20 +0100 (CET) To: Robert Watson References: From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) Date: Sun, 01 Feb 2004 05:46:20 +0100 In-Reply-To: (Robert Watson's message of "Sat, 31 Jan 2004 21:23:46 -0500 (EST)") Message-ID: User-Agent: Gnus/5.090024 (Oort Gnus v0.24) Emacs/21.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Checker-Version: SpamAssassin 2.61 (1.212.2.1-2003-12-09-exp) on flood.des.no X-Spam-Level: ss X-Spam-Status: No, hits=2.6 required=5.0 tests=RCVD_IN_DYNABLOCK, RCVD_IN_SORBS autolearn=no version=2.61 cc: current@FreeBSD.org Subject: Re: Coalescing pipe allocation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 01 Feb 2004 04:46:47 -0000 Robert Watson writes: > I have not yet macro-benchmarked this change. It will either have a > slight performance benefit if half-closed pipes aren't used extensively or > it doesn't make a difference, be a wash, or if half-closed pipes turn up a > lot, potentially slightly negatively impact performance (seems unlikely). [you may want to check the -current archives for silby@'s excellent message about pipe memory usage last Tuesday] I think our biggest problem with pipes is not half-open pipes but rather their absence; most of the code that uses pipes only uses one direction and does not bother to shut the other down. This is certainly the case for sh(1) and make(1), which I suspect are the heaviest pipe consumers in a typical system (especially in one running our favorite benchmark tool, 'make buildworld'.) The result is that we use almost twice as much memory for pipes as we need to. * solution: allocate pipe buffers on first-use rather than at pipe creation time. Another problem with pipes is that the default pipe buffer size is too large. * solution: reduce the initial allocation to a single page, and grow the buffer later if necessary (up to a limit og four or eight pages) One problem we can't solve easily is the fact that pipe buffers are allocated directly from the pipe vm map, which is a very inefficient operation in FreeBSD. I have a tentative design in my notebook for a more efficient data structure for vm maps, allowing vm_map_find() to run in logarithmic rather than linear time, but this hasn't been high on my priority list as it's fairly hard to get right. BTW, as I am currently working on some of these issues, I'd be happy if you could commit your patches fairly quickly so I don't have to start all over again once they hit the tree. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no