Date: Sun, 01 Feb 2004 05:46:20 +0100 From: des@des.no (Dag-Erling =?iso-8859-1?q?Sm=F8rgrav?=) To: Robert Watson <rwatson@FreeBSD.org> Cc: current@FreeBSD.org Subject: Re: Coalescing pipe allocation Message-ID: <xzp1xpfiezn.fsf@dwp.des.no> In-Reply-To: <Pine.NEB.3.96L.1040131211448.17012A-100000@fledge.watson.org> (Robert Watson's message of "Sat, 31 Jan 2004 21:23:46 -0500 (EST)") References: <Pine.NEB.3.96L.1040131211448.17012A-100000@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Robert Watson <rwatson@FreeBSD.org> writes: > I have not yet macro-benchmarked this change. It will either have a > slight performance benefit if half-closed pipes aren't used extensively or > it doesn't make a difference, be a wash, or if half-closed pipes turn up a > lot, potentially slightly negatively impact performance (seems unlikely). [you may want to check the -current archives for silby@'s excellent message about pipe memory usage last Tuesday] I think our biggest problem with pipes is not half-open pipes but rather their absence; most of the code that uses pipes only uses one direction and does not bother to shut the other down. This is certainly the case for sh(1) and make(1), which I suspect are the heaviest pipe consumers in a typical system (especially in one running our favorite benchmark tool, 'make buildworld'.) The result is that we use almost twice as much memory for pipes as we need to. * solution: allocate pipe buffers on first-use rather than at pipe creation time. Another problem with pipes is that the default pipe buffer size is too large. * solution: reduce the initial allocation to a single page, and grow the buffer later if necessary (up to a limit og four or eight pages) One problem we can't solve easily is the fact that pipe buffers are allocated directly from the pipe vm map, which is a very inefficient operation in FreeBSD. I have a tentative design in my notebook for a more efficient data structure for vm maps, allowing vm_map_find() to run in logarithmic rather than linear time, but this hasn't been high on my priority list as it's fairly hard to get right. BTW, as I am currently working on some of these issues, I'd be happy if you could commit your patches fairly quickly so I don't have to start all over again once they hit the tree. DES --=20 Dag-Erling Sm=F8rgrav - des@des.no
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?xzp1xpfiezn.fsf>