From owner-freebsd-hackers Sat Feb 13 11:23:56 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id LAA06102 for freebsd-hackers-outgoing; Sat, 13 Feb 1999 11:23:56 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from iquest3.iquest.net (iquest3.iquest.net [209.43.20.203]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id LAA06097 for ; Sat, 13 Feb 1999 11:23:54 -0800 (PST) (envelope-from toor@y.dyson.net) Received: (qmail 11912 invoked from network); 13 Feb 1999 19:23:46 -0000 Received: from dyson.iquest.net (HELO y.dyson.net) (198.70.144.127) by iquest3.iquest.net with SMTP; 13 Feb 1999 19:23:46 -0000 Received: (from root@localhost) by y.dyson.net (8.9.1/8.9.1) id OAA05392; Sat, 13 Feb 1999 14:23:45 -0500 (EST) Message-Id: <199902131923.OAA05392@y.dyson.net> Subject: Re: PIPE_BUF In-Reply-To: from Tony Finch at "Feb 13, 99 02:33:51 pm" To: dot@dotat.at (Tony Finch) Date: Sat, 13 Feb 1999 14:23:45 -0500 (EST) Cc: hackers@FreeBSD.ORG From: "John S. Dyson" Reply-To: dyson@iquest.net X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Tony Finch said: > Mike Smith wrote: > >> I've been looking at the Apache code for doing buffered writes to > >> logs, which it attempts to do in such a way that log records are not > >> split across buffer boundaries. It therefore buffers up to PIPE_BUF > >> bytes to be written in one go. > > > >If it's actually writing into a pipe, it should write as much as > >possible at once under FreeBSD to get best performance. > > How likely am I to fall foul of concurrency issues when doing that? > The reason Apache does what it does is to avoid writes from different > httpds to the same pipe getting confused with each other. > > Why should a large PIPE_BUF have a bad effect on interactive pipes? Is > it simply because of the way stdio buffers writes? > It isn't the PIPE_BUF size precisely, but if stdio makes the buffer size too big, the delay for the flush is sometimes too long for interactive use in command line operations. I am running with a special version of libc that causes pipe stdio buffers to flush more often, even though the maximum pipe size is large. I suggest making PIPE_BUF at least 4K (maybe even 8K), and then modifying stdio so that performs it's writes by default in smaller chunks (512bytes or 1K) if the process has a tty, and the file descriptor is a pipe. The reads should have the largest buffer that makes sense (at least the large PIPE_BUF size.) I really should take a look at the priority mgmt in sys_pipe to properly handle the case of small writes, so that the context switch rate for massive transfers isn't seriously adversely affected. It seems that the optimum I/O size in the stat block should still be large, so that applications that are not stdio based can still decide on their own the optimum I/O size for throughput. Those applications, since they are not stdio based, are taking more responsibility, and if they need to provide lower latency or higher throughput, then they can craft a more complete solution. -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message