Date: Sat, 13 Feb 1999 14:23:45 -0500 (EST) From: "John S. Dyson" <dyson@iquest.net> To: dot@dotat.at (Tony Finch) Cc: hackers@FreeBSD.ORG Subject: Re: PIPE_BUF Message-ID: <199902131923.OAA05392@y.dyson.net> In-Reply-To: <E10Bg8l-0007WQ-00@fanf.noc.demon.net> from Tony Finch at "Feb 13, 99 02:33:51 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
Tony Finch said: > Mike Smith <mike@smith.net.au> wrote: > >> I've been looking at the Apache code for doing buffered writes to > >> logs, which it attempts to do in such a way that log records are not > >> split across buffer boundaries. It therefore buffers up to PIPE_BUF > >> bytes to be written in one go. > > > >If it's actually writing into a pipe, it should write as much as > >possible at once under FreeBSD to get best performance. > > How likely am I to fall foul of concurrency issues when doing that? > The reason Apache does what it does is to avoid writes from different > httpds to the same pipe getting confused with each other. > > Why should a large PIPE_BUF have a bad effect on interactive pipes? Is > it simply because of the way stdio buffers writes? > It isn't the PIPE_BUF size precisely, but if stdio makes the buffer size too big, the delay for the flush is sometimes too long for interactive use in command line operations. I am running with a special version of libc that causes pipe stdio buffers to flush more often, even though the maximum pipe size is large. I suggest making PIPE_BUF at least 4K (maybe even 8K), and then modifying stdio so that performs it's writes by default in smaller chunks (512bytes or 1K) if the process has a tty, and the file descriptor is a pipe. The reads should have the largest buffer that makes sense (at least the large PIPE_BUF size.) I really should take a look at the priority mgmt in sys_pipe to properly handle the case of small writes, so that the context switch rate for massive transfers isn't seriously adversely affected. It seems that the optimum I/O size in the stat block should still be large, so that applications that are not stdio based can still decide on their own the optimum I/O size for throughput. Those applications, since they are not stdio based, are taking more responsibility, and if they need to provide lower latency or higher throughput, then they can craft a more complete solution. -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199902131923.OAA05392>