Date: Sat, 18 Oct 2008 17:40:58 -0700 From: Tim Kientzle <kientzle@freebsd.org> To: Ivan Voras <ivoras@freebsd.org> Cc: freebsd-hackers@freebsd.org, Dan Nelson <dnelson@allantgroup.com> Subject: Re: Pipes, cat buffer size Message-ID: <48FA821A.8070200@freebsd.org> In-Reply-To: <9bbcef730810181719x4387a14yec74bdb6893d1a2a@mail.gmail.com> References: <gddjoj$apg$1@ger.gmane.org> <20081018213502.GL99270@dan.emsphone.com> <gddmip$im0$1@ger.gmane.org> <20081018231201.GM99270@dan.emsphone.com> <9bbcef730810181719x4387a14yec74bdb6893d1a2a@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
>>>... the writer could write 1-byte buffers and >>>the reader will be forced to read each byte individually. >> >>No; take a look at /sys/kern/sys_pipe.c . Depending on how much data >>is in the pipe, it switches between async in-kernel buffering (<8192 >>bytes), and direct page wiring between sender and receiver (basically >>zero-copy). > > Ok, maybe it's just not behaving as I thought it should. See this test program: However, when I add "sleep(1)" to your test program, I see the following output: $ dd bs=1 if=/dev/zero | ./reader read 2282 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes read 65536 bytes In your original example, because your sample reader does nothing, it sees each write separately. If the reader was actually doing some work then the pipe would buffer up data while your reader was busy. This looks like exactly the right behavior: The reader will only block if there is no data in the pipe at all; the writer will only block if it gets "too far ahead" of the reader. Except in those cases, each program gets to do I/O as fast as it can. If your program needs larger blocks, it should keep reading until it gets enough data. (The -B option of GNU tar is an example of this sort of behavior.) Tim
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48FA821A.8070200>