Date: Tue, 4 Apr 2023 01:01:16 +0200 From: Christian Weisgerber <naddy@mips.inka.de> To: Pete <freebsd-questions-3@voidcaptain.com> Cc: questions@freebsd.org Subject: Re: Clogged pipe? Message-ID: <ZCtavPuXQtYoV0TC@lorvorc.mips.inka.de> In-Reply-To: <f8a7a545-9731-7481-4fb2-bfc77b8ad6f0@slagle.net> References: <f8a7a545-9731-7481-4fb2-bfc77b8ad6f0@slagle.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Pete: > On FreeBSD 13.1-RELEASE-p7 I have a small test file, named testfile, > which contains three short lines of text, "one," "two," and "three" > without the quotes. > > This command waits indefinitely without producing any output: > > tail -f testfile | cat -n | sed -u 's/^/X/' Stdio buffering. See setbuf(3). By default, output using stdio functions is buffered. If it goes to a tty, it is line-buffered, i.e., output is saved up until a whole line terminated by '\n' is complete, then that line is written. Output to files or pipes is block-buffered. Output is saved up until 8 kB or 16 kB or such, and only then is it written. The buffer is also flushed on program exit. > But, these five commands all work as expected, immediately outputting > versions of the three lines of text: > > tail -f testfile | cat -n tail(1) specifically avoids stdio buffering. cat(1) here writes to a tty, so output is only line-buffered. > cat -n testfile | sed -u 's/^/X/' cat(1) terminates, so it writes all its output. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail -f testfile | sed -u 's/^/X/' tail(1) specifically avoids stdio buffering. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail testfile | cat -n | sed -u 's/^/X/' tail(1) terminates, so it writes all its output. cat(1) terminates, so it writes all its output. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail -f testfile | cat | sed -u 's/^/X/' tail(1) specifically avoids stdio buffering. Looks like plain cat(1) also avoids buffering and only introduces buffering when you specify line-oriented filter functions such as -n. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > Interestingly (or maybe not), the issue doesn't occur on my Debian box; > all six commands produce immediate output there in both dash and bash. Differences in default buffering behavior. Note that you can explicitly disable cat(1)'s output buffering with the -u option. There is also stdbuf(1). -- Christian "naddy" Weisgerber naddy@mips.inka.de
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ZCtavPuXQtYoV0TC>