From nobody Mon Apr 3 23:01:16 2023 X-Original-To: questions@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pr61Q3WhPz43Spr for ; Mon, 3 Apr 2023 23:05:14 +0000 (UTC) (envelope-from naddy@mips.inka.de) Received: from mail.inka.de (mail.inka.de [IPv6:2a04:c9c7:0:1073:217:a4ff:fe3b:e77c]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pr61Q17R6z3MpH for ; Mon, 3 Apr 2023 23:05:13 +0000 (UTC) (envelope-from naddy@mips.inka.de) Authentication-Results: mx1.freebsd.org; none Received: from mips.inka.de (naddy@[127.0.0.1]) by mail.inka.de with uucp (rmailwrap 0.5) id 1pjTEX-005nDj-7q; Tue, 04 Apr 2023 01:05:05 +0200 Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.17.1/8.17.1) with ESMTP id 333N1GtG074733; Tue, 4 Apr 2023 01:01:16 +0200 (CEST) (envelope-from naddy@lorvorc.mips.inka.de) Received: (from naddy@localhost) by lorvorc.mips.inka.de (8.17.1/8.17.1/Submit) id 333N1GnZ074732; Tue, 4 Apr 2023 01:01:16 +0200 (CEST) (envelope-from naddy) Date: Tue, 4 Apr 2023 01:01:16 +0200 From: Christian Weisgerber To: Pete Cc: questions@freebsd.org Subject: Re: Clogged pipe? Message-ID: References: List-Id: User questions List-Archive: https://lists.freebsd.org/archives/freebsd-questions List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4Pr61Q17R6z3MpH X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:202113, ipnet:2a04:c9c7::/32, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Pete: > On FreeBSD 13.1-RELEASE-p7 I have a small test file, named testfile, > which contains three short lines of text, "one," "two," and "three" > without the quotes. > > This command waits indefinitely without producing any output: > > tail -f testfile | cat -n | sed -u 's/^/X/' Stdio buffering. See setbuf(3). By default, output using stdio functions is buffered. If it goes to a tty, it is line-buffered, i.e., output is saved up until a whole line terminated by '\n' is complete, then that line is written. Output to files or pipes is block-buffered. Output is saved up until 8 kB or 16 kB or such, and only then is it written. The buffer is also flushed on program exit. > But, these five commands all work as expected, immediately outputting > versions of the three lines of text: > > tail -f testfile | cat -n tail(1) specifically avoids stdio buffering. cat(1) here writes to a tty, so output is only line-buffered. > cat -n testfile | sed -u 's/^/X/' cat(1) terminates, so it writes all its output. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail -f testfile | sed -u 's/^/X/' tail(1) specifically avoids stdio buffering. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail testfile | cat -n | sed -u 's/^/X/' tail(1) terminates, so it writes all its output. cat(1) terminates, so it writes all its output. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > tail -f testfile | cat | sed -u 's/^/X/' tail(1) specifically avoids stdio buffering. Looks like plain cat(1) also avoids buffering and only introduces buffering when you specify line-oriented filter functions such as -n. The -u option to sed(1) disables buffering, although it would default to line-buffering here anyway if output goes to a tty. > Interestingly (or maybe not), the issue doesn't occur on my Debian box; > all six commands produce immediate output there in both dash and bash. Differences in default buffering behavior. Note that you can explicitly disable cat(1)'s output buffering with the -u option. There is also stdbuf(1). -- Christian "naddy" Weisgerber naddy@mips.inka.de