From owner-freebsd-current@FreeBSD.ORG Sat Feb 19 19:10:38 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9C19F10657D6 for ; Sat, 19 Feb 2011 19:10:38 +0000 (UTC) (envelope-from jeremie@le-hen.org) Received: from smtpfb1-g21.free.fr (smtpfb1-g21.free.fr [212.27.42.9]) by mx1.freebsd.org (Postfix) with ESMTP id AE5998FC15 for ; Sat, 19 Feb 2011 19:10:35 +0000 (UTC) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by smtpfb1-g21.free.fr (Postfix) with ESMTP id B318F2D2BF for ; Sat, 19 Feb 2011 19:50:55 +0100 (CET) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id 89434D480B3 for ; Sat, 19 Feb 2011 19:50:45 +0100 (CET) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id 43F4333CF9 for ; Sat, 19 Feb 2011 18:50:44 +0000 (UTC) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id 10783A14D1; Sat, 19 Feb 2011 18:50:44 +0000 (UTC) Date: Sat, 19 Feb 2011 19:50:43 +0100 From: Jeremie Le Hen To: freebsd-current@FreeBSD.org Message-ID: <20110219185043.GA6573@felucia.tataz.chchile.org> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="u3/rZRmxL6MmkK24" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Subject: [RFC] Force stdio output streams to line-buffered mode X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 Feb 2011 19:10:38 -0000 --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hi, I've been annoyed multiple time when running a command such like iostat -x 1 | grep -v ad10 | cat -n The problem stems from two factors: - grep's stdio sees that its stdout is not a terminal, so stdout is full buffered and not line-buffered; - iostat produces output too slowly so the aforementioned buffer takes numerous seconds to be filled and flushed to the last command. This problems is not specific to FreeBSD, it is actually a consequence of POSIX specification. I've checked this on Solaris and Linux. I've attached a small patch for stdio, so if the environment variable STDIO_IOLBF is set, the output streams will be line-oriented by default. iostat -x 1 | env STDIO_IOLBF=1 grep -v ad10 | cat -n Before send it as a PR, I would like to hear your comments about this, especially: - the variable name (no bikeshed please, I just ask this if there is a naming convention I'm not aware of); - the documentation: I've put a hint in stdio(3) manpage and put the full explanation in setvbuf(3). Thanks. Regards, -- Jeremie Le Hen Humans are born free and equal. But some are more equal than others. Coluche --u3/rZRmxL6MmkK24 Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="STDIO_IOLBF.diff" diff -rup /usr/src.orig/lib/libc/stdio/makebuf.c /usr/src/lib/libc/stdio/makebuf.c --- /usr/src.orig/lib/libc/stdio/makebuf.c 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/makebuf.c 2011-02-19 19:09:56.000000000 +0100 @@ -59,6 +59,7 @@ __smakebuf(fp) FILE *fp; { void *p; + char *bmode; int flags; size_t size; int couldbetty; @@ -79,7 +80,8 @@ __smakebuf(fp) flags |= __SMBF; fp->_bf._base = fp->_p = p; fp->_bf._size = size; - if (couldbetty && isatty(fp->_file)) + if (((bmode = getenv("STDIO_IOLBF")) && bmode[0] != '\0') || + (couldbetty && isatty(fp->_file))) flags |= __SLBF; fp->_flags |= flags; } diff -rup /usr/src.orig/lib/libc/stdio/setbuf.3 /usr/src/lib/libc/stdio/setbuf.3 --- /usr/src.orig/lib/libc/stdio/setbuf.3 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/setbuf.3 2011-02-19 19:09:13.000000000 +0100 @@ -79,7 +79,9 @@ and an optimally-sized buffer is obtaine If a stream refers to a terminal (as .Dv stdout -normally does) it is line buffered. +normally does), or the environment variable +.Ev STDIO_IOLBF +is set, it is line buffered. The standard error stream .Dv stderr is always unbuffered. @@ -176,6 +178,12 @@ The function returns what the equivalent .Fn setvbuf would have returned. +.Sh ENVIRONMENT +.Bl -tag -width ".Ev STDIO_IOLBF" +If the environment variable +.Ev STDIO_IOLBF +is set, output streams will be line-buffered by default +even when not referring to a terminal. .Sh SEE ALSO .Xr fclose 3 , .Xr fopen 3 , diff -rup /usr/src.orig/lib/libc/stdio/stdio.3 /usr/src/lib/libc/stdio/stdio.3 --- /usr/src.orig/lib/libc/stdio/stdio.3 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/stdio.3 2011-02-19 12:56:00.000000000 +0100 @@ -137,7 +137,8 @@ an interactive or .Dq terminal device, as determined by the .Xr isatty 3 -function. +function (this can be overriden with an environment variable, see +.Xr setvbuf 3 ) . In fact, .Em all freshly-opened streams that refer to terminal devices --u3/rZRmxL6MmkK24--