From owner-freebsd-bugs@FreeBSD.ORG Sun Feb 20 17:00:23 2011 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 69B991065674 for ; Sun, 20 Feb 2011 17:00:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A9ADF8FC15 for ; Sun, 20 Feb 2011 17:00:21 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p1KH0LLa073952 for ; Sun, 20 Feb 2011 17:00:21 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p1KH0LCF073946; Sun, 20 Feb 2011 17:00:21 GMT (envelope-from gnats) Resent-Date: Sun, 20 Feb 2011 17:00:21 GMT Resent-Message-Id: <201102201700.p1KH0LCF073946@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Jeremie Le Hen Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 487FD106566C for ; Sun, 20 Feb 2011 16:50:40 +0000 (UTC) (envelope-from jlh@felucia.tataz.chchile.org) Received: from smtpfb1-g21.free.fr (smtpfb1-g21.free.fr [212.27.42.9]) by mx1.freebsd.org (Postfix) with ESMTP id A29158FC14 for ; Sun, 20 Feb 2011 16:50:37 +0000 (UTC) Received: from smtp5-g21.free.fr (smtp5-g21.free.fr [212.27.42.5]) by smtpfb1-g21.free.fr (Postfix) with ESMTP id 9B98F2E22B for ; Sun, 20 Feb 2011 17:32:31 +0100 (CET) Received: from endor.tataz.chchile.org (unknown [82.233.239.98]) by smtp5-g21.free.fr (Postfix) with ESMTP id 46ED0D481B8; Sun, 20 Feb 2011 17:32:24 +0100 (CET) Received: from felucia.tataz.chchile.org (felucia.tataz.chchile.org [192.168.1.9]) by endor.tataz.chchile.org (Postfix) with ESMTP id E9A8D33CF9; Sun, 20 Feb 2011 16:32:22 +0000 (UTC) Received: by felucia.tataz.chchile.org (Postfix, from userid 1000) id CAEBBA1498; Sun, 20 Feb 2011 16:32:22 +0000 (UTC) Message-Id: <20110220163222.CAEBBA1498@felucia.tataz.chchile.org> Date: Sun, 20 Feb 2011 16:32:22 +0000 (UTC) From: Jeremie Le Hen To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: jeremie@le-hen.org Subject: bin/154915: [patch] Force stdio output streams to line-buffered mode X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Jeremie Le Hen List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Feb 2011 17:00:23 -0000 >Number: 154915 >Category: bin >Synopsis: [patch] Force stdio output streams to line-buffered mode >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Sun Feb 20 17:00:20 UTC 2011 >Closed-Date: >Last-Modified: >Originator: Jeremie Le Hen >Release: FreeBSD 8.1-STABLE i386 >Organization: >Environment: System: FreeBSD felucia.tataz.chchile.org >Description: I've been annoyed multiple time when running a command such like iostat -x 1 | cat -n | grep -v ad10 The problem stems from two factors: - cat's stdio sees that its stdout is not a terminal, so stdout is full buffered and not line-buffered; - iostat produces output too slowly so the aforementioned buffer takes numerous seconds to be filled and flushed to the last command. This problems is not specific to FreeBSD, it is actually a consequence of POSIX specification. I've checked this on Solaris and Linux. I've attached a small patch for stdio, so if the environment variable STDIO_IOLBF is set, the output streams will be line-oriented by default. iostat -x 1 | env STDIO_IOLBF=1 cat -n | grep -v ad10 >How-To-Repeat: >Fix: --- STDIO_IOLBF.diff begins here --- diff -rup /usr/src.orig/lib/libc/stdio/makebuf.c /usr/src/lib/libc/stdio/makebuf.c --- /usr/src.orig/lib/libc/stdio/makebuf.c 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/makebuf.c 2011-02-19 19:09:56.000000000 +0100 @@ -59,6 +59,7 @@ __smakebuf(fp) FILE *fp; { void *p; + char *bmode; int flags; size_t size; int couldbetty; @@ -79,7 +80,8 @@ __smakebuf(fp) flags |= __SMBF; fp->_bf._base = fp->_p = p; fp->_bf._size = size; - if (couldbetty && isatty(fp->_file)) + if (((bmode = getenv("STDIO_IOLBF")) && bmode[0] != '\0') || + (couldbetty && isatty(fp->_file))) flags |= __SLBF; fp->_flags |= flags; } diff -rup /usr/src.orig/lib/libc/stdio/setbuf.3 /usr/src/lib/libc/stdio/setbuf.3 --- /usr/src.orig/lib/libc/stdio/setbuf.3 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/setbuf.3 2011-02-19 19:09:13.000000000 +0100 @@ -79,7 +79,9 @@ and an optimally-sized buffer is obtaine If a stream refers to a terminal (as .Dv stdout -normally does) it is line buffered. +normally does), or the environment variable +.Ev STDIO_IOLBF +is set, it is line buffered. The standard error stream .Dv stderr is always unbuffered. @@ -176,6 +178,12 @@ The function returns what the equivalent .Fn setvbuf would have returned. +.Sh ENVIRONMENT +.Bl -tag -width ".Ev STDIO_IOLBF" +If the environment variable +.Ev STDIO_IOLBF +is set, output streams will be line-buffered by default +even when not referring to a terminal. .Sh SEE ALSO .Xr fclose 3 , .Xr fopen 3 , diff -rup /usr/src.orig/lib/libc/stdio/stdio.3 /usr/src/lib/libc/stdio/stdio.3 --- /usr/src.orig/lib/libc/stdio/stdio.3 2009-08-03 10:13:06.000000000 +0200 +++ /usr/src/lib/libc/stdio/stdio.3 2011-02-19 12:56:00.000000000 +0100 @@ -137,7 +137,8 @@ an interactive or .Dq terminal device, as determined by the .Xr isatty 3 -function. +function (this can be overriden with an environment variable, see +.Xr setvbuf 3 ) . In fact, .Em all freshly-opened streams that refer to terminal devices --- STDIO_IOLBF.diff ends here --- >Release-Note: >Audit-Trail: >Unformatted: