From owner-freebsd-bugs@FreeBSD.ORG Fri Feb 10 19:20:13 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4A76E106564A for ; Fri, 10 Feb 2012 19:20:13 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 343128FC17 for ; Fri, 10 Feb 2012 19:20:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q1AJKCZr081890 for ; Fri, 10 Feb 2012 19:20:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q1AJKCtb081889; Fri, 10 Feb 2012 19:20:12 GMT (envelope-from gnats) Date: Fri, 10 Feb 2012 19:20:12 GMT Message-Id: <201202101920.q1AJKCtb081889@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org From: Martin Cracauer Cc: Subject: Re: bin/164947: tee looses data when writing to non-blocking file descriptors X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Martin Cracauer List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Feb 2012 19:20:13 -0000 The following reply was made to PR bin/164947; it has been noted by GNATS. From: Martin Cracauer To: Diomidis Spinellis Cc: freebsd-gnats-submit@freebsd.org Subject: Re: bin/164947: tee looses data when writing to non-blocking file descriptors Date: Fri, 10 Feb 2012 14:17:36 -0500 Diomidis Spinellis wrote on Fri, Feb 10, 2012 at 07:04:41AM +0000: > > >Number: 164947 > >Category: bin > >Synopsis: tee looses data when writing to non-blocking file descriptors > >Confidential: no > >Severity: serious > >Priority: medium > >Responsible: freebsd-bugs > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Fri Feb 10 07:10:09 UTC 2012 > >Closed-Date: > >Last-Modified: > >Originator: Diomidis Spinellis > >Release: 8.1 > >Organization: > AUEB > >Environment: > FreeBSD istlab.dmst.aueb.gr 8.1-RELEASE-p6 FreeBSD 8.1-RELEASE-p6 #0: Tue Nov 1 15:16:34 EET 2011 dds@istlab.dmst.aueb.gr:/usr/obj/usr/src/sys/ISTLAB i386 > You have new mail in /var/mail/dds > > >Description: > When tee(1) tries to write to a file descriptor that has been set to non-blocking mode the write(2) call may fail with EAGAIN. Instead of retrying the operation, tee will throw that chunk of data away. > >How-To-Repeat: > Run the following: > #!/usr/local/bin/bash > # bash needed for the >(...) functionality > # ssh apparently sets O_NONBLOCK > # Remove the 2>/dev/null to see tee complaining > dd count=100000 if=/dev/zero | > tee >(ssh localhost dd of=/dev/null) 2>/dev/null | > (ssh localhost dd of=/dev/null) I don't think it is ssh that is causing this. If you use a named pipe explicitly and hook ssh up to that the error doesn't appear. Seems to be something that bash is doing there. That doesn't mean I am opposed to handling EAGAIN. The way I normally do it is a simple retry loop, not using select. I'm aware of the tradeoffs, so far I was always better off not investing a second system call into every retry. Martin > 100000+0 records in > 100000+0 records out > 51200000 bytes transferred in 9.224390 secs (5550503 bytes/sec) > 100000+0 records in > 100000+0 records out > 51200000 bytes transferred in 9.061471 secs (5650297 bytes/sec) > 92080+0 records in > 92080+0 records out > 47144960 bytes transferred in 9.101738 secs (5179776 bytes/sec) > > >Fix: > I attach a patch that fixes the problem. > > Patch attached with submission follows: > > --- tee.c 2012/02/08 14:50:10 1.1 > +++ tee.c 2012/02/08 14:59:10 > @@ -46,8 +46,10 @@ > #endif /* not lint */ > > #include > +#include > #include > #include > +#include > #include > #include > #include > @@ -64,6 +66,7 @@ > > void add(int, const char *); > static void usage(void); > +static void waitfor(int fd); > > int > main(int argc, char *argv[]) > @@ -110,9 +113,14 @@ > bp = buf; > do { > if ((wval = write(p->fd, bp, n)) == -1) { > - warn("%s", p->name); > - exitval = 1; > - break; > + if (errno == EAGAIN) { > + waitfor(p->fd); > + wval = 0; > + } else { > + warn("%s", p->name); > + exitval = 1; > + break; > + } > } > bp += wval; > } while (n -= wval); > @@ -141,3 +149,15 @@ > p->next = head; > head = p; > } > + > +/* Wait for the specified fd to be ready for writing */ > +static void > +waitfor(int fd) > +{ > + fd_set writefds; > + > + FD_ZERO(&writefds); > + FD_SET(fd, &writefds); > + if (select(fd + 1, NULL, &writefds, NULL, NULL) == -1) > + err(1, "select"); > +} > > > >Release-Note: > >Audit-Trail: > >Unformatted: > _______________________________________________ > freebsd-bugs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-bugs > To unsubscribe, send any mail to "freebsd-bugs-unsubscribe@freebsd.org" -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/