From owner-freebsd-hackers@FreeBSD.ORG Sat Mar 1 15:40:28 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D5211065675 for ; Sat, 1 Mar 2008 15:40:28 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from mx.utwente.nl (unknown [IPv6:2001:610:1908:1000:204:23ff:feb7:b8fe]) by mx1.freebsd.org (Postfix) with ESMTP id DF3AA8FC2D for ; Sat, 1 Mar 2008 15:40:27 +0000 (UTC) (envelope-from pieter@degoeje.nl) Received: from lux.student.utwente.nl (lux.student.utwente.nl [130.89.170.81]) by mx.utwente.nl (8.12.10/SuSE Linux 0.7) with ESMTP id m21Fe3Xs023453; Sat, 1 Mar 2008 16:40:03 +0100 From: Pieter de Goeje To: freebsd-hackers@freebsd.org Date: Sat, 1 Mar 2008 16:40:02 +0100 User-Agent: KMail/1.9.7 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200803011640.02961.pieter@degoeje.nl> X-UTwente-MailScanner-Information: Scanned by MailScanner. Contact servicedesk@icts.utwente.nl for more information. X-UTwente-MailScanner: Found to be clean X-UTwente-MailScanner-From: pieter@degoeje.nl X-Spam-Status: No Cc: Martin Laabs Subject: Re: SIGPIPE propagation X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Mar 2008 15:40:28 -0000 On Saturday 01 March 2008, Martin Laabs wrote: > Hello, > > as you probably know I want to expand dump in such > a way it enableds multivolume dumps also when the > output data is written to stdout/a pipe. > Now I ran into a serious problem. > Dump counts the written bytes to remember where > it has to proceede after the volumen change. > Now the SIGPIPE signal does not propagate fast > enought through the system. Thus dump writes > some time after the pipe has got broken data into > the pipe while it doesn't exist anymore. This short > time it does not receive any error about that. > (Until the signal propagates to dump.) > I wrote a small test progam to demonstrate this > effect (and get clear about it by myself). It just > write data to stdout and counts it until the pipe > is broken (and the write call will return -1) This > is the output of a few successively runs: This problem cannot be solved using sigpipe. You will always have a race condition because the reader's only way of notifying the writer that it shouldn't write anymore is through closing the pipe, after which the reader cannot read any remaining data that was just written to the pipe buffer. Given ./a.out | asdf Consider the fact that when you create a pipe, both ends are open. At this point it is possible to write a small amount of data to the pipe without blocking, because of the pipe buffer. The reader intends to close the pipe as soon as possible (command does not exist). Which of these events occurs first is however undefined, so sometimes a.out succeeds in writing data and sometimes it doesn't. writer reader | | write() (succeeds) | | close() | | sigpipe() | write() (fails) | The data in the pipe is now lost. > > > $ ./a.out |asdf > $ asdf: command not found > cought broken pipe > i: 12 could only wrote -1 bytes. total wrote: 61440 > > $ ./a.out |asdf > $ asdf: command not found > cought broken pipe > i: 1 could only wrote -1 bytes. total wrote: 5120 > > $ ./a.out |asdf > $ asdf: command not found > cought broken pipe > i: 12 could only wrote -1 bytes. total wrote: 61440 > > $ ./a.out |asdf > $ asdf: command not found > cought broken pipe > i: 0 could only wrote -1 bytes. total wrote: 0 -- Pieter de Goeje