From owner-freebsd-hackers@FreeBSD.ORG Wed Nov 5 18:13:12 2008 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E7B75106567A for ; Wed, 5 Nov 2008 18:13:12 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 7CC5D8FC16 for ; Wed, 5 Nov 2008 18:13:12 +0000 (UTC) (envelope-from peterjeremy@optushome.com.au) Received: from server.vk2pj.dyndns.org (c122-106-215-175.belrs3.nsw.optusnet.com.au [122.106.215.175]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id mA5IDAbd029339 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 6 Nov 2008 05:13:10 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id mA5ID9d7051388; Thu, 6 Nov 2008 05:13:09 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id mA5ID9Pt051387; Thu, 6 Nov 2008 05:13:09 +1100 (EST) (envelope-from peter) Date: Thu, 6 Nov 2008 05:13:09 +1100 From: Peter Jeremy To: rihad Message-ID: <20081105181309.GB51239@server.vk2pj.dyndns.org> References: <4911A23B.7050104@mail.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="H1spWtNR+x+ondvy" Content-Disposition: inline In-Reply-To: <4911A23B.7050104@mail.ru> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 18:13:13 -0000 --H1spWtNR+x+ondvy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2008-Nov-05 17:40:11 +0400, rihad wrote: >Imagine this shell pipeline: > >sh prog1 | sh prog2 > > >As given above, prog1 blocks if prog2 hasn't yet read previously written >data (actually, newline separated commands) or is busy. What I want is >for prog1 to never block: > >sh prog1 | buffer | sh prog2 There's also misc/mbuffer which is supposed to be an enhancement of misc/buffer - though I haven't used either. I have a program I wrote to do this but it's not in a releasable state. >Wouldn't such an intermediary tool be a great way to boost performance >for certain types of solutions? I've found that for dump|restore or dump|gzip, I can get quite significant speedups by adding a buffer that is several hundred MB in the middle. --=20 Peter Jeremy Please excuse any delays as the result of my ISP's inability to implement an MTA that is either RFC2821-compliant or matches their claimed behaviour. --H1spWtNR+x+ondvy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkkR4jUACgkQ/opHv/APuId41ACfagHvmVdaJpqFJgbehUiudfTb HewAn1OqR5EoTgFpiJnaoTWJqvz+pkbR =5DA6 -----END PGP SIGNATURE----- --H1spWtNR+x+ondvy--