Date: Wed, 05 Nov 2008 23:21:13 +0400 From: rihad <rihad@mail.ru> To: Peter Jeremy <peterjeremy@optushome.com.au> Cc: freebsd-hackers@freebsd.org Subject: Re: Asynchronous pipe I/O Message-ID: <4911F229.2010502@mail.ru> In-Reply-To: <20081105181309.GB51239@server.vk2pj.dyndns.org> References: <4911A23B.7050104@mail.ru> <20081105181309.GB51239@server.vk2pj.dyndns.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Peter Jeremy wrote: > On 2008-Nov-05 17:40:11 +0400, rihad <rihad@mail.ru> wrote: >> Imagine this shell pipeline: >> >> sh prog1 | sh prog2 >> >> >> As given above, prog1 blocks if prog2 hasn't yet read previously written >> data (actually, newline separated commands) or is busy. What I want is >> for prog1 to never block: >> >> sh prog1 | buffer | sh prog2 > > There's also misc/mbuffer which is supposed to be an enhancement of > misc/buffer - though I haven't used either. I have a program I wrote > to do this but it's not in a releasable state. > That thing only works for certain types of devices (tape?): $ while :; do echo hi; sleep 1; done | mbuffer -q | cat warning: Could not stat output device (unsupported by system)! This can result in incorrect written data when using multiple volumes. Continue at your own risk! ^C $ While writing this email I tried another incantation, and buffer finally worked! $ while :; do echo hi; sleep 1; done | buffer -s1 -b100 | cat hi hi hi hi hi ^C (each line was output one second apart). Once again thank to everybody participating! >> Wouldn't such an intermediary tool be a great way to boost performance >> for certain types of solutions? > > I've found that for dump|restore or dump|gzip, I can get quite significant > speedups by adding a buffer that is several hundred MB in the middle. > Well, if OS buffers aren't good enough, then throwing some memory at disk I/O surely helps ;)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4911F229.2010502>