From owner-freebsd-questions@FreeBSD.ORG Wed Nov 5 12:23:09 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AEE8106567C for ; Wed, 5 Nov 2008 12:23:09 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx28.mail.ru (mx28.mail.ru [194.67.23.67]) by mx1.freebsd.org (Postfix) with ESMTP id DB01C8FC0A for ; Wed, 5 Nov 2008 12:23:08 +0000 (UTC) (envelope-from rihad@mail.ru) Received: from mx34.mail.ru (mx34.mail.ru [194.67.23.200]) by mx28.mail.ru (mPOP.Fallback_MX) with ESMTP id 6CBA277A762 for ; Wed, 5 Nov 2008 14:13:17 +0300 (MSK) Received: from [217.25.27.27] (port=46291 helo=[217.25.27.27]) by mx34.mail.ru with asmtp id 1KxgK3-000AW0-00 for freebsd-questions@freebsd.org; Wed, 05 Nov 2008 14:13:15 +0300 Message-ID: <49117FCA.5030107@mail.ru> Date: Wed, 05 Nov 2008 15:13:14 +0400 From: rihad User-Agent: Icedove 1.5.0.14eol (X11/20080724) MIME-Version: 1.0 To: freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam: Not detected X-Mras: OK Subject: Asynchronous pipe I/O X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: rihad@mail.ru List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2008 12:23:09 -0000 Imagine this shell pipeline: sh prog1 | sh prog2 As given above, prog1 blocks if prog2 hasn't yet read previously written data (actually, newline separated commands) or is busy. What I want is for prog1 to never block: sh prog1 | buffer | sh prog2 I first thought that the aptly named misc/buffer port would do exactly what I wanted: buffering prog1 output for prog2 to read it at its earliest convenience. That way prog1 would never block (unless it hit buffer's memory limits). Alas, misc/buffer was originally designed for tape backups, and despite its author's stating its applicability to other uses: "This is a program designed initially to speed up writing tapes on remote tape drives, but may be used as a general pipe buffering utility." buffer never starts writing unless the limit given by -s is crossed, which is 10 kbytes by default and cannot be less than 496 bytes, which is too much for me. Ideally I want it to start writing immediately, whenever new data hits its pools. Unfortunately, the -p 0 option doesn't work either: -s size Size in bytes of each block. The default blocksize is 10k to match the normal output of the tar(1) program. -p percentage Only start a write when the given percentage of the internal queue is full. A percentage around 75 often proves best. Defaults to zero. Wouldn't such an intermediary tool be a great way to boost performance for certain types of solutions? Thanks for any tips (Sorry if this was an inappropriate place to ask)