From owner-freebsd-questions Tue Dec 18 15:13:33 2001 Delivered-To: freebsd-questions@freebsd.org Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by hub.freebsd.org (Postfix) with ESMTP id B898837B417 for ; Tue, 18 Dec 2001 15:13:28 -0800 (PST) Received: from chk.phattydomain.com ([12.225.230.182]) by rwcrmhc51.attbi.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20011218231234.SAAW19716.rwcrmhc51.attbi.com@chk.phattydomain.com> for ; Tue, 18 Dec 2001 23:12:34 +0000 To: freebsd-questions@freebsd.org Subject: sh redirect caching issue with fifos Message-Id: <20011218231234.SAAW19716.rwcrmhc51.attbi.com@chk.phattydomain.com> Date: Tue, 18 Dec 2001 23:13:28 +0000 From: chkno@dork.com Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG English: A function named "fout" which will read from stdin, break up input by lines, and output each line one at a time to the given named pipe (fifo). sh: fout () { while read a;do echo $a > $1;done; } The idea being that you could, for example: $ mkfifo blarg $ cat /usr/share/dict/words | fout blarg (then, elsewhere) $ cat blarg A $ cat blarg a $ cat blarg aa $ cat blarg aal $ cat blarg aalii etc., getting one word at a time. However, if you go & try this, it doesn't operate as outlined above. For each access of the fifo, it returns anywhere from 1 to ~5000 words. sh caches the lines redirected with ">", and then writes them in chunks. This is a great optimization for flat files, but it goes aginst the intention of the code when used with fifos. Changing the function to: fout () { while read a;do echo $a > $1;sleep .001;done; } fixes this issue, and makes the fifo accesses return one line at a time, but only if sh gets a chance to run often enough. If the load average goes above 10 or so, multiple lines per read will start to slip through. Bumping the sleep value all the way up to .1 buys you some more reliability, but breakes down (on my system) at about a load average of 30. Bumping the value higher makes the code unacceptably slow, and is still prone to breakage if the LA rises high enough. Options, ideas, other workarounds? Am I doing something stupid? -- chkno chkno@dork.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message