Date: Thu, 18 Apr 2002 10:11:02 -0500 From: Dan Nelson <dnelson@allantgroup.com> To: jconner@enterit.com Cc: "Jim C." <notjames@concon.homeip.net>, Michael E Mercer <mmercer@nc.rr.com>, Taylor Dondich <thexder@lvcm.com>, questions@FreeBSD.ORG Subject: Re: More of a scripting question I guess. Message-ID: <20020418151102.GC21402@dan.emsphone.com> In-Reply-To: <1019113236.72577.84.camel@snafu.concon.homeip.net> References: <000901c1e678$bc14fa80$6600a8c0@penguin> <20020418015926.GI72244@dan.emsphone.com> <3CBE2F51.41E19E8A@nc.rr.com> <20020418023400.GJ72244@dan.emsphone.com> <1019106179.72577.32.camel@snafu.concon.homeip.net> <20020418044346.GB21402@dan.emsphone.com> <1019113236.72577.84.camel@snafu.concon.homeip.net>
next in thread | previous in thread | raw e-mail | index | archive | help
In the last episode (Apr 18), Jim C. said: > On Thu, 2002-04-18 at 01:43, Dan Nelson wrote: > > In the last episode (Apr 18), Jim C. said: > > > On Wed, 2002-04-17 at 23:34, Dan Nelson wrote: > > > > In the last episode (Apr 17), Michael E Mercer said: > > > > > another way to do it is this with the same outcome: > > > > > > > > > > while read address ; do > > > > > somecommand $address > > > > > done < file.txt > > > > > > > > But that puts the input file at the end of the command, which > > > > makes it harder to see where the input is coming from, > > > > especially if you have a lot of stuff in your loop. > > > > > > But its still more resource intensive. If you pipe a huge file > > > into while the machine has to store that file into memory before > > > processing it whereas with the redirection it simply gets > > > filtered straight into the while loop line by line. > > > > No it doesn't; > > It does. (not significantly but it does) > > > the 'read' command will pull one line at a time out of the pipe. > > Correct. However, the pipe "fills up" before read starts to take lines > out of it. This is where the resources get taken. Pipes aren't infinite buffers. The pipe fills up at 16k, and simply blocks cat until the script empties it a bit. The resources are taken by the extra syscalls and the context switches between the shell and cat. > Based on these tests I guarantee you the pipe is taking more system > resources (albeit not much) than the redirect is. No argument there. But I'd have switched to C long before trying to process a 15MB file line-by-line in a scripting language :) I think we've strayed into bikeshed territory anyhow. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020418151102.GC21402>