From owner-freebsd-current@FreeBSD.ORG Fri Feb 13 19:45:24 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5727D16A4CE for ; Fri, 13 Feb 2004 19:45:24 -0800 (PST) Received: from endif.cjb.net (65-101-229-205.dnvr.qwest.net [65.101.229.205]) by mx1.FreeBSD.org (Postfix) with SMTP id B254E43D1F for ; Fri, 13 Feb 2004 19:45:23 -0800 (PST) (envelope-from end@endif.cjb.net) Received: (qmail 25983 invoked from network); 14 Feb 2004 03:45:22 -0000 Received: from localhost (127.0.0.1) by localhost with SMTP; 14 Feb 2004 03:45:22 -0000 Date: Fri, 13 Feb 2004 20:45:21 -0700 From: Robin Schoonover To: Steve Young In-Reply-To: References: <200402090704.i1974IVa032065@the-macgregors.org> X-Mailer: Sylpheed version 0.9.9claws (GTK+ 1.2.10; i386-portbld-freebsd5.2.1) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Message-Id: <20040214034523.B254E43D1F@mx1.FreeBSD.org> cc: freebsd-current@freebsd.org Subject: Re: grep bug X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2004 03:45:24 -0000 On Fri, 13 Feb 2004 19:51:23 -0700, Steve Young wrote: > > Arguably grep shouldn't eat up all your > memory when this happens but that's more of a GNU issue than a FreeBSD > issue, and really it would be kind of hard to handle > gracefully/portably. Have you reported this to the gnu bugs mailing > address? It may conceivably use 100% CPU but it shouldn't try and grab > 100% memory too. > grep has to pull in the entire line, and /dev/zero is a line with no end, so it tries pulling it all into memory until there is no more memory for a line to go into. (This part has been mentioned on this list before.) The reason is simple: regular expressions. You may have a regex that matches from beginning to end (^ to $), which means we need the entire line before we can determine if there is a match or not. plain grep (as far as I know) shouldn't require an entire line to operate on, just the length of the search. (There may be other reasons too) However, you rarely have lines long enough to make enough of a difference to cause problems. (1000 char lines? 1000000 char lines?) IMHO, it wouldn't be worth making the grep code more complex to handle such rare cases. -- Robin Schoonover (aka End) # "The POP3 server service depends on the SMTP server service, which # failed to start because of the following error: # The operation completed successfully." -- Windows NT Server v3.51