From owner-freebsd-questions@FreeBSD.ORG Thu Sep 13 19:37:19 2007 Return-Path: Delivered-To: questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D877116A41B for ; Thu, 13 Sep 2007 19:37:19 +0000 (UTC) (envelope-from kurt.buff@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.233]) by mx1.freebsd.org (Postfix) with ESMTP id 74FA113C45D for ; Thu, 13 Sep 2007 19:37:19 +0000 (UTC) (envelope-from kurt.buff@gmail.com) Received: by wr-out-0506.google.com with SMTP id 70so317689wra for ; Thu, 13 Sep 2007 12:37:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=//Ug4oqp8CP+JLFMjKA7HqEpt1p7wTQUv4K6LuKeRcw=; b=gF76Z3awsDEsSFutIJcvnC5RbozVlbq2KFmrZpwDnsKiUUpd0OKfL5+OW7GBu1tvP0SOnZmMGk+CbG7Chg4u0pJXAHB0hcHBT7vrOxdocy4W78fxdQTVUyV9d/r/AZeey/iPJBWlCI4w2sz+etZSK1aaTqoDu+wSQHHVfjpdimE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=qAEOum0/4faI7WcKKZLirHqWNhqN5hRyZbv6GmG0XBf9uSFPHlQygO2UMaE5lgsf4Ay0d9JL468j2YmlBKBPvOyyNRLYbwwM+C0xr9Ahg0UZXeZEREAmCtz0EJmSHlA+chBQRoE6X04xrITzFN1PrgU6d/23OGktd1aIflyZ2YQ= Received: by 10.142.234.12 with SMTP id g12mr228396wfh.1189712237586; Thu, 13 Sep 2007 12:37:17 -0700 (PDT) Received: by 10.142.78.21 with HTTP; Thu, 13 Sep 2007 12:37:17 -0700 (PDT) Message-ID: Date: Thu, 13 Sep 2007 12:37:17 -0700 From: "Kurt Buff" To: "Craig Whipp" In-Reply-To: <62309.65.121.28.16.1189710531.squirrel@whippsthroughlife.servebeer.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20070913172001.GA78799@gizmo.acns.msu.edu> <20070913175510.GA78984@gizmo.acns.msu.edu> <62309.65.121.28.16.1189710531.squirrel@whippsthroughlife.servebeer.com> Cc: questions@freebsd.org Subject: Re: Scripting question X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Sep 2007 19:37:19 -0000 On 9/13/07, Craig Whipp wrote: > > On 9/13/07, Jerry McAllister wrote: > >> > The only space is the one separating the SMTP address from the OK or > >> NO. > >> > >> Then you should be able to tell it to sort on the first token in > >> the string with white space as a separator and to eliminate > >> duplicates. It has been a long time since I had need of sort. I > >> don't remember the arguments/flags but am sure that type of thing can be > >> done. > >> > >> ////jerry > > > > Ya know, it's really easy to get wrapped around the axle on this stuff. > > > > I think I may have a better solution. The file I'm trying to massage > > has a predecessor - the non-unique lines are the result of a > > concatenation of two files. > > > > Silly me, it's better to 'grep -v' with the one file vs. the second > > rather than trying to merge, sort and further massage the result. The > > fix will be to use sed against the first file to remove the ' NO', > > thus providing a clean argument for grepping the other file. > > > > Sigh. > > > > Kurt > > > It sounds like you've found your solution, but how about the below shell > script? Probably woefully inefficient, but should work. > > - Craig > > ########### begin script ############## > #!/bin/sh > # Read in an input list of 2 column data pairs and output the pairs where > the first columns are unique. > > INPUT_FILE="list.txt" > OUTPUT_FILE="new_list.txt" > NON_UNIQ_LIST="" > > for NON_UNIQ in `cat $INPUT_FILE | awk '{print $1}' | sort | uniq -c | > grep -vE '^ *1' | awk '{print $2}'` > do > NON_UNIQ_LIST=$NON_UNIQ_LIST"|"$NON_UNIQ > done > > NON_UNIQ_LIST=`echo $NON_UNIQ_LIST | sed 's/^.//'` > > cat $INPUT_FILE | grep -vE $NON_UNIQ_LIST > $OUTPUT_FILE > ########### end script ############## I'll fiddle with this too, but I like the perl better.