From owner-freebsd-questions@freebsd.org Sat Apr 8 17:00:18 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3BDFAD357CF for ; Sat, 8 Apr 2017 17:00:18 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0123FCD7 for ; Sat, 8 Apr 2017 17:00:18 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-io0-x235.google.com with SMTP id t68so31111604iof.0 for ; Sat, 08 Apr 2017 10:00:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-transfer-encoding; bh=OhXmFVUiWC0EMqv46F2CY7/DuJBPOrIMfn2eZbn37VE=; b=INVCH8eH8dWN8YMq7kMXV99nna0fAovQI5rNm3Rhmz9fJFkOsEJFJ5djdM258et4T6 cL4vCeWA5+t1yqR4GeDHy8z9j9xn3p04Fib4RAWrGEA6dqACZdqytqdyvCTqE6CvPxl7 uRebphRF4+iDz2tMDpulO8oGCNMVLpec5FmEBbnYQ5K8q9dlTNvaAsmavLD+TGWPEczg 5xX/RapnAKpezDG9g8OmAyhzpDxybsyGB+UjCfgB/rmNVlwluId800hoDKc93Gu7zzPK GXIl1af3CiIiXhrNULkrdpxxDxXWHUe6QzQcyIR55U3YL/DTNG/qVDDnTsGGWpP3BP1W 6m5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :cc:subject:references:in-reply-to:content-transfer-encoding; bh=OhXmFVUiWC0EMqv46F2CY7/DuJBPOrIMfn2eZbn37VE=; b=ADKwfJEqatIYaONT1lJoyI1HVAOzqA4VEovn8geRx+MzXi2Di32iKfDIzZQHq8FUa2 kZvxQOB7bO4tTRDj2c3fSH46fwQ51G1MImkh+/FJHrG0JO9xHmcVDfDXXMjQNkIZzFf9 HAhTX//nV4nMm9xa0YQzida3+7V9Fz1k2iHE+iK6/lYaEvAD/Kf+noWcn5uImlF/j9uw kU+afagSJkMwU89zm9W7cwkSDAi7o1FFgeNasyhqoiNKrB6sIGP542Ihd9Cj/YpzeZov dallv4pgtlatGr4nJmSO6ctX3WJ36fQzOjJ6P9SfraXmL0kmCTOLUh8jcDO4JwYfAngi XDLw== X-Gm-Message-State: AFeK/H2atluYqhQIzoLjc58xHYjTPmG5I6t5NcUL0249cVHQ3cz3wwNAjKrYUMDzq1B1VQ== X-Received: by 10.107.47.130 with SMTP id v2mr41448763iov.201.1491670817285; Sat, 08 Apr 2017 10:00:17 -0700 (PDT) Received: from [10.0.10.3] (cpe-74-141-88-57.neo.res.rr.com. [74.141.88.57]) by smtp.googlemail.com with ESMTPSA id h91sm4017045ioi.24.2017.04.08.10.00.16 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 08 Apr 2017 10:00:16 -0700 (PDT) Message-ID: <58E9171F.3060405@gmail.com> Date: Sat, 08 Apr 2017 13:00:15 -0400 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: RW CC: freebsd-questions@freebsd.org Subject: Re: Is there a database built into the base system References: <58E696BD.6050503@gmail.com> <69607026-F68C-4D9D-A826-3EFE9ECE12AB@mac.com> <58E69E59.6020108@gmail.com> <20170406210516.c63644064eb99f7b60dbd8f4@sohara.org> <58E6AFC0.2080404@gmail.com> <20170407001101.GA5885@tau1.ceti.pl> <20170407210629.GR2787@mailboy.kipshouse.net> <58E83E19.8010709@gmail.com> <20170408145503.69ddf649@gumby.homeunix.com> In-Reply-To: <20170408145503.69ddf649@gumby.homeunix.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Apr 2017 17:00:18 -0000 RW via freebsd-questions wrote: > On Fri, 07 Apr 2017 21:34:17 -0400 > Ernie Luzar wrote: > > he op I have been reading all the replies. I know that awk >> exists, but never used it because the man page is so hard to >> understand. I like this manual but this online version is hard to >> navigate. > > I were you I'd start by looking for a shorter tutorial with some good > examples. > > What's nice about awk is the way it's optimized for use in pipelines. > You can run a series of action (blocks of code) against each line, each > action can be preceded by an optional test. There also a BEGIN{} block > where you can do initialization and an END{} block where you can tie > things up. If you don't want to use it that way you can simply put all > the code inside BEGIN{}. > > Beyond this unusual structure it's a pretty straightforward scripting > language. Here is my first try at using awk to Read every record in the input file and drop duplicates records from output file. This what the data looks like. /etc >cat /ip.org.sorted 1.121.136.228; 1.186.172.200; 1.186.172.210; 1.186.172.218; 1.186.172.218; 1.186.172.218; 1.34.169.204; 101.109.155.81; 101.109.155.81; 101.109.155.81; 101.109.155.81; 104.121.89.129; /etc >cat /root/bin/ipf.table.awk.dup #! /bin/sh file_in="/ip.org.sorted" file_out="/ip.no-dups" awk '{ in_ip = $1 }' END { (if in_ip = prev_ip) next else prev_ip > $file_out prev_ip = in_ip } $file_in When I run this script it just hangs there. I have to ctrl/c to break out of it. What is wrong with my awk command?