From owner-freebsd-questions@freebsd.org Sun Sep 10 13:32:07 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41645E113BD for ; Sun, 10 Sep 2017 13:32:07 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 08A887473E for ; Sun, 10 Sep 2017 13:32:07 +0000 (UTC) (envelope-from luzar722@gmail.com) Received: by mail-io0-x236.google.com with SMTP id v36so285318ioi.1 for ; Sun, 10 Sep 2017 06:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=message-id:date:from:user-agent:mime-version:to:subject :content-transfer-encoding; bh=9gVgKjzpXrf7odLLQx76Ip2eSLT3YLRs6TskzVPaGlw=; b=fMKXJ7IWFLu0YqSlK59lT5EgRHwvSk4F7XKsNkU+mWRFpfAvjPYZqr8Dda8ZjIGd2V J4UyIXznmN2C4mVlZaA4oI2oP3N/I25jEMCtcPWyia9HDfAm9wJqlDwEKyViY/sMaMab 7zb0f69Gd6NQhvGQR6acQhBbREYvjxWFTpUGUQoxinVlylIsfFbLNlSk4x3ICftisf1U i4539E+Z4z0X5w3daMiBtWpMcvgHTgs3wBslRcAj62TzxHrti/AkMMk8cfoMBKioYtqT +mBQ/5rOSbG6NQo9eLlPGFuZkHXN031+X1FCoi3M/eErl4VtLJGfsnpA8iSQL9XZToK/ MQSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:content-transfer-encoding; bh=9gVgKjzpXrf7odLLQx76Ip2eSLT3YLRs6TskzVPaGlw=; b=OgIXcDIa5LAy4HTLV+upeMRK0vIVlFcJNu2H0+vz0vFzwQw+7OQd8RfqyNQnbfjDFW 7jpfnFI5zA2MqpDV9Sj/9jgeXd7fgWeDPubArHc97BRy+JDrCRS/ZZGqay/8gD6McW43 UzzwHlIoWD/B9jQEsxOsDup6Vi35OPBx6slN+XX8ulYDceysfxYbKaARa/lLxXpVS9nD 2MkbEmZLD6KLmxpiPekB2DH+lOSBRDHRARwMiOsoHCEVZ1Ab7aQ/f1UWaCGHPRYY3vV/ pZLmwAnEJpPgIGIguaqigwovCQ/XoZM0BXUUHbwGDqXq96XLbELueeZSr8PAwBQ9BZog ke3Q== X-Gm-Message-State: AHPjjUgdnnkFTLlCdiK16aZunLc423D8q6JCsdqGqmfhI1GyjGt9GfiS IeI46hXTuKPg7RqF X-Google-Smtp-Source: AOwi7QC0DwC3GRCxr/9ry3msWeuAS0KQf0YexbIGbGPkIk7QgPSWYd7nEjI8OqfhRV7yFIkKmcB2tg== X-Received: by 10.107.82.10 with SMTP id g10mr11006296iob.52.1505050326287; Sun, 10 Sep 2017 06:32:06 -0700 (PDT) Received: from [10.0.10.3] (cpe-74-141-88-147.neo.res.rr.com. [74.141.88.147]) by smtp.googlemail.com with ESMTPSA id k189sm3175905itb.41.2017.09.10.06.32.05 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sun, 10 Sep 2017 06:32:05 -0700 (PDT) Message-ID: <59B53ED2.3000409@gmail.com> Date: Sun, 10 Sep 2017 09:32:02 -0400 From: Ernie Luzar User-Agent: Thunderbird 2.0.0.24 (Windows/20100228) MIME-Version: 1.0 To: "freebsd-questions@freebsd.org" Subject: awk help Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Sep 2017 13:32:07 -0000 Hello list; I wrote the following script to read a hosts file I downloaded containing known bad sex domain names intended for addition to /etc/hosts file. This script drops some useless records and builds a local-zone: record for input to unbound. The file contains 55,000 records and this script takes 7 minutes to process it. I know awk can do this same thing must faster. I have searched awk and can not find any reference that would do what the tr '\r' ' '` command does. Would some one point me to documentation with example of how to get the same result with awk as the tr '\r' ' '` command does. Thanks. #! /bin/sh date host_out="$1" host_in="$2" truncate -s 0 $host_out # Make the input file read a line at a time, not a field at a time. IFS=$'\n' set -f for line in `cat $host_in`; do # Locate and replace carriage return with blank. line=`echo -n "${line}" | tr '\r' ' '` # Locate and replace tab with blank. line=`echo -n "${line}" | tr '\t' ' '` # Drop blank lines. blank_line=`echo -n $line | cut -c 1-1` if [ "$blank_line" = " " ]; then continue fi # Drop lines with localhost in it. localhost=`echo -n $line | cut -w -f 2` if [ "$localhost" = "localhost" ]; then continue fi # Drop line with # in cloumn 1 as a comment. comment1=`echo -n $line | cut -c 1-1` if [ "$comment1" = "#" ]; then continue fi # Drop line with word Malvertising starting in cloumn 1 comment1=`echo -n $line | cut -w -f 1` if [ "$comment1" = "Malvertising" ]; then continue fi # Out put record. ip=`echo -n $line | cut -w -f 1` $trace_on echo "ip = ${ip}" if [ "$ip" = "127.0.0.1" -o "$ip" = "0.0.0.0" ]; then domain_name=`echo -n $line | cut -w -f 2` echo "local-zone: \"${domain_name}\" always_nxdomain" >> $host_out continue else domain_name=`echo -n $line | cut -w -f 1` echo "local-zone: \"${domain_name}\" always_nxdomain" >> $host_out fi done date exit 0