From owner-freebsd-questions@freebsd.org Sat Sep 16 14:20:13 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF688E19621 for ; Sat, 16 Sep 2017 14:20:13 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mailrelay13.qsc.de (mailrelay13.qsc.de [212.99.187.253]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.antispameurope.com", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5185677A0F for ; Sat, 16 Sep 2017 14:20:12 +0000 (UTC) (envelope-from freebsd@edvax.de) Received: from mx01.qsc.de ([213.148.129.14]) by mailrelay13.qsc.de; Sat, 16 Sep 2017 16:20:03 +0200 Received: from r56.edvax.de (port-92-195-1-252.dynamic.qsc.de [92.195.1.252]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx01.qsc.de (Postfix) with ESMTPS id C1EF23CC75; Sat, 16 Sep 2017 16:20:02 +0200 (CEST) Received: from r56.edvax.de (localhost [127.0.0.1]) by r56.edvax.de (8.14.5/8.14.5) with SMTP id v8GEK1jq002011; Sat, 16 Sep 2017 16:20:01 +0200 (CEST) (envelope-from freebsd@edvax.de) Date: Sat, 16 Sep 2017 16:20:01 +0200 From: Polytropon To: Ernie Luzar Cc: RW , freebsd-questions@freebsd.org Subject: Re: Help scripting dns lookup using awk Message-Id: <20170916162001.4ad176b3.freebsd@edvax.de> In-Reply-To: <59BC6036.8040709@gmail.com> References: <59BB24E4.6060908@gmail.com> <20170915143019.2e02d386@gumby.homeunix.com> <59BC6036.8040709@gmail.com> Reply-To: Polytropon Organization: EDVAX X-Mailer: Sylpheed 3.1.1 (GTK+ 2.24.5; i386-portbld-freebsd8.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-cloud-security-sender: freebsd@edvax.de X-cloud-security-recipient: freebsd-questions@freebsd.org X-cloud-security-Virusscan: CLEAN X-cloud-security-disclaimer: This E-Mail was scanned by E-Mailservice on mailrelay13.qsc.de with 08992683517 X-cloud-security-connect: mx01.qsc.de[213.148.129.14], TLS=1, IP=213.148.129.14 X-cloud-security: scantime:.1364 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Sep 2017 14:20:14 -0000 On Fri, 15 Sep 2017 19:20:22 -0400, Ernie Luzar wrote: > RW via freebsd-questions wrote: > > On Thu, 14 Sep 2017 20:55:00 -0400 > > Ernie Luzar wrote: > > > >> The following sh script works, but runs very slow. > > > > Almost certainly the reason it's slow is that you are doing sequential > > synchronous lookups. Switching to another language isn't going help > > much. To speed it up you either need to switch to a language with a > > DNS library that supports asynchronous lookups or fire-off parallel > > child processes. The latter is easier. > > > > How would I go about coding a sh script to fire-off parallel child > processes? There are several methods. One would be to "pre-sort" per TLD or per IP. You can do this with awk or a sort | grep contruct. Then you iterate over the result files and send the batch lookup processes into background with the & appended. When everything is done, re-combine the result files as needed. -- Polytropon Magdeburg, Germany Happy FreeBSD user since 4.0 Andra moi ennepe, Mousa, ...