Date: Fri, 31 Mar 2017 11:17:37 +0100 From: krad <kraduk@gmail.com> To: David Mehler <dave.mehler@gmail.com> Cc: Ultima <ultima1252@gmail.com>, freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: shell script guru Message-ID: <CALfReyc%2BCDfNozHbx8OF=rHMh06FQKxGGhw4j_0GZDXx0X1_dA@mail.gmail.com> In-Reply-To: <CAPORhP73=2_5nfOaR=a=TZTOyquBSZRS===FakeJWMPLjpNMjw@mail.gmail.com> References: <CAPORhP5ESqJL%2BkK4tfSD5t5=fnFjsCNXGdUhAjMpezq4WdjKyw@mail.gmail.com> <CADbyKk61wyYj1Jgc9daFTbXE_9s5xPLEYHa4p=KF8FhngzOQ3Q@mail.gmail.com> <CAPORhP6%2Bu4DpUq=9WJ9XmSHDYSJSmXaa6_o7NnVtOq=n_g0v=w@mail.gmail.com> <CAFsnNZL8EgYQK9u_mz4BB%2BULwo9xgsPFT%2BP-4uD4-tqHd%2Bn2QQ@mail.gmail.com> <CANJ8om6svf%2B6sgrV4UW8F=aidaHhWce%2BfNO4-g4Lfa2QteYa7w@mail.gmail.com> <CAPORhP73=2_5nfOaR=a=TZTOyquBSZRS===FakeJWMPLjpNMjw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
you can use the "-O -" option on wget to pipe the file contents to stdout and get rid of the cat and rm. You could also use fetch rather than wget/curl and save installing a port. Use '-o -' for that though. 8) On 31 March 2017 at 04:56, David Mehler <dave.mehler@gmail.com> wrote: > Hello, > > My thanks to everyone who helped me. > > I've got a solution, and have set it in my monthly periodic checks in > /etc/periodic/monthly. > > I've got two solutions both of which retrieve the country database. In > pf I have a table that blocks the ip's in the table. Here's the perl > script: > > #!/usr/bin/env perl > open(ZONES, "<zonesfile"); > > while(<ZONES>) { > > chomp; > > system("wget -4 --no-proxy --no-cookies --no-cache > http://ipdeny.com/ipblocks/data/countries/$_.zone"); > > system("cat $_.zone >>blocked_zones"); > > unlink($_); > > sleep(2); > > } > > close(ZONES); > > ssystem("mv blocked_zones /etc/pf"); > ssystem("pfctl -f /etc/pf.conf)"; > > pf.conf: > table <blocked_countries persist "/etc/pf/blocked_countries" > block in quick from <blocked_tables> > > The zonesfile contains countries in quotes one per line: > > "al" > "cz" > "ch" > ... > > etc > > Here's the script that I put in monthly: > > #!/bin/sh > # > # Monthly retrieve the selected country IP block lists > # Retrieves dns zones from ipdeny.com > # Adds the zones to a country block file > # Then adds them to a pf block table > > # If there is a global system configuration file, suck it in. > # > if [ -r /etc/defaults/periodic.conf ] > then > . /etc/defaults/periodic.conf > source_periodic_confs > fi > > case "$monthly_country_blocks_enable" in > [Yy][Ee][Ss]) > cd /tmp > echo "Retrieving Zones" > for i in "af" "al" "dz" "am" "az" "ba" "br" "kh" "cf" "cn" "co" "cr" > "hr" "cu" "cy" "cz" "do" "eg" "fr" "gi" "ht" "ir" "iq" "jp" "jo" "kz" > "kp" "kr" "kw" "lb" "li" "ni" "ne" "ng" "om" "pk" "qa" "ro" "ru" "sa" > "rs" "so" "za" "sy" "tj" "tr" "tm" "ae" "uz" "vn" "ye" ; > do > wget -4 --no-proxy --no-cookies --no-cache > --append-output=/var/log/wget.log > http://ipdeny.com/ipblocks/data/countries/$i.zone > cat $i.zone >>/tmp/blocked_countries > rm $i.zone > sleep 2 > done > > echo "Removing all *.zone files" > echo "Moving the temp file in to place" > mv /tmp/blocked_countries /etc/pf > # Restarting pf > pfctl -f /etc/pf.conf > echo "Complete" > esac > > exit $rc > > Hope this is useful to someone else. > > Thanks again. > Dave. > > > On 3/30/17, Ultima <ultima1252@gmail.com> wrote: > > Curl is probably the correct utility for this job. With curl the cat and > rm > > command can be negated entirely, although I'm not sure it has the same > > option set if explicitly required. Just stdout to the desired file. If a > > fresh list each use of the command is needed, add an rm before the for. > > > > On Thu, Mar 30, 2017 at 8:19 PM, William Dudley <wfdudley@gmail.com> > wrote: > > > >> for i in "vn.zone" "uz.zone" "tm.zone" ; > >> do > >> wget -4 --no-proxy --no-cookies --no-cache \ > >> http://ipdeny.com/ipblocks/data/countries/$i > >> cat $i >>blocked_zones > >> rm $i > >> sleep 2 > >> done > >> > >> Like that? > >> > >> Bill > >> > >> This email is free of malware because I run Linux. > >> > >> On Thu, Mar 30, 2017 at 8:02 PM, David Mehler <dave.mehler@gmail.com> > >> wrote: > >> > >> > Hello, > >> > > >> > My question is regarding a shell script and pf. > >> > > >> > What I'm wanting to do is take a selected list of countries and cat > >> > them in to a file and use that as pf input. Here's a sequential > >> > example: > >> > > >> > #!/bin/sh > >> > # > >> > PATH=/bin:/usr/local/bin:/sbin > >> > cd /tmp > >> > mkdir zones > >> > cd zones > >> > # -4 = use IPv4 only > >> > # --no-proxy = don't care for proxies > >> > # --no-cookies = don't accept cookies > >> > # --no-cache = no cached files > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/cn.zone # CHINA > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/az.zone # > AZERBAIJAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/by.zone # BELARUS > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/kz.zone # > KAZAKHSTAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/kg.zone # > KYRGYZSTAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/ru.zone # RUSSIAN > >> > FEDERATION > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/tj.zone # > TAJIKISTAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/tm.zone # > >> > TURKMENISTAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/uz.zone # > UZBEKISTAN > >> > sleep 2 > >> > wget -4 --no-proxy --no-cookies --no-cache \ > >> > http://ipdeny.com/ipblocks/data/countries/vn.zone # VIET NAM > >> > # > >> > cat cn.zone > blocked_zones > >> > cat az.zone >> blocked_zones > >> > cat by.zone >> blocked_zones > >> > cat kz.zone >> blocked_zones > >> > cat kg.zone >> blocked_zones > >> > cat ru.zone >> blocked_zones > >> > cat tj.zone >> blocked_zones > >> > cat tm.zone >> blocked_zones > >> > cat uz.zone >> blocked_zones > >> > cat vn.zone >> blocked_zones > >> > # > >> > rm *.zone > >> > # > >> > mv blocked_zones /etc/pf/ > >> > pfctl -f /etc/pf.conf > >> > > >> > There are 250 plus zones just in the ipv4 space, and about the same in > >> > the ipv6 space. I do not want to manually take down each domain, three > >> > times, that's error prown and very easy to miss one. I thought about > >> > doing an array, and feeding that to a loop which would cut down the > >> > number of lines of repeative code. > >> > > >> > Help appreciated. > >> > > >> > Thanks. > >> > Dave. > >> > > >> > > >> > On 3/30/17, Rajarajan Rajamani <r.rajamani@gmail.com> wrote: > >> > > Ask your question and I am sure someone will answer! > >> > > > >> > > On Mar 30, 2017 7:37 PM, "David Mehler" <dave.mehler@gmail.com> > >> > > wrote: > >> > > > >> > >> Hello, > >> > >> > >> > >> Any shell scripting gurus here please contact me offlist. I have a > >> > >> question that I can't figure out. > >> > >> > >> > >> Thanks. > >> > >> Dave. > >> > >> _______________________________________________ > >> > >> freebsd-questions@freebsd.org mailing list > >> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-questions > >> > >> To unsubscribe, send any mail to "freebsd-questions- > >> > >> unsubscribe@freebsd.org" > >> > >> > >> > > > >> > _______________________________________________ > >> > freebsd-questions@freebsd.org mailing list > >> > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > >> > To unsubscribe, send any mail to "freebsd-questions- > >> > unsubscribe@freebsd.org" > >> > > >> _______________________________________________ > >> freebsd-questions@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-questions > >> To unsubscribe, send any mail to "freebsd-questions- > >> unsubscribe@freebsd.org" > >> > > > _______________________________________________ > freebsd-questions@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions- > unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALfReyc%2BCDfNozHbx8OF=rHMh06FQKxGGhw4j_0GZDXx0X1_dA>