Date: Tue, 18 Apr 2017 19:40:11 +0200 From: Andreas Perstinger <andipersti@gmail.com> To: freebsd-questions@freebsd.org Subject: Re: awk help Message-ID: <aed3ad4b-7013-471f-8b11-bc717230cff0@gmail.com> In-Reply-To: <58F61027.3090100@gmail.com> References: <58F25A01.1060208@gmail.com> <7951DF71-5CD3-4B53-9CB4-13CAA8945983@huiekin.org> <58F4CD14.7090008@gmail.com> <c95e03d2-986d-3c3c-198a-a28ab862dc70@gmail.com> <58F53EEA.2030206@gmail.com> <7b381f8f-e2a5-26ea-075e-96ae35efb25d@rogers.com> <58F61027.3090100@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I think awk is the better tool for your task but you could still
simplify your shell script a little bit:
> hits_rpt="/etc/ipf_pool_hits_rpt" > rm $hits_rpt> touch $hits_rpt> >
hits_new="/etc/ipf_pool.hits.yes"> rm $hits_new> touch
$hits_new> > hits_no="/etc/ipf_pool.hits.no"> rm $hits_no>
touch $hits_no
Using the "truncate" command you could reduce these lines to
hits_rpt="/etc/ipf_pool_hits_rpt"
hits_new="/etc/ipf_pool.hits.yes"
hits_no="/etc/ipf_pool.hits.no"
truncate -s 0 $hits_rpt $hits_yes $hits_no
> ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null > > for line in `cat $hits_rpt`; do> > # drop the first 3
rpt lines> poollist_line=""> poollist_line=`echo -n $line
| grep poollist`> [ -n "${poollist_line}" ] && continue> >
role_line=""> role_line=`echo -n $line | grep Role`> [ -n
"${role_line}" ] && continue> > nodes_line="">
nodes_line=`echo -n $line | grep Nodes`> [ -n "${nodes_line}" ]
&& continue
Assuming that your file always contains the three unneeded lines at the
beginning you could use the "tail" command to start at line 4. And I
would also use a while-loop in combination with the "read" built-in
function. Then you are able to process pairs of lines:
tail -n +4 $hits_rpt | while read line
do
> in_line1=`echo -n $line | grep Address:`
> [ -n "${in_line1}" ] && save_in_line1="${in_line1}"
>
> in_line2=`echo -n $line | grep Hits`
> [ -n "${in_line2}" ] && save_in_line2="${in_line2}"
>
> if [ "${save_in_line1}" -a "${save_in_line2}" ]; then
> build_line1=${save_in_line1##*:}
> build_line1=${build_line1%%/*}
> build_line1="${build_line1};"
> build_line2=${save_in_line2##*Hits }
> # So remove everything to the right of the word Bytes.
> build_line2=${build_line2%%Bytes*}
Using sed instead of grep would simplify the extraction of the IP
address and the number of hits:
ip=$(echo $line | sed -e 's/ *Address: *\([^/]*\).*/\1/')
read line
hits=$(echo $line | sed -e 's/ *Hits *\([[:digit:]*]\).*/\1/')
> if [ ${build_line2} -gt 0 ]; then
> db_rec="$added_date ${build_line1}"
> echo "${db_rec}" >> $hits_new
> fi
> build_line="${build_line2} ${build_line1}"
> echo "${build_line}" >> $hits_no
> in_line1=""
> in_line2=""
> save_in_line1=""
> save_in_line2=""
> else
> continue
> fi
> done
> exit 0
So the complete shell script would be:
#!/bin/sh
added_date="`date +%Y%m%d`"
hits_rpt="/etc/ipf_pool_hits_rpt"
hits_new="/etc/ipf_pool.hits.yes"
hits_no="/etc/ipf_pool.hits.no"
truncate -s 0 $hits_rpt $hits_yes $hits_no
ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null
tail -n +4 $hits_rpt | while read line
do
ip=$(echo $line | sed -e 's/ *Address: *\([^/]*\).*/\1/')
read line
hits=$(echo $line | sed -e 's/ *Hits *\([[:digit:]*]\).*/\1/')
if [ "$hits" -gt 0 ]; then
echo "$added_date ${ip};" >> $hits_new
fi
echo "$hits ${ip};" >> $hits_no
done
exit 0
Does this still take minutes to process your data?
Bye, Andreas
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?aed3ad4b-7013-471f-8b11-bc717230cff0>