Date: Tue, 18 Apr 2017 09:09:59 -0400 From: Ernie Luzar <luzar722@gmail.com> To: Mike Jeays <mj001@rogers.com> Cc: "freebsd-questions@FreeBSD. ORG" <freebsd-questions@FreeBSD.ORG> Subject: Re: awk help Message-ID: <58F61027.3090100@gmail.com> In-Reply-To: <7b381f8f-e2a5-26ea-075e-96ae35efb25d@rogers.com> References: <58F25A01.1060208@gmail.com> <7951DF71-5CD3-4B53-9CB4-13CAA8945983@huiekin.org> <58F4CD14.7090008@gmail.com> <c95e03d2-986d-3c3c-198a-a28ab862dc70@gmail.com> <58F53EEA.2030206@gmail.com> <7b381f8f-e2a5-26ea-075e-96ae35efb25d@rogers.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mike Jeays wrote:
> snip
> That is an amazing difference in performance - I might have expected a
> five to ten times improvement, but not 300+ times.
> I don't see anything very time-consuming in the script above. Is it
> possible for you to post the equivalent csh and awk scripts? Either I or
> someone with more experience with csh might be able to spot the problem.
Here are the 2 scripts
#! /bin/sh
# This script is designed to process the o/p from the 2 line hit count
# Make the input file read a line at a time, not a field at a time.
IFS=$'\n'
set -f
added_date="`date +%Y%m%d`"
hits_rpt="/etc/ipf_pool_hits_rpt"
rm $hits_rpt
touch $hits_rpt
hits_new="/etc/ipf_pool.hits.yes"
rm $hits_new
touch $hits_new
hits_no="/etc/ipf_pool.hits.no"
rm $hits_no
touch $hits_no
ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null
for line in `cat $hits_rpt`; do
# drop the first 3 rpt lines
poollist_line=""
poollist_line=`echo -n $line | grep poollist`
[ -n "${poollist_line}" ] && continue
role_line=""
role_line=`echo -n $line | grep Role`
[ -n "${role_line}" ] && continue
nodes_line=""
nodes_line=`echo -n $line | grep Nodes`
[ -n "${nodes_line}" ] && continue
in_line1=`echo -n $line | grep Address:`
[ -n "${in_line1}" ] && save_in_line1="${in_line1}"
in_line2=`echo -n $line | grep Hits`
[ -n "${in_line2}" ] && save_in_line2="${in_line2}"
if [ "${save_in_line1}" -a "${save_in_line2}" ]; then
build_line1=${save_in_line1##*:}
build_line1=${build_line1%%/*}
build_line1="${build_line1};"
build_line2=${save_in_line2##*Hits }
# So remove everything to the right of the word Bytes.
build_line2=${build_line2%%Bytes*}
if [ ${build_line2} -gt 0 ]; then
db_rec="$added_date ${build_line1}"
echo "${db_rec}" >> $hits_new
fi
build_line="${build_line2} ${build_line1}"
echo "${build_line}" >> $hits_no
in_line1=""
in_line2=""
save_in_line1=""
save_in_line2=""
else
continue
fi
done
exit 0
#! /bin/sh
hits_rpt="/etc/ipf_pool_hits"
# ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null
awk 'BEGIN { "date +%Y%m%d" | getline date
hits_yes = "/etc/ipf_pool_awk_hits_yes"
hits_no = "/etc/ipf_pool_awk_hits_no"
system("rm " hits_yes)
system("touch " hits_yes)
system("rm " hits_no)
system("touch " hits_no)
}
/Address/ { address = $2; sub("/32", ";", address); got_address
= 1; }
/Hits/ { if (got_address)
{ hits = $2;
if (hits > 0)
{ print date, " ", address > hits_yes;
got_address = 0;
}
else
{ print "hit no", hits, address > hits_no;
got_address = 0;
}
}
}' $hits_rpt
Note; the
ippool -l -d -m probing_ips > $hits_rpt 2> /dev/null
command is run in the first script and the file it creates in used as
input in the second script so both scripts process the same data. This
ippool command is not the source of the big time difference between the
2 scripts, where the first one takes 5 minutes and the second takes 1
second.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58F61027.3090100>