Date: Fri, 31 Mar 2017 11:17:37 +0100 From: krad <kraduk@gmail.com> To: David Mehler <dave.mehler@gmail.com> Cc: Ultima <ultima1252@gmail.com>, freebsd-questions <freebsd-questions@freebsd.org> Subject: Re: shell script guru Message-ID: <CALfReyc%2BCDfNozHbx8OF=rHMh06FQKxGGhw4j_0GZDXx0X1_dA@mail.gmail.com> In-Reply-To: <CAPORhP73=2_5nfOaR=a=TZTOyquBSZRS===FakeJWMPLjpNMjw@mail.gmail.com> References: <CAPORhP5ESqJL%2BkK4tfSD5t5=fnFjsCNXGdUhAjMpezq4WdjKyw@mail.gmail.com> <CADbyKk61wyYj1Jgc9daFTbXE_9s5xPLEYHa4p=KF8FhngzOQ3Q@mail.gmail.com> <CAPORhP6%2Bu4DpUq=9WJ9XmSHDYSJSmXaa6_o7NnVtOq=n_g0v=w@mail.gmail.com> <CAFsnNZL8EgYQK9u_mz4BB%2BULwo9xgsPFT%2BP-4uD4-tqHd%2Bn2QQ@mail.gmail.com> <CANJ8om6svf%2B6sgrV4UW8F=aidaHhWce%2BfNO4-g4Lfa2QteYa7w@mail.gmail.com> <CAPORhP73=2_5nfOaR=a=TZTOyquBSZRS===FakeJWMPLjpNMjw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
you can use the "-O -" option on wget to pipe the file contents to stdout
and get rid of the cat and rm. You could also use fetch rather than
wget/curl and save installing a port. Use '-o -' for that though. 8)
On 31 March 2017 at 04:56, David Mehler <dave.mehler@gmail.com> wrote:
> Hello,
>
> My thanks to everyone who helped me.
>
> I've got a solution, and have set it in my monthly periodic checks in
> /etc/periodic/monthly.
>
> I've got two solutions both of which retrieve the country database. In
> pf I have a table that blocks the ip's in the table. Here's the perl
> script:
>
> #!/usr/bin/env perl
> open(ZONES, "<zonesfile");
>
> while(<ZONES>) {
>
> chomp;
>
> system("wget -4 --no-proxy --no-cookies --no-cache
> http://ipdeny.com/ipblocks/data/countries/$_.zone");
>
> system("cat $_.zone >>blocked_zones");
>
> unlink($_);
>
> sleep(2);
>
> }
>
> close(ZONES);
>
> ssystem("mv blocked_zones /etc/pf");
> ssystem("pfctl -f /etc/pf.conf)";
>
> pf.conf:
> table <blocked_countries persist "/etc/pf/blocked_countries"
> block in quick from <blocked_tables>
>
> The zonesfile contains countries in quotes one per line:
>
> "al"
> "cz"
> "ch"
> ...
>
> etc
>
> Here's the script that I put in monthly:
>
> #!/bin/sh
> #
> # Monthly retrieve the selected country IP block lists
> # Retrieves dns zones from ipdeny.com
> # Adds the zones to a country block file
> # Then adds them to a pf block table
>
> # If there is a global system configuration file, suck it in.
> #
> if [ -r /etc/defaults/periodic.conf ]
> then
> . /etc/defaults/periodic.conf
> source_periodic_confs
> fi
>
> case "$monthly_country_blocks_enable" in
> [Yy][Ee][Ss])
> cd /tmp
> echo "Retrieving Zones"
> for i in "af" "al" "dz" "am" "az" "ba" "br" "kh" "cf" "cn" "co" "cr"
> "hr" "cu" "cy" "cz" "do" "eg" "fr" "gi" "ht" "ir" "iq" "jp" "jo" "kz"
> "kp" "kr" "kw" "lb" "li" "ni" "ne" "ng" "om" "pk" "qa" "ro" "ru" "sa"
> "rs" "so" "za" "sy" "tj" "tr" "tm" "ae" "uz" "vn" "ye" ;
> do
> wget -4 --no-proxy --no-cookies --no-cache
> --append-output=/var/log/wget.log
> http://ipdeny.com/ipblocks/data/countries/$i.zone
> cat $i.zone >>/tmp/blocked_countries
> rm $i.zone
> sleep 2
> done
>
> echo "Removing all *.zone files"
> echo "Moving the temp file in to place"
> mv /tmp/blocked_countries /etc/pf
> # Restarting pf
> pfctl -f /etc/pf.conf
> echo "Complete"
> esac
>
> exit $rc
>
> Hope this is useful to someone else.
>
> Thanks again.
> Dave.
>
>
> On 3/30/17, Ultima <ultima1252@gmail.com> wrote:
> > Curl is probably the correct utility for this job. With curl the cat and
> rm
> > command can be negated entirely, although I'm not sure it has the same
> > option set if explicitly required. Just stdout to the desired file. If a
> > fresh list each use of the command is needed, add an rm before the for.
> >
> > On Thu, Mar 30, 2017 at 8:19 PM, William Dudley <wfdudley@gmail.com>
> wrote:
> >
> >> for i in "vn.zone" "uz.zone" "tm.zone" ;
> >> do
> >> wget -4 --no-proxy --no-cookies --no-cache \
> >> http://ipdeny.com/ipblocks/data/countries/$i
> >> cat $i >>blocked_zones
> >> rm $i
> >> sleep 2
> >> done
> >>
> >> Like that?
> >>
> >> Bill
> >>
> >> This email is free of malware because I run Linux.
> >>
> >> On Thu, Mar 30, 2017 at 8:02 PM, David Mehler <dave.mehler@gmail.com>
> >> wrote:
> >>
> >> > Hello,
> >> >
> >> > My question is regarding a shell script and pf.
> >> >
> >> > What I'm wanting to do is take a selected list of countries and cat
> >> > them in to a file and use that as pf input. Here's a sequential
> >> > example:
> >> >
> >> > #!/bin/sh
> >> > #
> >> > PATH=/bin:/usr/local/bin:/sbin
> >> > cd /tmp
> >> > mkdir zones
> >> > cd zones
> >> > # -4 = use IPv4 only
> >> > # --no-proxy = don't care for proxies
> >> > # --no-cookies = don't accept cookies
> >> > # --no-cache = no cached files
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/cn.zone # CHINA
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/az.zone #
> AZERBAIJAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/by.zone # BELARUS
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/kz.zone #
> KAZAKHSTAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/kg.zone #
> KYRGYZSTAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/ru.zone # RUSSIAN
> >> > FEDERATION
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/tj.zone #
> TAJIKISTAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/tm.zone #
> >> > TURKMENISTAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/uz.zone #
> UZBEKISTAN
> >> > sleep 2
> >> > wget -4 --no-proxy --no-cookies --no-cache \
> >> > http://ipdeny.com/ipblocks/data/countries/vn.zone # VIET NAM
> >> > #
> >> > cat cn.zone > blocked_zones
> >> > cat az.zone >> blocked_zones
> >> > cat by.zone >> blocked_zones
> >> > cat kz.zone >> blocked_zones
> >> > cat kg.zone >> blocked_zones
> >> > cat ru.zone >> blocked_zones
> >> > cat tj.zone >> blocked_zones
> >> > cat tm.zone >> blocked_zones
> >> > cat uz.zone >> blocked_zones
> >> > cat vn.zone >> blocked_zones
> >> > #
> >> > rm *.zone
> >> > #
> >> > mv blocked_zones /etc/pf/
> >> > pfctl -f /etc/pf.conf
> >> >
> >> > There are 250 plus zones just in the ipv4 space, and about the same in
> >> > the ipv6 space. I do not want to manually take down each domain, three
> >> > times, that's error prown and very easy to miss one. I thought about
> >> > doing an array, and feeding that to a loop which would cut down the
> >> > number of lines of repeative code.
> >> >
> >> > Help appreciated.
> >> >
> >> > Thanks.
> >> > Dave.
> >> >
> >> >
> >> > On 3/30/17, Rajarajan Rajamani <r.rajamani@gmail.com> wrote:
> >> > > Ask your question and I am sure someone will answer!
> >> > >
> >> > > On Mar 30, 2017 7:37 PM, "David Mehler" <dave.mehler@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> Hello,
> >> > >>
> >> > >> Any shell scripting gurus here please contact me offlist. I have a
> >> > >> question that I can't figure out.
> >> > >>
> >> > >> Thanks.
> >> > >> Dave.
> >> > >> _______________________________________________
> >> > >> freebsd-questions@freebsd.org mailing list
> >> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> >> > >> To unsubscribe, send any mail to "freebsd-questions-
> >> > >> unsubscribe@freebsd.org"
> >> > >>
> >> > >
> >> > _______________________________________________
> >> > freebsd-questions@freebsd.org mailing list
> >> > https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> >> > To unsubscribe, send any mail to "freebsd-questions-
> >> > unsubscribe@freebsd.org"
> >> >
> >> _______________________________________________
> >> freebsd-questions@freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> >> To unsubscribe, send any mail to "freebsd-questions-
> >> unsubscribe@freebsd.org"
> >>
> >
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "freebsd-questions-
> unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALfReyc%2BCDfNozHbx8OF=rHMh06FQKxGGhw4j_0GZDXx0X1_dA>