Date: Sun, 10 Aug 2003 01:22:18 -0300 From: =?ISO-8859-1?Q?Jo=E3o_Carlos_Mendes_Lu=EDs?= <jonny@jonny.eng.br> To: Oliver Fromme <olli@lurza.secnetix.de> Cc: Garrett Wollman <wollman@khavrinen.lcs.mit.edu> Subject: Re: Requirements Final Draft Attempt #2 :-/ Message-ID: <3F35C87A.4050004@jonny.eng.br> In-Reply-To: <200308082326.h78NQsuu065234@lurza.secnetix.de> References: <200308082326.h78NQsuu065234@lurza.secnetix.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Oliver,
Oliver Fromme wrote:
> João Carlos Mendes Luís wrote:
> > Oliver Fromme wrote:
> > > [...]
> > > All of that might sound complicated, but it really isn't
> > > that bad. The collector can be a small shell script using
> > > /usr/bin/fetch or automated ncftp. The extension to
> > > sysinstall shouldn't be too difficult either. I would be
> > > willing to work on the collector thing at least, and maybe
> > > also on the sysinstall part (if time permits).
> >
> > You should not need to get the files, only list them.
>
> Correct, that's what I meant.
>
> > Wouldn't this be much easier in perl?
>
> I do not like perl, but that task is indeed a bit heavy for
> a shell script.
>
> When I had a few minutes of free time, I coded a rough draft
> of such a script. I took the liberty to write it in Python.
No problem with python. I was just worried about bourne shell and
fetch.
> It doesn't do any DNS zone transfers, so it is not affected
> by reluctant DNS servers. Instead, the list of CC subdomains
> is hardcoded. For each subdomain, it starts at ftp.$DOMAIN,
> ftp1.$DOMAIN and increases the number until a host cannot be
> resolved. That seems to be a better approach than relying
> on zone transfers.
Good, and even better after the DNS hole fix.
> The script produces ASCII output in a very simple format.
> It could easily be fed into a database (maybe I'll try it
> with PostgreSQL when I have a few more minutes of free time).
>
> I also wrote a web page which reads that ASCII file and
> presents the data in HTML tables. It doesn't look very
> pretty (I'm a programmer, not an artist), but serves well
> as a rough draft and proof-of-concept.
>
> http://www.secnetix.de/~olli/fbsd-ftp/
I really liked it. It has even helped me find problems in my own
mirror! ;-)
Here's what it found about the mirror I maintain:
tp.br.freebsd.org
FTP Install Trees
4-STABLE - i386 - BAD
4.8-RELEASE - i386
4.8-STABLE - i386 - BAD
5.1-RELEASE - i386
CD ISO9660 Images
4.8-RELEASE - i386
5.1-RELEASE - i386
Software Packages
4.7-RELEASE - i386 - BAD
4.8-RELEASE - i386
- 4-stable and 4-8stable were there only to hold links to latest
packages (and sometime ago, XFree86 links), to allow usage of
/stand/sysinstall after cvsup recompiling.
- 4.7 packages was a temporary link I have done, to 4.8 packages, for
the same reason above, that I forgot to remove.
These may be used as an example of some changes that could be useful:
- After discovering some "content directory", verify it's contents to
see if it's true. This is a little intrusive, but could be done by
verifying the presence and size of some file, for example.
- If a problem is found, send an email to the CC.freebsd.br hostmaster,
and to postmaster@ftpX.CC.freebsd.org. Also, do not list it to the public.
The discussion about ftp holes and links reminded me about other
possible problem: Some ftp DNS names may point to multiple IP Addresses
(I've seen this already, but do not know if it's valid), and some
multiple DNS names could point to the same IP (ftp3.de and ftp6.de
example). Your program is surely the best place to check for this, and
probably list same IPs only once.
In the overal, I really liked it! Thanks a lot for your help!!!
Jonny
--
João Carlos Mendes Luís - Networking Engineer - jonny@jonny.eng.br
--
"the West won the world not by the superiority of its ideas or values
or religion but rather by its superiority in applying organized
violence. Westerners often forget this fact, non-Westerners never do."
-- Samuel P. Huntington
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3F35C87A.4050004>