Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 4 Apr 2006 10:45:49 +0100
From:      Shaun Amott <shaun@inerd.com>
To:        freebsd-ports@FreeBSD.org
Subject:   Re: portscout - new distfile scanner
Message-ID:  <20060404094549.GA751@picobyte.net>
In-Reply-To: <20060403190532.GA966@picobyte.net>
References:  <20060403190532.GA966@picobyte.net>

next in thread | previous in thread | raw e-mail | index | archive | help
Thanks for all the feedback everyone. It's been very useful.

I'm slightly disappointed by all the false positives. The only
explanation I have for portscout finding versions that don't exist is
hosts lieing about files. Edwin's script dealt with this by using a host
blacklist - portscout tries to avoid the problem by sending dummy files
to the master site and looking for "ok" responses. I have a feeling that
portscout is being too aggressive and causing connection failures by
flooding the network.

There seems to be another bug that has surfaced which affects certain
files with version number components beginnings with "0". I'm looking
into this.

Someone noted that portscout found 0.811 as a update for 0.9 --
unfortunately, when I tried to account for weird versions like this
before, it didn't work out well at all. So, for now, portscout assumes
version numbers count up.

I did another test run last night with improved results. Version string
extraction has improved considerably; there are more results than
before, and from what I can tell, less false positives. The results have
been uploaded.

Building the database from scratch takes a long time - I haven't timed
it recently, but I think it finishes in around three hours. That is
essentially converting all the information from the ports tree into a
usable, easily-accessible format.

Re-building after a cvsup/index takes a minute or two.

The check last night took 2:40:51; the machine is a PIII 750MHz with
256MB RAM and very fast disks. It usually has a load average of 0 - 1;
Running portscout with 30 parallel processes puts the load average up to
12.0. Average download bandwidth used was around 780Kbps.

I will do another run tonight with less processes and see if the phantom
updates disappear.

    -Shaun

-- 
Shaun Amott [ PGP: 0x6B387A9A ]
    Scientia Est Potentia.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060404094549.GA751>