Date: Mon, 30 Mar 1998 12:31:30 +0200 From: Wolfram Schneider <wosch@cs.tu-berlin.de> To: shimon@simon-shapiro.org, Wolfram Schneider <wosch@cs.tu-berlin.de> Cc: freebsd-database@FreeBSD.ORG, andreas@klemm.gtn.com, scrappy@hub.org, Satoshi Asami <asami@FreeBSD.ORG>, Amancio Hasty <hasty@rah.star-gate.com> Subject: Re: [PORTS] Pgaccess doesn't run on -current anymore, Update Message-ID: <19980330123130.39177@caramba.cs.tu-berlin.de> In-Reply-To: <XFMail.980329135730.shimon@simon-shapiro.org>; from Simon Shapiro on Sun, Mar 29, 1998 at 01:57:30PM -0800 References: <p1i3eg5jdbb.fsf@panke.panke.de> <XFMail.980329135730.shimon@simon-shapiro.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 1998-03-29 13:57:30 -0800, Simon Shapiro wrote: > We have been playing with the idea of normalizing the archive into an > RDBMS. Some of the benefits are: > > * no need to update the threads database. It will always be updated. > * Users can create, easily, their own thread logic with no impact on > system performance. > * Searching on normalized fields are many times faster, and much less > costly in system resources. Some figures ... The FreeBSD mailing list archive is 620MB large. There are currently 270,000 messages. The archive grow with 100,000 messages/year. If you plan to use a real SQL database, you should consider at least 500,000 data sets, better 1 million. You need 2GB for the raw E-Mails and 2-4GB for the index. I don't know if there are free available databases which can handle this large data. That was the hardware part. You must hire a database expert, a Web designer and a cgi script programmer. All people should be willing to work for at least 2-3 years on this project. This is not an easy task. A full update of the thread database took 6 min on hub (Pentium Pro), thats 100MB/min ;-) An update for the last week took 3-6 seconds. -- Wolfram Schneider <wosch@freebsd.org> http://www.freebsd.org/~wosch/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-database" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19980330123130.39177>