Date: Wed, 4 Aug 2004 12:07:23 -0400 From: William Mansfield <william.mansfield@gmail.com> To: Kathy Quinlan <kat-free@kaqelectronics.dyndns.org> Cc: freebsd-hardware@freebsd.org Subject: Re: Big Problem Message-ID: <95fd7dc704080409077deea4e9@mail.gmail.com> In-Reply-To: <4110C9C7.6080506@kaqelectronics.dyndns.org> References: <4110C9C7.6080506@kaqelectronics.dyndns.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I had a similar problem with a database environment a while ago. Throughput on the DB (5-6TB) was aweful due to some really poor coding, so we needed options to speed up the indexes and such. This was on a 16way Itanium2 Unisys box attached to EMC 8830 storage to give people an idea how bad the DB and the queries were set up. We eventually scrapped the project due to overruns in cost, time, and efficiency. The only real solution we could come up with was a product from Texas Memory Systems called TerraRAMSan. It's a bit spendy and power hungry, but allows you to have ram based LUNs addressable via the SAN. Stripe 12 or so of these together into a volume and you should be able to get some performance gain. http://www.superssd.com/products/tera-ramsan/ On Wed, 04 Aug 2004 19:34:31 +0800, Kathy Quinlan <kat-free@kaqelectronics.dyndns.org> wrote: > Hi Guys and Gals, > > First off I am not a troll, this is a serious email. I can not go into > to many fine points as I am bound by an NDA. > > The problem: > > I need to hold a text file in ram, the text file in the forseable future > could be up to 10TB in size. > > My Options: > > Design a computer (probably multiple AMD 64's) to handle 10TB of memory > (+ a few extra Gb of ram for system overhead) and hold the file in one > physical computer system. > > Build a server farm and have each server hold a portion eg 4GB each > Server (250 servers (plus a few extra for system overhead) > > The reason the file needs to be in ram is that I need speed of search > for paterns in the data (less than 1 second to pull out relevent chunks) > > I am sure I have missed some options, right now I am just kicking ideas > around, the software will be based on FreeBSD with some major > modifications to address the large amount of ram (probably set it up as > a virtual drive with one file) > > Regards, > > Kat. > _______________________________________________ > freebsd-hardware@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hardware > To unsubscribe, send any mail to "freebsd-hardware-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?95fd7dc704080409077deea4e9>