Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 28 Nov 2008 14:45:31 -0500 (EST)
From:      vogelke+software@pobox.com (Karl Vogel)
To:        freebsd-questions@freebsd.org
Subject:   Re: 5 TB server
Message-ID:  <20081128194531.7B17FB7C5@kev.msw.wpafb.af.mil>
In-Reply-To: <139b44430811280548x36915301i766bfb15f162c8ca@mail.gmail.com> (valentin.bud@gmail.com)

next in thread | previous in thread | raw e-mail | index | archive | help
>> On Fri, 28 Nov 2008 15:48:45 +0200, 
>> "Valentin Bud" <valentin.bud@gmail.com> said:

V> I have to come up with a solution for a company that has as we speak 4 TB
V> of data spread among 3 computers with lots of HDDs.  Of course I've
V> recommend them to buy a server for that storage capacity and for data
V> organization.

   Good idea.  We had a similar problem here; a SuperMicro server tanked,
   and the company that provided the warranty went bankrupt, so all we had
   were 12 perfectly good 400-Gb SATA drives and nowhere to put them.

   The drives were the big-cost item, so I didn't want to just dump them.
   We finally bought two empty IBM x3400 8-bay enclosures plus some IBM
   SAS 3.5" hot-swap trays, and it's working like a charm.

V> I thought of going on the ZFS way (on FreeBSD of course) with some raidz.

   In my experience, completely new filesystems or operating systems need
   at least 5 years in the field to weed out all the weird corner-cases.
   I might trust ZFS on Sun hardware (*with* vendor support) at this point,
   but I'd wait awhile before trying it on anything else.

   This isn't a slam at ZFS or the FreeBSD porters, it's just recognition
   of the fact that some types of software development are *not* time-
   compressible, regardless of who's doing the work.

V> One of the problems is that the server will stay in their office so it
V> has to be quite silent.

   Not a good idea, especially if this data is their bread-and-butter.
   You can walk out the door with a system this size on your shoulder,
   so I'd recommend a locked room with reasonable cooling and *clean*
   power.  You don't need a 10-foot-tall zillion-dollar Liebert A/C,
   but you absolutely need a UPS that can take care of power spikes;
   the more moderate the environment, the less likely you are to have a
   hardware failure.

   I don't use disk mirroring because 99% of our problems come from humans
   rather than hardware.  If someone zaps the wrong file, a mirror will
   simply replicate that mistake; we have two matching servers in separate
   rooms, and we run rsync nightly to back up the production box without
   deleting any files.  I also run hourly backups on the production box
   to store anything that's been modified in the last 60 minutes, which
   gives us a nice file history and takes care of most recovery problems.

   With two servers, I can use basic UFS filesystems and get fine I/O
   performance with minimum maintenance.

-- 
Karl Vogel                      I don't speak for the USAF or my company

The first product Motorola started to develop was a record player for cars.
At that time, the most known player on the market was Victrola, so they
called themselves Motorola.   --possibly-true item for a lull in conversation



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20081128194531.7B17FB7C5>