Date: Fri, 22 Apr 2016 16:57:38 -0500 (CDT) From: "Valeri Galtsev" <galtsev@kicp.uchicago.edu> To: "Mehmet Erol Sanliturk" <m.e.sanliturk@gmail.com> Cc: "FreeBSD Questions Mailing List" <freebsd-questions@freebsd.org> Subject: Re: Storage cluster advise, anybody? Message-ID: <52604.128.135.52.6.1461362258.squirrel@cosmo.uchicago.edu> In-Reply-To: <CAOgwaMtBxeptoC4Kcg0WMKbhc6xRgutHCXESfGmkq5ZYrsznhw@mail.gmail.com> References: <29462.128.135.52.6.1461352625.squirrel@cosmo.uchicago.edu> <CAOgwaMtBxeptoC4Kcg0WMKbhc6xRgutHCXESfGmkq5ZYrsznhw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, April 22, 2016 2:38 pm, Mehmet Erol Sanliturk wrote:
> On Fri, Apr 22, 2016 at 12:17 PM, Valeri Galtsev
> <galtsev@kicp.uchicago.edu>
> wrote:
>
>> Dear Experts,
>> I would like to ask everybody: what would you advise to use as a
storage
>> cluster, or as a distributed filesystem.
<snip>
>> <rant>
>> I really didn't mean to write this, but I figure it probably will
surface
>> once I start getting your advices, so here it is. I did my research
having
>> my requirements in mind and came up with the solution: moosefs. It is
not
>> reviewed much, no reviews with criticism at all, and not much you can
("I
>> could" I should say) find howtos about customizations, performance
tuning
>> etc. It installs without a hitch. It runs well, until you start stress
writing a lot to it in parallel, then it started performing
>> exponentially
>> badly for me. Here is where extensive attempts to find performance
tuning
>> documentation faces lack of success. What made my decision to never
ever
>> use it in a future was the following. I started migrating data back
from
>> moosefs to local UFS (that is FreeBSD box) filesystem using rsync
command.
>> What I observed was: source files after they have been touched by rsync
changed their timestamps. As if instead of creation timestamp it is an
access timestamp on moosefs. This renders rsync from moosefs useless, as
>> you can not re-run failed rsync, and you obliterate some of metadata of
the source ("creation" timestamp). I wrote e-mail to sourceforge
moosefs
>> mail list, mentioning all this and the fact that I am using open source
moosefs. Next day they replied asking whether I use version 3."this" or
version 3."that", as they want to know in which of them they have a bug.
>> Whereas latest open source version they have everywhere, including
sourceforge is older version: 2.0.88.
>> Basically, my decision was made. Sorry for venting it out here, but I
figured, it will happen some moment when I will get your advises.
</rant>
>> Thanks a lot for all your advises!
>> Valeri
> In page :
> http://moosefs.org/download.html
>
> there are the following lines with download links :
>
> Current release version 3.0.73-1 (GPLv2) 2016-03-04
> Current release version 3.0.69-1 (GPLv2) 2016-01-12
> Current release version 3.0.55-1 (GPLv2) 2015-10-21
> Current release version 3.0.37-1 (GPLv2) 2015-05-15
> Current release version 3.0.17-1 (GPLv2) 2015-04-21
> .
> .
> .
>
>
>
> Perhaps they assumed you are using one of them .
Thanks for pointing that out. I see now that they are there under the name
"current" as opposed to 2.0.88 "stable" - pretty much consistent with what
FreeBSD terminology would be. So, 2.0.88 _is_ the latest "stable".
No, I'm using "stable" on all production everything (my laptop doesn't
count). And the weirdness I see is in production version (built on my
machine from port I must admit).
Thanks!
Valeri
>
>
> Mehmet Erol Sanliturk
>
++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52604.128.135.52.6.1461362258.squirrel>