Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 24 Dec 2013 18:00:58 -0800
From:      Dennis Glatting <freebsd@pki2.com>
To:        iamatt <iamatt@gmail.com>
Cc:        FreeBSD Mailing List <freebsd-questions@freebsd.org>
Subject:   Re: InfiniBand in 10.0?
Message-ID:  <1387936858.71651.137.camel@btw.pki2.com>
In-Reply-To: <CAEeRwNXzCVx_5uw7-KKjHr5RPZS6jJJ82pL5f=_6JUW9jTiRgw@mail.gmail.com>
References:  <1387915155.71651.118.camel@btw.pki2.com> <CAEeRwNXsxT_pbqOOF3tfHSecjFzRqqb6tgxZVc5nwMWqoyBOxw@mail.gmail.com> <1387920273.71651.119.camel@btw.pki2.com> <CAEeRwNU1nM+ueRz-yAAOf0XNZMJ8ZqFYNAH8ZsFX_jqy33A-dw@mail.gmail.com> <1387923358.71651.122.camel@btw.pki2.com> <CAEeRwNXzCVx_5uw7-KKjHr5RPZS6jJJ82pL5f=_6JUW9jTiRgw@mail.gmail.com>

Next in thread | Previous in thread | Raw E-Mail | Index | Archive | Help
On Tue, 2013-12-24 at 19:42 -0600, iamatt wrote:
> And then you need to see what mpi are you going to use.  I'm curious
> which one you plan on using and how well it works on FreeBSD.
> 

I've used MPIch2 under FreeBSD with John The Ripper about a year ago. It
eventually worked but it was a struggle to get there. I have not been
able to get OpenMPI to work at my former job (also FreeBSD). I've twice
came close but in each case the project was turned off before I could
finish.

I'm running oclHascat under CentOS 6.5 in my home lab where I have >50T
(uncompressed) of words. Most of my servers are FreeBSD of various
versions.



> 
> On Dec 24, 2013 4:16 PM, "Dennis Glatting" <freebsd@pki2.com> wrote:
>         On Tue, 2013-12-24 at 15:46 -0600, iamatt wrote:
>         > A good batch of mellanox cables.  Melanic had a bad run of
>         FDR early
>         > in the year.  Most FDR centers have been plagued with them.
>          Even the
>         > back end of our DDN lustres has had issues.  They typically
>         do not
>         > just fail but downgrade to QDR rate.
>         >
>         
>         Ah. I though they were stripping the technology InfinBand.
>         
>         
>         
>         > On Dec 24, 2013 3:24 PM, "Dennis Glatting"
>         <freebsd@pki2.com> wrote:
>         >         On Tue, 2013-12-24 at 14:43 -0600, iamatt wrote:
>         >         > Not sure if I would even bother with running IB on
>         BSD.  We
>         >         have enough
>         >         > problems just getting things stable with Linux and
>         mellanox
>         >         ofed .  Add all
>         >         > of the documented build problems with the mellanox
>         FDR
>         >         cables and recalls
>         >         > and you are guaranteed a nightmare.  TACC, navy,
>         and almost
>         >         every center I
>         >         > am aware of has replaced all or most or some of
>         their FDR,
>         >         including us.
>         >         >
>         >
>         >         Replaced them with what?
>         >
>         >
>         >         > As far as cards go it's all mellanox.  Qlogic lost
>         the game,
>         >         still dealing
>         >         > with a 12800  qlogic switch which is a total
>         nightmare.
>         >          Stick with
>         >         > mellanox.  Different vendors used to make their
>         own fabric
>         >         managers in the
>         >         > early time of IB but now its best to stick with
>         open ibd.
>         >         >
>         >         > FreeBSD and IB gives me the chills just thinking
>         about it.
>         >         > On Dec 24, 2013 1:59 PM, "Dennis Glatting"
>         >         <freebsd@pki2.com> wrote:
>         >         >
>         >         > >
>         >         > > This site appears dated:
>         >         > >
>         >         > >         https://wiki.freebsd.org/InfiniBand
>         >         > >
>         >         > > I have a couple of questions.
>         >         > >
>         >         > > 1) Under 10.0-RC2, is InfiniBand a part of
>         GENERIC or did
>         >         I still have
>         >         > > to modify the conf as specified in the wiki? I
>         don't see
>         >         any of the
>         >         > > options from the wiki in 10.0-RC2 GENERIC.
>         >         > >
>         >         > > 2) Any recommendation on cards? What works, what
>         doesn't?
>         >         > >
>         >         > > 3) What things to watch out for?
>         >         > >
>         >         > >
>         >         > > Thanks.
>         >         > >
>         >         > >
>         >         > >
>         >         > >
>         >         > > _______________________________________________
>         >         > > freebsd-questions@freebsd.org mailing list
>         >         > >
>         >
>         http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>         >         > > To unsubscribe, send any mail to "
>         >         > > freebsd-questions-unsubscribe@freebsd.org"
>         >         > >
>         >         > _______________________________________________
>         >         > freebsd-questions@freebsd.org mailing list
>         >         >
>         http://lists.freebsd.org/mailman/listinfo/freebsd-questions
>         >         > To unsubscribe, send any mail to
>         >         "freebsd-questions-unsubscribe@freebsd.org"
>         >
>         >
>         
>         





Want to link to this message? Use this URL: <http://docs.FreeBSD.org/cgi/mid.cgi?1387936858.71651.137.camel>