From owner-freebsd-questions@FreeBSD.ORG Tue Dec 24 21:46:33 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 19963B00 for ; Tue, 24 Dec 2013 21:46:33 +0000 (UTC) Received: from mail-wg0-x233.google.com (mail-wg0-x233.google.com [IPv6:2a00:1450:400c:c00::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9B68C12DC for ; Tue, 24 Dec 2013 21:46:32 +0000 (UTC) Received: by mail-wg0-f51.google.com with SMTP id b13so6153045wgh.18 for ; Tue, 24 Dec 2013 13:46:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=spnSzhsTGmq3eJAV+fcNgFi9462Uz6TB3P9m78djY/Q=; b=0HaxdzlZIWPqJzFs2od1qDw04KyIdEh6S4pTD6PhNOxvcqWey7mRxK/pAyBjzBr+b1 SR0Q4i0JbLXtyGgIy/zNvP0KU6IuFWB1qvqWkzLCprZVjd0BvneMDiTGVnEHaF3/zIYV 7EnaLtpxrXTq8s2bdAZYAI2C2WSYsj2RP2UW2szd3VejJW/8mt2H9Axjf1Vf6tKr/bUt l5nlu7gEEoM/f+DEieHyq257MvycNrm2d1Fj2lmmUdm0b9mK527ewzTIUioWVzOUtDZq s7ZelTRESUgXP/KWoSVaSduWUuXB0A+MB1URLa/kBTJCS2WOHJzMCbvVzdsEEXPIJ2ng llmg== MIME-Version: 1.0 X-Received: by 10.180.36.8 with SMTP id m8mr24576638wij.3.1387921590890; Tue, 24 Dec 2013 13:46:30 -0800 (PST) Received: by 10.216.108.195 with HTTP; Tue, 24 Dec 2013 13:46:30 -0800 (PST) Received: by 10.216.108.195 with HTTP; Tue, 24 Dec 2013 13:46:30 -0800 (PST) In-Reply-To: <1387920273.71651.119.camel@btw.pki2.com> References: <1387915155.71651.118.camel@btw.pki2.com> <1387920273.71651.119.camel@btw.pki2.com> Date: Tue, 24 Dec 2013 15:46:30 -0600 Message-ID: Subject: Re: InfiniBand in 10.0? From: iamatt To: Dennis Glatting Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: FreeBSD Mailing List X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Dec 2013 21:46:33 -0000 A good batch of mellanox cables. Melanic had a bad run of FDR early in the year. Most FDR centers have been plagued with them. Even the back end of our DDN lustres has had issues. They typically do not just fail but downgrade to QDR rate. On Dec 24, 2013 3:24 PM, "Dennis Glatting" wrote: > On Tue, 2013-12-24 at 14:43 -0600, iamatt wrote: > > Not sure if I would even bother with running IB on BSD. We have enough > > problems just getting things stable with Linux and mellanox ofed . Add > all > > of the documented build problems with the mellanox FDR cables and recalls > > and you are guaranteed a nightmare. TACC, navy, and almost every center > I > > am aware of has replaced all or most or some of their FDR, including us. > > > > Replaced them with what? > > > > As far as cards go it's all mellanox. Qlogic lost the game, still > dealing > > with a 12800 qlogic switch which is a total nightmare. Stick with > > mellanox. Different vendors used to make their own fabric managers in > the > > early time of IB but now its best to stick with open ibd. > > > > FreeBSD and IB gives me the chills just thinking about it. > > On Dec 24, 2013 1:59 PM, "Dennis Glatting" wrote: > > > > > > > > This site appears dated: > > > > > > https://wiki.freebsd.org/InfiniBand > > > > > > I have a couple of questions. > > > > > > 1) Under 10.0-RC2, is InfiniBand a part of GENERIC or did I still have > > > to modify the conf as specified in the wiki? I don't see any of the > > > options from the wiki in 10.0-RC2 GENERIC. > > > > > > 2) Any recommendation on cards? What works, what doesn't? > > > > > > 3) What things to watch out for? > > > > > > > > > Thanks. > > > > > > > > > > > > > > > _______________________________________________ > > > freebsd-questions@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > > > To unsubscribe, send any mail to " > > > freebsd-questions-unsubscribe@freebsd.org" > > > > > _______________________________________________ > > freebsd-questions@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > > To unsubscribe, send any mail to " > freebsd-questions-unsubscribe@freebsd.org" > > >