From owner-freebsd-infiniband@FreeBSD.ORG Tue Feb 12 16:43:11 2013 Return-Path: Delivered-To: freebsd-infiniband@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7AD3B312 for ; Tue, 12 Feb 2013 16:43:11 +0000 (UTC) (envelope-from accornehl@gmail.com) Received: from mail-ee0-f46.google.com (mail-ee0-f46.google.com [74.125.83.46]) by mx1.freebsd.org (Postfix) with ESMTP id 0039B117 for ; Tue, 12 Feb 2013 16:43:10 +0000 (UTC) Received: by mail-ee0-f46.google.com with SMTP id e49so152323eek.19 for ; Tue, 12 Feb 2013 08:43:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=XK1oow+2a5oHwtr4JxwcLX5jLAsI1n+nP1qdYVnBnRs=; b=PFv+j85bxcwPBtKsGI84enXG7/AELPNOOWhHuwKAEF9kEML6Z6cjtt3MV0Xf+trNXI 09yNiU3glFUAJwt63UtyXLDmWP/AVIVJ7RH/4cJFJmAvbylT75cQOlzQPbF124b/gEOu lDQb3JY/nu6C0EaJwZXPASwvpRHCIZOVJhjhQqlm4N2ItxWtaYvj8bsaOCV4qEvi/Vwb NrlJLnYtcHr6o22r9YcXqGF4QejtV1oC6MiyCjQSGBbdAmVvkTABgrgcuuW9PF6cTVCl sL3ay5vjTWmp/wKwmAT4rHFEXQS7vYaFR3gozdGRJUM3ybbDQ8yJFrpFkYI1FmUN6w/g Gefg== MIME-Version: 1.0 X-Received: by 10.14.218.71 with SMTP id j47mr64239245eep.28.1360686980876; Tue, 12 Feb 2013 08:36:20 -0800 (PST) Received: by 10.223.18.14 with HTTP; Tue, 12 Feb 2013 08:36:20 -0800 (PST) Received: by 10.223.18.14 with HTTP; Tue, 12 Feb 2013 08:36:20 -0800 (PST) In-Reply-To: <33AE1DB1-7BA9-4329-8AF0-EC7FEDCFBE7F@polycoresoftware.com> References: <33AE1DB1-7BA9-4329-8AF0-EC7FEDCFBE7F@polycoresoftware.com> Date: Tue, 12 Feb 2013 08:36:20 -0800 Message-ID: Subject: Re: OFED setup and Mellanox cards From: Anthony Cornehl To: Tamer Abuelata Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-infiniband@freebsd.org X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Feb 2013 16:43:11 -0000 The /sys folder is abstracted to live in the sysctls. There is some abstraction in the Linux-compat layer, in /usr/src/sys/ofed/include, that can allow the use of folder and sysctl in your program from Linux to FreeBSD. I'll get a test setup this week. On Feb 11, 2013 1:29 PM, "Tamer Abuelata" wrote: > Hi, > > I'm trying to get OFED up and running on a FreeBSD 9.1 machine with 2 > Mellanox cards (MT26428). > We have compiled the Kernel with the following options (per > https://wiki.freebsd.org/InfiniBand ): > > options OFED # Infiniband protocol stack and support > options SDP # Sockets Direct Protocol for infiniband > options IPOIB_CM # Use connect mode ipoib > > device ipoib # IP over IB devices > device mlx4ib # ConnectX Infiniband support > device mlxen # ConnectX Ethernet support > device mthca # Infinihost cards > > I gave an IP to each card. > # ifconfig > ib0: flags=3D8043 metric 0 mtu 65520 > options=3D80018 > lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.d9 > inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 > nd6 options=3D29 > ib1: flags=3D8043 metric 0 mtu 65520 > options=3D80018 > lladdr 80.0.0.48.fe.80.0.0.0.0.0.0.0.25.90.ff.ff.7.70.f1 > inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 > nd6 options=3D29 > > After doing a build world and build kernel I try to run an example > application and get the following warnings/errors: > > # ./server > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs1 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs1 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs1 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > listening on port 29823. > > Then I launch the client from another terminal: > # ./client 10.0.0.2 29823 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs1 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs1 > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > [=E2=80=A6 shows several more times =E2=80=A6] > Segmentation fault (core dumped) > > So the sample app compiles fine but I get this runtime error which seems > to come from the libibverbs. This /sys/class/infiniband_verbs folder does > not exist. I'm not sure if I'm missing configuration files. However I hav= e > devices at /dev/uverbs0 and /dev/uverbs1. > > The example app I am using is from: > > http://thegeekinthecorner.wordpress.com/2010/08/14/building-an-rdma-capab= le-application-with-ib-verbs-part-2-the-server/ > > The code is linked at the bottom of the page. Direct link: > > https://sites.google.com/a/bedeir.com/home/basic-rdma-client-server.tar.g= z?attredirects=3D0&d=3D1 > > I would appreciate any help or pointers from someone who managed to get > OFED to run on FreeBSD. > > Best regards, > Tamer > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to " > freebsd-infiniband-unsubscribe@freebsd.org" >