From owner-freebsd-infiniband@FreeBSD.ORG Sat Jun 13 18:43:40 2015 Return-Path: Delivered-To: freebsd-infiniband@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 341E85CB for ; Sat, 13 Jun 2015 18:43:40 +0000 (UTC) (envelope-from unixro@gmail.com) Received: from mail-wi0-x22b.google.com (mail-wi0-x22b.google.com [IPv6:2a00:1450:400c:c05::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0DE67C0 for ; Sat, 13 Jun 2015 18:43:39 +0000 (UTC) (envelope-from unixro@gmail.com) Received: by wibut5 with SMTP id ut5so42445706wib.1 for ; Sat, 13 Jun 2015 11:43:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=iEgc93ipLznSyXu8OmDUnb2JqqoqbaPJJev7mL4qc+k=; b=WiQF36GDxVqnLMRHhd3tACBe2tuGDbBrAUNX1iB9moJSw4ncUix5lX3PJNh8Fbprl/ avaXI/fUmFEKTZplKX1Ae3O4XvjAVpwh3zLSn8u3cKFF3kRteUwG7xkKeoEW3t6RTk8F XSUWOTa+856Rv5hj3bB1C4bBD20yhpR5bGGTTATiKUg+/kqKpUz3lg0d+7hRE7FLK33w PTbf2JvF2KqRGRcb515MdcffsOqrsbymQpCXO1oS+oAZMt76YHVule3gsBbkjyyhX/sw yqXtQmWOpQj5gYBpEbasuIG9iPrsktMPEcQYDwfgeXmrA39ht1GU9PbwJFQVnd6uB066 vLng== X-Received: by 10.180.36.170 with SMTP id r10mr17916280wij.10.1434221018202; Sat, 13 Jun 2015 11:43:38 -0700 (PDT) Received: from [192.168.3.105] ([193.148.0.35]) by mx.google.com with ESMTPSA id u9sm11418496wju.44.2015.06.13.11.43.37 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 13 Jun 2015 11:43:37 -0700 (PDT) Message-ID: <557C79D8.8040403@gmail.com> Date: Sat, 13 Jun 2015 21:43:36 +0300 From: Mihai Vintila User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: freebsd-infiniband@freebsd.org Subject: Infiniband subnetmanager and driver compilation on FreeBSD 10.1 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jun 2015 18:43:40 -0000 Hi, I've posted in the stable mailing list but i guess this is where i should ask. Can you tell me what is the proper way to get subnet manager working on FreeBSD 10.1 ? I've used RELEASE branch to compile opensm and drivers from Mellanox but i get: ibportstate -G 0xe61d2dfffe0a2130 1 enable ibpanic: [868] madrpc_init: can't open UMAD port ((null):0): (m) opensm ------------------------------------------------- OpenSM 3.3.1 Reading Cached Option File: /etc/opensm/opensm.conf Command Line Arguments: Log File: /var/log/opensm.log ------------------------------------------------- OpenSM 3.3.1 Entering DISCOVERING state Using default GUID 0x0 Error from osm_opensm_bind (0x2A) Perhaps another instance of OpenSM is already running Exiting SM kldstat Id Refs Address Size Name 1 27 0xffffffff80200000 1756638 kernel 2 1 0xffffffff81957000 2681e8 zfs.ko 3 2 0xffffffff81bc0000 6780 opensolaris.ko 4 2 0xffffffff81bc7000 17d60 nvme.ko 5 1 0xffffffff81bdf000 4210 nvd.ko 6 1 0xffffffff81e11000 357f ums.ko 7 3 0xffffffff81e15000 37656 mlx4.ko 8 2 0xffffffff81e4d000 4e348 ibcore.ko 9 1 0xffffffff81e9c000 2f72 toecore.ko 10 1 0xffffffff81e9f000 238fa mlx4ib.ko 11 1 0xffffffff81ec3000 12482 mlxen.ko Driver that i'm using: mlx4_core0: mem 0xc7400000-0xc74fffff,0xc5000000-0xc57fffff irq 32 at device 0.0 on pci2 mlx4_core: Initializing mlx4_core: Mellanox ConnectX VPI driver v2.1.6 (Jun 12 2015) mlx4_core0: mlx4_ib_add: allocated counter index 1 for port 1 mlx4_en mlx4_core0: Activating port:1 mlxen0: Ethernet address: e4:1d:2d:0a:21:30 mlx4_en: mlx4_core0: Port 1: Using 12 TX rings mlxen0: link state changed to DOWN mlx4_en: mlx4_core0: Port 1: Using 8 RX rings mlx4_en: mlxen0: Using 12 TX rings mlx4_en: mlxen0: Using 8 RX rings mlx4_en: mlxen0: Initializing port -- Best regards, Vintila Mihai Alexandru From owner-freebsd-infiniband@FreeBSD.ORG Sat Jun 13 18:58:48 2015 Return-Path: Delivered-To: freebsd-infiniband@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 22D5D975 for ; Sat, 13 Jun 2015 18:58:48 +0000 (UTC) (envelope-from hps@selasky.org) Received: from mail.turbocat.net (mail.turbocat.net [IPv6:2a01:4f8:d16:4514::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DC624A00 for ; Sat, 13 Jun 2015 18:58:47 +0000 (UTC) (envelope-from hps@selasky.org) Received: from laptop015.home.selasky.org (cm-176.74.213.204.customer.telag.net [176.74.213.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.turbocat.net (Postfix) with ESMTPSA id C204B1FE023; Sat, 13 Jun 2015 20:58:43 +0200 (CEST) Message-ID: <557C7D94.9020207@selasky.org> Date: Sat, 13 Jun 2015 20:59:32 +0200 From: Hans Petter Selasky User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Mihai Vintila , freebsd-infiniband@freebsd.org Subject: Re: Infiniband subnetmanager and driver compilation on FreeBSD 10.1 References: <557C79D8.8040403@gmail.com> In-Reply-To: <557C79D8.8040403@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Jun 2015 18:58:48 -0000 On 06/13/15 20:43, Mihai Vintila wrote: > Hi, > I've posted in the stable mailing list but i guess this is where i > should ask. > Can you tell me what is the proper way to get subnet manager working on > FreeBSD 10.1 ? > I've used RELEASE branch to compile opensm and drivers from Mellanox but > i get: > ibportstate -G 0xe61d2dfffe0a2130 1 enable > ibpanic: [868] madrpc_init: can't open UMAD port ((null):0): (m) > > opensm > ------------------------------------------------- > OpenSM 3.3.1 > Reading Cached Option File: /etc/opensm/opensm.conf > Command Line Arguments: > Log File: /var/log/opensm.log > ------------------------------------------------- > OpenSM 3.3.1 > > Entering DISCOVERING state > > Using default GUID 0x0 > > Error from osm_opensm_bind (0x2A) > Perhaps another instance of OpenSM is already running > Exiting SM > > > > kldstat > Id Refs Address Size Name > 1 27 0xffffffff80200000 1756638 kernel > 2 1 0xffffffff81957000 2681e8 zfs.ko > 3 2 0xffffffff81bc0000 6780 opensolaris.ko > 4 2 0xffffffff81bc7000 17d60 nvme.ko > 5 1 0xffffffff81bdf000 4210 nvd.ko > 6 1 0xffffffff81e11000 357f ums.ko > 7 3 0xffffffff81e15000 37656 mlx4.ko > 8 2 0xffffffff81e4d000 4e348 ibcore.ko > 9 1 0xffffffff81e9c000 2f72 toecore.ko > 10 1 0xffffffff81e9f000 238fa mlx4ib.ko > 11 1 0xffffffff81ec3000 12482 mlxen.ko > > Driver that i'm using: > mlx4_core0: mem 0xc7400000-0xc74fffff,0xc5000000-0xc57fffff > irq 32 at device 0.0 on pci2 > mlx4_core: Initializing mlx4_core: Mellanox ConnectX VPI driver v2.1.6 > (Jun 12 2015) > > mlx4_core0: mlx4_ib_add: allocated counter index 1 for port 1 > mlx4_en mlx4_core0: Activating port:1 > mlxen0: Ethernet address: e4:1d:2d:0a:21:30 > mlx4_en: mlx4_core0: Port 1: Using 12 TX rings > mlxen0: link state changed to DOWN > mlx4_en: mlx4_core0: Port 1: Using 8 RX rings > mlx4_en: mlxen0: Using 12 TX rings > mlx4_en: mlxen0: Using 8 RX rings > mlx4_en: mlxen0: Initializing port > Hi, Did you check your ports are configured for "ib" and not "eth" ? For example: sysctl sys.device.mlx4_core0.mlx4_port1=ib --HPS