From owner-freebsd-infiniband@freebsd.org Sat Apr 16 14:56:05 2016 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 20D7FAED7C0 for ; Sat, 16 Apr 2016 14:56:05 +0000 (UTC) (envelope-from menyy@mellanox.com) Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1on0056.outbound.protection.outlook.com [157.56.112.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 744C51BB5 for ; Sat, 16 Apr 2016 14:56:03 +0000 (UTC) (envelope-from menyy@mellanox.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=JYueaUnkcxYMm8nLwVqRVZzwY3RZM/d2mGNH43UcqF0=; b=G1zubrPotEqZFQEUBvgJkguYMpsOt8FoeUqfOPp/FTPQhSZW8ZTNLrRmI3h2cz9ziUm5XQPPmlO11a0IbhJRQcsMXR6HYWqPOvDzqmhtZlxSN1IFgzxsHclwBQCgexysjpRqNMIfjiCWy6xW23FPDVGXjGzGI815Y9OGoGMJAa8= Received: from DB5PR05MB1765.eurprd05.prod.outlook.com (10.165.7.23) by DB5PR05MB1302.eurprd05.prod.outlook.com (10.162.157.151) with Microsoft SMTP Server (TLS) id 15.1.466.19; Sat, 16 Apr 2016 14:55:54 +0000 Received: from DB5PR05MB1765.eurprd05.prod.outlook.com ([10.165.7.23]) by DB5PR05MB1765.eurprd05.prod.outlook.com ([10.165.7.23]) with mapi id 15.01.0453.031; Sat, 16 Apr 2016 14:55:54 +0000 From: Meny Yossefi To: "owner-freebsd-infiniband@freebsd.orgOn" , "freebsd-infiniband@freebsd.org" , Hans Petter Selasky , Yuval Bason Subject: RE: How to force IB cards to come up in IB mode? Thread-Topic: How to force IB cards to come up in IB mode? Thread-Index: AQHRl9FGkcYw8QOXCky3Yxke7zEzY5+McuvlgAA2bcA= Date: Sat, 16 Apr 2016 14:55:53 +0000 Message-ID: References: <9C08621D-AAE9-42D0-8079-716CDD9BC5E0@postgresql.org> <223ee8f3dd5240779d39bf78c383c50c@AMSPR05MB0580.eurprd05.prod.outlook.com> In-Reply-To: <223ee8f3dd5240779d39bf78c383c50c@AMSPR05MB0580.eurprd05.prod.outlook.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.orgOn; dkim=none (message not signed) header.d=none; freebsd.orgOn; dmarc=none action=none header.from=mellanox.com; x-originating-ip: [79.178.38.82] x-ms-office365-filtering-correlation-id: 665fc7ce-4c0f-4d7d-5173-08d3660735fc x-microsoft-exchange-diagnostics: 1; DB5PR05MB1302; 5:6MApQlxtNNbWH/80byvbnRiX1ENC62tJJXM+10STD6IQL2PjTRCWMgXG+WLxB1u7Ng/FpdQw904zuqdWNvHtD4+TO0AkzFZEujoWnKFiTsFzhE9CH/Z6Xq2g4BZAHlHsa5oYth3Q2Pz6HKKJ4GnwjjoNFoI/yn+fzgWdBg08WE4pxcTxeduf/a8iKbZP5L9Q; 24:w+X0EIBmlXru+Pzpah+c/c4/vv791vWsV5fDZIGdlEiEYKHd3fhBicm7CWiZGNy7o9Bm/cRp7mKrjwcUt+FY0oFeBBV5JTJsPO9glWmMfpY=; 7:dm1DpcqEIcmCiLOp038GtTeSRnZe22XKUKpAKMYzoDcR4RwZuSiBBmVsDFkJz4OuEFsRImXIqelLrtaJhzeexi+Ib7IrdBzxnfqdgtzgUjLJ+qUlIcktCy6CTZ7n1+Mea0i5Mfj58FcJcQL31SN/FWNy9vgjDD7gJkdVm/w8r8wNCm7TZ0moSvMv/G1O0CKHG1QzkT1L5Z5+T8l9BX5/1w== x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:DB5PR05MB1302; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(9101521026)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001); SRVR:DB5PR05MB1302; BCL:0; PCL:0; RULEID:; SRVR:DB5PR05MB1302; x-forefront-prvs: 09144DB0F7 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(53754006)(377454003)(3660700001)(5008740100001)(5004730100002)(11100500001)(33656002)(110136002)(189998001)(81166005)(107886002)(66066001)(19617315012)(19625215002)(106116001)(2906002)(16236675004)(5003600100002)(19300405004)(3280700002)(5002640100001)(92566002)(87936001)(50986999)(1096002)(1220700001)(2900100001)(3846002)(790700001)(6116002)(102836003)(122556002)(15975445007)(76176999)(2950100001)(10400500002)(2501003)(76576001)(4001450100002)(9686002)(19580405001)(19580395003)(77096005)(86362001)(586003)(74316001)(54356999); DIR:OUT; SFP:1101; SCL:1; SRVR:DB5PR05MB1302; H:DB5PR05MB1765.eurprd05.prod.outlook.com; FPR:; SPF:None; MLV:sfv; LANG:en; MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Apr 2016 14:55:53.5747 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5PR05MB1302 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Apr 2016 14:56:05 -0000 Hi Justin, It seems as the pre-boot configuration didn't work. loader.conf doesn't work on all sysctl params AFAIK. Is it a pre-boot only issue? What is the current sysctl output for those ports? Have you tried switching to IB after system was loaded? If I remember correctly, the driver was set for Eth configuration by defaul= t. Of course, this is reversible.. Regards, Meny Yossefi | SW Manager | FreeBSD Team Mellanox Technologies Ltd Work: +972-74-7129121, Cell: +972-52-8379557 ________________________________________ From: owner-freebsd-infiniband@freebsd.orgOn Behalf OfJustin Clift Sent: Saturday, April 16, 2016 11:15:11 AM (UTC) Monrovia, Reykjavik To: freebsd-infiniband@freebsd.org Subject: How to force IB cards to come up in IB mode? Hi all, I'm trying to get a new FreeBSD 10.3-STABLE install to work well with my MH= GH28-XTC (dual port ConnectX DDR) cards. The problem is they're coming up in eth mode, not ib mode, and I can't seem= to get that fixed. :( >From boot: # ifconfig mlxen0: flags=3D8802 metric 0 mtu 1500 options=3D6d07bb ether 00:02:c9:00:ca:cd nd6 options=3D29 media: Ethernet autoselect status: no carrier mlxen1: flags=3D8802 metric 0 mtu 1500 options=3D6d07bb ether 00:02:c9:00:ca:ce nd6 options=3D29 media: Ethernet autoselect status: no carrier My loader.conf file seems like it should be correct: # cat /boot/loader.conf sys.device.mlx4_core0.mlx4_port1=3D"ib" sys.device.mlx4_core0.mlx4_port2=3D"ib" As background, this is a new install (coming from CentOS 6.x, where it all = works well), so I know there's no problem with the cards. They're plugged = into a 24 port DDR switch (Voltaire ISR 9024D-M) using active optical cable= s. Any ideas? Regards and best wishes, Justin Clift -- "My grandfather once told me that there are two kinds of people: those who = work and those who take the credit. He told me to try to be in the first gr= oup; there was less competition there." - Indira Gandhi _______________________________________________ freebsd-infiniband@freebsd.org maili= ng list https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.or= g"