From owner-freebsd-infiniband@freebsd.org Thu Aug 20 01:45:38 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3D8B9BEEA4 for ; Thu, 20 Aug 2015 01:45:38 +0000 (UTC) (envelope-from jon.the.wong@gmail.com) Received: from mail-ob0-x229.google.com (mail-ob0-x229.google.com [IPv6:2607:f8b0:4003:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E95DD6C for ; Thu, 20 Aug 2015 01:45:38 +0000 (UTC) (envelope-from jon.the.wong@gmail.com) Received: by obkg7 with SMTP id g7so20259549obk.3 for ; Wed, 19 Aug 2015 18:45:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=CWARYO7h+7jTcdjzC4VOGOGEFeEe/48vhRTvUTruato=; b=nzxck/WEVu/d27RdebTA4ZmTpY4e7RROPGxlyCD24S8yWF2sJ55LatYkAdb1z7Ma7x qIjahgOA8LWGnlkQ+uUdXXbdlNbkiR2QsvaCJYITwCYavUf+E0/a0fdC6u21BxRfrB8e ErqmEGU2pqFRsB/egK1a9NaCU0rsLbqnX4/023G4fhkmIPr+9HJZRvS2n2CTmgkp4v9U FuEdJUVmz5gG+J3B42Ud5FNOk98A8/y8pd9kyCWekJ9XrQGjjfV/M0Gb3tNGVQBRMrCR sP3JI1lxdKWdKzJD0V2ZRmbdd+Hd78l6nO80khWRIxVIhvBmdJcVUR+Dl60bEGmFsnb9 3eeQ== MIME-Version: 1.0 X-Received: by 10.60.132.45 with SMTP id or13mr535389oeb.20.1440035137799; Wed, 19 Aug 2015 18:45:37 -0700 (PDT) Received: by 10.202.172.197 with HTTP; Wed, 19 Aug 2015 18:45:37 -0700 (PDT) Date: Wed, 19 Aug 2015 18:45:37 -0700 Message-ID: Subject: Switchless FreeBSD/Ubuntu "eth" mode From: Jonathan Wong To: freebsd-infiniband@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 01:45:38 -0000 I've been able to get ipoib working in "ib" mode between a FreeBSD and Ubuntu machine. The machines are directly connected without a switch, so in "ib" mode I run SMs. I've been able to ping and transfer files over the ib interfaces via static ip addresses I've statically defined. In "eth" mode, I can't seem to get the machines connected or ping either machine using static ip addresses I've assigned to the interfaces. "ibstat" shows that at least one set of interfaces is up. tcpdump on mlxen0, and mlxen1 (freebsd) and eth1 (ubuntu) show outgoing ARP requests when pinging, but no replies. Is there a utility to debug whether eth mode is working in a switchless configuration? Thanks From owner-freebsd-infiniband@freebsd.org Thu Aug 20 06:34:23 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9759F9BEF37 for ; Thu, 20 Aug 2015 06:34:23 +0000 (UTC) (envelope-from odeds@mellanox.com) Received: from emea01-db3-obe.outbound.protection.outlook.com (mail-db3on0058.outbound.protection.outlook.com [157.55.234.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "MSIT Machine Auth CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 20FDD1A0C for ; Thu, 20 Aug 2015 06:34:21 +0000 (UTC) (envelope-from odeds@mellanox.com) Received: from DB3PR05MB026.eurprd05.prod.outlook.com (10.242.137.13) by DB3PR05MB028.eurprd05.prod.outlook.com (10.242.137.23) with Microsoft SMTP Server (TLS) id 15.1.225.19; Thu, 20 Aug 2015 06:18:55 +0000 Received: from DB3PR05MB026.eurprd05.prod.outlook.com ([169.254.2.112]) by DB3PR05MB026.eurprd05.prod.outlook.com ([169.254.2.112]) with mapi id 15.01.0243.020; Thu, 20 Aug 2015 06:18:56 +0000 From: Oded Shanoon To: Jonathan Wong , "freebsd-infiniband@freebsd.org" Subject: RE: Switchless FreeBSD/Ubuntu "eth" mode Thread-Topic: Switchless FreeBSD/Ubuntu "eth" mode Thread-Index: AQHQ2unuM2Ypw8xDQ0ORJjksObvABZ4UacZQ Date: Thu, 20 Aug 2015 06:18:55 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=odeds@mellanox.com; x-originating-ip: [193.47.165.251] x-microsoft-exchange-diagnostics: 1; DB3PR05MB028; 5:9T8hmDRN71Af7FTtPHNe4E7JpOzV1e+s+fR8gI8CwEEmOD0mYX4nWGnjF83/PxWG7RIdT0MwbTcXlYNlJU75LePHLZMp3qblVcPIJeMBFjfqfLXd/PbWDvqwk5BN3HnbbZXZ+hSBFdFRD/wvsVvijQ==; 24:Og+HyX0oY18Ci+poCwI6ZExK0Z+PBp0/1iE+e9RJE+t3gdb7P5ERbewGJMOkLzvaaUkJaySl8LA5zCJJ5hAG72c6c8SlG7GqOMpECcADeK8=; 20:Qg40xfqZt8lAJ+UPt/CCvqRyQtfS7fhNAcK131W+N5IaQ7Cmmakgzmv5bgqufH/Cbay5xQW2KpPfp5o4SRZPxA== x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(42134001)(42139001); SRVR:DB3PR05MB028; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(601004)(8121501046)(5005006)(3002001); SRVR:DB3PR05MB028; BCL:0; PCL:0; RULEID:; SRVR:DB3PR05MB028; x-forefront-prvs: 0674DC6DD3 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(189002)(199003)(13464003)(377454003)(77156002)(97736004)(5002640100001)(68736005)(2900100001)(102836002)(19580395003)(76176999)(2950100001)(19580405001)(62966003)(189998001)(122556002)(74316001)(92566002)(15975445007)(5003600100002)(101416001)(76576001)(5001770100001)(50986999)(81156007)(4001540100001)(5001830100001)(54356999)(5001860100001)(107886002)(5001960100002)(86362001)(46102003)(64706001)(33656002)(66066001)(10400500002)(2501003)(40100003)(2656002)(87936001)(106116001)(5007970100001)(105586002)(106356001); DIR:OUT; SFP:1101; SCL:1; SRVR:DB3PR05MB028; H:DB3PR05MB026.eurprd05.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:23 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Aug 2015 06:18:55.9521 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR05MB028 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 06:34:23 -0000 What do you mean by Eth mode? The Mellanox Ethernet driver (which produces the mlxen interfaces) is worki= ng on FreeBSD. What kind of cards do you have? Which FreeBSD version and Mellanox driver are you using?=20 Regards, Oded Shanoon OFED-FreeBSD Team Leader Mellanox Technologies, Raanana -----Original Message----- From: owner-freebsd-infiniband@freebsd.org [mailto:owner-freebsd-infiniband= @freebsd.org] On Behalf Of Jonathan Wong Sent: Thursday, August 20, 2015 4:46 AM To: freebsd-infiniband@freebsd.org Subject: Switchless FreeBSD/Ubuntu "eth" mode I've been able to get ipoib working in "ib" mode between a FreeBSD and Ubun= tu machine. The machines are directly connected without a switch, so in "ib= " mode I run SMs. I've been able to ping and transfer files over the ib int= erfaces via static ip addresses I've statically defined. In "eth" mode, I can't seem to get the machines connected or ping either ma= chine using static ip addresses I've assigned to the interfaces. "ibstat" shows that at least one set of interfaces is up. tcpdump on mlxen0, and mlxen1 (freebsd) and eth1 (ubuntu) show outgoing ARP requests when pinging,= but no replies. Is there a utility to debug whether eth mode is working in a switchless con= figuration? Thanks _______________________________________________ freebsd-infiniband@freebsd.org mailing list https://lists.freebsd.org/mailm= an/listinfo/freebsd-infiniband To unsubscribe, send any mail to "freebsd-infiniband-unsubscribe@freebsd.or= g" From owner-freebsd-infiniband@freebsd.org Thu Aug 20 07:50:31 2015 Return-Path: Delivered-To: freebsd-infiniband@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0ABE9BE154 for ; Thu, 20 Aug 2015 07:50:30 +0000 (UTC) (envelope-from jon.the.wong@gmail.com) Received: from mail-oi0-x235.google.com (mail-oi0-x235.google.com [IPv6:2607:f8b0:4003:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B6FD61FCA for ; Thu, 20 Aug 2015 07:50:30 +0000 (UTC) (envelope-from jon.the.wong@gmail.com) Received: by oio137 with SMTP id 137so18284235oio.0 for ; Thu, 20 Aug 2015 00:50:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xdMt4PcI6/EzZtqa76440wPQ6ORV65aUgvIVcIozNdA=; b=WYBJ3r6wP83+6w9ykctL6uXtG+0roPxOc5cV3nwdIPGoCkkmuENSN31QSmOm6k4sbz Ks5oWSbgxlzfzfOHOUNWFSiYVNoRFyWTtQX8ViwelJtX/gDxr0gfG3y6N8XRGSKb7rJ1 ucdmifPEPPRS/vkv0JyumaBepXhxjs0qyWJzIKjR+4IY71jx5o8WQdhlmT3UXzwC1FbP 0CR3HgyHUaMIA0BN18AIJrol1s7+2eNHZejzIyYyEzwYqR1q2SrwykzYoGhJxkmpMnD7 YfbVXroVMhgXZKlFmvcnDISD/Do9n+OZ40YSmLqsc5evgcB2jUEGRmhwUCm4u17YlfAe 7nSQ== MIME-Version: 1.0 X-Received: by 10.202.204.66 with SMTP id c63mr1534618oig.12.1440057029735; Thu, 20 Aug 2015 00:50:29 -0700 (PDT) Received: by 10.202.172.197 with HTTP; Thu, 20 Aug 2015 00:50:29 -0700 (PDT) In-Reply-To: References: Date: Thu, 20 Aug 2015 00:50:29 -0700 Message-ID: Subject: Re: Switchless FreeBSD/Ubuntu "eth" mode From: Jonathan Wong To: Oded Shanoon Cc: "freebsd-infiniband@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 07:50:31 -0000 The cards were dual port ConnectX 3 cards. I managed to resolve the error which appears to be a routing related issue because of the way I wired the switchless setup. Thanks. On Wed, Aug 19, 2015 at 11:18 PM, Oded Shanoon wrote: > What do you mean by Eth mode? > The Mellanox Ethernet driver (which produces the mlxen interfaces) is > working on FreeBSD. > > What kind of cards do you have? > Which FreeBSD version and Mellanox driver are you using? > > > Regards, > > Oded Shanoon > OFED-FreeBSD Team Leader > Mellanox Technologies, Raanana > > -----Original Message----- > From: owner-freebsd-infiniband@freebsd.org [mailto: > owner-freebsd-infiniband@freebsd.org] On Behalf Of Jonathan Wong > Sent: Thursday, August 20, 2015 4:46 AM > To: freebsd-infiniband@freebsd.org > Subject: Switchless FreeBSD/Ubuntu "eth" mode > > I've been able to get ipoib working in "ib" mode between a FreeBSD and > Ubuntu machine. The machines are directly connected without a switch, so in > "ib" mode I run SMs. I've been able to ping and transfer files over the ib > interfaces via static ip addresses I've statically defined. > > In "eth" mode, I can't seem to get the machines connected or ping either > machine using static ip addresses I've assigned to the interfaces. "ibstat" > shows that at least one set of interfaces is up. tcpdump on mlxen0, and > mlxen1 (freebsd) and eth1 (ubuntu) show outgoing ARP requests when > pinging, but no replies. > > Is there a utility to debug whether eth mode is working in a switchless > configuration? > > Thanks > _______________________________________________ > freebsd-infiniband@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-infiniband > To unsubscribe, send any mail to " > freebsd-infiniband-unsubscribe@freebsd.org" >