From owner-freebsd-infiniband@FreeBSD.ORG Sun Jun 14 19:12:14 2015 Return-Path: Delivered-To: freebsd-infiniband@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A1483559 for ; Sun, 14 Jun 2015 19:12:14 +0000 (UTC) (envelope-from unixro@gmail.com) Received: from mail-wg0-x235.google.com (mail-wg0-x235.google.com [IPv6:2a00:1450:400c:c00::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 362C8EDC for ; Sun, 14 Jun 2015 19:12:14 +0000 (UTC) (envelope-from unixro@gmail.com) Received: by wgbhy7 with SMTP id hy7so20869616wgb.2 for ; Sun, 14 Jun 2015 12:12:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=o8ZzaZjEHwD0/FhsI00TgmqvvdfrUqUFN7P+LU/L2K8=; b=RBkCp6Ee8PZB2mZ1Qu55Sjs0PiXgyxNQqTEuMikWTy+KWntJx6mHO1LkqwZzhCPD2B 8nY1rJHqHWE0oaJjuGgYeu/gM8yT7dDO3UEwqjG6s2SOtwYkJkGUDP+qH5asS4cayc2O 0sx3S+gt3zXQxHwlQoQhdURkRJzCSxgJjOmAj2fMgByWqs6t9y+n2sahnmAJMM3yvbeQ pF2OCByvt88aJrI1tPdEcXQNxGaFVEWB5nzUZ2dRkT3ArPnlxnIwoxMGaW2fwldpzIRH 4UeOtBI94L8Zu/G3Hs+cJk/vAqhK2R+ENez76ZpvHFxeVpw5VJeWWWH71pX+sZKk5iAQ ELmg== X-Received: by 10.180.7.169 with SMTP id k9mr25435318wia.84.1434309132665; Sun, 14 Jun 2015 12:12:12 -0700 (PDT) Received: from [192.168.3.105] ([193.148.0.35]) by mx.google.com with ESMTPSA id di9sm12487217wib.16.2015.06.14.12.12.11 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 14 Jun 2015 12:12:12 -0700 (PDT) Message-ID: <557DD20B.7020508@gmail.com> Date: Sun, 14 Jun 2015 22:12:11 +0300 From: Mihai Vintila User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:31.0) Gecko/20100101 Thunderbird/31.7.0 MIME-Version: 1.0 To: Hans Petter Selasky , freebsd-infiniband@freebsd.org Subject: Re: Ib0 not working unless i unload ipoib and load it again. References: <557C79D8.8040403@gmail.com> <557C7D94.9020207@selasky.org> <557DCA80.50203@gmail.com> <557DCEA7.3030703@selasky.org> In-Reply-To: <557DCEA7.3030703@selasky.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-infiniband@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Infiniband on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 Jun 2015 19:12:14 -0000 Same behavior: sysctl -w sys.device.mlx4_core0.mlx4_port1=ib sys.device.mlx4_core0.mlx4_port1: eth -> ib ifconfig ib0 ib0: flags=8043 metric 0 mtu 2044 options=8009b lladdr 0.0.0.48.fe.80.0.0.0.0.0.0.e4.1d.2d.3.0.a.21.31 inet 10.105.0.225 netmask 0xffffff00 broadcast 10.105.0.255 nd6 options=29 ping 10.105.0.15 PING 10.105.0.15 (10.105.0.15): 56 data bytes ping: sendto: No route to host ping: sendto: No route to host Might worth to mention that this host also holds opensm, and i actually need to run opensm first next unload/load the module. But i rather expect that the ib0 state changes along mlx4_core0 state. Best regards, Vintila Mihai Alexandru On 6/14/2015 9:57 PM, Hans Petter Selasky wrote: > Hi, > > On 06/14/15 20:40, Mihai Vintila wrote: >> Another strange issue. Is that i have: >> /boot/loader.conf >> mlx4_core="YES" >> ibcore_load="YES" >> mlxen_load="YES" >> mlx4ib_load="YES" >> ipoib_load="YES" > > If you remove this line from sysctl.conf and run it from the command > line instead, does it work then? > >> >> and in sysctl.conf >> sys.device.mlx4_core0.mlx4_port1=ib >> >> Only when it boots ib0 is not functional. From what i've noticed if >> after mlx4_core0 is up i remove ipoib module and add it again it's >> working. >> >> Bellow is a small output from dmesg in which i unload/load ipoib at boot >> in rc.local and interface still doesn't work and after port is up i do >> it manually and it starts working: >> ib0: Attached to mlx4_0 port 1 >> ums0: >> on usbus0 >> ums0: 3 buttons and [Z] coordinates ID=0 >> ib0: link state changed to DOWN >> ib0: Attached to mlx4_0 port 1 >> mlx4_core0: mlx4_ib: Port 1 logical link is up >> ib0: link state changed to DOWN >> ib0: Attached to mlx4_0 port 1 >> ib0: link state changed to UP >> >> >> in ifconfig only difference is a new lladdr for ib0. >> > > --HPS >