From owner-freebsd-virtualization@freebsd.org Mon Feb 22 20:53:55 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2F6FCAB0FC2; Mon, 22 Feb 2016 20:53:55 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by mx1.freebsd.org (Postfix) with ESMTP id 0369C1D6B; Mon, 22 Feb 2016 20:53:54 +0000 (UTC) (envelope-from jeffrey.e.pieper@intel.com) Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga103.jf.intel.com with ESMTP; 22 Feb 2016 12:53:48 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,486,1449561600"; d="scan'208";a="892042241" Received: from orsmsx106.amr.corp.intel.com ([10.22.225.133]) by orsmga001.jf.intel.com with ESMTP; 22 Feb 2016 12:53:48 -0800 Received: from orsmsx153.amr.corp.intel.com (10.22.226.247) by ORSMSX106.amr.corp.intel.com (10.22.225.133) with Microsoft SMTP Server (TLS) id 14.3.248.2; Mon, 22 Feb 2016 12:53:48 -0800 Received: from orsmsx111.amr.corp.intel.com ([169.254.12.9]) by ORSMSX153.amr.corp.intel.com ([169.254.12.170]) with mapi id 14.03.0248.002; Mon, 22 Feb 2016 12:53:47 -0800 From: "Pieper, Jeffrey E" To: Ultima , Eric Joyner CC: "freebsd-current@freebsd.org" , "freebsd-virtualization@freebsd.org" Subject: RE: sr-iov issues, reset_hw() failed with error -100 Thread-Topic: sr-iov issues, reset_hw() failed with error -100 Thread-Index: AQHRbawE8t/NRDwpiUa/C5k4Foam5p84irWw Date: Mon, 22 Feb 2016 20:53:46 +0000 Message-ID: <2A35EA60C3C77D438915767F458D656880942E9A@ORSMSX111.amr.corp.intel.com> References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-titus-metadata-40: eyJDYXRlZ29yeUxhYmVscyI6IiIsIk1ldGFkYXRhIjp7Im5zIjoiaHR0cDpcL1wvd3d3LnRpdHVzLmNvbVwvbnNcL0ludGVsMyIsImlkIjoiYjZkZGUyMGEtMmU0Yy00M2JmLThiY2MtOTVjMWZmN2FiNWUyIiwicHJvcHMiOlt7Im4iOiJDVFBDbGFzc2lmaWNhdGlvbiIsInZhbHMiOlt7InZhbHVlIjoiQ1RQX0lDIn1dfV19LCJTdWJqZWN0TGFiZWxzIjpbXSwiVE1DVmVyc2lvbiI6IjE1LjkuNi42IiwiVHJ1c3RlZExhYmVsSGFzaCI6IlJJZUdWcHFLUW1xSHAzQndmOVY1dm5uZlwvcVhwOTBZSEhadEF2bHljN2hFPSJ9 x-ctpclassification: CTP_IC x-originating-ip: [10.22.254.138] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Feb 2016 20:53:55 -0000 Just out of curiosity, how many cores does your system have? Jeff -----Original Message----- From: owner-freebsd-current@freebsd.org [mailto:owner-freebsd-current@freeb= sd.org] On Behalf Of Ultima Sent: Monday, February 22, 2016 12:02 PM To: Eric Joyner Cc: freebsd-current@freebsd.org; freebsd-virtualization@freebsd.org Subject: Re: sr-iov issues, reset_hw() failed with error -100 After reboot... ifconfig ix1 up dhclient ix1 DHCPDISCOVER on ix1 to 255.255.255.255 port 67 interval 4 DHCPOFFER from 192.168.1.1 DHCPREQUEST on ix1 to 255.255.255.255 port 67 DHCPACK from 192.168.1.1 bound to 192.168.1.145 -- renewal in 21600 seconds. ix0 down ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1): 56 data bytes 64 bytes from 192.168.1.1: icmp_seq=3D0 ttl=3D64 time=3D0.149 ms 64 bytes from 192.168.1.1: icmp_seq=3D1 ttl=3D64 time=3D0.171 ms 64 bytes from 192.168.1.1: icmp_seq=3D2 ttl=3D64 time=3D0.167 ms iovctl -Cf /etc/iovctl.conf ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1): 56 data bytes ^C --- 192.168.1.1 ping statistics --- 29 packets transmitted, 0 packets received, 100.0% packet loss ifconfig ix1 up ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1): 56 data bytes ^C --- 192.168.1.1 ping statistics --- 12 packets transmitted, 0 packets received, 100.0% packet loss ix1 is no longer usable until a restart... iovctl -Dd ix1 ifconfig ix1 up ping 192.168.1.1 PING 192.168.1.1 (192.168.1.1): 56 data bytes ^C --- 192.168.1.1 ping statistics --- 9 packets transmitted, 0 packets received, 100.0% packet loss Is there anything else that maybe useful? here is my ifconfig at the end (after ifconfig ix0 up) ix0: flags=3D8943 metric 0 mtu 1500 options=3De400b9 ether -Hidden- inet 192.168.1.8 netmask 0xffffff00 broadcast 192.168.1.255 inet 192.168.1.9 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=3D29 media: Ethernet autoselect (10Gbase-T ) status: active ix1: flags=3D8843 metric 0 mtu 1500 options=3De407bb ether -Hidden- inet 192.168.1.145 netmask 0xffffff00 broadcast 192.168.1.255 nd6 options=3D29 media: Ethernet autoselect (10Gbase-T ) status: active lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo bridge0: flags=3D8843 metric 0 mtu 1500 ether -Hidden- nd6 options=3D9 groups: bridge id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: ix0 flags=3D143 ifmaxaddr 0 port 1 priority 128 path cost 2000 member: epair0a flags=3D143 ifmaxaddr 0 port 5 priority 128 path cost 2000 epair0a: flags=3D8943 metri= c 0 mtu 1500 options=3D8 ether -Hidden- inet6 fe80::ff:70ff:fe00:50a%epair0a prefixlen 64 scopeid 0x5 nd6 options=3D21 media: Ethernet 10Gbase-T (10Gbase-T ) status: active groups: epair On Mon, Feb 22, 2016 at 1:51 PM, Eric Joyner wrote: > Did you do an ifconfig up on ix1 before loading the VF driver? > > On Sat, Feb 20, 2016 at 11:57 AM Ultima wrote: > >> Decided to do some testing with iovctl to see how sr-iov is coming alon= g. >> Currently when adding the vf's there are a couple errors, and the networ= k >> no longer function after iovctl is started. My guess is the reset_hw() >> call >> that is failing. Any ideas why this call would fail? I tested this on bo= th >> ports, ix1 is detached and unused for this test, however inserting a cab= le >> results in an unusable port. iovctl -Dd ix1 removes the vf's, however >> functionality is still not restored without a system restart. >> >> FreeBSD S1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r295736: Wed Feb 17 >> 21:17:28 EST 2016 root@S1:/usr/obj/usr/src/sys/MYKERNEL amd64 >> >> /boot/loader.conf >> hw.ix.num_queues=3D"4" >> >> /etc/iovctl.conf >> PF { >> device : ix1; >> num_vfs : 31; >> } >> >> DEFAULT { >> passthrough : true; >> } >> VF-0 { >> passthrough : false; >> } >> VF-1 { >> passthrough : false; >> } >> >> # iovctl -C -f /etc/iovctl.conf >> >> dmesg >> ixv0: > 1.4.6-k> at device 0.129 on pci12 >> ixv0: Using MSIX interrupts with 2 vectors >> ixv0: ixgbe_reset_hw() failed with error -100 >> device_attach: ixv0 attach returned 5 >> ixv0: > 1.4.6-k> at device 0.131 on pci12 >> ixv0: Using MSIX interrupts with 2 vectors >> ixv0: ixgbe_reset_hw() failed with error -100 >> device_attach: ixv0 attach returned 5 >> pci12: at device 0.133 (no driver attached) >> pci12: at device 0.135 (no driver attached) >> pci12: at device 0.137 (no driver attached) >> pci12: at device 0.139 (no driver attached) >> pci12: at device 0.141 (no driver attached) >> pci12: at device 0.143 (no driver attached) >> pci12: at device 0.145 (no driver attached) >> pci12: at device 0.147 (no driver attached) >> pci12: at device 0.149 (no driver attached) >> pci12: at device 0.151 (no driver attached) >> pci12: at device 0.153 (no driver attached) >> pci12: at device 0.155 (no driver attached) >> pci12: at device 0.157 (no driver attached) >> pci12: at device 0.159 (no driver attached) >> pci12: at device 0.161 (no driver attached) >> pci12: at device 0.163 (no driver attached) >> pci12: at device 0.165 (no driver attached) >> pci12: at device 0.167 (no driver attached) >> pci12: at device 0.169 (no driver attached) >> pci12: at device 0.171 (no driver attached) >> pci12: at device 0.173 (no driver attached) >> pci12: at device 0.175 (no driver attached) >> pci12: at device 0.177 (no driver attached) >> pci12: at device 0.179 (no driver attached) >> pci12: at device 0.181 (no driver attached) >> pci12: at device 0.183 (no driver attached) >> pci12: at device 0.185 (no driver attached) >> pci12: at device 0.187 (no driver attached) >> pci12: at device 0.189 (no driver attached) >> >> pciconf -lv >> ix1@pci0:129:0:1: class=3D0x020000 card=3D0x00001458 chip=3D0x1528= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'Ethernet Controller 10-Gigabit X540-AT2' >> class =3D network >> subclass =3D ethernet >> none155@pci0:129:0:129: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> none156@pci0:129:0:131: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt0@pci0:129:0:133: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt1@pci0:129:0:135: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt2@pci0:129:0:137: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt3@pci0:129:0:139: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt4@pci0:129:0:141: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt5@pci0:129:0:143: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt6@pci0:129:0:145: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt7@pci0:129:0:147: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt8@pci0:129:0:149: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt9@pci0:129:0:151: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt10@pci0:129:0:153: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt11@pci0:129:0:155: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt12@pci0:129:0:157: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt13@pci0:129:0:159: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt14@pci0:129:0:161: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt15@pci0:129:0:163: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt16@pci0:129:0:165: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt17@pci0:129:0:167: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt18@pci0:129:0:169: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt19@pci0:129:0:171: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt20@pci0:129:0:173: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt21@pci0:129:0:175: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt22@pci0:129:0:177: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt23@pci0:129:0:179: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt24@pci0:129:0:181: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt25@pci0:129:0:183: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt26@pci0:129:0:185: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt27@pci0:129:0:187: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> ppt28@pci0:129:0:189: class=3D0x020000 card=3D0x00001458 chip=3D0x1515= 8086 >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'Intel Corporation' >> device =3D 'X540 Ethernet Controller Virtual Function' >> class =3D network >> subclass =3D ethernet >> >> Ultima >> _______________________________________________ >> freebsd-current@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-current >> To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.or= g >> " >> > _______________________________________________ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"