Date: Thu, 16 May 2013 01:02:31 -0700 From: Jack Vogel <jfvogel@gmail.com> To: GemaEvor <legumen@foxmail.com> Cc: freebsd-virtualization <freebsd-virtualization@freebsd.org> Subject: Re: KVM+FreeBSD+82599 10G NIC doesn't work. Message-ID: <CAFOYbcnM_45OYkYbyrvQtTt9Em3-J2h05TGhU7Cvk87AP_3u_Q@mail.gmail.com> In-Reply-To: <tencent_2F83E9141799B0FD4B9F7355@qq.com> References: <tencent_2F83E9141799B0FD4B9F7355@qq.com>
next in thread | previous in thread | raw e-mail | index | archive | help
The environment that I tested in when the virtualization support was put into the driver was using SRIOV, not pass-through, we have never validated the latter, and whenever it crops up it seems to be having problems. I can put it into my queue to look into, but I can't say how quickly I can get to it, very busy here. Jack On Thu, May 16, 2013 at 12:45 AM, GemaEvor <legumen@foxmail.com> wrote: > Hi gurus: > > > I'm working on a FreeBSD virtulizaion project under KVM. > Everything works fine, except the 10G Intel 82599 SFP NIC. > > > I'm sure there are some body else who have encountered this problem > before. See: > FreeBSD guest with VTD NIC not passing traffic > http://comments.gmane.org/gmane.comp.emulators.kvm.devel/83806 > VT-d not working for FreeBSD 9.0 guest > http://permalink.gmane.org/gmane.comp.emulators.kvm.devel/91074 > > > But, unfortunately, none of them solved it or gave the clearly way to > solve it. :< > > > Firstly, let's make it clearly. This problem could not be reproduced with: > 1. Xen hypervisor + FreeBSD guest os > 2. or KVM + Linux guest os (e.g. Fedora 12) > So, it must be something wrong with KVM or FreeBSD. > > > Here is my environment: > Host OS: > OpenSUSE 12.2 > Linux linux-kj2u 3.4.6-2.10-desktop #1 SMP PREEMPT Thu Jul 26 09:36:26 UTC > 2012 (641c197) x86_64 x86_64 x86_64 GNU/Linux > QEMU emulator version 1.1.1 (kvm-1.1.1-1.8.1), Copyright (c) 2003-2008 > Fabrice Bellard > > > Guest OS: > FreeBSD 9.1 amd64 > > > What I did: > 1.Make the pci device of 82599SFP assignale: > [CODE] > 0d:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ > Network Connection (rev 01) > Subsystem: QLogic, Corp. Device 00e4 > Flags: fast devsel, IRQ 25 > Memory at f8fe0000 (64-bit, prefetchable) [size=128K] > I/O ports at cc00 [size=32] > Memory at f8fdc000 (64-bit, prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Capabilities: [70] MSI-X: Enable- Count=64 Masked- > Capabilities: [a0] Express Endpoint, MSI 00 > Capabilities: [e0] Vital Product Data > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 00-00-00-ff-ff-00-00-00 > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > Kernel driver in use: pci-stub > [/CODE] > > > 2. Passthrough it to the guest FreeBSD os. Here is my vm configuration file > [FILE] > <domain type='kvm'> > <name>FreeBSD9.1</name> > <uuid>4d56613e-400c-f37d-2a9a-e0332d81ca63</uuid> > <memory>2097152</memory> > <currentMemory>2097152</currentMemory> > <vcpu>2</vcpu> > <os> > <type arch='x86_64' machine='pc'>hvm</type> > <boot dev='hd'/> > </os> > <features> > <pae/> > <apic/> > <acpi/> > </features> > <clock offset='utc'> > </clock> > <on_poweroff>destroy</on_poweroff> > <on_reboot>restart</on_reboot> > <on_crash>destroy</on_crash> > <devices> > <emulator>/usr/bin/qemu-kvm</emulator> > <disk type='file' device='disk'> > <driver name='qemu' type='raw' cache='default'/> > <source file='/home/vm/FreeBSD9.1/FreeBSD9.1.raw'/> > <target dev='hdc'/> > </disk> > <interface type='bridge'> > <source bridge='br0'/> > <mac address='52:54:00:0d:92:6f'/> > <model type='rtl8139'/> > </interface> > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x0d' slot='0x00' function='0x0'/> > </source> > </hostdev> > <input type='mouse' bus='ps2'/> > <graphics type='vnc' port='-1' autoport='yes'/> > > </devices> > </domain> > [/FILE] > > > 3.The device could be probed and attached by FreeBSD: > [CODE] > ix0: <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 2.4.8> port > 0xc020-0xc03f mem 0xfe000000-0xfe01ffff,0xfe020000-0xfe023fff irq 11 at > device 4.0 on pci0 > ix0: Using MSIX interrupts with 2 vectors > ix0: Ethernet address: 00:0c:bd:05:7d:4a > ix0: PCI Express Bus: Speed 5.0Gb/s Width x8 > [/CODE] > > > But no interrupt could be found: > [CODE] > interrupt total rate > irq1: atkbd0 143 0 > irq4: uart0 177 1 > irq15: ata1 1593 10 > cpu0:timer 7140 46 > Total 9053 58 > [/CODE] > > > And ix0 keeps to be "no carrier" (even if I config an IP address on it and > ping): > [CODE] > ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > > options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO> > ether 00:0c:bd:05:7d:4a > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet autoselect > status: no carrier > [/CODE] > > > 4.In the host side, kvm could see the change made by the guest on the > device, but no irq: > [CODE] > linux-kj2u:~ # cat /proc/interrupts > CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 > CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 > CPU13 CPU14 CPU15 CPU16 CPU17 CPU18 > CPU19 CPU20 CPU21 CPU22 CPU23 > 210: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 > 0 0 0 0 0 PCI-MSI-edge > kvm:0000:0d:00.0 > 211: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 > 0 0 0 0 0 PCI-MSI-edge > kvm:0000:0d:00.0 > [/CODE] > > > > > > > So, anyone who has the same problem or has a clue to solve it, or has some > ideas to do a debug job, please help. > > > Thanks! > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFOYbcnM_45OYkYbyrvQtTt9Em3-J2h05TGhU7Cvk87AP_3u_Q>