From owner-freebsd-virtualization@FreeBSD.ORG Thu May 16 08:02:32 2013 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AC9CAB90 for ; Thu, 16 May 2013 08:02:32 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ve0-x22b.google.com (mail-ve0-x22b.google.com [IPv6:2607:f8b0:400c:c01::22b]) by mx1.freebsd.org (Postfix) with ESMTP id 70F4062C for ; Thu, 16 May 2013 08:02:32 +0000 (UTC) Received: by mail-ve0-f171.google.com with SMTP id m1so2737469ves.16 for ; Thu, 16 May 2013 01:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=VhBXqem30W6+nGINdpjQDAbV6XbjMGWWy2HZuMkf22Q=; b=hV5lsUkXnLKh633GUHCX7/X8/tqeKJmyv8gw5UqZ0q3tyufoyJoObbUdYf/zQ/y3Bt 1KhRLxozS2pH4ruxpW2WfXTz2WJprAXwZhv3259XzOmYxOhg/TPGUSbhWn1pxyVpBLNf E8yv9+78gvP4j0a96zPkfWWdLcN5DjhdvpcvMIso9S9/eHqvWyn56U5B4vNGz85ZMzdG op9cevQgp7uNnyEJKx9OWI+ptQjllolKf/2jsigZLmeFsB/yFAQzUHxdstffJG9Ioqlt wfxzI4URnF9AjyZBHltPxShnvygoHuGsnYuYhoV1dweX0H4PcjZsJuPBihqT2GgIMzFD 0Atg== MIME-Version: 1.0 X-Received: by 10.220.223.202 with SMTP id il10mr27567761vcb.4.1368691351972; Thu, 16 May 2013 01:02:31 -0700 (PDT) Received: by 10.220.55.143 with HTTP; Thu, 16 May 2013 01:02:31 -0700 (PDT) In-Reply-To: References: Date: Thu, 16 May 2013 01:02:31 -0700 Message-ID: Subject: Re: KVM+FreeBSD+82599 10G NIC doesn't work. From: Jack Vogel To: GemaEvor Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-virtualization X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 08:02:32 -0000 The environment that I tested in when the virtualization support was put into the driver was using SRIOV, not pass-through, we have never validated the latter, and whenever it crops up it seems to be having problems. I can put it into my queue to look into, but I can't say how quickly I can get to it, very busy here. Jack On Thu, May 16, 2013 at 12:45 AM, GemaEvor wrote: > Hi gurus: > > > I'm working on a FreeBSD virtulizaion project under KVM. > Everything works fine, except the 10G Intel 82599 SFP NIC. > > > I'm sure there are some body else who have encountered this problem > before. See: > FreeBSD guest with VTD NIC not passing traffic > http://comments.gmane.org/gmane.comp.emulators.kvm.devel/83806 > VT-d not working for FreeBSD 9.0 guest > http://permalink.gmane.org/gmane.comp.emulators.kvm.devel/91074 > > > But, unfortunately, none of them solved it or gave the clearly way to > solve it. :< > > > Firstly, let's make it clearly. This problem could not be reproduced with: > 1. Xen hypervisor + FreeBSD guest os > 2. or KVM + Linux guest os (e.g. Fedora 12) > So, it must be something wrong with KVM or FreeBSD. > > > Here is my environment: > Host OS: > OpenSUSE 12.2 > Linux linux-kj2u 3.4.6-2.10-desktop #1 SMP PREEMPT Thu Jul 26 09:36:26 UTC > 2012 (641c197) x86_64 x86_64 x86_64 GNU/Linux > QEMU emulator version 1.1.1 (kvm-1.1.1-1.8.1), Copyright (c) 2003-2008 > Fabrice Bellard > > > Guest OS: > FreeBSD 9.1 amd64 > > > What I did: > 1.Make the pci device of 82599SFP assignale: > [CODE] > 0d:00.0 Ethernet controller: Intel Corporation 82599EB 10-Gigabit SFI/SFP+ > Network Connection (rev 01) > Subsystem: QLogic, Corp. Device 00e4 > Flags: fast devsel, IRQ 25 > Memory at f8fe0000 (64-bit, prefetchable) [size=128K] > I/O ports at cc00 [size=32] > Memory at f8fdc000 (64-bit, prefetchable) [size=16K] > Capabilities: [40] Power Management version 3 > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Capabilities: [70] MSI-X: Enable- Count=64 Masked- > Capabilities: [a0] Express Endpoint, MSI 00 > Capabilities: [e0] Vital Product Data > Capabilities: [100] Advanced Error Reporting > Capabilities: [140] Device Serial Number 00-00-00-ff-ff-00-00-00 > Capabilities: [150] Alternative Routing-ID Interpretation (ARI) > Capabilities: [160] Single Root I/O Virtualization (SR-IOV) > Kernel driver in use: pci-stub > [/CODE] > > > 2. Passthrough it to the guest FreeBSD os. Here is my vm configuration file > [FILE] > > FreeBSD9.1 > 4d56613e-400c-f37d-2a9a-e0332d81ca63 > 2097152 > 2097152 > 2 > > hvm > > > > > > > > > > destroy > restart > destroy > > /usr/bin/qemu-kvm > > > > > > > > > > > > >
> > > > > > > > [/FILE] > > > 3.The device could be probed and attached by FreeBSD: > [CODE] > ix0: port > 0xc020-0xc03f mem 0xfe000000-0xfe01ffff,0xfe020000-0xfe023fff irq 11 at > device 4.0 on pci0 > ix0: Using MSIX interrupts with 2 vectors > ix0: Ethernet address: 00:0c:bd:05:7d:4a > ix0: PCI Express Bus: Speed 5.0Gb/s Width x8 > [/CODE] > > > But no interrupt could be found: > [CODE] > interrupt total rate > irq1: atkbd0 143 0 > irq4: uart0 177 1 > irq15: ata1 1593 10 > cpu0:timer 7140 46 > Total 9053 58 > [/CODE] > > > And ix0 keeps to be "no carrier" (even if I config an IP address on it and > ping): > [CODE] > ix0: flags=8802 metric 0 mtu 1500 > > options=401bb > ether 00:0c:bd:05:7d:4a > nd6 options=29 > media: Ethernet autoselect > status: no carrier > [/CODE] > > > 4.In the host side, kvm could see the change made by the guest on the > device, but no irq: > [CODE] > linux-kj2u:~ # cat /proc/interrupts > CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 > CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12 > CPU13 CPU14 CPU15 CPU16 CPU17 CPU18 > CPU19 CPU20 CPU21 CPU22 CPU23 > 210: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 > 0 0 0 0 0 PCI-MSI-edge > kvm:0000:0d:00.0 > 211: 0 0 0 0 0 0 > 0 0 0 0 0 0 0 > 0 0 0 0 0 0 > 0 0 0 0 0 PCI-MSI-edge > kvm:0000:0d:00.0 > [/CODE] > > > > > > > So, anyone who has the same problem or has a clue to solve it, or has some > ideas to do a debug job, please help. > > > Thanks! > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" >