From owner-freebsd-virtualization@FreeBSD.ORG Thu May 31 00:46:12 2012 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 72B2D106566C for ; Thu, 31 May 2012 00:46:12 +0000 (UTC) (envelope-from sheshas@gmail.com) Received: from mail-lb0-f182.google.com (mail-lb0-f182.google.com [209.85.217.182]) by mx1.freebsd.org (Postfix) with ESMTP id E17F88FC16 for ; Thu, 31 May 2012 00:46:11 +0000 (UTC) Received: by lbon10 with SMTP id n10so572360lbo.13 for ; Wed, 30 May 2012 17:46:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=w/OLfWF5H6srT5u9PlsZIckKntRbsyJ4tED0WKvKNjk=; b=o3zCq7BEM3CWoQnyp2TsRg1MXGlg+VoEIHkpVls2mfiF7PuFE8o5KNAJ4kpfVR4TE7 ldQSDGspMNIx5LGOKeCbLKBCCyvaRA3Ch36K6o4MnCIQXfCTPa8T7+0RNQmdks7OqDwx clocw0B8R8M/BSEDNjbXj6yjQu15BZlViMMXkkf1cZkXoz3I9DA57dC44Twfjzv9+Z1X Qmd6h4MgxrGWwSJt8rNUz+l/szfMtsyfOqfuSTO+TKGMmQEH5AHPPfcaD+GpnyzmYWWI +IhIXWCZyZnlKkfQKbFqwQx5O7Y51AxPdVmYlFfeDrqi15Nniym1oh3VuEVlnvD8z11/ VpPg== MIME-Version: 1.0 Received: by 10.152.48.6 with SMTP id h6mr17702632lan.30.1338425170584; Wed, 30 May 2012 17:46:10 -0700 (PDT) Received: by 10.112.58.7 with HTTP; Wed, 30 May 2012 17:46:10 -0700 (PDT) Date: Wed, 30 May 2012 17:46:10 -0700 Message-ID: From: Shesha Sreenivasamurthy To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: VT-d issues with FreeBSD 9.0 guest using 82599 HW X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 May 2012 00:46:12 -0000 Hi, Earlier I had sent the same email to freebsd general discussion. However, I felt this forum is more appropriate. I apologize for double email. I have an issue with enabling VT-d with freebsd guest and was wondering of anyone out there are experiencing the similar issues. I am using the following setup HOST: Centos 6.2 Hypervisor: KVM Hardware: ixgbe 82599EB (10 GB nic) Guest: freebsd 9.0 Guest Driver version 2.4.4 When I use MSI/MSIX the link status is shown as "no carrier". When I disable MSI and MSIX in the guest so that it can fall back on INTx, I see that the link becomes active. However not packets are sent or received. When pinged from outside, sysctl shows dev.ix.0.queue0.txd_head: 0 dev.ix.0.queue0.rxd_head: 0 dev.ix.0.queue0.rxd_tail: 2047 dev.ix.0.queue0.rx_packets: 0 dev.ix.0.mac_stats.good_octets_rcvd: 320 dev.ix.0.mac_stats.good_pkts_rcvd: 5 When pinged from inside, sysctl shows dev.ix.0.queue0.txd_head: 0 dev.ix.0.queue0.rxd_head: 0 dev.ix.0.queue0.txd_tail: 4 dev.ix.0.queue0.rxd_tail: 2047 dev.ix.0.queue0.tx_packets: 4 dev.ix.0.queue0.rx_packets: 0 dev.ix.0.mac_stats.good_octets_rcvd: 320 dev.ix.0.mac_stats.good_pkts_rcvd: 5 dev.ix.0.mac_stats.good_octets_txd: 0 dev.ix.0.mac_stats.good_pkts_txd: 0 I guess, hardware is getting the packets but not put in the ringbuffer. If I install freebsd 9.0 on bare metel and have the same hardware and same driver running, the link comes up and packets are transferred. Therefore, in case of VT-d (PCI passthrough) the driver is not setting up things correctly. Another data point is, if Centos guests is used instead of freebsd guest, the hardware comes up correctly and can send/receive packets. The freebsd driver that I am using is the one posted on the intel's website. http://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&ProdId=3189&DwnldID=14688&ProductFamily=Ethernet+Components&ProductLine=Ethernet+Controllers&ProductProduct=Intel%C2%AE+82599+10+Gigabit+Ethernet+Controller&DownloadType=Drivers&OSFullname=FreeBSD*eng