From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 31 05:59:22 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 15728619 for ; Tue, 31 Mar 2015 05:59:22 +0000 (UTC) Received: from plane.gmane.org (plane.gmane.org [80.91.229.3]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C54E79EF for ; Tue, 31 Mar 2015 05:59:21 +0000 (UTC) Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1YcpCh-00016K-6c for freebsd-virtualization@freebsd.org; Tue, 31 Mar 2015 07:59:11 +0200 Received: from ip184-189-251-175.sb.sd.cox.net ([184.189.251.175]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Mar 2015 07:59:11 +0200 Received: from madoka by ip184-189-251-175.sb.sd.cox.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 31 Mar 2015 07:59:11 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-virtualization@freebsd.org From: Julian Hsiao Subject: Re: Bhyve storage improvements (was: Several bhyve quirks) Date: Mon, 30 Mar 2015 22:59:01 -0700 Lines: 43 Message-ID: References: <5515270A.7050408@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: 8bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: ip184-189-251-175.sb.sd.cox.net User-Agent: Unison/2.1.10 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 05:59:22 -0000 On 2015-03-27 09:46:50 +0000, Alexander Motin said: > [snip] > > Also both virtio-blk and ahci-hd drivers now report to guest logical and > physical block sizes of underlying storage, that allow guests properly > align partitions and I/Os for best compatibility and performance. Hi Alexander, In a previous reply from Peter Grehan, he said that ahci-hd should already report the correct block size in 10.1. I had some time to try it out today, but I'm still having issues: $ zfs create \ -o compression=off \ -o primarycache=metadata \ -o secondarycache=metadata \ -o volblocksize=4096 \ -o refreservation=none \ -V 10G \ zroot/usr/bhyve/test/img $ geli init -B none -e AES-XTS -K test.key -l 128 -P -s 4096 \ zvol/zroot/usr/bhyve/test/img $ geli attach -p -k test.key zvol/zroot/usr/bhyve/test/img [set up device map, grub-bhyve, etc.] $ bhyve -A -c 1 -H -P -m 256 \ -s 0:0,hostbridge \ -s 1:0,ahci-hd,img.eli \ -s 2:0,ahci-cd,ubuntu-14.10-server-amd64.iso \ -s 31,lpc -l com1,stdio \ test [boot guest to recovery console] $ fdisk -l /dev/sda fdisk: cannot open /dev/sda: Input/output error And syslog shows a lot of errors accessing sda. Note that the actual HDD has 512-byte sectors, so perhaps bhyve is getting the sector size from the hardware and not from geli / ZFS? Julian Hsiao From owner-freebsd-virtualization@FreeBSD.ORG Tue Mar 31 14:29:19 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1449DB35 for ; Tue, 31 Mar 2015 14:29:19 +0000 (UTC) Received: from mail-ob0-x22a.google.com (mail-ob0-x22a.google.com [IPv6:2607:f8b0:4003:c01::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D62BBCB5 for ; Tue, 31 Mar 2015 14:29:18 +0000 (UTC) Received: by obbec2 with SMTP id ec2so28923378obb.3 for ; Tue, 31 Mar 2015 07:29:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=zX+2ueFI4RyuQZUaNoOy25iFlLB14JtSEZ4vqIPz2Os=; b=imB5QJS1GXwPtxIyuljivhqXJ5eA+ddJ3HA+lXvqpv6hM++MJjkx19WHXtbfNCRYm9 O5Aj6Ly2pIeVuZwYFBZtZw6cc6HmPDZMp/tiG7AQ9NEM6rGuacwLbW028h20CMLeTrt8 EManp1B70Isr4UZYdVT9sz3Np33uNCgD7HRPBWlPbr8pe5BbBRV3oJR7fI4Tlhj2n+Cx F2AEmuF8QTFl8zYRyNhK7Qf56n/Vj8PIE7Hm3MyDgQd+BnvIf8cVYob0zb0O24oxM9zG irSM1oUmPxxuH2D3ISeM+R6kljkXfZB8fD02QPAs7a8rFiyxRflekJE9KtRzEfkyG/LD iKzg== MIME-Version: 1.0 X-Received: by 10.60.73.38 with SMTP id i6mr34039194oev.27.1427812144823; Tue, 31 Mar 2015 07:29:04 -0700 (PDT) Received: by 10.60.82.233 with HTTP; Tue, 31 Mar 2015 07:29:04 -0700 (PDT) Date: Tue, 31 Mar 2015 17:29:04 +0300 Message-ID: Subject: Bhyve: Investigating poor guest performance when host is busy From: Stefan Andritoiu To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 14:29:19 -0000 Problem intro: - Started a make -j 20 buildworld on the host to peg the cpus - Started a 2 and 4 vcpu FreeBSD guest and time how long it takes to boot up - Started a 1 vcpu FreeBSD guest and time how long it takes to boot up. It is always much faster than the 2 or 4 vcpus. My suspicion: I suspect that the ULE scheduler makes some bad decisions with scheduling because it is not aware that bhyve threads are actually running vcpus on top of them, and Lock-holder-preemption may occur. The classic way to solve this is with gang scheduling to ensure that all vcpus are running simultaneously. 1. Can anyone tell me what the cause might be? What may be happening? 2. Do you know if there is currently any work in investigation this problem? Or anything related? 3. Is Gang Scheduling or Coscheduling implemented in FreeBSD? 4. Do you know of any other solution to this kind of problem? 5. Can you recommend me any papers/videos/links in anyway related to this? I'm sorry if the question seem vague. But I don't understand the problem so well either and I'm relatively new to FreeBSD From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 00:06:27 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 043958DA for ; Wed, 1 Apr 2015 00:06:27 +0000 (UTC) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B29ADDBA for ; Wed, 1 Apr 2015 00:06:26 +0000 (UTC) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 4AFAE280F84 for ; Wed, 1 Apr 2015 10:06:23 +1000 (EST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au X-Virus-Scanned: amavisd-new at iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8JNnR6fgVsn4 for ; Wed, 1 Apr 2015 10:06:23 +1000 (EST) Received: from Peters-MacBook-Pro.local (c-76-126-65-88.hsd1.ca.comcast.net [76.126.65.88]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 858FD280F74; Wed, 1 Apr 2015 10:06:20 +1000 (EST) Message-ID: <551B367B.4020104@freebsd.org> Date: Tue, 31 Mar 2015 17:06:19 -0700 From: Peter Grehan User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Stefan Andritoiu Subject: Re: Bhyve: Investigating poor guest performance when host is busy References: In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 00:06:27 -0000 > 1. Can anyone tell me what the cause might be? What may be happening? > 2. Do you know if there is currently any work in investigation this > problem? Or anything related? > 3. Is Gang Scheduling or Coscheduling implemented in FreeBSD? > 4. Do you know of any other solution to this kind of problem? > 5. Can you recommend me any papers/videos/links in anyway related to this? I answered these in the FreeBSD forums post, but reproduced again here for the list: 1. The main issue is 'lock holder preemption', where a vCPU that is holding a spinlock has been pre-empted by the host scheduler, resulting in other vCPUs that are trying to acquire that lock to spin for full quantums. Booting is a variant of this for FreeBSD since the AP spins on a memory location waiting for a BSP to start up. 2. There's some minor investigation going on. 3. No. 4. I don't know that 'classic' gang scheduling is the answer (see 5). What has been thought of for bhyve at least is to a) have the concept of vCPU 'groups' in the scheduler, b) provide metrics to assist the scheduler in trying to spread out threads associated with a vCPU group so they don't end up on the same physical CPU (avoidance of lock-holder preemption), and c) implement pause-loop exits (see the Intel SDM, 24.6.13) in the hypervisor and provide that information to the scheduler so it can give a temporary priority boost to vCPUs that have been preempted but aren't currently running. 5. The classic reference on this is VMWare's scheduler paper: www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf later, Peter. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 01:24:17 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8AFDE688 for ; Wed, 1 Apr 2015 01:24:17 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 672498DC for ; Wed, 1 Apr 2015 01:24:17 +0000 (UTC) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [127.0.1.5]) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9) with ESMTP id t311OHY3002312 for ; Wed, 1 Apr 2015 01:24:17 GMT (envelope-from root@phabric-backend.isc.freebsd.org) Received: (from root@localhost) by phabric-backend.isc.freebsd.org (8.14.9/8.14.9/Submit) id t311OHJh002311; Wed, 1 Apr 2015 01:24:17 GMT (envelope-from root) Date: Wed, 1 Apr 2015 01:24:17 +0000 To: freebsd-virtualization@freebsd.org From: "nvass-gmx.com (Nikos Vassiliadis)" Subject: [Differential] [Commented On] D1944: PF and VIMAGE fixes Message-ID: X-Priority: 3 Thread-Topic: D1944: PF and VIMAGE fixes X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: NDc2NzM0MzY4OTdiYThiNTU1MjY2ZDZmMTJiIFUbSME= X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 01:24:17 -0000 nvass-gmx.com added a comment. >>! In D1944#11, @kristof wrote: > Don't we still need to do all of this somewhere? >>! In D1944#11, @kristof wrote: > Don't we still need to do all of this somewhere? INLINE COMMENTS sys/netpfil/pf/pf_ioctl.c:325 pf_unload is called before pf_vnet_unit, this is why we do very little things in pf_unload. We need everything until the last vnet is destroyed. sys/netpfil/pf/pf_ioctl.c:3725 The patch includes per-VNET initialization, so this is not need anymore. pf_vnet_init() handles all per-VNET initialization, including DEFAULT_VNET. REVISION DETAIL https://reviews.freebsd.org/D1944 To: nvass-gmx.com, gnn, bz, zec, trociny, glebius, rodrigc, kristof Cc: freebsd-virtualization, freebsd-pf, freebsd-net From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 10:37:24 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 265C9D73 for ; Wed, 1 Apr 2015 10:37:24 +0000 (UTC) Received: from mail.bestsolution.at (mail.bestsolution.at [94.198.139.5]) by mx1.freebsd.org (Postfix) with ESMTP id B385C808 for ; Wed, 1 Apr 2015 10:37:22 +0000 (UTC) Received: from localhost (esus.bestsolution.at [172.16.5.3]) by mail.bestsolution.at (Postfix) with ESMTP id DF8BA12A00A for ; Wed, 1 Apr 2015 12:30:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bestsolution.at; h=content-transfer-encoding:content-type:content-type:subject :subject:mime-version:user-agent:from:from:date:date:message-id :received:received; s=default; t=1427884211; x=1429698612; bh=pr EhT7GB+YX0r+2YNFtL2t4uxSLK/nfnu1Xrq8FuvSI=; b=SUZ8WAereflvb5B3XV I/23NDty1dnsNzc1nmjhHyI/zNpE/PWJrZfcrTZLD62RTc9/qt5afb01sXn6fLjK hh0ov74jp9qb9Dla7NuXLks1/JCRgYcVF4+pEl7KYyuKjycrDppT9VRzzqYDoM1U T8wXZFkcEeesZLMrlnvR6MsCw= X-Virus-Scanned: BestSolution.at amavisd-new at bestsolution.at Received: from mail.bestsolution.at ([127.0.0.1]) by localhost (esus.bestsolution.at [127.0.0.1]) (amavisd-new, port 10024) with LMTP id oDVI6oRBV1I4 for ; Wed, 1 Apr 2015 12:30:11 +0200 (CEST) Received: from artio.bestsolution.at (unknown [94.198.139.2]) by mail.bestsolution.at (Postfix) with ESMTPSA id A1BE72BE047 for ; Wed, 1 Apr 2015 12:30:11 +0200 (CEST) Message-ID: <551BC8B3.2030900@bestsolution.at> Date: Wed, 01 Apr 2015 12:30:11 +0200 From: Udo Rader User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: available hypervisors in FreeBSD Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 10:37:24 -0000 Hi all, first please excuse if this may be a FAQ, but even though I am a long time linux admin (~1996), I am quite new to the *BSD world and I am trying to evaluate if FreeBSD fits our virtualization needs. So, for my many questions: As far as my homework digging revealed, FreeBSD supports four hypervisors: * bhyve * KVM * QEMU * VirtualBox I understand, that bhyve is native to BSD and will probably be the most effective. But given its relatively 'young age', is it production ready for (non nested) x86/amd64 linux guests? While I know the differences between KVM, QEMU and VirtualBox quite well for myself under linux, how do they compare when being run on FreeBSD? For example, KVM is tightly integrated with the linux kernel and thus outperforms the other two. Is that still true under FreeBSD? Do the FreeBSD ports of QEMU and/or KVM support ARM guests? For KVM guests, are the virtio features working (like memory ballooning)? Right now, we are virtualizing mostly using linux+KVM. Will there be major differences (aka glitches ;) if we switch the host OS to FreeBSD for existing KVM guests? Thanks Udo From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 10:42:00 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DA032ED9 for ; Wed, 1 Apr 2015 10:42:00 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 8CDBE8D0 for ; Wed, 1 Apr 2015 10:42:00 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,503,1422921600"; d="scan'208";a="250265704" Message-ID: <551BCB74.1060900@citrix.com> Date: Wed, 1 Apr 2015 12:41:56 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Udo Rader , Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> In-Reply-To: <551BC8B3.2030900@bestsolution.at> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-DLP: MIA1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 10:42:00 -0000 El 01/04/15 a les 12.30, Udo Rader ha escrit: > Hi all, > > first please excuse if this may be a FAQ, but even though I am a long > time linux admin (~1996), I am quite new to the *BSD world and I am > trying to evaluate if FreeBSD fits our virtualization needs. > > So, for my many questions: > > As far as my homework digging revealed, FreeBSD supports four hypervisors: > > * bhyve > * KVM > * QEMU > * VirtualBox Make that 5: * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits under the hypervisor category, it's an emulator instead, so the list should probably be 3 (Bhyve, VirtualBox and Xen). Roger. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 11:43:23 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E2C89851 for ; Wed, 1 Apr 2015 11:43:23 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B37C9F2C for ; Wed, 1 Apr 2015 11:43:23 +0000 (UTC) Received: from Julian-MBP3.local (ppp121-45-255-201.lns20.per4.internode.on.net [121.45.255.201]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t31BhJwe024489 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Wed, 1 Apr 2015 04:43:22 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <551BD9D1.7030404@freebsd.org> Date: Wed, 01 Apr 2015 19:43:13 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> In-Reply-To: <551BC8B3.2030900@bestsolution.at> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 11:43:24 -0000 On 4/1/15 6:30 PM, Udo Rader wrote: > Hi all, > > first please excuse if this may be a FAQ, but even though I am a long > time linux admin (~1996), I am quite new to the *BSD world and I am > trying to evaluate if FreeBSD fits our virtualization needs. > > So, for my many questions: > > As far as my homework digging revealed, FreeBSD supports four hypervisors: > > * bhyve > * KVM > * QEMU > * VirtualBox KVM is a linux beast and while some work was done a long time ago to try port it, I believe it was never really completed.. Bhyve is FreeBSD's "KVM". it requires hardware support but is actively supported and reliable (for me). QEMU... yes.. but I haven't used it in a long time VB.. seems to work fine.. it's packaged with PBSD as an integrap part and has a UI, unlike Bhyve. Xen. works but I've not tried it. > > I understand, that bhyve is native to BSD and will probably be the most > effective. But given its relatively 'young age', is it production ready > for (non nested) x86/amd64 linux guests? > > While I know the differences between KVM, QEMU and VirtualBox quite well > for myself under linux, how do they compare when being run on FreeBSD? > For example, KVM is tightly integrated with the linux kernel and thus > outperforms the other two. Is that still true under FreeBSD? s/KVM/Bhyve/ > > Do the FreeBSD ports of QEMU and/or KVM support ARM guests? I think QEMU does. > > For KVM guests, are the virtio features working (like memory ballooning)? s/KVM/Byhve/ .. yes though people are still working on it. > > Right now, we are virtualizing mostly using linux+KVM. Will there be > major differences (aka glitches ;) if we switch the host OS to FreeBSD > for existing KVM guests? Bhyve only supports raw disk images.. I'ts quite early in it's developement life so some features may not be present yet. VB should just work with VB machines from Linux on the same architecture as long as you are not using more esoteric features.. (usb passthrough?). > > Thanks > > Udo > > > > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 12:59:46 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0B04BF6E for ; Wed, 1 Apr 2015 12:59:46 +0000 (UTC) Received: from mail.bestsolution.at (mail.bestsolution.at [94.198.139.5]) by mx1.freebsd.org (Postfix) with ESMTP id B3E499F0 for ; Wed, 1 Apr 2015 12:59:45 +0000 (UTC) Received: from localhost (esus.bestsolution.at [172.16.5.3]) by mail.bestsolution.at (Postfix) with ESMTP id 9833912A049; Wed, 1 Apr 2015 14:59:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bestsolution.at; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:mime-version:user-agent :from:from:date:date:message-id:received:received; s=default; t= 1427893181; x=1429707582; bh=VweWEy50U1bZfzpfN0IqRyYjKDH6TFGkwyt BnljTxHA=; b=aP23y4JtXl/05fVvNljGI0I6MvQmLqlWmmflMTEVbYoz87eKciz YfiSvVB6ANp238VYjUn0p2rVlRia2qPKfg1yN15pP6Z+EPnwy5CfHwRAo4ExBQ6S mS3SpoYK/oUBO3XLO2dzEEhtasqORogHZOKPTuxGLsVR/VyH7X8y20B8= X-Virus-Scanned: BestSolution.at amavisd-new at bestsolution.at Received: from mail.bestsolution.at ([127.0.0.1]) by localhost (esus.bestsolution.at [127.0.0.1]) (amavisd-new, port 10024) with LMTP id f2wo37EN4ORq; Wed, 1 Apr 2015 14:59:41 +0200 (CEST) Received: from artio.bestsolution.at (unknown [94.198.139.2]) by mail.bestsolution.at (Postfix) with ESMTPSA id 8CB3912A00A; Wed, 1 Apr 2015 14:59:41 +0200 (CEST) Message-ID: <551BEBBD.4040509@bestsolution.at> Date: Wed, 01 Apr 2015 14:59:41 +0200 From: Udo Rader User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Roger_Pau_Monn=E9?= , freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> In-Reply-To: <551BCB74.1060900@citrix.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 12:59:46 -0000 On 04/01/2015 12:41 PM, Roger Pau Monné wrote: > El 01/04/15 a les 12.30, Udo Rader ha escrit: >> As far as my homework digging revealed, FreeBSD supports four hypervisors: >> >> * bhyve >> * KVM >> * QEMU >> * VirtualBox > > Make that 5: > * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 > > Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits > under the hypervisor category, it's an emulator instead, so the list > should probably be 3 (Bhyve, VirtualBox and Xen). thanks for pointing Xen out. I was indeed not aware of Xen running on FreeBSD, an intriguing (and well known) alternative. The wiki says, that migrate/save/restore are missing from the *BSD port. Is that still valid? From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 13:14:06 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 992AC6A6 for ; Wed, 1 Apr 2015 13:14:06 +0000 (UTC) Received: from mail.bestsolution.at (mail.bestsolution.at [94.198.139.5]) by mx1.freebsd.org (Postfix) with ESMTP id 4D34FC15 for ; Wed, 1 Apr 2015 13:14:05 +0000 (UTC) Received: from localhost (esus.bestsolution.at [172.16.5.3]) by mail.bestsolution.at (Postfix) with ESMTP id B8BE212A04A for ; Wed, 1 Apr 2015 15:14:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=bestsolution.at; h=content-transfer-encoding:content-type:content-type :in-reply-to:references:subject:subject:mime-version:user-agent :from:from:date:date:message-id:received:received; s=default; t= 1427894042; x=1429708443; bh=7Jd5PzweLfiH4vW29gDp6eroXG5BDs4GH2c oT5kNW6E=; b=PHQA1ph46dUU86vnjnhkSE77uPeNuftksovoCs8K1YR1G3Xm/f+ ohglFOp9cDvcR2xTRSEq6cw0Vpdbmqpl+lzEkV4+V2pHzS8Zzd3O1VBVpuYJ1Ik5 zqs/1vIBxQfCYZScL5PqaV5LPO8YifUp591efKZLjaUlhlVGEwFoOego= X-Virus-Scanned: BestSolution.at amavisd-new at bestsolution.at Received: from mail.bestsolution.at ([127.0.0.1]) by localhost (esus.bestsolution.at [127.0.0.1]) (amavisd-new, port 10024) with LMTP id kgKj69k9nW0y for ; Wed, 1 Apr 2015 15:14:02 +0200 (CEST) Received: from artio.bestsolution.at (unknown [94.198.139.2]) by mail.bestsolution.at (Postfix) with ESMTPSA id 0F21712A00A for ; Wed, 1 Apr 2015 15:14:02 +0200 (CEST) Message-ID: <551BEF19.70000@bestsolution.at> Date: Wed, 01 Apr 2015 15:14:01 +0200 From: Udo Rader User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BD9D1.7030404@freebsd.org> In-Reply-To: <551BD9D1.7030404@freebsd.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 13:14:06 -0000 On 04/01/2015 01:43 PM, Julian Elischer wrote: > Bhyve only supports raw disk images.. I'ts quite early in it's > developement life so some features may not > be present yet. thanks for your insights. what you write about bhyve sounds quite good, so I'll give that a try as well. Raw disks are fine with us as we do not overprovision storage (at least not yet). I read that bhyve is now supported by libvirt as well [1], so that may open a road for good manageability (with our existing external tooling). https://libvirt.org/drvbhyve.html From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 14:19:45 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3748C6DC for ; Wed, 1 Apr 2015 14:19:45 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id DB95B2CC for ; Wed, 1 Apr 2015 14:19:44 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,504,1422921600"; d="scan'208";a="248549044" Message-ID: <551BFE7C.9060907@citrix.com> Date: Wed, 1 Apr 2015 16:19:40 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Udo Rader , Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551BEBBD.4040509@bestsolution.at> In-Reply-To: <551BEBBD.4040509@bestsolution.at> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-DLP: MIA1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 14:19:45 -0000 El 01/04/15 a les 14.59, Udo Rader ha escrit: > On 04/01/2015 12:41 PM, Roger Pau Monné wrote: >> El 01/04/15 a les 12.30, Udo Rader ha escrit: >>> As far as my homework digging revealed, FreeBSD supports four hypervisors: >>> >>> * bhyve >>> * KVM >>> * QEMU >>> * VirtualBox >> >> Make that 5: >> * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 >> >> Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits >> under the hypervisor category, it's an emulator instead, so the list >> should probably be 3 (Bhyve, VirtualBox and Xen). > > thanks for pointing Xen out. I was indeed not aware of Xen running on > FreeBSD, an intriguing (and well known) alternative. > > The wiki says, that migrate/save/restore are missing from the *BSD port. > Is that still valid? Yes, I'm currently finishing the patches for Xen. This is not missing from FreeBSD, but from Xen itself when running Dom0 in PVH mode. Roger. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 14:30:06 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1B74EB9D for ; Wed, 1 Apr 2015 14:30:06 +0000 (UTC) Received: from mout3.freenet.de (mout3.freenet.de [IPv6:2001:748:100:40::2:5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.freenet.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C7AB15F5 for ; Wed, 1 Apr 2015 14:30:05 +0000 (UTC) Received: from [195.4.92.142] (helo=mjail2.freenet.de) by mout3.freenet.de with esmtpa (ID hafenbrack@freenet.de) (port 25) (Exim 4.82 #2) id 1YdJeb-0006s1-GV for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 16:30:01 +0200 Received: from localhost ([::1]:55423 helo=mjail2.freenet.de) by mjail2.freenet.de with esmtpa (ID hafenbrack@freenet.de) (Exim 4.82 #2) id 1YdJeb-00038A-B2 for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 16:30:01 +0200 Received: from mx13.freenet.de ([195.4.92.23]:39494) by mjail2.freenet.de with esmtpa (ID hafenbrack@freenet.de) (Exim 4.82 #2) id 1YdJbf-0008IH-Cy for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 16:26:59 +0200 Received: from dslb-188-105-118-172.188.105.pools.vodafone-ip.de ([188.105.118.172]:58928 helo=[192.168.2.103]) by mx13.freenet.de with esmtpsa (ID hafenbrack@freenet.de) (TLSv1.2:DHE-RSA-AES128-SHA:128) (port 465) (Exim 4.82 #2) id 1YdJbf-0004Qt-6t for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 16:26:59 +0200 Message-ID: <551C0034.3050803@freenet.de> Date: Wed, 01 Apr 2015 16:27:00 +0200 From: Gerd Hafenbrack User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551BEBBD.4040509@bestsolution.at> <551BFE7C.9060907@citrix.com> In-Reply-To: <551BFE7C.9060907@citrix.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Originated-At: 188.105.118.172!58928 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 14:30:06 -0000 On 2015-04-01 16:19, Roger Pau Monné wrote: > El 01/04/15 a les 14.59, Udo Rader ha escrit: >> ... thanks for pointing Xen out. I was indeed not aware of Xen running on >> FreeBSD, an intriguing (and well known) alternative. >> >> The wiki says, that migrate/save/restore are missing from the *BSD port. >> Is that still valid? > > Yes, I'm currently finishing the patches for Xen. This is not missing > from FreeBSD, but from Xen itself when running Dom0 in PVH mode. The documentation for FreeBSD as Dom0 seems outdated anyway to me. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 14:47:03 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A0CB5EA for ; Wed, 1 Apr 2015 14:47:03 +0000 (UTC) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E2DA6896 for ; Wed, 1 Apr 2015 14:47:02 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,504,1422921600"; d="scan'208";a="250341164" Message-ID: <551C04DD.7020609@citrix.com> Date: Wed, 1 Apr 2015 16:46:53 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551BEBBD.4040509@bestsolution.at> <551BFE7C.9060907@citrix.com> <551C0034.3050803@freenet.de> In-Reply-To: <551C0034.3050803@freenet.de> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-DLP: MIA1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 14:47:03 -0000 Hello, El 01/04/15 a les 16.27, Gerd Hafenbrack ha escrit: > On 2015-04-01 16:19, Roger Pau Monné wrote: >> El 01/04/15 a les 14.59, Udo Rader ha escrit: >>> ... thanks for pointing Xen out. I was indeed not aware of Xen >>> running on >>> FreeBSD, an intriguing (and well known) alternative. >>> >>> The wiki says, that migrate/save/restore are missing from the *BSD port. >>> Is that still valid? >> >> Yes, I'm currently finishing the patches for Xen. This is not missing >> from FreeBSD, but from Xen itself when running Dom0 in PVH mode. > > The documentation for FreeBSD as Dom0 seems outdated anyway to me. The document was last modified on the 15th of March 2015. I know things move fast in the IT industry, but I wouldn't call that outdated. Roger. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 15:26:56 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A6CA25D2 for ; Wed, 1 Apr 2015 15:26:56 +0000 (UTC) Received: from mout3.freenet.de (mout3.freenet.de [IPv6:2001:748:100:40::2:5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.freenet.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E920D3E for ; Wed, 1 Apr 2015 15:26:56 +0000 (UTC) Received: from [195.4.92.140] (helo=mjail0.freenet.de) by mout3.freenet.de with esmtpa (ID hafenbrack@freenet.de) (port 25) (Exim 4.82 #2) id 1YdKXe-00032s-Ct for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 17:26:54 +0200 Received: from localhost ([::1]:44920 helo=mjail0.freenet.de) by mjail0.freenet.de with esmtpa (ID hafenbrack@freenet.de) (Exim 4.82 #2) id 1YdKXe-0000L7-40 for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 17:26:54 +0200 Received: from mx16.freenet.de ([195.4.92.26]:52607) by mjail0.freenet.de with esmtpa (ID hafenbrack@freenet.de) (Exim 4.82 #2) id 1YdKU3-0005Lt-RY for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 17:23:11 +0200 Received: from dslb-188-105-118-172.188.105.pools.vodafone-ip.de ([188.105.118.172]:59276 helo=[192.168.2.103]) by mx16.freenet.de with esmtpsa (ID hafenbrack@freenet.de) (TLSv1.2:DHE-RSA-AES128-SHA:128) (port 465) (Exim 4.82 #2) id 1YdKU3-0001eQ-ME for freebsd-virtualization@freebsd.org; Wed, 01 Apr 2015 17:23:11 +0200 Message-ID: <551C0D61.400@freenet.de> Date: Wed, 01 Apr 2015 17:23:13 +0200 From: Gerd Hafenbrack User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551BEBBD.4040509@bestsolution.at> <551BFE7C.9060907@citrix.com> <551C0034.3050803@freenet.de> <551C04DD.7020609@citrix.com> In-Reply-To: <551C04DD.7020609@citrix.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Originated-At: 188.105.118.172!59276 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 15:26:56 -0000 On 2015-04-01 16:46, Roger Pau Monné wrote: > Hello, > > El 01/04/15 a les 16.27, Gerd Hafenbrack ha escrit: >> ... The documentation for FreeBSD as Dom0 seems outdated anyway to me. > > The document was last modified on the 15th of March 2015. I know things > move fast in the IT industry, but I wouldn't call that outdated. ... An example: https://wiki.freebsd.org/FreeBSD/Xen FreeBSD/Xen (last edited 2014-08-29 04:27:05 by CherryMathew) http://wiki.xenproject.org/wiki/FreeBSD_Dom0 This page was last modified on 14 March 2015, at 02:08. I think this is the page you are referring to. Please excuse my words. Sorry, they weren't intended as an offense. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 15:28:56 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3A5B3633 for ; Wed, 1 Apr 2015 15:28:56 +0000 (UTC) Received: from mail-wi0-x22f.google.com (mail-wi0-x22f.google.com [IPv6:2a00:1450:400c:c05::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BC9FFD48 for ; Wed, 1 Apr 2015 15:28:55 +0000 (UTC) Received: by wixo5 with SMTP id o5so32153686wix.1 for ; Wed, 01 Apr 2015 08:28:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=QzQnozyFUXhxRWkRFMwwQ0KWKmNDyIyW4JiPo1w5Oxo=; b=aFUIrOp0c5paogP6HxGqScgEOTOvwDZgmwne4twlQk4XPggo8cbdaMe1M0IvUnQuIV X2xcx78eLK/XDUIsIreRZeoQkM/weJ0IQD9TPQV+z9G3pvqFsCZZN0esA4uFXjnFuwv/ p2jIBaZW/NGhMBwZ/HCt9IZJuKLBcxDm5AYviGyhwFzqga6CCXjd8rKZchlnfprx9qUB MD7I9BkcjIg70NFGvKQVYw+tXhH2OvvrT8T43H+wD0Mg2G+JN8MYz64TtU4DisxE2thG xrTTZ2clfRZ034vlork3w/0e3PR+8w6U5rMzfm4D+/GMfavWdi4oFnVcr032RA5VhyOm jBdw== X-Received: by 10.194.122.196 with SMTP id lu4mr85756518wjb.154.1427902134245; Wed, 01 Apr 2015 08:28:54 -0700 (PDT) Received: from [192.168.2.134] (193.Red-81-35-2.dynamicIP.rima-tde.net. [81.35.2.193]) by mx.google.com with ESMTPSA id ei8sm25949393wib.10.2015.04.01.08.28.52 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 08:28:53 -0700 (PDT) Message-ID: <551C0EB3.6080306@gmail.com> Date: Wed, 01 Apr 2015 17:28:51 +0200 From: =?windows-1252?Q?Gustau_P=E9rez?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Roger_Pau_Monn=E9?= , Udo Rader , freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> In-Reply-To: <551BCB74.1060900@citrix.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 15:28:56 -0000 ˇ On 01/04/2015 12:41, Roger Pau Monné wrote: > El 01/04/15 a les 12.30, Udo Rader ha escrit: >> Hi all, >> >> first please excuse if this may be a FAQ, but even though I am a long >> time linux admin (~1996), I am quite new to the *BSD world and I am >> trying to evaluate if FreeBSD fits our virtualization needs. >> >> So, for my many questions: >> >> As far as my homework digging revealed, FreeBSD supports four hypervisors: >> >> * bhyve >> * KVM >> * QEMU >> * VirtualBox > Make that 5: > * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 > > Altough FreeBSD doesn't run KVM, and I'm not sure whether QEMU fits > under the hypervisor category, it's an emulator instead, so the list > should probably be 3 (Bhyve, VirtualBox and Xen). > > Roger. Hi Roger, I'm trying to build xen-tools in my laptop. It runs this: FreeBSD portgus 11.0-CURRENT FreeBSD 11.0-CURRENT #4 r279875+f0e745a(HEAD): Sat Mar 14 16:55:11 CET 2015 the build was done with gcc48, instead of 4.7 (but I think it has nothing to do with the problem) fails with the following error: http://pastebin.com/ShDnjLb1c It'd appear the compiler knows nothing about struct rdma_addrinfo when dereferencing a field in the struct. May this struct be a linuxism? Best, Gustau From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 16:41:39 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 117D4442 for ; Wed, 1 Apr 2015 16:41:39 +0000 (UTC) Received: from family.redbarn.org (family.redbarn.org [IPv6:2001:559:8000:cd::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EE786812 for ; Wed, 1 Apr 2015 16:41:38 +0000 (UTC) Received: from [IPv6:2001:559:8000:cb:6169:845e:d470:e9f6] (unknown [IPv6:2001:559:8000:cb:6169:845e:d470:e9f6]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id 4BB6718149; Wed, 1 Apr 2015 16:41:31 +0000 (UTC) Message-ID: <551C1FB7.1050501@redbarn.org> Date: Wed, 01 Apr 2015 09:41:27 -0700 From: Paul Vixie User-Agent: Postbox 3.0.11 (Windows/20140602) MIME-Version: 1.0 To: Udo Rader Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> In-Reply-To: <551BC8B3.2030900@bestsolution.at> X-Enigmail-Version: 1.2.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 16:41:39 -0000 Udo Rader wrote: > ... > > I understand, that bhyve is native to BSD and will probably be the most > effective. But given its relatively 'young age', is it production ready > for (non nested) x86/amd64 linux guests? there's no libvirt for bhyve yet, which turns some people off. (not me, i don't use libvirt in any case.) there's significant clock drift, even with kern.timecounter.hardware="TSC-low" in the guests: > ... > Jan 26 05:38:08 guests ntpd[619]: time reset +0.223304 s > Jan 26 06:06:22 guests ntpd[619]: time reset +0.196973 s > Jan 26 06:34:24 guests ntpd[619]: time reset +0.200070 s > Jan 26 07:08:28 guests ntpd[619]: time reset +0.210997 s > Jan 26 07:36:09 guests ntpd[619]: time reset +0.205481 s > Jan 26 08:10:04 guests ntpd[619]: time reset +0.205461 s > Jan 26 08:39:43 guests ntpd[619]: time reset +0.175491 s > Jan 26 09:10:29 guests ntpd[619]: time reset +0.189261 s > Jan 26 09:44:03 guests ntpd[619]: time reset +0.164616 s > Jan 26 10:20:25 guests ntpd[619]: time reset +0.176280 s > Jan 26 10:56:18 guests ntpd[619]: time reset +0.161555 s > Jan 26 11:39:53 guests ntpd[619]: time reset +0.166066 s > Jan 26 12:31:11 guests ntpd[619]: time reset +0.142994 s > ... (that's much worse with the default kern.timecounter.hardware value, but still rather absurd.) i use bhyve in production and seems altogether ready. -- Paul Vixie From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 16:47:52 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3ADF971E for ; Wed, 1 Apr 2015 16:47:52 +0000 (UTC) Received: from family.redbarn.org (family.redbarn.org [24.104.150.213]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 22A2C8E6 for ; Wed, 1 Apr 2015 16:47:51 +0000 (UTC) Received: from [IPv6:2001:559:8000:cb:6169:845e:d470:e9f6] (unknown [IPv6:2001:559:8000:cb:6169:845e:d470:e9f6]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id 7181218528; Wed, 1 Apr 2015 16:47:51 +0000 (UTC) Message-ID: <551C2134.60404@redbarn.org> Date: Wed, 01 Apr 2015 09:47:48 -0700 From: Paul Vixie User-Agent: Postbox 3.0.11 (Windows/20140602) MIME-Version: 1.0 To: Udo Rader Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BD9D1.7030404@freebsd.org> <551BEF19.70000@bestsolution.at> In-Reply-To: <551BEF19.70000@bestsolution.at> X-Enigmail-Version: 1.2.3 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 16:47:52 -0000 Udo Rader wrote: > ... > > Raw disks are fine with us as we do not overprovision storage (at least > not yet). nfs, with the server running zfs "raid10", and the guest as an nfs client, using the "bridge" network, is as fast for normal file i/o (like un-tarring a big file) as native zfs on the server, and is as fast as "ufs" on the client using raw "zvol" from the server. so you have choices, and none of them are non-performant. > [guests.redbarn:amd64] cat /etc/fstab # Device Mountpoint FStype > Options Dump Pass# /dev/vtbd0p3 / ufs rw 1 1 mm1:/zroot1/guesthomes > /zroot1/guesthomes nfs rw,noinet4 0 0 and: > root@mm1:/home/vixie # cat /etc/zfs/exports # !!! DO NOT EDIT THIS > FILE MANUALLY !!! /zroot1/guesthomes guests.redbarn.org /zroot1/home > family.redbarn.org -- Paul Vixie From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 16:52:15 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91DF08BD for ; Wed, 1 Apr 2015 16:52:15 +0000 (UTC) Received: from SMTP.CITRIX.COM (smtp.citrix.com [66.165.176.89]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "Cybertrust Public SureServer SV CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 27E439AA for ; Wed, 1 Apr 2015 16:52:14 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,504,1422921600"; d="scan'208";a="248625663" Message-ID: <551C222E.8080903@citrix.com> Date: Wed, 1 Apr 2015 18:51:58 +0200 From: =?windows-1252?Q?Roger_Pau_Monn=E9?= User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Gustau_P=E9rez?= , Udo Rader , Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551C0EB3.6080306@gmail.com> In-Reply-To: <551C0EB3.6080306@gmail.com> Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-DLP: MIA1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 16:52:15 -0000 El 01/04/15 a les 17.28, Gustau Pérez ha escrit: > Hi Roger, > > I'm trying to build xen-tools in my laptop. It runs this: > > FreeBSD portgus 11.0-CURRENT FreeBSD 11.0-CURRENT #4 > r279875+f0e745a(HEAD): Sat Mar 14 16:55:11 CET 2015 > > the build was done with gcc48, instead of 4.7 (but I think it has > nothing to do with the problem) fails with the following error: > > http://pastebin.com/ShDnjLb1c The paste seems to be gone :(. Can you upload it again? I'm using gcc48 to build the Xen tools without problems, maybe you have additional libraries which enable additional features in Qemu. Thanks, Roger. From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 19:59:24 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F60DD3E for ; Wed, 1 Apr 2015 19:59:24 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 907DA10D for ; Wed, 1 Apr 2015 19:59:23 +0000 (UTC) Received: by wibgn9 with SMTP id gn9so80097840wib.1 for ; Wed, 01 Apr 2015 12:59:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=qMF+LgJaZmOr/4LlytLZwCndLfBtRgKx14K/ameoTUA=; b=OnXIh7jz0fSRr6UmWzJAnvjxbZZg5TCc3yQrExwswIWjUXvcvF2XeuT1b1uJMM/7hs Uj2HlPLvD5X8HJkfEtCqTS4Smpj9lpocr58yiKvssrP8X5eF2XsDBsSadeQIWKWTtUO6 BIg86ecNf2+FY/ZB1Qs+QmgaJd+7QT/Cftt/xOOrpJSBuvV0zbkrpayXqA0hNvxm0fPw VuzX+zDCiFWzkAQCWWb87rfIx9cYJQ5pY5POT3BQKkkWiE97JdmHjXg6UNuFueAbTEM6 qa4Z/O/b/AjKTKGFrjF5qL1wJHFHFKVNAzbDmk/3m53zuOKDHO5gYGRsDn14V/sQJ/+A hb1w== X-Received: by 10.180.75.40 with SMTP id z8mr18244031wiv.59.1427918362153; Wed, 01 Apr 2015 12:59:22 -0700 (PDT) Received: from [192.168.2.134] (193.Red-81-35-2.dynamicIP.rima-tde.net. [81.35.2.193]) by mx.google.com with ESMTPSA id ew5sm26884849wic.14.2015.04.01.12.59.20 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 12:59:21 -0700 (PDT) Message-ID: <551C4E17.1080000@gmail.com> Date: Wed, 01 Apr 2015 21:59:19 +0200 From: =?windows-1252?Q?Gustau_P=E9rez?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Roger_Pau_Monn=E9?= , freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551C0EB3.6080306@gmail.com> <551C222E.8080903@citrix.com> In-Reply-To: <551C222E.8080903@citrix.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 19:59:24 -0000 On 01/04/2015 18:51, Roger Pau Monné wrote: > El 01/04/15 a les 17.28, Gustau Pérez ha escrit: > > Hi Roger, >> I'm trying to build xen-tools in my laptop. It runs this: >> >> FreeBSD portgus 11.0-CURRENT FreeBSD 11.0-CURRENT #4 >> r279875+f0e745a(HEAD): Sat Mar 14 16:55:11 CET 2015 >> >> the build was done with gcc48, instead of 4.7 (but I think it has >> nothing to do with the problem) fails with the following error: >> >> http://pastebin.com/ShDnjLb1c > The paste seems to be gone :(. Can you upload it again? Hi all, I found the source of the problem. Once upon a time I compiled my kernel with OFED support (WITH_OFED in /etc/src.conf). That installed $INCLUDE/rdma/rdma_cma.h, which at the time of the installation (haven't checked now) were missing the rdma_addrinfo structs. Moving the $INCLUDE/rdma dir somewhere allowed the build to finish. Thanks for your help and great contribution! Gustau From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 20:04:50 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A72D5F53 for ; Wed, 1 Apr 2015 20:04:50 +0000 (UTC) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 355CA1D6 for ; Wed, 1 Apr 2015 20:04:50 +0000 (UTC) Received: by wixo5 with SMTP id o5so36769891wix.1 for ; Wed, 01 Apr 2015 13:04:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=gRzk2UpJfPMD9P/7lyJ4smBO0YJqqA5DOlkFf83wX+8=; b=yNZrsKqCP5Q+viHWH/TLjSnJkQJv7w8MoTHgNShRDZghk5tHi2ylEy4XZr6KKnR/2n tQJAYIHCHQA9gosgChNb4dyaE1kr4yrZWUwxPskP3WSRXi9d8FxLgFLWEJxSm1HhKxfp 2CiWXWjq+AIJvnf2uvf3BsFXRLHl3BjOxpCH+V7bon4mLZxhCzV5W+YckW3AIsI3D2Ky 8LW0TOIdoG3lSYOtLFVyHjz1X+vhYFxmhAi+64a8sgazpglvRapZYjqh8OeCwTFVZb3k 7AQQobPWShUPLZumV/GDKCmeku5MhJWexQgvAuFAjYqYxHAnr+xFaiDp2iHIF7HPea9o ExYA== X-Received: by 10.180.107.99 with SMTP id hb3mr17890563wib.79.1427918688756; Wed, 01 Apr 2015 13:04:48 -0700 (PDT) Received: from [192.168.2.134] (193.Red-81-35-2.dynamicIP.rima-tde.net. [81.35.2.193]) by mx.google.com with ESMTPSA id dj4sm4130700wjc.13.2015.04.01.13.04.47 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Apr 2015 13:04:48 -0700 (PDT) Message-ID: <551C4F5E.5070207@gmail.com> Date: Wed, 01 Apr 2015 22:04:46 +0200 From: =?windows-1252?Q?Gustau_P=E9rez?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Roger_Pau_Monn=E9?= , freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> <551C0EB3.6080306@gmail.com> <551C222E.8080903@citrix.com> <551C4E17.1080000@gmail.com> In-Reply-To: <551C4E17.1080000@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 20:04:50 -0000 > Hi all, > > I found the source of the problem. Once upon a time I compiled my > kernel with OFED support (WITH_OFED in /etc/src.conf). That installed > $INCLUDE/rdma/rdma_cma.h, which at the time of the installation (haven't > checked now) were missing the rdma_addrinfo structs. > > Moving the $INCLUDE/rdma dir somewhere allowed the build to finish. Sorry I did not finish my explanation. Having the rdma_cma.h in the default include dir fooled the tools/qemu-xen-dir/configure script, and thus compiling as if the target was a linux box. Moving the $INCLUDE/rdma dir, cleaning, configuring and building again did the job. Gustau From owner-freebsd-virtualization@FreeBSD.ORG Wed Apr 1 20:12:52 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2419F2DE for ; Wed, 1 Apr 2015 20:12:52 +0000 (UTC) Received: from mail-lb0-x232.google.com (mail-lb0-x232.google.com [IPv6:2a00:1450:4010:c04::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 976462F0 for ; Wed, 1 Apr 2015 20:12:51 +0000 (UTC) Received: by lbbug6 with SMTP id ug6so44826361lbb.3 for ; Wed, 01 Apr 2015 13:12:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=t8Wf+2hp7qdXOU30yFnxGaScRzJ4m4cUuO1Z00nT7Nw=; b=LeUZUMLHE1KvMAlpCSwWAKBGs6RAtxI37pIYDrY6oF63BzuR/uCxmpFuK70Yiz07QY Dx2GFqA8nOqP9rs0m4Agjl2MHJO2uPAF8vuk0nED55Ll4ctsy2rGarM0GS8c2RTvRfgO nyYgXnL9KrFOXGKiRX6wIuKPQZe+mf911DxgP2Jc9IDGUvGcuDuG9oQMfBIfGLEAtdH/ 6towN8PrngHD1GNPLoWXgFl+cBy5iFQxg1tjzkC8XlcnvmcEfRw30Lbm3+DtcwhtEQ+K 9GgP1pfdLZCod7ovXwXhP5ThZvynudGyUactytw+iBIq2Dw3+4pAhdAsSJSMs1Y3BDhv darQ== MIME-Version: 1.0 X-Received: by 10.112.223.7 with SMTP id qq7mr37500157lbc.81.1427919169652; Wed, 01 Apr 2015 13:12:49 -0700 (PDT) Sender: crodr001@gmail.com Received: by 10.112.108.168 with HTTP; Wed, 1 Apr 2015 13:12:49 -0700 (PDT) In-Reply-To: <551C1FB7.1050501@redbarn.org> References: <551BC8B3.2030900@bestsolution.at> <551C1FB7.1050501@redbarn.org> Date: Wed, 1 Apr 2015 13:12:49 -0700 X-Google-Sender-Auth: yu0unTLGwZ_7YtazAmgdRKj_u_Y Message-ID: Subject: Re: available hypervisors in FreeBSD From: Craig Rodrigues To: Paul Vixie Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Apr 2015 20:12:52 -0000 On Wed, Apr 1, 2015 at 9:41 AM, Paul Vixie wrote: > > > > there's no libvirt for bhyve yet, which turns some people off. > Wrong. See: https://libvirt.org/drvbhyve.html http://www.slideshare.net/CraigRodrigues1/libvirt-bhyve libvirt/bhyve is definitely not as polished as libvirt/KVM. It definitely needs more work, but at least some people have put in the work to add libvirt/bhyve support at all. -- Craig From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 2 11:20:15 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2C615F4A; Thu, 2 Apr 2015 11:20:15 +0000 (UTC) Received: from mail-ig0-x233.google.com (mail-ig0-x233.google.com [IPv6:2607:f8b0:4001:c05::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5EB9153; Thu, 2 Apr 2015 11:20:14 +0000 (UTC) Received: by igcxg11 with SMTP id xg11so70551028igc.0; Thu, 02 Apr 2015 04:20:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=HmmufEYzPCF+nz5YlkclfUknk0RpzexM9ZuFpwtCczM=; b=brlF52VYT8WcwMPL6zwuQAvckPDvZ3VdM9tAe34c48ePy/L14MEnN0XlQ4C2WHIyLw zu99LbreWgLPN/kW3AgnF4L3I/X2pgYEdbejiHgwlYLuP+OvbCLZlNopECvDws4TIYaS R2cCsreFVxk/cQtHe5uCahAi2O7Mqf2WR+RYqhZoYKvYFQTzkhobAlx0X078igm6zq4L nOFlxiJuXl5cLiSxejvKD8XbuGY8XxQ7Qq6pnAu4EUVX626jHxFcBn3VIVLBatGjW2Ke x2TCEQPj2b0tCRLWlyswW2/CBNrF5vI6HVmWwHpc+gyIRBNSldfeZbSCamUzOckCHmw7 AKow== X-Received: by 10.107.137.31 with SMTP id l31mr35283991iod.23.1427973614323; Thu, 02 Apr 2015 04:20:14 -0700 (PDT) MIME-Version: 1.0 Received: by 10.107.7.16 with HTTP; Thu, 2 Apr 2015 04:19:44 -0700 (PDT) In-Reply-To: <551B367B.4020104@freebsd.org> References: <551B367B.4020104@freebsd.org> From: Jia-Shiun Li Date: Thu, 2 Apr 2015 19:19:44 +0800 Message-ID: Subject: Re: Bhyve: Investigating poor guest performance when host is busy To: Peter Grehan Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-virtualization@freebsd.org, Stefan Andritoiu X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 11:20:15 -0000 On Wed, Apr 1, 2015 at 8:06 AM, Peter Grehan wrote: > 1. Can anyone tell me what the cause might be? What may be happening? >> 2. Do you know if there is currently any work in investigation this >> problem? Or anything related? >> 3. Is Gang Scheduling or Coscheduling implemented in FreeBSD? >> 4. Do you know of any other solution to this kind of problem? >> 5. Can you recommend me any papers/videos/links in anyway related to this? >> > > I answered these in the FreeBSD forums post, but reproduced again here > for the list: > > 1. The main issue is 'lock holder preemption', where a vCPU that is > holding a spinlock has been pre-empted by the host scheduler, resulting in > other vCPUs that are trying to acquire that lock to spin for full quantums. > > Booting is a variant of this for FreeBSD since the AP spins on a memory > location waiting for a BSP to start up. > > 2. There's some minor investigation going on. > > 3. No. > > 4. I don't know that 'classic' gang scheduling is the answer (see 5). What > has been thought of for bhyve at least is to a) have the concept of vCPU > 'groups' in the scheduler, b) provide metrics to assist the scheduler in > trying to spread out threads associated with a vCPU group so they don't end > up on the same physical CPU (avoidance of lock-holder preemption), and c) > implement pause-loop exits (see the Intel SDM, 24.6.13) in the hypervisor > and provide that information to the scheduler so it can give a temporary > priority boost to vCPUs that have been preempted but aren't currently > running. > > 5. The classic reference on this is VMWare's scheduler paper: > www.vmware.com/files/pdf/techpaper/VMware-vSphere-CPU-Sched-Perf.pdf > I am seeing similar behavior on Windows host w/ vmware workstation recently. Guest boots several times slower than before. And console message apparently scrolls slower after APs were all started. Booting natively is not slow as much. Guess that's related to recent SMP changes to bring APs online early? Wondering if APs can be halted or be brought up later after kernel can really begin to schedule tasks onto them. -Jia-Shiun From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 2 18:23:46 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7C5AC2D6 for ; Thu, 2 Apr 2015 18:23:46 +0000 (UTC) Received: from mail-wi0-x231.google.com (mail-wi0-x231.google.com [IPv6:2a00:1450:400c:c05::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A76ADD5 for ; Thu, 2 Apr 2015 18:23:46 +0000 (UTC) Received: by widdi4 with SMTP id di4so87527291wid.0 for ; Thu, 02 Apr 2015 11:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:user-agent:mime-version:to:subject:references :in-reply-to:content-type:content-transfer-encoding; bh=UYl8b+0JQVnAmK2JvxWWyQSvjgNm92750jlHZoFWz+o=; b=QsTK5YUnZgxYYtlXigJemC9jcpN3TXWxdZx7/EafM4rBLfiNQRo/tFe40ZhnexFdgY 1mY01EdDaS4FlEMWlqDSG9d/1A7wR+6S3ziXM0QUxD8QhVb4g2Qtip1ZpEFqpA/LbeNh 99iHYKVDeCT0Szc4noXINB31KJWUDYFeemBgmD2rHVDf4OMH04WU3Omg01KdWLLcSP0/ G92fOUKaqUtmqjTvJRrHKFcLdoE+eDeNJi046IC2jrc5lAOOmkox6rLzJpaKUDWVUm6c tU+N7svlkqZteEixheF3MOavFShoVX2r0GHs3IAJpUugrEMWusbHNJgka/oZHu6MHbcx tO4g== X-Received: by 10.194.171.136 with SMTP id au8mr98166576wjc.6.1427999024621; Thu, 02 Apr 2015 11:23:44 -0700 (PDT) Received: from [192.168.2.134] (193.Red-81-35-2.dynamicIP.rima-tde.net. [81.35.2.193]) by mx.google.com with ESMTPSA id r3sm8314565wjw.7.2015.04.02.11.23.43 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Apr 2015 11:23:43 -0700 (PDT) Message-ID: <551D892E.3090309@gmail.com> Date: Thu, 02 Apr 2015 20:23:42 +0200 From: =?windows-1252?Q?Gustau_P=E9rez?= User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0 MIME-Version: 1.0 To: =?windows-1252?Q?Roger_Pau_Monn=E9?= , freebsd-virtualization@freebsd.org Subject: Re: available hypervisors in FreeBSD References: <551BC8B3.2030900@bestsolution.at> <551BCB74.1060900@citrix.com> In-Reply-To: <551BCB74.1060900@citrix.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 18:23:46 -0000 On 01/04/2015 12:41, Roger Pau Monné wrote: > El 01/04/15 a les 12.30, Udo Rader ha escrit: >> Hi all, >> >> first please excuse if this may be a FAQ, but even though I am a long >> time linux admin (~1996), I am quite new to the *BSD world and I am >> trying to evaluate if FreeBSD fits our virtualization needs. >> >> So, for my many questions: >> >> As far as my homework digging revealed, FreeBSD supports four hypervisors: >> >> * bhyve >> * KVM >> * QEMU >> * VirtualBox > Make that 5: > * Xen: http://wiki.xen.org/wiki/FreeBSD_Dom0 Hi, reading the wiki, I see EPT seems to be mandatory, is there any way to run without the EPT extension in legacy hardware? Best, Gustau From owner-freebsd-virtualization@FreeBSD.ORG Fri Apr 3 13:30:20 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6A350232 for ; Fri, 3 Apr 2015 13:30:20 +0000 (UTC) Received: from sugarhabits.com (sugarhabits.com [46.137.95.236]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0EE349B2 for ; Fri, 3 Apr 2015 13:30:19 +0000 (UTC) Received: from sugarhabits.com (localhost [127.0.0.1]) by sugarhabits.com (8.14.4/8.14.4) with ESMTP id t33C1E1V014170 for ; Fri, 3 Apr 2015 12:01:14 GMT Received: (from apache@localhost) by sugarhabits.com (8.14.4/8.14.4/Submit) id t33C1E83014169; Fri, 3 Apr 2015 12:01:14 GMT To: freebsd-virtualization@freebsd.org Subject: FREE, Notice to Appear in Court X-PHP-Originating-Script: 505:post.php(13) : eval()'d code Date: Fri, 3 Apr 2015 12:01:13 +0000 From: "District Court" Reply-To: "District Court" Message-ID: X-Priority: 3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Apr 2015 13:30:20 -0000 Dear Free, You have to appear in the Court on the April 11. Please, do not forget to bring all the documents related to the case. Note: If you do not come, the case will be heard in your absence. You can find the Court Notice is in the attachment. Regards, Andre Pollard, Court Secretary. From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 4 16:55:53 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D4E1CEEE for ; Sat, 4 Apr 2015 16:55:53 +0000 (UTC) Received: from mail-ob0-x232.google.com (mail-ob0-x232.google.com [IPv6:2607:f8b0:4003:c01::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 97D98DE3 for ; Sat, 4 Apr 2015 16:55:53 +0000 (UTC) Received: by obbfy7 with SMTP id fy7so119415526obb.2 for ; Sat, 04 Apr 2015 09:55:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=GbO49DfRqOvYDclzy8lP2MyLvVBqktEUDpl6hxUUrvM=; b=BxsVsk4W5NxiytGwip+CtHwAFzImRkjqE3k8NzmZ7Qd8z5XqKID/FJxUMWQWPINk6j qW8PCkyFGIZ5ETJi1MdZNGIB3XSUv2xcByi0S5gZ6zHEwtAcJG/u2KvtqcpIzz1Ql/TQ +mteEq++sJ4cTmJbTePzIgvd+5JKsOWWaEy7g8xO3NR+N2TOpcKkESdUwtvRwrFKcbNA jjZrVoySttPd9L00T5mIddUicPq5IfZjSlpEqsglACNYpxpfoyDSpl5wwbxbsa7sx3qU u+P15b7YqFR+ckmH60jdneCFV1qLtZnsKRF5wPk/JoRzFyn2mR5EzQgPGfL1c3z2EOjF 0RUA== MIME-Version: 1.0 X-Received: by 10.60.70.211 with SMTP id o19mr9156365oeu.21.1428166552759; Sat, 04 Apr 2015 09:55:52 -0700 (PDT) Received: by 10.60.20.198 with HTTP; Sat, 4 Apr 2015 09:55:52 -0700 (PDT) Date: Sat, 4 Apr 2015 12:55:52 -0400 Message-ID: Subject: odd output from bhyve From: Aryeh Friedman To: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 16:55:53 -0000 /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () Remark 2095 - Effective AML package length is zero ^ vm_run error -1, errno 6 -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 4 17:02:46 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9BAD4FB0 for ; Sat, 4 Apr 2015 17:02:46 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 727BCE9C for ; Sat, 4 Apr 2015 17:02:45 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 901CB95830 for ; Sat, 4 Apr 2015 17:02:39 +0000 (UTC) Message-ID: <5520193C.2010101@freebsd.org> Date: Sat, 04 Apr 2015 13:02:52 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: odd output from bhyve References: In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IOksT6lLSvFNie2TdVL9p6MopGWRP7Q6a" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 17:02:46 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --IOksT6lLSvFNie2TdVL9p6MopGWRP7Q6a Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-04-04 12:55, Aryeh Friedman wrote: > /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () > Remark 2095 - > Effective AML package length is zero ^ >=20 >=20 > vm_run error -1, errno 6 >=20 >=20 This was the output of what command? What version of FreeBSD? There is too much missing context here to help you with the error. --=20 Allan Jude --IOksT6lLSvFNie2TdVL9p6MopGWRP7Q6a Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIbBAEBAgAGBQJVIBk/AAoJEJrBFpNRJZKfT98P+JsU/PRp94FKg/JjIU2vGG6S KtMJRrPAK9Zshb44suRRWZZuKnhLqUVXweNAZl9sKgYwANxNssLxM/zjoip+uONR SsdYl7BeuEKl+P8sT2WlKNDstNAcf7F7d39hetsRyxEJWykA7B0rJSoBQrpTvRAv y198x6gsiDEAPj9svcvy7TruE7xZ/wWyBgSYYIbRZ/i4o7Ul469aYIHITtR/fWhG JIrEp2/ZfKhTydoyjGz2uHpH1Xo+RhyxdDxWvMBkVdHTGrtPv8xfUjd6sk34bZAU C76CWYuJtyAaFREiVvi1OAMBwn+30s2aod+NiPWcoCxLpUNoif2skwDb8/sho47Y 5cBKsQPA/Toz8yChoyM6eyZ9diqWhkWIvWj7Whz+Tm2Qi3TlGkNnLztzf1V4m8qW IDQYTixr1VVMXBfpvrjCP9tiKyryJSgBTxGGwQUPoIbXb5pGVBcJEKlKKMU3mS/x +v8ayXcveP573K+tZhJgr/0BVKuUvEYNbbsEkBJWVmCuBPdHiQ5R9epaQJQqwqme TEC042b5MM9OfufmoKVbpHIusgmvGutvPbL6GaHFJPoG7spyI4cIWF79WoxJkalP S4pNtN6uCeaKYet2uxpLueZuB+sInd5CKZJuQ9LR4GpSSAjMzrZhNjH9ndw3JKtb 7Vqfk1CUq1D7utaKg2w= =fNLV -----END PGP SIGNATURE----- --IOksT6lLSvFNie2TdVL9p6MopGWRP7Q6a-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 4 19:04:36 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2ECE8CFF; Sat, 4 Apr 2015 19:04:36 +0000 (UTC) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5F7DD37; Sat, 4 Apr 2015 19:04:35 +0000 (UTC) Received: by obvd1 with SMTP id d1so375314obv.0; Sat, 04 Apr 2015 12:04:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=GucpE5Fw8hjaS09fJRRMKTNHUJ1BWlsQzjoqN+kfMZc=; b=C74sgXWkdKcXipSIlAofiBaS1Z/AxFp2AY9K0cb8/pGPEn71vglTTUHGn91G6qdF6k mKZXqOPDZNj90tSLX07WEypBlJNaq2qlK/I2jzUjmk52n+y5pDFXlA4el2sh3n2iZ/UW jQ8sOIHkegLsU4R0+xrmjn5++MTWdofyZTlozZKONjNf7i5qgzy2lzEVcF9wEpPtYesq vugSNTJzTqV6wujTpK16H5tyKWNrhuEjpdZZ7yMN4XrpODrFM5s3L9bz9rFNvJjCDlu+ rLC3aVPdQzxdjn6r9+0zxOYK5pNjyimgysID0IxIpDS6Po/o9z3BPI8a1yLVxe9Wvy1L B9pA== MIME-Version: 1.0 X-Received: by 10.182.39.195 with SMTP id r3mr9699531obk.44.1428174275271; Sat, 04 Apr 2015 12:04:35 -0700 (PDT) Received: by 10.60.20.198 with HTTP; Sat, 4 Apr 2015 12:04:35 -0700 (PDT) In-Reply-To: <5520193C.2010101@freebsd.org> References: <5520193C.2010101@freebsd.org> Date: Sat, 4 Apr 2015 15:04:35 -0400 Message-ID: Subject: Re: odd output from bhyve From: Aryeh Friedman To: Allan Jude Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 19:04:36 -0000 FreeBSD prometheus 10.1-RELEASE-p8 FreeBSD 10.1-RELEASE-p8 #0 r281026: Fri Apr 3 14:24:14 EDT 2015 root@prometheus:/usr/obj/usr/src/sys/GENERIC amd64 #!/bin/sh # # Generated by PetiteCloud 0.2.6 # ifconfig tap56 destroy ifconfig tap56 create ifconfig tap56 up sleep 5 ifconfig bridge0 addm tap56 up bhyvectl --destroy --vm=hjtm24l65 /usr/sbin/bhyveload -m 4096 -d /data/pri/4miq904k2q hjtm24l65 /usr/sbin/bhyve -c 2 -m 4096 -AI -H -P -g 0 -s 0:0,hostbridge -s 1,virtio-net,tap56 -s 2,virtio-blk,/data/pri/4miq904k2q -s 31,uart hjtm24l65& echo $!>/var/run/petitecloud/hjtm24l65 On Sat, Apr 4, 2015 at 1:02 PM, Allan Jude wrote: > On 2015-04-04 12:55, Aryeh Friedman wrote: >> /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () >> Remark 2095 - >> Effective AML package length is zero ^ >> >> >> vm_run error -1, errno 6 >> >> > > This was the output of what command? What version of FreeBSD? > > There is too much missing context here to help you with the error. > > -- > Allan Jude > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 4 19:24:31 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 80F2A436 for ; Sat, 4 Apr 2015 19:24:31 +0000 (UTC) Received: from mx1.scaleengine.net (beauharnois2.bhs1.scaleengine.net [142.4.218.15]) by mx1.freebsd.org (Postfix) with ESMTP id 57A49F86 for ; Sat, 4 Apr 2015 19:24:31 +0000 (UTC) Received: from [192.168.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 219BD9596E for ; Sat, 4 Apr 2015 19:24:30 +0000 (UTC) Message-ID: <55203A7B.9020703@freebsd.org> Date: Sat, 04 Apr 2015 15:24:43 -0400 From: Allan Jude User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: freebsd-virtualization@freebsd.org Subject: Re: odd output from bhyve References: <5520193C.2010101@freebsd.org> In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="e35CDLVlrNTS8IgXtUpUENVXkqsiqu3EK" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 19:24:31 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --e35CDLVlrNTS8IgXtUpUENVXkqsiqu3EK Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2015-04-04 15:04, Aryeh Friedman wrote: > FreeBSD prometheus 10.1-RELEASE-p8 FreeBSD 10.1-RELEASE-p8 #0 r281026: > Fri Apr 3 14:24:14 EDT 2015 > root@prometheus:/usr/obj/usr/src/sys/GENERIC amd64 >=20 >=20 >=20 > #!/bin/sh > # > # Generated by PetiteCloud 0.2.6 > # >=20 > ifconfig tap56 destroy > ifconfig tap56 create > ifconfig tap56 up > sleep 5 > ifconfig bridge0 addm tap56 up > bhyvectl --destroy --vm=3Dhjtm24l65 > /usr/sbin/bhyveload -m 4096 -d /data/pri/4miq904k2q hjtm24l65 > /usr/sbin/bhyve -c 2 -m 4096 -AI -H -P -g 0 -s 0:0,hostbridge -s > 1,virtio-net,tap56 -s 2,virtio-blk,/data/pri/4miq904k2q -s 31,uart > hjtm24l65& > echo $!>/var/run/petitecloud/hjtm24l65 >=20 > On Sat, Apr 4, 2015 at 1:02 PM, Allan Jude wrot= e: >> On 2015-04-04 12:55, Aryeh Friedman wrote: >>> /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () >>> Remark 2095 = - >>> Effective AML package length is zero ^ >>> >>> >>> vm_run error -1, errno 6 >>> >>> >> >> This was the output of what command? What version of FreeBSD? >> >> There is too much missing context here to help you with the error. >> >> -- >> Allan Jude >> >=20 >=20 >=20 You are missing a space before the ampersand at the end of your bhyve invocation --=20 Allan Jude --e35CDLVlrNTS8IgXtUpUENVXkqsiqu3EK Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJVIDp+AAoJEJrBFpNRJZKfv7wP/jJOUpMKp5aDOk5np+mFIRvm rm6EtF6YqRwAsYqxJBMExET27rmfT7vipo4AY/70wWHyNkuo4JW7X3V4Zfkidqw8 ZGyujryCKNx722tp/eWCc6BiCymy4TlRhTTRjmrCXQKhBapDa9vI7u/BL9Qour6q hkC6nmXuHb2elX2ACuI0qr740q/w/vXWCdAhMJLcf1nUyfavFeqIihh51lYQSRO1 Zti6EUoBUqy2ABtOeYW2FtJWVFDunO8jdu34wsWciwcWsYqqA5NjCdRoPEPhZOLP xHHHDMjiNPtQF/3OLX8Ld3q25PohWzB/077xdIZOYfvBA8oRU0repeAF1k+2wHXO lkM5pR2Dp2w/IR8dIkOSZOC/FzfoGbTzOfWSiZAIpb5/Za6832mSzjhRpnrPv55y LkmIlqgIpRzwn6exmEgKW5Jvb6FUN+27uWJ9Q1xRF9cyQhrDlGQf7jKGYWub0uEt RtHi1h693Q1dJJcMda6VPD1lxjePIwMQsQ9sIwvTqf/CNxTmAU7PnPLmAMJsVRzM 6+vjxaXujCGhfSrGcT9BLQ8bB9Rf/TxMVsZtcdi1ZcPclhIWxlbB7x6vhevLqxEP aXIkDJx0MMLC4IkDdZShrhy/etlWWXrTWniCCp0eKhw62OQci4uZ5hejU9jQMVIS VNm8+q+eKiPZZ7XuFWXl =Ctpo -----END PGP SIGNATURE----- --e35CDLVlrNTS8IgXtUpUENVXkqsiqu3EK-- From owner-freebsd-virtualization@FreeBSD.ORG Sat Apr 4 19:26:37 2015 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DAAB74A3; Sat, 4 Apr 2015 19:26:37 +0000 (UTC) Received: from mail-ob0-x229.google.com (mail-ob0-x229.google.com [IPv6:2607:f8b0:4003:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C081FA9; Sat, 4 Apr 2015 19:26:37 +0000 (UTC) Received: by obbfy7 with SMTP id fy7so662025obb.2; Sat, 04 Apr 2015 12:26:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=o7+oui2to1JiZ3T7gO+Mwi162fSTIuAF2xT4iAdp3po=; b=ETCT+YFTyeJLxb+ITeH8FPljpmej2OT4SCBFxF7U8TEKK/bpT/sSSGhnlolPCYnjaV 70hXUwmputZBEKx+dxi94v/qBVgLgOa206uoLFi46mYxnWs8l/y7xpKWNsfGij6y7fnB OMGUxdmEqtrCvJ5FeFb6NKW6As7T+m0Itr6Iy26CVu/MgsONRyX1MUni73QTGJSXL3GX 3N+3iQm5bTGF4i2PHOhCxjloP3lz/NYAu+vqCq+eIZSwF6DJ2v3GgO3LIbUeH1n8bX/R S4qw3lhVrsh8vDXQpdcHfshHg2xwhk3K+VP9MoVSKb6tgk+CD5qSB7tvW75Wnx7cQgpe XHGA== MIME-Version: 1.0 X-Received: by 10.182.91.35 with SMTP id cb3mr9760269obb.87.1428175596872; Sat, 04 Apr 2015 12:26:36 -0700 (PDT) Received: by 10.60.20.198 with HTTP; Sat, 4 Apr 2015 12:26:36 -0700 (PDT) In-Reply-To: <55203A7B.9020703@freebsd.org> References: <5520193C.2010101@freebsd.org> <55203A7B.9020703@freebsd.org> Date: Sat, 4 Apr 2015 15:26:36 -0400 Message-ID: Subject: Re: odd output from bhyve From: Aryeh Friedman To: Allan Jude Content-Type: text/plain; charset=UTF-8 Cc: "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Apr 2015 19:26:37 -0000 PetiteCloud generated it I took it off for debugging reasons (same error with or without) On Sat, Apr 4, 2015 at 3:24 PM, Allan Jude wrote: > On 2015-04-04 15:04, Aryeh Friedman wrote: >> FreeBSD prometheus 10.1-RELEASE-p8 FreeBSD 10.1-RELEASE-p8 #0 r281026: >> Fri Apr 3 14:24:14 EDT 2015 >> root@prometheus:/usr/obj/usr/src/sys/GENERIC amd64 >> >> >> >> #!/bin/sh >> # >> # Generated by PetiteCloud 0.2.6 >> # >> >> ifconfig tap56 destroy >> ifconfig tap56 create >> ifconfig tap56 up >> sleep 5 >> ifconfig bridge0 addm tap56 up >> bhyvectl --destroy --vm=hjtm24l65 >> /usr/sbin/bhyveload -m 4096 -d /data/pri/4miq904k2q hjtm24l65 >> /usr/sbin/bhyve -c 2 -m 4096 -AI -H -P -g 0 -s 0:0,hostbridge -s >> 1,virtio-net,tap56 -s 2,virtio-blk,/data/pri/4miq904k2q -s 31,uart >> hjtm24l65& >> echo $!>/var/run/petitecloud/hjtm24l65 >> >> On Sat, Apr 4, 2015 at 1:02 PM, Allan Jude wrote: >>> On 2015-04-04 12:55, Aryeh Friedman wrote: >>>> /tmp/bhyve.WwKTaeP 84: Name (PPRT, Package () >>>> Remark 2095 - >>>> Effective AML package length is zero ^ >>>> >>>> >>>> vm_run error -1, errno 6 >>>> >>>> >>> >>> This was the output of what command? What version of FreeBSD? >>> >>> There is too much missing context here to help you with the error. >>> >>> -- >>> Allan Jude >>> >> >> >> > > You are missing a space before the ampersand at the end of your bhyve > invocation > > -- > Allan Jude > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org