From owner-freebsd-testing@FreeBSD.ORG Mon Jun 16 06:44:19 2014 Return-Path: Delivered-To: freebsd-testing@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26A20A34; Mon, 16 Jun 2014 06:44:19 +0000 (UTC) Received: from mail-la0-x231.google.com (mail-la0-x231.google.com [IPv6:2a00:1450:4010:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2DEC12CD8; Mon, 16 Jun 2014 06:44:18 +0000 (UTC) Received: by mail-la0-f49.google.com with SMTP id gf5so2124367lab.22 for ; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=60b/mZEKfhsOeWiaBPKnwOQRRossbb9/NKT4QBNRVbM=; b=lC1DlHTis9GiJ5Aqztei3sxY7ju6VwYJ13dBa8jhgJ0CqAEJc/7DmPEsQ48wMO4bEw xHUaRI8sKWBmaZly9Y1gJg321G6WcijlFndMZzfBPeV/ht5NWckrV3i3Zh3q30hrFXy3 6tMqejLkA7RMEtchwgvpDpil9Q3LTdOlwZqWfB10+ZCRvXrvfF4no5M/3Zm3kyEBkFxr DdH2THIK7CXJFQO89tKbgSe+hBGE3S8ARaKuaYWfB1rT8IHIhihjVQc2ny24KJlbHylt 4yb8Jqjhb6BP1EWeQGn/eEf++8J3LQmrx1DSIMsWbqy3922gDldB/zSWrLmD1SGt8g7c Y2DA== X-Received: by 10.112.17.102 with SMTP id n6mr12120948lbd.39.1402901056011; Sun, 15 Jun 2014 23:44:16 -0700 (PDT) Received: from dev.san.ru (dev.san.ru. [88.147.129.60]) by mx.google.com with ESMTPSA id s15sm7882189lbp.42.2014.06.15.23.44.14 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 15 Jun 2014 23:44:15 -0700 (PDT) Sender: Roman Bogorodskiy Date: Mon, 16 Jun 2014 10:44:07 +0400 From: Roman Bogorodskiy To: Craig Rodrigues Subject: Re: libvirt and bhyve problems Message-ID: <20140616064406.GA41585@dev.san.ru> References: <20140611192514.GE2855@kloomba> <20140612080057.GA93486@kloomba> <20140612190042.GA2858@kloomba> <20140615152008.GA19985@kloomba> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: Mutt/1.5.22 (2013-10-16) Cc: "freebsd-testing@freebsd.org" , Roman Bogorodskiy , "freebsd-virtualization@freebsd.org" X-BeenThere: freebsd-testing@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Testing on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jun 2014 06:44:19 -0000 Craig Rodrigues wrote: > On Sun, Jun 15, 2014 at 8:20 AM, Roman Bogorodskiy wr= ote: > > > > I have updated the port: > > > > http://people.freebsd.org/~novel/misc/libvirt_port2.tgz >=20 >=20 > OK, I deinstall the old port, took your port, built it, and installed it. >=20 > When I started libvirtd, I got this: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 >=20 > Trying to start bhyve did not work. >=20 > After some investigation, I did the following: >=20 > service local_unbound stop >=20 >=20 > Then I started libvirtd again: >=20 > 2014-06-15 18:47:07.236+0000: 34485605376: info : libvirt version: 1.2.5 > 2014-06-15 18:47:07.236+0000: 34485605376: info : > dnsmasqCapsSetFromBuffer:685 : dnsmasq version is 2.71, --bind-dynamic > is present, SO_BINDTODEVICE is in use > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkReloadFirewallRules:1750 : Reloading iptables rules > 2014-06-15 18:47:07.237+0000: 34485605376: info : > networkRefreshDaemons:1722 : Refreshing network daemons > 2014-06-15 18:47:07.395+0000: 34485605376: error : virCommandWait:2426 > : internal error: Child process (/usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf) > unexpected exit status 2: > dnsmasq: failed to create listening socket for 127.0.0.1: Address already= in use >=20 > 2014-06-15 18:47:07.484+0000: 34485605376: error : virExec:417 : > Cannot find 'pm-is-supported' in path: No such file or directory > 2014-06-15 18:47:07.484+0000: 34485605376: warning : > virQEMUCapsInit:948 : Failed to get host power management capabilities > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/var/run/libvirt/qemu > 2014-06-15 18:47:07.495+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/qemu > 2014-06-15 18:47:07.583+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18249 : Scanning for configs in > /usr/local/etc/libvirt/bhyve > 2014-06-15 18:47:07.584+0000: 34485605376: info : > virDomainObjListLoadAllConfigs:18273 : Loading config file 'bhyve.xml' >=20 >=20 > At this point, I could see the virbr0 device: >=20 > virbr0: flags=3D8843 metric 0 mtu= 1500 > ether 02:b8:18:5f:0f:00 > inet 192.168.122.1 netmask 0xffffff00 broadcast 192.168.122.255 > nd6 options=3D1 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 4 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >=20 >=20 > I was then able to do: >=20 > virsh -c 'bhyve:///system' > start bhyve >=20 > and start a VM. > This is good progress! >=20 > Once thing I observed is that if I killed the libvirtd process, > it did not clean up the dnsmasq process that it started: >=20 > /usr/local/sbin/dnsmasq > --conf-file=3D/usr/local/var/lib/libvirt/dnsmasq/default.conf >=20 > and a subsequent restart of libvirtd would fail. I think it's intentional not to kill dnsmasq. There are persistent and transient objects in libvirt (such as domains and networks), and persistent objects live forever, while transient objects live until manually destroyed or host is restarted. So libvirt doesn't kill persistent VMs and, I guess, it doesn't stop dnsmasq to keep persistent networks functional. It should not affect restart. I'll fail to start the network, but as it's here already, we're fine. BTW, I'm still yet to write the code to catch up previously started bhyve VMs between libvirtd restarts... > The devel/libvirt port is looking a lot better with your changes. > Can you push these changes into the port? >=20 > I filed bugs 190767 and 190783, but did not get any response > from the port maintainer, jgh@. I'll send the patch, but I need to fix some issues first. I noticed that check-orphans complains about the orphaned libvirtd.conf file. Also, I need to have a poudriere run as well to make sure I didn't miss anything else. Roman Bogorodskiy