From owner-freebsd-xen@freebsd.org Tue Jan 24 11:45:12 2017 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B6BCBCBF8B9 for ; Tue, 24 Jan 2017 11:45:12 +0000 (UTC) (envelope-from prvs=19068b32a=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.ctxuk.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F180686B; Tue, 24 Jan 2017 11:45:08 +0000 (UTC) (envelope-from prvs=19068b32a=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.33,278,1477958400"; d="scan'208";a="39262165" Date: Tue, 24 Jan 2017 11:44:44 +0000 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Alexander Nusov CC: , Subject: Re: Xen on FreeBSD 11: Cannot boot from QCOW2 properly (waiting for XENBUS, xenbusb_nop_confighook_cb) Message-ID: <20170124114444.xdl3qj35lwebkso7@dhcp-3-221.uk.xensource.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: User-Agent: NeoMutt/20170113 (1.7.2) X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2017 11:45:12 -0000 On Mon, Jan 23, 2017 at 06:25:23PM +0300, Alexander Nusov wrote: > Hello, > Sorry for cross-posting, since it's related to Xen hypervisor I'm forward= ing this message to the freebsd-xen mailing list. >=20 > I'm trying to launch a HVM DomU guest from QCOW2 image by using PV driver= on FreeBSD 11 Dom0. > The issue is that guest cannot connect to the device/vbd and requires to = wait for 4 minutes to proceed, it goes through the countdown and starts fin= e (disk, networking) >=20 > [ 6.684115] XENBUS: Waiting for devices to initialise: 25s...20s...15s= =2E..10s...5s...0s...235s...230s...225s...220s...215s...210s...205s...200s.= =2E.195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...1= 45s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...= 90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s= =2E..25s...20s...15s...10s...5s...0s... > [ 271.591403] XENBUS: Timeout connecting to device: device/vbd/51712 (lo= cal state 3, remote state 1) > [ 271.599963] XENBUS: Device with no driver: device/vkbd/0 > [ 271.604249] Magic number: 1:453:334 > ... > login:=20 >=20 >=20 > Unlike Linux It's impossible to boot FreeBSD 11 guests from QCOW2 (xenbus= b_nop_confighook_cb timeout) >=20 > Steps to reproduce: > 1. Download qcow2 cirros image (small linux)=20 > http://secure-web.cisco.com/1RtWXk7FS6UyRxe5A9ejZVfU2VK9512Yeds80mlP_0LoL= wQJLjbPBoILL2e4QyF2OaMyO8fbrQrYVmfOybSYvHgWH1sRz1gK4Zi5XDAzBDUluwhlU4NxVQ7S= 17tH6vSTrIJmBJ_NO-sdA1Ph_KfSOsNvMkw-swwuKHD9ophVFfqEe6Lnt_PIP4H-LhZfp-5_aCu= qbPYciXtMyxWbF1Od65jiFe-_LfmPRt53aCscOk7cBRI_-o603sb7htpDHH06Y8-M4oG5Lt4s1j= r1XcKTrIWhnD-Lhz55blWyc2FnCgvk/http%3A%2F%2Fdownload.cirros-cloud.net%2F0.3= =2E4%2Fcirros-0.3.4-x86_64-disk.img > > # file cirros-0.3.4-x86_64-disk.img=20 > cirros-0.3.4-x86_64-disk.img: QEMU QCOW Image (v2), 41126400 bytes > 2. create DomU from config bellow xl create -c config.cfg >=20 > builder =3D "hvm" > memory =3D 512 > vcpus =3D 2 > name =3D "cirros" > disk =3D [ 'file:qcow2:/root/cirros-0.3.4-x86_64-disk.img,xvda,w' ] > boot =3D "c"=20 > vnc =3D 1 > vnclisten =3D "0.0.0.0" > usbdevice =3D 'tablet' > on_poweroff =3D 'destroy' > on_reboot =3D 'restart' > on_crash =3D 'restart' > acpi =3D 1 > serial =3D 'pty' >=20 > I've also tried multiple configurations like tap:qcow2:. tap2:qcow2:, aio= :, switching from xen bus to ide. didn't work. > The only driver that had no issues was PHY but it supports only RAW image= s. >=20 > Is that a bug or I'm missing something? >=20 > tested both STABLE snapshot and 11.0-RELEASE >=20 > # uname -a > FreeBSD xen 11.0-STABLE FreeBSD 11.0-STABLE #0 r311441: Thu Jan 5 22:45:= 20 UTC 2017 >=20 > # pkg info | grep xen > xen-4.7.0_2 Xen Hypervisor meta port > xen-kernel-4.7.1_3 Hypervisor using a microkernel design > xen-tools-4.7.1_1 Xen management tool, based on LibXenlight So just that I understand this correctly, this is a FreeBSD 11.0-STABLE Dom= 0, plus the Xen packages from pkg? If that's the case, it's not going to work, FreeBSD 11.0 Dom0 doesn't yet support qcow image format for HVM/PV guests, you will have to use FreeBSD 12 (HEAD) as your Dom0, or backport r308128 into STABLE (should be self contai= ned, so I don't expect any conflicts). You will also have to apply the following patch to the xen-tools package and recompile: https://lists.freebsd.org/pipermail/freebsd-xen/2016-August/002819.html IIRC the right syntax to specify the disk device is: 'format=3Dqcow2,vdev=3Dxvda,access=3Drw,backendtype=3Dqdisk,target=3D/root/= cirros-0.3.4-x86_64-disk.img' I'm also adding Akshay to the conversation, who did the gntdev implementati= on. Roger.