Date: Mon, 14 May 2018 14:04:45 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= <royger@FreeBSD.org> To: Kai Otto <kai_otto@duckster.net> Cc: freebsd-xen@freebsd.org Subject: Re: Linux domU only works with xen_platform_pci=0 ? Message-ID: <20180514130445.ahqk5ol3kdhriqju@MacBook-Pro-de-Roger.local> In-Reply-To: <0749df4b-1614-dcdf-1bf2-1bbad1ae5743@duckster.net> References: <a5a066b7-625f-d18f-6ea6-663256c09e59@duckster.net> <20180513151649.4ls73myegkhm3cep@MacBook-Pro-de-Roger.local> <0749df4b-1614-dcdf-1bf2-1bbad1ae5743@duckster.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, May 13, 2018 at 07:33:03PM +0200, Kai Otto wrote: > On 13/05/18 17:16, Roger Pau Monné wrote: > > On Sun, May 13, 2018 at 03:51:36PM +0200, Kai Otto wrote: > >> Hello, > >> > >> I'm trying to set up a FreeBSD 11.1 system as Xen virtualization host. > >> Following a combination of handbook [1] and wiki [2], I was able to get > >> get a FreeBSD dom0 in PVH mode, and FreeBSD domU in HVM mode running, > >> including installation to disk and networking. > >> > >> It seemed to me as if the switch 'xen_platform_pci' doesn't have an > >> effect on FreeBSD domU's, as I see e.g. a xenpci0 and xbd0 in dmesg no > >> matter if I set it to 1 or 0. > >> > >> Do I understand it correctly, that this switch makes the difference > >> between HVM and PVHVM mode? > >> According to xl.cfg(5), it 'enables a guest Operating System [...] to > >> make use of paravirtualization features such as disk and network devices'. > >> > >> > >> Afterwards, I tried to create a Linux domU. Both Centos 7 and Alpine > >> Linux only detected the harddisk with xen_platform_pci=0. > >> > >> For Alpine Linux, with xen_platform_pci=1, I get the following messages > >> on the console: > >> > >> vbd vbd-5632: 19 xenbus_dev_probe on device/vbd/5632 > > That's ENODEV IIRC, I think there's something wrong with FreeBSD disk > > backend. > > > >> vbd vbd-5632: failed to write error node for device/vbd/5632 (19 > >> xenbus_dev_probe on device/vbd/5632) > >> > >> After waiting for a couple minutes it boots, but doesn't detect the disk. > >> > >> Looking in /var/log/xen/qemu-dm-alpine-hvm.log, I see the following > >> messages: > >> > >> xen be core: can't open gnttab device > >> xen be core: can't open gnttab device > >> xen be: vkbd-0: initalize() failed > >> xen be: vkbd-0: initalize() failed > >> xen be: vkbd-0: initalize() failed > >> > >> > >> > >> Googling around yielded [3], where someone apparently ran into the same > >> problem, and used xen_platform_pci=0 as workaround. > >> > >> > >> > >> So my question is: > >> Is xen_platform_pci=1 required for PVHVM mode? > > No, it shouldn't be. > > > >> If yes, is PVHVM mode only available for FreeBSD domU's? > >> If no, how can I enable it for Linux guests? > > If you boot FreeBSD Dom0 kernel with boot_verbose=YES (in > > /boot/loader.conf), can you paste the messages you get when trying to > > boot the PVHVM guest with xen_platform_pci=1? > > > > Also, can you paste the output of `xenstore-ls -fp` executed on Dom0 > > after the PVHVM guest has failed to boot but is still running? > > > Hello again, > thanks for your response Roger! > > I attached both. Because I don't know how the mailinglist handles > attachments, here is a pastebin of xenstore-ls: > https://pastebin.com/s7wa1ee7 > And a direct paste of dmesg messages: > > random: harvesting attach, 8 bytes (4 bits) from xbbd0 > tap0: bpf attached > tap0: Ethernet address: 00:bd:ee:99:ff:00 > tap0: link state changed to UP > tap0: changing name to 'xnb2.0-emu' > xnb(xnb_probe:1127): Claiming device 0, xnb > xnb2.0: bpf attached > xnb(xnb_attach:1271): Attaching to backend/vif/2/0 > random: harvesting attach, 8 bytes (4 bits) from xnb0 > xnb(xnb_frontend_changed:1395): frontend_state=Initialising, > xnb_state=InitWait > xnb2.0: link state changed to DOWN > xnb2.0: link state changed to UP > xnb2.0: link state changed to DOWN > xnb2.0: promiscuous mode enabled > xnb2.0: link state changed to UP > xnb2.0-emu: promiscuous mode enabled > nd6_dad_timer: cancel DAD on xnb2.0 because of ND6_IFF_IFDISABLED. > xnb(xnb_frontend_changed:1395): frontend_state=Connected, xnb_state=InitWait > xnb(xnb_connect_comms:788): rings connected! Can you please give a try to the following patch? I think it should apply cleanly to stable/11 (11.1-RELEASE). You will have to rebuild and install the kernel. Thanks, Roger. ---8<--- diff --git a/sys/dev/xen/blkback/blkback.c b/sys/dev/xen/blkback/blkback.c index 9744c8da73cb..e9c7afe7d18c 100644 --- a/sys/dev/xen/blkback/blkback.c +++ b/sys/dev/xen/blkback/blkback.c @@ -806,6 +806,9 @@ struct xbb_softc { /** Watch to wait for hotplug script execution */ struct xs_watch hotplug_watch; + + /** Got the needed data from hotplug scripts? */ + bool hotplug_done; }; /*---------------------------- Request Processing ----------------------------*/ @@ -3310,10 +3313,9 @@ xbb_connect(struct xbb_softc *xbb) { int error; - if (xenbus_get_state(xbb->dev) != XenbusStateInitialised) - return; - - if (xbb_collect_frontend_info(xbb) != 0) + if (!xbb->hotplug_done || + (xenbus_get_state(xbb->dev) != XenbusStateInitWait) || + (xbb_collect_frontend_info(xbb) != 0)) return; xbb->flags &= ~XBBF_SHUTDOWN; @@ -3412,6 +3414,7 @@ xbb_shutdown(struct xbb_softc *xbb) free(xbb->hotplug_watch.node, M_XENBLOCKBACK); xbb->hotplug_watch.node = NULL; } + xbb->hotplug_done = false; if (xenbus_get_state(xbb->dev) < XenbusStateClosing) xenbus_set_state(xbb->dev, XenbusStateClosing); @@ -3692,8 +3695,7 @@ xbb_attach_disk(struct xs_watch *watch, const char **vec, unsigned int len) return; } - /* Tell the front end that we are ready to connect. */ - xenbus_set_state(dev, XenbusStateInitialised); + xbb->hotplug_done = true; } /** @@ -3757,6 +3759,7 @@ xbb_attach(device_t dev) * We need to wait for hotplug script execution before * moving forward. */ + KASSERT(!xbb->hotplug_done, ("Hotplug scripts already executed")); watch_path = xs_join(xenbus_get_node(xbb->dev), "physical-device-path"); xbb->hotplug_watch.callback_data = (uintptr_t)dev; xbb->hotplug_watch.callback = xbb_attach_disk;
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180514130445.ahqk5ol3kdhriqju>