From owner-freebsd-xen@freebsd.org Tue May 15 02:34:58 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 767A4EB1F7E for ; Tue, 15 May 2018 02:34:58 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: from mail-pg0-x22e.google.com (mail-pg0-x22e.google.com [IPv6:2607:f8b0:400e:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE7947A111; Tue, 15 May 2018 02:34:57 +0000 (UTC) (envelope-from nathan.friess@gmail.com) Received: by mail-pg0-x22e.google.com with SMTP id l2-v6so6307277pgc.7; Mon, 14 May 2018 19:34:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language; bh=XtBIjZ3KT8GXBlthLRfC/HiaSOZFwHt9zLnKiIvH0yI=; b=KpRnUs5fU0RUP8CUHh15/Z5F9+waPhoebtGPGl5MmGTq+XZMkx5F5jaM23JDiFfadr kvNYK91cWh+Yyav5QjsuBxIzQNHyONVlVTbg5wF7//m6xL7SBX5KLczQW9ZLMQ7tXXkJ ZmWnRoR/Rl9AhCC3mT+ugRQKIAg6E1JKahK1RLFo6LMAIWyMzbvMaCb6p4QlZwoeyY8k DdNX6NYN9xpCX4gI9IH7ywe+O+btA6sSZng4KpMsJavZP8tcActxbqBWfLBVN5RTS0hY PwDCFrQGyVrnobzZhMgMs5Hc0+TQBM3UtKFCECzuqUS5RZINUC3PB81Wq8K08qaagc0d Pbyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language; bh=XtBIjZ3KT8GXBlthLRfC/HiaSOZFwHt9zLnKiIvH0yI=; b=gNbF1Uuh18RvRf2mTQhVnBy1VhbzwAVt5QlzrZZe2IbFPUdUaMElbrH0Vl3retGPMn GvrxguMTqXUGtx8Zi6v6qXEivVUB356CZ7S83jy5zA/vqRuzqS5hQ0K6ck1SS4BZZ2gB cUyI0lNeT8wTdB7HtHCFknr4woDoFVrKJeaMlsTtQtfA5nyeTtNaoNhj1qA1A915zGl6 f462QK4XTUCN8tE7plf37HhV2iqvM5X99UDP+PhErBMj/3Vf3GqhQexlllp7RRZIttZm AsVvF5xMScYnZlPJzbeUMeocv5f2btGRiKS2WTWt1kuI27CALIjfSkvqyiK1xn3D7QR0 W/Gg== X-Gm-Message-State: ALKqPwcOSVhB2cxnTfJTxwMXbUMWDvFHyPJb1KtDlzPyOTEvJZ7IDvTB USa4gx0OzPygkAGZ/5tLdYLSniMR X-Google-Smtp-Source: AB8JxZqRDwQLQVxEjSzPbf2qcTbHzm9YC1XdYyptuRBpu+E0OUWvX5M1Q8DHbHK16CN2wk6EVjdkkg== X-Received: by 2002:a63:302:: with SMTP id 2-v6mr10317941pgd.98.1526351696765; Mon, 14 May 2018 19:34:56 -0700 (PDT) Received: from [10.2.1.1] (S01060018e7c4b870.cg.shawcable.net. [70.72.182.108]) by smtp.gmail.com with ESMTPSA id f4-v6sm6564064pgo.77.2018.05.14.19.34.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 May 2018 19:34:56 -0700 (PDT) Subject: Re: Linux domU only works with xen_platform_pci=0 ? To: =?UTF-8?Q?Roger_Pau_Monn=c3=a9?= , Kai Otto Cc: freebsd-xen@freebsd.org References: <20180513151649.4ls73myegkhm3cep@MacBook-Pro-de-Roger.local> <0749df4b-1614-dcdf-1bf2-1bbad1ae5743@duckster.net> <20180514130445.ahqk5ol3kdhriqju@MacBook-Pro-de-Roger.local> From: Nathan Friess Message-ID: <6c0e1f5a-3e7d-054e-298c-5ec3d97e6141@gmail.com> Date: Mon, 14 May 2018 20:34:54 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <20180514130445.ahqk5ol3kdhriqju@MacBook-Pro-de-Roger.local> Content-Type: multipart/mixed; boundary="------------E874B936718ADF8811C4B06C" Content-Language: en-US X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2018 02:34:58 -0000 This is a multi-part message in MIME format. --------------E874B936718ADF8811C4B06C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 2018-05-14 07:04 AM, Roger Pau Monné wrote: > On Sun, May 13, 2018 at 07:33:03PM +0200, Kai Otto wrote: >> On 13/05/18 17:16, Roger Pau Monné wrote: >>> On Sun, May 13, 2018 at 03:51:36PM +0200, Kai Otto wrote: >>>> Hello, >>>> >>>> I'm trying to set up a FreeBSD 11.1 system as Xen virtualization host. >>>> Following a combination of handbook [1] and wiki [2], I was able to get >>>> get a FreeBSD dom0 in PVH mode, and FreeBSD domU in HVM mode running, >>>> including installation to disk and networking. >>>> >>>> It seemed to me as if the switch 'xen_platform_pci' doesn't have an >>>> effect on FreeBSD domU's, as I see e.g. a xenpci0 and xbd0 in dmesg no >>>> matter if I set it to 1 or 0. >>>> >>>> Do I understand it correctly, that this switch makes the difference >>>> between HVM and PVHVM mode? >>>> According to xl.cfg(5), it 'enables a guest Operating System [...] to >>>> make use of paravirtualization features such as disk and network devices'. >>>> >>>> >>>> Afterwards, I tried to create a Linux domU. Both Centos 7 and Alpine >>>> Linux only detected the harddisk with xen_platform_pci=0. >>>> >>>> For Alpine Linux, with xen_platform_pci=1, I get the following messages >>>> on the console: >>>> >>>> vbd vbd-5632: 19 xenbus_dev_probe on device/vbd/5632 >>> That's ENODEV IIRC, I think there's something wrong with FreeBSD disk >>> backend. >>> >>>> vbd vbd-5632: failed to write error node for device/vbd/5632 (19 >>>> xenbus_dev_probe on device/vbd/5632) >>>> >>>> After waiting for a couple minutes it boots, but doesn't detect the disk. I had similar issues with Linux domUs being unable to detect their disks when FreeBSD 11.1-RELEASE is the backend: https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002924.html What seems to be happening is that on my system the frontend and backend may go from state InitWait to Initialised in different orders and so either end may end up getting stuck waiting for the other side to change state when the other side already has done so. I have been running with the attached patch since my last message above and Linux domUs have been working perfectly since then. I realize that the call to pause() is not the correct solution but it demonstrates that some fine tuning of how the states are handled will help. Nathan --------------E874B936718ADF8811C4B06C Content-Type: text/x-patch; name="missed-notifications.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="missed-notifications.patch" Index: sys/dev/xen/blkback/blkback.c =================================================================== --- sys/dev/xen/blkback/blkback.c (revision 321598) +++ sys/dev/xen/blkback/blkback.c (working copy) @@ -3692,6 +3692,11 @@ /* Tell the front end that we are ready to connect. */ xenbus_set_state(dev, XenbusStateInitialised); + + /* The front end may have finished first, so we should proceed as well. */ + if (xenbus_get_otherend_state(xbb->dev) == XenbusStateInitialised) { + xbb_connect(xbb); + } } /** @@ -3769,6 +3774,14 @@ return (error); } + /* + * We may need to wait for Linux frontend to become ready + * so it will not miss us going into InitWait. + */ + if (xenbus_get_otherend_state(xbb->dev) != XenbusStateInitialising) { + pause("xbb_attach", 100); + } + /* Tell the toolstack blkback has attached. */ xenbus_set_state(dev, XenbusStateInitWait); --------------E874B936718ADF8811C4B06C--