Date: Tue, 22 May 2018 11:01:24 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= <royger@FreeBSD.org> To: Nathan Friess <nathan.friess@gmail.com> Cc: freebsd-xen@freebsd.org Subject: Re: Linux domU only works with xen_platform_pci=0 ? Message-ID: <20180522090124.onimqynage4hys53@MBP-de-Roger> In-Reply-To: <723f10d0-62c0-e48e-e8f4-b137378f0a25@gmail.com> References: <a5a066b7-625f-d18f-6ea6-663256c09e59@duckster.net> <20180513151649.4ls73myegkhm3cep@MacBook-Pro-de-Roger.local> <0749df4b-1614-dcdf-1bf2-1bbad1ae5743@duckster.net> <20180514130445.ahqk5ol3kdhriqju@MacBook-Pro-de-Roger.local> <6c0e1f5a-3e7d-054e-298c-5ec3d97e6141@gmail.com> <20180515080836.kufr3q3mk5mxwx4q@MacBook-Pro-de-Roger.local> <c834ba9c-251a-ed00-dfaa-f3b012e3c302@gmail.com> <20180519080250.67fl66gqbew7xfzp@MacBook-Pro-de-Roger.local> <723f10d0-62c0-e48e-e8f4-b137378f0a25@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, May 21, 2018 at 01:51:13PM -0600, Nathan Friess wrote: > On 2018-05-19 02:02 AM, Roger Pau Monné wrote: > > On Thu, May 17, 2018 at 06:10:15PM -0600, Nathan Friess wrote: > > > I tried the patch on and I my Linux domU did was not able to complete the > > > attachment to the FreeBSD backend. I applied the patch to the 11-RELEASE > > > kernel. (I couldn't get 11-STABLE to compile.) > > > > > > With xen_platform_pci=0 the frontend and backend vbds were both stuck in > > > state 1. With xen_platform_pci=1 the frontend was in state 1 and backend in > > > state 3. > > > > Thanks for the testing! I however think you had some issue applying > > the patch or building/installing the kernel, with the attached patch > > the backend should never go into state 3. > > > > Can you confirm that you have the patch correctly applied to the > > kernel you are running? > > > > And if so, can you try to debug why the backend goes into state 3? > > > > FWIW, the patch works fine for me and I've been able to boot Debian > > and Alpine Linux guests without having to add xen_platform_pci=0. > > > > Thanks, Roger. > > > > Sorry, I don't which run I was looking at there. Here is the order of > events. There are two zvols exposed to the domU, > /dev/zvol/zroot/vmachines/smyth/rootfs and .../swap, so the messages are > interspersed between the two. > > xbb(xbb_attach:3738): Attaching to backend/vbd/10/51713 > xbb(xbb_attach:3802): Attach done, set state InitWait > xbb(xbb_attach_disk:3612): Enter xbb_attach_disk > xbb(xbb_attach_disk:3620): Read physical-device-path failed > xbb(xbb_frontend_changed:3918): frontend_state=Initialising, > xbb_state=InitWait > xbb(xbb_attach:3738): Attaching to backend/vbd/10/51714 > xbb(xbb_attach:3802): Attach done, set state InitWait > xbb(xbb_attach_disk:3612): Enter xbb_attach_disk > xbb(xbb_attach_disk:3620): Read physical-device-path failed > xbb(xbb_frontend_changed:3918): frontend_state=Initialising, > xbb_state=InitWait > (Shell script) Start block script /local/domain/9/backend/vbd/10/51714 add > (Shell script) Start block script /local/domain/9/backend/vbd/10/51713 add > [1] xbb(xbb_frontend_changed:3918): frontend_state=Initialised, > xbb_state=InitWait > xbb(xbb_connect:3316): Enter xbb_connect: hotplug=0, get_state=2, > collect_frontend=0 > [1] xbb(xbb_frontend_changed:3918): frontend_state=Initialised, > xbb_state=InitWait > xbb(xbb_connect:3316): Enter xbb_connect: hotplug=0, get_state=2, > collect_frontend=0 > (Shell script) Write /dev/zvol/zroot/vmachines/smyth/rootfs to > /local/domain/9/backend/vbd/10/51713/physical-device-path > xbb(xbb_attach_disk:3612): Enter xbb_attach_disk > xbb(xbb_open_backend:2687): opening > dev=/dev/zvol/zroot/vmachines/smyth/rootfs > xbb(xbb_open_backend:2757): opened > dev=/dev/zvol/zroot/vmachines/smyth/rootfs sector_size=512 > media_size=21474836480 > xbb(xbb_attach_disk:3718): Set hotplug done, attach_disk done > (Shell script) Write /dev/zvol/zroot/vmachines/smyth/swap to > /local/domain/9/backend/vbd/10/51714/physical-device-path > xbb(xbb_attach_disk:3612): Enter xbb_attach_disk > xbb(xbb_open_backend:2687): opening dev=/dev/zvol/zroot/vmachines/smyth/swap > xbb(xbb_open_backend:2757): opened dev=/dev/zvol/zroot/vmachines/smyth/swap > sector_size=512 media_size=2147483648 > xbb(xbb_attach_disk:3718): Set hotplug done, attach_disk done > > ... is stuck here... > > At [1] the frontend has transitioned to Initialised but the block script has > not completed yet so hotplug_done is 0 and the notification about the > frontend status change is lost. Calling xbb_connect at the end of > xbb_attach_disk like in the attached patch will catch this case. Now with > your recent change and the attached file I can use Linux domUs successfully! Your Linux kernel is certainly very fast to boot, so that it wins the race with blkback processing the result from the hotplug script execution. I've now committed the fix to HEAD, will MFC in a week if everything goes fine: https://svnweb.freebsd.org/base?view=revision&revision=334027 TYVM for the testing and the fix. > On a related note, did these patches ever make it into 11-stable? I don't > see them at svn.freebsd.org/base/stable/11 but I may have missed something. > > https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002918.html I don't think they are on HEAD either? I don't have a lot of time ATM, so if you want to pick the patches, rebase them onto HEAD, test them and give me a branch I'm more than happy to review and commit them. > There were 4 patches, the third one being required for xl devd to trigger > block scripts. I have been using them as well for over a year with no > issues: > > https://lists.freebsd.org/pipermail/freebsd-xen/2016-December/002918.html The Xen side of this is already upstream AFAIK, see: http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=7ff99b232e0f91a5189f429498868bfccf8d7154 Thanks, Roger.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180522090124.onimqynage4hys53>