Date: Sun, 1 Jul 2018 01:34:01 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Mark Johnston <markj@freebsd.org> Cc: Mihai Carabas <mihai.carabas@gmail.com>, freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Inspect pages created after a vm_object is marked as copy-on-write Message-ID: <20180630223401.GW2430@kib.kiev.ua> In-Reply-To: <20180630215956.GA1282@pesky.lan> References: <CAGOCPLjrbv5nyXTQwqrsJhF9wFEFACeEyuS1aW0jdHycWNYz8g@mail.gmail.com> <20180629225209.GA4238@pesky.lan> <CANg1yUupb0hQb7jQHjD%2BvY8=3m4v%2BcH819qNGS7TjDUVFMgL=Q@mail.gmail.com> <20180630215956.GA1282@pesky.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jun 30, 2018 at 05:59:56PM -0400, Mark Johnston wrote: > On Sat, Jun 30, 2018 at 10:38:21AM +0300, Mihai Carabas wrote: > > On Sat, Jun 30, 2018 at 1:52 AM, Mark Johnston <markj@freebsd.org> wrote: > > > On Fri, Jun 29, 2018 at 11:58:31AM +0300, Elena Mihailescu wrote: > > >> Is there anything I am doing wrong? Maybe I misunderstood something about > > >> the way the virtual memory works in FreeBSD. > > > > > > I'll note that inspecting and manipulating vm_map_entry and vm_object > > > structures in the bhyve code constitutes something of an abstraction > > > violation, though it's reasonable to proceed this way while working on a > > > prototype of the feature. That is, I think you should keep trying your > > > current approach, but just be aware that you are using the copy-on-write > > > mechanism in a way that the VM system isn't really expecting. > > > > > > > Can you point out the right approach in our case? > > I am merely suggesting that once the required VM interactions are fully > understood, the mechanism implemented for bhyve should be generalized > and lifted into the VM code. It's hard to say what the "right" approach > is, since I don't fully understand the proposed algorithm. It sounds > like you might be attempting something like: > > 1. mark the mappings of to-be-migrated objects as NEEDS_COW, so that a > subsequent write fault triggers creation of a shadow object It is actually MAP_ENTRY_COW | MAP_ENTRY_NEEDS_COPY. Note that setting an entry to COW changes the behaviour of mprotect(2), at least. > 2. invalidate all physical mappings of pages in the object to be copied, > so that subsequent writes trigger a fault I do not think this is needed to detect writes after the COW is set. It is enough to remove the write permissions. Same as fork() does, see the vm_map_copy_entry() code for the handling of MAP_ENTRY_NEEDS_COPY case. > 3. copy pages from the backing object to the destination As I understand, this is done right after the entry is marked as COW. > 4. copy any pages from the shadow object to the desination And this is done after all backing data is copied and the process is suspended. > 5. collapse the backing object into the shadow > 6. if the shadow object exists and was non-empty before the collapse, > goto 1 Are you trying to describe how to undo the COW marking ? Marking an entry as COW really changes its semantic, and we do not need the undo operation in the base so far. Collapsing the objects would lesser the pressure on the system pollution with objects, but it does not change back the meaning of mappings, e.g. their behaviour on inheritance on fork. > > Is that at all accurate? I'm not familiar with the mechanisms used to > implement live migration in other hypervisors. > _______________________________________________ > freebsd-amd64@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-amd64 > To unsubscribe, send any mail to "freebsd-amd64-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180630223401.GW2430>