Date: Tue, 11 Apr 2017 15:37:26 +0300 From: Flavius Anton <f.v.anton@gmail.com> To: freebsd-hackers@freebsd.org Subject: On COW memory mapping in d_mmap_single Message-ID: <CANXdjjYajtvWK%2Bq3OK4j5uPFR4sVUrhrQD8zZSpoJ1hwZhVS5Q@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi everyone, I'll start by giving some context, so you can better understand what is the problem I'm trying to solve. I=E2=80=99ve been working for a while o= n bhyve trying to implement save/restore [1]. We've currently managed to get it working for VMs using a ramdisk and no devices, so just vCPU and memory states are saved and restored so far. Last week I started looking into network devices, specifically virtio-net devices. The problem was that when I issue a checkpoint operation, the guest virtio driver stops working. After digging for a while, I figured out the problem is marking VM memory as COW. If I don't do this, the driver continues with no problem after checkpointing. Each VM has an associated vmspace and a /dev/vmm/VM_NAME device. When the user space does a mmap on the /dev device, we would like to mark VM memory as COW, thus the VM can continue touching pages while the user space is writing the 'freezed', COW marked memory to a persistent storage. We do this by iterating through all vm_entries from VM's vmspace, we find which entry is mapping the object that has VM memory and then we roughly just set MAP_ENTRY_COW and MAP_ENTRY_NEEDS_COPY on that entry. You can see the code here [2]. I'm not sure if the above is sufficient for our purpose. In other words, how would you do this? You have a vm_object that is referenced via a vm_entry by process A (the user space). Somebody else, process B let's say, does an mmap() on your device and you'd like to freeze that object, such that process B can see a consistent snapshot of it, while you want process A to be able to continue reading and writing from/to it. I've also read through Design Elements of the FreeBSD VM system [3], but I am still afraid (I am sure) that I have some misunderstandings. Thank you very much for bearing with me and going through this wall of text= . -- Flavius [1] https://github.com/flaviusanton/freebsd/tree/bhyve-save-restore [2] https://github.com/flaviusanton/freebsd/blob/bhyve-save-restore/sys/amd= 64/vmm/vmm_dev.c#L862 [3] https://www.freebsd.org/doc/en/articles/vm-design/index.html
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANXdjjYajtvWK%2Bq3OK4j5uPFR4sVUrhrQD8zZSpoJ1hwZhVS5Q>