Date: Sun, 1 Jul 2018 04:29:19 -0400 From: Mark Johnston <markj@freebsd.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: Mihai Carabas <mihai.carabas@gmail.com>, freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Subject: Re: Inspect pages created after a vm_object is marked as copy-on-write Message-ID: <20180701082919.GB3926@pesky.lan> In-Reply-To: <20180630223401.GW2430@kib.kiev.ua> References: <CAGOCPLjrbv5nyXTQwqrsJhF9wFEFACeEyuS1aW0jdHycWNYz8g@mail.gmail.com> <20180629225209.GA4238@pesky.lan> <CANg1yUupb0hQb7jQHjD%2BvY8=3m4v%2BcH819qNGS7TjDUVFMgL=Q@mail.gmail.com> <20180630215956.GA1282@pesky.lan> <20180630223401.GW2430@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jul 01, 2018 at 01:34:01AM +0300, Konstantin Belousov wrote: > On Sat, Jun 30, 2018 at 05:59:56PM -0400, Mark Johnston wrote: > > On Sat, Jun 30, 2018 at 10:38:21AM +0300, Mihai Carabas wrote: > > > On Sat, Jun 30, 2018 at 1:52 AM, Mark Johnston <markj@freebsd.org> wrote: > > > > On Fri, Jun 29, 2018 at 11:58:31AM +0300, Elena Mihailescu wrote: > > > >> Is there anything I am doing wrong? Maybe I misunderstood something about > > > >> the way the virtual memory works in FreeBSD. > > > > > > > > I'll note that inspecting and manipulating vm_map_entry and vm_object > > > > structures in the bhyve code constitutes something of an abstraction > > > > violation, though it's reasonable to proceed this way while working on a > > > > prototype of the feature. That is, I think you should keep trying your > > > > current approach, but just be aware that you are using the copy-on-write > > > > mechanism in a way that the VM system isn't really expecting. > > > > > > > > > > Can you point out the right approach in our case? > > > > I am merely suggesting that once the required VM interactions are fully > > understood, the mechanism implemented for bhyve should be generalized > > and lifted into the VM code. It's hard to say what the "right" approach > > is, since I don't fully understand the proposed algorithm. It sounds > > like you might be attempting something like: > > > > 1. mark the mappings of to-be-migrated objects as NEEDS_COW, so that a > > subsequent write fault triggers creation of a shadow object > It is actually MAP_ENTRY_COW | MAP_ENTRY_NEEDS_COPY. Indeed. > Note that setting an entry to COW changes the behaviour of mprotect(2), > at least. > > > 2. invalidate all physical mappings of pages in the object to be copied, > > so that subsequent writes trigger a fault > I do not think this is needed to detect writes after the COW is set. > It is enough to remove the write permissions. Same as fork() does, > see the vm_map_copy_entry() code for the handling of MAP_ENTRY_NEEDS_COPY > case. Ah, right. > > 3. copy pages from the backing object to the destination > As I understand, this is done right after the entry is marked > as COW. > > > 4. copy any pages from the shadow object to the desination > And this is done after all backing data is copied and the process is > suspended. Right, I see. Some reading suggests that in general we might perform multiple iterations of this procedure before suspending the process and performing any remaining copies. > > 5. collapse the backing object into the shadow > > 6. if the shadow object exists and was non-empty before the collapse, > > goto 1 > Are you trying to describe how to undo the COW marking ? Marking an > entry as COW really changes its semantic, and we do not need the undo > operation in the base so far. Collapsing the objects would lesser > the pressure on the system pollution with objects, but it does not > change back the meaning of mappings, e.g. their behaviour on inheritance > on fork. I was just thinking about how to avoid creating a long chain of objects if multiple iterations are in fact needed.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180701082919.GB3926>