Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 1 Jul 2018 04:29:19 -0400
From:      Mark Johnston <markj@freebsd.org>
To:        Konstantin Belousov <kostikbel@gmail.com>
Cc:        Mihai Carabas <mihai.carabas@gmail.com>, freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org
Subject:   Re: Inspect pages created after a vm_object is marked as copy-on-write
Message-ID:  <20180701082919.GB3926@pesky.lan>
In-Reply-To: <20180630223401.GW2430@kib.kiev.ua>
References:  <CAGOCPLjrbv5nyXTQwqrsJhF9wFEFACeEyuS1aW0jdHycWNYz8g@mail.gmail.com> <20180629225209.GA4238@pesky.lan> <CANg1yUupb0hQb7jQHjD%2BvY8=3m4v%2BcH819qNGS7TjDUVFMgL=Q@mail.gmail.com> <20180630215956.GA1282@pesky.lan> <20180630223401.GW2430@kib.kiev.ua>

next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jul 01, 2018 at 01:34:01AM +0300, Konstantin Belousov wrote:
> On Sat, Jun 30, 2018 at 05:59:56PM -0400, Mark Johnston wrote:
> > On Sat, Jun 30, 2018 at 10:38:21AM +0300, Mihai Carabas wrote:
> > > On Sat, Jun 30, 2018 at 1:52 AM, Mark Johnston <markj@freebsd.org> wrote:
> > > > On Fri, Jun 29, 2018 at 11:58:31AM +0300, Elena Mihailescu wrote:
> > > >> Is there anything I am doing wrong? Maybe I misunderstood something about
> > > >> the way the virtual memory works in FreeBSD.
> > > >
> > > > I'll note that inspecting and manipulating vm_map_entry and vm_object
> > > > structures in the bhyve code constitutes something of an abstraction
> > > > violation, though it's reasonable to proceed this way while working on a
> > > > prototype of the feature.  That is, I think you should keep trying your
> > > > current approach, but just be aware that you are using the copy-on-write
> > > > mechanism in a way that the VM system isn't really expecting.
> > > >
> > > 
> > > Can you point out the right approach in our case?
> > 
> > I am merely suggesting that once the required VM interactions are fully
> > understood, the mechanism implemented for bhyve should be generalized
> > and lifted into the VM code.  It's hard to say what the "right" approach
> > is, since I don't fully understand the proposed algorithm.  It sounds
> > like you might be attempting something like:
> > 
> > 1. mark the mappings of to-be-migrated objects as NEEDS_COW, so that a
> >    subsequent write fault triggers creation of a shadow object
> It is actually MAP_ENTRY_COW | MAP_ENTRY_NEEDS_COPY.

Indeed.

> Note that setting an entry to COW changes the behaviour of mprotect(2),
> at least.
> 
> > 2. invalidate all physical mappings of pages in the object to be copied,
> >    so that subsequent writes trigger a fault
> I do not think this is needed to detect writes after the COW is set.
> It is enough to remove the write permissions.  Same as fork() does,
> see the vm_map_copy_entry() code for the handling of MAP_ENTRY_NEEDS_COPY
> case.

Ah, right.

> > 3. copy pages from the backing object to the destination
> As I understand, this is done right after the entry is marked
> as COW.
> 
> > 4. copy any pages from the shadow object to the desination
> And this is done after all backing data is copied and the process is
> suspended.

Right, I see.  Some reading suggests that in general we might perform
multiple iterations of this procedure before suspending the process and
performing any remaining copies.

> > 5. collapse the backing object into the shadow
> > 6. if the shadow object exists and was non-empty before the collapse,
> >    goto 1
> Are you trying to describe how to undo the COW marking ?  Marking an
> entry as COW really changes its semantic, and we do not need the undo
> operation in the base so far.  Collapsing the objects would lesser
> the pressure on the system pollution with objects, but it does not
> change back the meaning of mappings, e.g. their behaviour on inheritance
> on fork.

I was just thinking about how to avoid creating a long chain of objects
if multiple iterations are in fact needed.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180701082919.GB3926>