From owner-freebsd-virtualization@freebsd.org Fri Jun 29 08:59:03 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47C5B1035237; Fri, 29 Jun 2018 08:59:03 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: from mail-oi0-x22c.google.com (mail-oi0-x22c.google.com [IPv6:2607:f8b0:4003:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4E3782BC8; Fri, 29 Jun 2018 08:59:02 +0000 (UTC) (envelope-from elenamihailescu22@gmail.com) Received: by mail-oi0-x22c.google.com with SMTP id w126-v6so517819oie.7; Fri, 29 Jun 2018 01:59:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to:cc; bh=btaGFOIewTKa9yTmziEpM/4891mrOI3TGYGhUw7YPEs=; b=WHPN8I0fDlxS+IrbEMfuTdylWED+yGUkqXSH9Q8veuJyEG4go7sO6YrCWmslhnEkqA Sg6P3s+3GiokpDLwVUxoZtNjgfL9Ci6hWTTvAERjbH4LBiMNzj0kkK7+1tqyd6GNMW/J niNNdCicublQ68y11VVRvRwPP3QvwBHiTMIdUfUM+ZNz5VThyof/9ZGijkTPNXJLJFch gf6QlhHtYgwXpoykqhn1fjMOu/T0TDvde2JOKdWVSnSB4PqK55vd3i2YSQO0g7gMP3NZ GBZr+9fXUxrdG6ZV30tIgcqqEcUY1+57pD1KOV5oQ8Gw7yAvRbU3SknmbWquNqJKOu6N Xerw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc; bh=btaGFOIewTKa9yTmziEpM/4891mrOI3TGYGhUw7YPEs=; b=oqgXXdLKDcb+0X1srQCgpltpTOgm/l738cC5b5Fdx3xt1Q8khFHmciKZYFigtsH0D3 VdNhEYzH2ZvE7capCO4DaxLS1j6OKTuuSSvPcjCcFgxU5YMShdZMUqWSflFZrfXY+P9p p0pXTdgYSzBt3TjVSU7edb5ryYys0bHZANqgnLDWsn3WHoBwyDJBq9Bw10uRcriujO5P rPCDvVvs4a8n0QB3A43Xh/MhKP8EhmevNyATlKSrv3jbcsoKofwMaH4bgH6JGACPQkqF 9NFr60plEXBVigu5/09fPt4xhmioY06Qh30VJHMAC8wWHflKGk3mxPPp266uMgn47ixZ RfEQ== X-Gm-Message-State: APt69E0ZvFRZxJ8zqQdlGUiqCI0CBl1FsYkGfzV7/bHGCiW7h4Sj1nK0 FKcH4weaLz6uvgxH/STX/zFppYt5ABLhuc38qgVgyQ== X-Google-Smtp-Source: AAOMgpeNkLg+B1jKTyyHZE6S0vKqsuyf606meQWvDo/5xH6FyrFf3wefAfpAIyyTnTf8UUjAnCiVAM3nUPu1xJkVbjw= X-Received: by 2002:aca:f409:: with SMTP id s9-v6mr8162984oih.102.1530262741744; Fri, 29 Jun 2018 01:59:01 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a9d:3c4e:0:0:0:0:0 with HTTP; Fri, 29 Jun 2018 01:58:31 -0700 (PDT) From: Elena Mihailescu Date: Fri, 29 Jun 2018 11:58:31 +0300 Message-ID: Subject: Inspect pages created after a vm_object is marked as copy-on-write To: freebsd-amd64@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jun 2018 08:59:03 -0000 Hello, I am interested if there is a method to inspect what pages/objects were created after a vm_object (the vm_map_entry associated with the object) is marked as copy-on-write. More specifically, I'm interested only in the pages that were copied when a write operation was proceed on a page that belongs to the object marked copy-on-write. I need this for a live migration feature for bhyve in order to send the pages that were modified between the iterations in which I migrate the guest's memory(the guest's memory will be migrated in rounds - firstly, all memory will be sent remote, then, only the pages that were modified and so on). What I want to implement is the following: Step 1: Given a vm_object *obj, mark its associated vm_map_entry *entry as copy-on-write. Step 2: After a while (a non-deterministic amount of time), inspect/retrieve the pages that were created based on information existent in the object. What I tried until now: I implemented a function in kernel that: - gets the vmspace structure pointer for the current process - gets the vm_map structure pointer for the vmspace - iterates through each vm_map_entry and based on the vm_offset_start and vm_offset_end determines vm_map_entry that contains the object I am interested in. - for this object, it prints some debug information such as: shadow_count, ref_count, whether if it has a backing_object or not. The code written is similar with the code from here (the way in which I get vmspace for the current process and the way I am iterating through vm_map_entry and objects): [0] https://github.com/FreeBSD-UPB/freebsd/blob/projects/bhyve_migration/sys/amd64/vmm/vmm_dev.c#L979 I have read the following documentation about FreeBSD's implementation for virtual memory: [1] https://www.freebsd.org/doc/en/books/arch-handbook/vm.html [2] https://www.freebsd.org/doc/en_US.ISO8859-1/articles/vm-design/article.html [3] https://people.freebsd.org/~neel/bhyve/bhyve_nested_paging.pdf [4] http://www.cse.chalmers.se/edu/year/2011/course/EDA203/unix4.pdf As far as I could tell after reading the documentation presented above, I should look for the object that the object I am interested in is a shadow of or an object that my object is shadow for. To do that, I should inspect the following fields from the vm_object structure (among others)( https://github.com/freebsd/freebsd/blob/master/sys/vm/vm_object.h#L98) : - int shadow_count; /* how many objects that this is a shadow for */ - struct vm_object *backing_object; /* object that I'm a shadow of */ But in all my tests, for the object I am interested in, the shadow_count is 0 and the backing_object is NULL. The code I use to mark the vm_map_entry for the object I am interested in copy-on-write is here: [5] https://github.com/FreeBSD-UPB/freebsd/blob/projects/bhyve_migration/sys/amd64/vmm/vmm_dev.c#L949 Is there anything I am doing wrong? Maybe I misunderstood something about the way the virtual memory works in FreeBSD. There is another way I could inspect what pages were created between the moment I mark an object (its vm_map_entry) as copy-on-write and a later moment? Thank you, Elena