Date: Thu, 25 May 2023 21:40:04 +0300 From: Vitaliy Gusev <gusev.vitaliy@gmail.com> To: Mario Marietto <marietto2008@gmail.com> Cc: Tomek CEDRO <tomek@cedro.info>, virtualization@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: BHYVE SNAPSHOT image format proposal Message-ID: <D243949C-4C2E-4C6E-8961-7534DD2CCD00@gmail.com> In-Reply-To: <CA%2B1FSijKkCnR5j1N9BN0CbqzzitibK7r9OA84jc2eZ=uJaMW-g@mail.gmail.com> References: <67FDC8A8-86A6-4AE4-85F0-FF7BEF9F2F06@gmail.com> <CAFYkXjng1LWy5wVyTnSo0xrEWOy%2BOx9ZjLcmFqQs5EVpT8J_uA@mail.gmail.com> <AF34E648-2D8A-46C7-82A5-B88006BBB8F6@gmail.com> <CAFYkXjkUjh8gEMv4XZgb2QQW=qM1fhxMoMxRYuc4p6HbBXsDCw@mail.gmail.com> <8FE14143-1AA9-418E-A497-FEFB99BF6B9F@gmail.com> <CA%2B1FSijKkCnR5j1N9BN0CbqzzitibK7r9OA84jc2eZ=uJaMW-g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] > On 25 May 2023, at 19:22, Mario Marietto <marietto2008@gmail.com> wrote: > > Vitaliy, > > what happens if I clone your repo as source code on my FreeBSD system. Can I test your code directly or not ? Anyway,I think that,before doing this,I need to follow some kind of tutorial,to understand how the workflow is. Otherwise I will be not able to test and stress it. You should build kernel and tools, install it. Then you can run bhyve, bhyvectl to deal with suspend/resume. Please follow 9.5. Building and Installing a Custom Kernel https://docs.freebsd.org/en/books/handbook/book/#kernelconfig-building Make sure that BHYVE_SNAPSHOT is enabled. Also look at build(7): https://man.freebsd.org/cgi/man.cgi?build(7) > > On Thu, May 25, 2023 at 3:40 PM Vitaliy Gusev <gusev.vitaliy@gmail.com <mailto:gusev.vitaliy@gmail.com>> wrote: >> >> >>> On 25 May 2023, at 04:30, Tomek CEDRO <tomek@cedro.info <mailto:tomek@cedro.info>> wrote: >>> >>> On Wed, May 24, 2023 at 5:11 PM Vitaliy Gusev wrote: >>>> Protecting requires more efforts and it should be clearly defined: what is purpose. If >>>> purpose is having checksum with 99.9% reliability, NVLIST HEADER can be widen >>>> to have “checksum” key/value for a Section. >>> >>> Well, this could be optional but useful to make sure snapshot did not >>> break somehow for instance backup medium error or something like >>> that.. even more maybe a way to fix it.. just a design stage idea :- >> >> Yes, new format can have checksum of a Section data if implemented. >> >>> >>> >>>> If purpose is having crypto verification - I believe sha256 program should be your choice. >>> >>> My question was more specific to availability of that feature >>> (integrity + repair) rather than specific format :-) >>> >>> The use case here is having a virtual machine (it was VirtualBox) with >>> a bare os installed, plus some common applications, that is snapshoted >>> at some point in time, then experimented a lot, restored from >>> snapshot, etc. I had a backup of such vm + snapshot backed up that got >>> broken somehow. It would be nice to know that something is broken, >>> what is broken, maybe a way to fix :-) >> >> >> “Integrity" is a very broad term. What checksum algorithm is fine enough? >> >> For the instance, ZFS has several options for checksum: >> >> checksum=on|off|fletcher2|fletcher4|sha256|noparity|sha512|skein|edonr >> >> >> Having checksum for a filesystem is strongly recommended. However, If consider image format, >> it doesn’t need to care about consistency in a file itself. As example (!) - binary files in a system. >> They don’t have checksum integrated, validation is done by another program - pkg or another. >> >> >>> >>> >>>> Why do you need modify snapshot image ? Could you describe more? Do you >>>> modify current 3 snapshot files? >>> >>> Analysis that require ram / nvram modification? Not sure if this is >>> already possible, but may come handy for experimenting with uefi and >>> maybe some OS (features) that will not run with unmodified nvram :-P >> >> >> Sorry I don’t get, why do you need to modify snapshot image, but not directly vmem on the running >> VM? >> >> Another question, checksum and modifying image - two mutual exclusive things. >> >>> >>> >>>> If you are talking about compatibility of a Image format - it should be compatible in >>>> both directions, at least for not so big format changes. >>>> >>>> If consider overall snapshot/resume compatibility - I believe forward compatibility >>>> is not case and target. Indeed, why do you need to resume an image created by >>>> a higher version of a program? >>> >>> This happens quite often. For instance there is a bug in application >>> and I need to revert to (at least) one step older version. Then I am >>> unable to work on a file that I just saved (or was autosaved for me). >>> Firefox profile settings let be the first example. KiCAD file format >>> is another example (sometimes I need to switch to a devel build to >>> evade a nasty blocker bug then anyone else that uses a release is >>> blocked for some months including me myself). >> >> Any additional thing has a cost of development, testing and support. Current >> Implementation doesn’t support compatibility at all. Having compatibility in both >> directions can be hard. >> >> For example, if some variable is removed in bhyve, backward compatibility is fine, >> but forward compatibly is not possible unless that removed variable is being saved >> into a snapshot image just for forward compatibility. And of course, it should be tested >> and verified as worked. >> >> Do you like that approach? I don’t think so. So I guess only backward compatibility >> should be supported to make the snapshot code simple and robust. >> >> Thanks, >> Vitaliy Gusev >> >> > > > -- > Mario. [-- Attachment #2 --] <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br><div><br><blockquote type="cite"><div>On 25 May 2023, at 19:22, Mario Marietto <marietto2008@gmail.com> wrote:</div><br class="Apple-interchange-newline"><div><div dir="ltr"><div>Vitaliy,</div><div><br></div><div>what happens if I clone your repo as source code on my FreeBSD system. Can I test your code directly or not ? Anyway,I think that,before doing this,I need to follow some kind of tutorial,to understand how the workflow is. Otherwise I will be not able to test and stress it. <br></div></div></div></blockquote><div><br></div><div><br></div>You should build kernel and tools, install it. Then you can run bhyve, bhyvectl to deal with suspend/resume.</div><div><br></div><div>Please follow </div><div><br></div><div> <span style="caret-color: rgb(68, 68, 68); color: rgb(68, 68, 68); font-family: "inter var", -apple-system, BlinkMacSystemFont, "avenir next", avenir, "segoe ui", "helvetica neue", helvetica, Cantarell, Ubuntu, roboto, noto, arial, sans-serif; font-size: var(--size-550);"><b>9.5. Building and Installing a Custom Kernel</b></span></div><div><br></div><div><a href="https://docs.freebsd.org/en/books/handbook/book/#kernelconfig-building">https://docs.freebsd.org/en/books/handbook/book/#kernelconfig-building</a><br></div><div><br></div><div><br></div><div>Make sure that BHYVE_SNAPSHOT is enabled.</div><div><br></div><div>Also look at build(7):</div><div><br></div><div><a href="https://man.freebsd.org/cgi/man.cgi?build(7)">https://man.freebsd.org/cgi/man.cgi?build(7)</a></div><div><br></div><div><br><blockquote type="cite"><div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, May 25, 2023 at 3:40 PM Vitaliy Gusev <<a href="mailto:gusev.vitaliy@gmail.com">gusev.vitaliy@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><br><div><br><blockquote type="cite"><div>On 25 May 2023, at 04:30, Tomek CEDRO <<a href="mailto:tomek@cedro.info" target="_blank">tomek@cedro.info</a>> wrote:</div><br><div><div>On Wed, May 24, 2023 at 5:11 PM Vitaliy Gusev wrote:<br><blockquote type="cite">Protecting requires more efforts and it should be clearly defined: what is purpose. If<br>purpose is having checksum with 99.9% reliability, NVLIST HEADER can be widen<br>to have “checksum” key/value for a Section.<br></blockquote><br>Well, this could be optional but useful to make sure snapshot did not<br>break somehow for instance backup medium error or something like<br>that.. even more maybe a way to fix it.. just a design stage idea :-</div></div></blockquote><br>Yes, new format can have checksum of a Section data if implemented.</div><div><br><blockquote type="cite"><div><div><br><br><blockquote type="cite">If purpose is having crypto verification - I believe sha256 program should be your choice.<br></blockquote><br>My question was more specific to availability of that feature<br>(integrity + repair) rather than specific format :-)<br><br>The use case here is having a virtual machine (it was VirtualBox) with<br>a bare os installed, plus some common applications, that is snapshoted<br>at some point in time, then experimented a lot, restored from<br>snapshot, etc. I had a backup of such vm + snapshot backed up that got<br>broken somehow. It would be nice to know that something is broken,<br>what is broken, maybe a way to fix :-)<br></div></div></blockquote><div><br></div><div><br></div><div> “Integrity" is a very broad term. What checksum algorithm is fine enough?</div><div> </div><div>For the instance, ZFS has several options for checksum:</div><div><br></div></div><blockquote style="margin:0px 0px 0px 40px;border:medium none;padding:0px"><div><div><div style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; font-size: 12px; line-height: normal; font-family: Menlo; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-feature-settings: normal; background-color: rgb(231, 238, 238);"><span style="font-variant-ligatures:no-common-ligatures;color:rgb(231,238,238);background-color:rgb(0,0,0)"><b>checksum</b></span><span style="font-variant-ligatures:no-common-ligatures">=<b>on</b>|<b>off</b>|<b>fletcher2</b>|<b>fletcher4</b>|<b>sha256</b>|<b>noparity</b>|<b>sha512</b>|<b>skein</b>|<b>edonr</b></span></div></div></div><div><div><p style="margin:0px;font-style:normal;font-variant-caps:normal;font-stretch:normal;font-size:12px;line-height:normal;font-family:Menlo;font-size-adjust:none;font-kerning:auto;font-variant-alternates:normal;font-variant-ligatures:normal;font-variant-numeric:normal;font-variant-east-asian:normal;font-feature-settings:normal;background-color:rgb(231,238,238)"><span style="font-variant-ligatures:no-common-ligatures"> </span></p></div></div></blockquote><div><br></div><div>Having checksum for a filesystem is strongly recommended. However, If consider image format,</div><div>it doesn’t need to care about consistency in a file itself. As example (!) - binary files in a system.</div><div>They don’t have checksum integrated, validation is done by another program - pkg or another.</div><div><br></div><div><br></div><div><blockquote type="cite"><div><div><br><br><blockquote type="cite">Why do you need modify snapshot image ? Could you describe more? Do you<br>modify current 3 snapshot files?<br></blockquote><br>Analysis that require ram / nvram modification? Not sure if this is<br>already possible, but may come handy for experimenting with uefi and<br>maybe some OS (features) that will not run with unmodified nvram :-P<br></div></div></blockquote><div><br></div><div><br></div>Sorry I don’t get, why do you need to modify snapshot image, but not directly vmem on the running</div><div>VM?</div><div><br></div><div>Another question, checksum and modifying image - two mutual exclusive things. </div><div><br><blockquote type="cite"><div><div><br><br><blockquote type="cite">If you are talking about compatibility of a Image format - it should be compatible in<br>both directions, at least for not so big format changes.<br><br>If consider overall snapshot/resume compatibility - I believe forward compatibility<br>is not case and target. Indeed, why do you need to resume an image created by<br>a higher version of a program?<br></blockquote><br>This happens quite often. For instance there is a bug in application<br>and I need to revert to (at least) one step older version. Then I am<br>unable to work on a file that I just saved (or was autosaved for me).<br>Firefox profile settings let be the first example. KiCAD file format<br>is another example (sometimes I need to switch to a devel build to<br>evade a nasty blocker bug then anyone else that uses a release is<br>blocked for some months including me myself).<br></div></div></blockquote><div><br></div><div>Any additional thing has a cost of development, testing and support. Current</div><div>Implementation doesn’t support compatibility at all. Having compatibility in both</div><div>directions can be hard.</div><div><br></div><div>For example, if some variable is removed in bhyve, backward compatibility is fine,</div><div>but forward compatibly is not possible unless that removed variable is being saved</div><div>into a snapshot image just for forward compatibility. And of course, it should be tested</div><div>and verified as worked.</div><div><br></div><div>Do you like that approach? I don’t think so. So I guess only backward compatibility</div><div>should be supported to make the snapshot code simple and robust.</div><div><br></div></div><div>Thanks,</div><div>Vitaliy Gusev</div><div><br></div><div><br></div></div></blockquote></div><br clear="all"><br><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature">Mario.<br></div> </div></blockquote></div><br></body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D243949C-4C2E-4C6E-8961-7534DD2CCD00>