Date: Fri, 14 Jul 2023 16:07:03 -0800 From: Rob Wing <rob.fx907@gmail.com> To: Matthew Grooms <mgrooms@shrew.net> Cc: "virtualization@freebsd.org" <virtualization@freebsd.org> Subject: Re: BHYVE SNAPSHOT image format proposal Message-ID: <CAF3%2Bn_czuJ=B2mok2wh7OPvkzMz7%2B9KRG8_NuCyEL9SMFLt6tw@mail.gmail.com> In-Reply-To: <3973013d-c183-360f-d7ca-ca822859c23d@shrew.net> References: <67FDC8A8-86A6-4AE4-85F0-FF7BEF9F2F06@gmail.com> <6b98da58a5bd8e83bc466efa20b5a900298210aa.camel@FreeBSD.org> <8387AC83-6667-48E5-A3FA-11475EA96A5F@gmail.com> <d92db9bfbea181d6eb9d57b579d67e8e118ef4de.camel@FreeBSD.org> <986A83D8-E0E0-4030-9369-A5B3500E27C6@gmail.com> <79fabe94-b800-c713-48ea-518da1f74e4d@shrew.net> <CAF3%2Bn_cc5ZpGsKCff%2Bu-rSjnJn%2BN1jdu9KW0Y5b6n_TieMsfng@mail.gmail.com> <3973013d-c183-360f-d7ca-ca822859c23d@shrew.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000005c7d1d06007b5815 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The review you referenced requires multiple files for a snapshot, which I think is a non-starter. I was trying to ensure that the limitations described in the original snapshot commit message are addressed appropriately - but maybe I've mis-interpreted the message or that I don't fully understand the problem. I don't want give unhelpful or contradictory suggestions to what you've received in the past. The best I can do at this point is defer to those who you have made these agreements with. On Friday, July 14, 2023, Matthew Grooms <mgrooms@shrew.net> wrote: > On 7/14/23 13:08, Rob Wing wrote: > >> >> >> On Fri, Jul 14, 2023 at 9:40=E2=80=AFAM Matthew Grooms <mgrooms@shrew.ne= t >> <mailto:mgrooms@shrew.net>> wrote: >> >> >> I see a need to define a format for bhyve so it's possible to mix >> different sections and encodings inside a unified stream. But all th= e >> data in your nvlist example above can be easily be represented as >> text. >> We already have JSON, YAML, XML, etc ... By adopting an preexisting >> format, we could retain the snapshot structure instead of lifting it >> up >> into the stream format. Even if we decide to break the structure up >> into >> different nvlist stream sections, using a common format would allow >> other tools to more easily parse and validate the structure inside >> these >> sections. Isn't that a good thing? Is there a reason we're trying to >> reinvent the wheel here? >> >> >> Does JSON support storing binary data? I'm under the impression that it >> does not. >> > > Hi Rob, > > When we spoke to John Baldwin and others in the community about being abl= e > to remove the #ifdef's from the snapshot code, we were told that copying > binary data structures to a state file was the wrong approach and that a > better method needed to be provided. We agreed. That's why the following > work was undertaken to provide a rich file format that describes the > component values of device's state instead of codifying the c structure i= n > the file format ... > > https://reviews.freebsd.org/D29262 > > When Vitaliy added comments to that review WRT an nvlist based approach, = I > assumed that meant preserving the decomposition of device information tha= t > the UPB team spent so much effort trying to extract. I should have read t= he > original file format proposal email before I replied as I tried too hard = to > interpret it info through that lens. My mistake. > > If Vitaliy, and apparently you, favor the continued practice of copying > data from c structure pointers directly into files to save device state, = I > have no more comments on that approach. Maybe John or UPB will chime in > here with feedback that's more helpful. > > -Matthew > --0000000000005c7d1d06007b5815 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The review you referenced requires multiple files for a snapshot, which I t= hink is a non-starter.<div><br></div><div>I was trying to ensure that the l= imitations described in the original snapshot commit message are addressed = appropriately - but maybe I've mis-interpreted the message or that I do= n't fully understand the problem.</div><div><br></div><div>I don't = want give unhelpful or contradictory suggestions to what you've receive= d in the past.</div><div><br></div><div>The best I can do at this point is = defer to those who you have made these agreements with.<br><br>On Friday, J= uly 14, 2023, Matthew Grooms <<a href=3D"mailto:mgrooms@shrew.net">mgroo= ms@shrew.net</a>> wrote:<br><blockquote class=3D"gmail_quote" style=3D"m= argin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On 7/14/23 13= :08, Rob Wing wrote:<br> <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p= x #ccc solid;padding-left:1ex"> <br> <br> On Fri, Jul 14, 2023 at 9:40=E2=80=AFAM Matthew Grooms <<a href=3D"mailt= o:mgrooms@shrew.net" target=3D"_blank">mgrooms@shrew.net</a> <mailto:<a = href=3D"mailto:mgrooms@shrew.net" target=3D"_blank">mgrooms@shrew.net</a>&g= t;> wrote:<br> <br> <br> =C2=A0 =C2=A0 I see a need to define a format for bhyve so it's possibl= e to mix<br> =C2=A0 =C2=A0 different sections and encodings inside a unified stream. But= all the<br> =C2=A0 =C2=A0 data in your nvlist example above can be easily be represente= d as text.<br> =C2=A0 =C2=A0 We already have JSON, YAML, XML, etc ... By adopting an preex= isting<br> =C2=A0 =C2=A0 format, we could retain the snapshot structure instead of lif= ting it up<br> =C2=A0 =C2=A0 into the stream format. Even if we decide to break the struct= ure up<br> =C2=A0 =C2=A0 into<br> =C2=A0 =C2=A0 different nvlist stream sections, using a common format would= allow<br> =C2=A0 =C2=A0 other tools to more easily parse and validate the structure i= nside<br> =C2=A0 =C2=A0 these<br> =C2=A0 =C2=A0 sections. Isn't that a good thing? Is there a reason we&#= 39;re trying to<br> =C2=A0 =C2=A0 reinvent the wheel here?<br> <br> <br> Does JSON support storing binary data? I'm under the impression that it= does not.<br> </blockquote> <br> Hi Rob,<br> <br> When we spoke to John Baldwin and others in the community about being able = to remove the #ifdef's from the snapshot code, we were told that copyin= g binary data structures to a state file was the wrong approach and that a = better method needed to be provided. We agreed. That's why the followin= g work was undertaken to provide a rich file format that describes the comp= onent values of device's state instead of codifying the c structure in = the file format ...<br> <br> <a href=3D"https://reviews.freebsd.org/D29262" target=3D"_blank">https://re= views.freebsd.org/D2<wbr>9262</a><br> <br> When Vitaliy added comments to that review WRT an nvlist based approach, I = assumed that meant preserving the decomposition of device information that = the UPB team spent so much effort trying to extract. I should have read the= original file format proposal email before I replied as I tried too hard t= o interpret it info through that lens. My mistake.<br> <br> If Vitaliy, and apparently you, favor the continued practice of copying dat= a from c structure pointers directly into files to save device state, I hav= e no more comments on that approach. Maybe John or UPB will chime in here w= ith feedback that's more helpful.<br> <br> -Matthew<br> </blockquote></div> --0000000000005c7d1d06007b5815--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAF3%2Bn_czuJ=B2mok2wh7OPvkzMz7%2B9KRG8_NuCyEL9SMFLt6tw>