Date: Tue, 6 Jun 2023 14:25:38 +0300 From: Vitaliy Gusev <gusev.vitaliy@gmail.com> To: =?utf-8?Q?Corvin_K=C3=B6hne?= <corvink@FreeBSD.org> Cc: virtualization@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: BHYVE SNAPSHOT image format proposal Message-ID: <8387AC83-6667-48E5-A3FA-11475EA96A5F@gmail.com> In-Reply-To: <6b98da58a5bd8e83bc466efa20b5a900298210aa.camel@FreeBSD.org> References: <67FDC8A8-86A6-4AE4-85F0-FF7BEF9F2F06@gmail.com> <6b98da58a5bd8e83bc466efa20b5a900298210aa.camel@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_A8750554-1230-4B80-AEEC-C03D927B7A32 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi Corvin,=20 Thanks for your comments and advices.=20 Answers are below, > On 5 Jun 2023, at 18:32, Corvin K=C3=B6hne <corvink@FreeBSD.org> = wrote: >=20 > On Tue, 2023-05-23 at 19:05 +0300, Vitaliy Gusev wrote: >> 2. HEADER PHYS format:=20 >>=20 >> 0 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+=20= >> | IDENT STRING - 64 BYTES | >> 64 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ =20= >> | NVLIST SIZE - 4 BYTES | NVLIST DATA | >> 72 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ >> | | >> | NVLIST DATA | >> | | >> 4096 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ >>=20 >>>=20 >>> IDENT STRING - Each producer can set its own value to specify >>> image. >>> NVLIST SIZE - The following packed header nvlist data size. >>> NVLIST DATA - Packed nvlist header data. >>>=20 >>> 4KB should be enough for the HEADER to keep basic information about >>> Sections. However, it can >>> be enlarged lately, without breaking backward compatibility.=20 >>>=20 >=20 > I can't see an advantage of using a fixed sized header of 4KB. You = have > to parse the offset and size of every section anyways. If it's for > alignment requirements you can still align all sections on save and = set > the offset accordingly. So, why complicating things by using a fixed > header size? You are right about 4KB restriction. I will correct it in updated format = proposal. Idea is to reserve enough space for HEADER and write it after all finished = stages at the beginning=20 of a snapshot file. Implementation (snapshot path) should know estimated maximum size of the = header and can use the possible maximum. Currently 4KB is enough and easily can be increased in the bhyve=E2=80=99s code without any problem.=20 Alignment is useful to debug and looking into snapshot image file. >=20 > The IDENT STRING seems to be very large. Even a GUID which should be a > global unique identifier uses just 16 Bytes. Additionally, it might be > worth using a dedicated ident and version field for an easier version > parsing. E.g.: Intention is to add enough space for the future version (as reservation) = and other producers and companies to specify it=E2=80=99s own ID string with possible add-on = information. So adding 64 bytes for the future is not so huge pay, but can be very useful. During resume, if IDENT string is not the same as in bhyve, resume can = fail before parsing other data, because it could be that internal format is not as expected. I would not to fix IDENT string format and just apply rule: During resume, bhyve compares its own IDENT string and IDENT string from = an Snapshot image. If it is not the same, further assumption about format = cannot be done, and resume should fail. >=20 > +------------------+-------------------+ > | IDENT - 56 BYTES | VERSION - 8 BYTES | > +------------------+-------------------+ >=20 > IDENT - "BHYVE CHECKPOINT IMAGE" > VERSION - 1 (as uint64_t) >=20 > Btw: I don't care but here we could leave some free space for possible > forward compatibility. E.g.: >=20 > +------------------+-------------------+-------------------------+ > | IDENT - 16 BYTES | VERSION - 8 BYTES | _FREE_SPACE_ - 40 BYTES | > = +------------------+-------------------+=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ ... >> 4. EXAMPLE: >>=20 >>=20 >> IDENT STRING: >>=20 >> "BHYVE CHECKPOINT IMAGE VERSION 1" >>=20 >> NVLIST HEADER:=20 >>=20 >> [config] >> config.offset =3D 0x1000 (4096) >> config.size =3D 0x1f6 (502) >> config.type =3D "text" >>=20 >=20 > Not sure if it's just an example for the "text" type. bhyve converts = it > into a nvlist, so it could be saved directly as nvlist. > Btw: I would only implement the "text" type if there's an usecase that > can't be solved by one of the other types. Intention is to use current engine to dump bhyve=E2=80=99s config and = read config from a file (-k option). Advantage of using =E2=80=9Ctext=E2=80=9D type - simple implementation = and as an example of flexibility of proposed image format. Image file can keep any types = that a producer would like to use: text, nvlist, binary, diff-pages, etc. >=20 > All in all, it looks good. Keep on your work! >=20 > Regards checksum feature: > We should focus on enabling this feature by default before adding > advanced features. So, keep it simple and small. Could you give a more example what you meant about =E2=80=9Cchecksum=E2=80= =9D feature? Did you mean as TAR=E2=80=99s checksum, i.e. only header? >=20 > Regards forward compatibility: > Backward compatibility is way more important than forward > compatibility. Nevertheless, forward compatibility would be nice to > have. So, we should keep it in mind when modifying the layout. For the > moment, just focus on a format which is backward compatible. >=20 It seems that having information about forward compatibility could be = very useful, at least to get it in advance if it is impossible to restore. I = will add it during implementing this format. Thanks, Vitaliy Gusev --Apple-Mail=_A8750554-1230-4B80-AEEC-C03D927B7A32 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;">Hi = Corvin, <div><br></div><div>Thanks for your comments and = advices. </div><div><br></div><div>Answers are = below,<br><div><div><br><blockquote type=3D"cite"><div>On 5 Jun 2023, at = 18:32, Corvin K=C3=B6hne <corvink@FreeBSD.org> wrote:</div><br = class=3D"Apple-interchange-newline"><div><meta charset=3D"UTF-8"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">On Tue, 2023-05-23 at 19:05 +0300, Vitaliy = Gusev wrote:</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><blockquote type=3D"cite" style=3D"font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;">2. HEADER PHYS = format: </blockquote><blockquote type=3D"cite" style=3D"font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><br> 0 = = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= + <br> <span = class=3D"Apple-converted-space"> </span>| = IDENT STRING - 64 BYTES = |<br> 64 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+ = <br> <span = class=3D"Apple-converted-space"> </span>| NVLIST SIZE - 4 = BYTES | NVLIST DATA |<br> 72 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+<br>= <span class=3D"Apple-converted-space"> </span>| = = = |<br> <span = class=3D"Apple-converted-space"> </span>| = NVLIST DATA = |<br> <span = class=3D"Apple-converted-space"> </span>| = = |<br> 4096 = +=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94+<br>= <br><blockquote type=3D"cite"><br>IDENT STRING - Each producer can set = its own value to specify<br>image.<br>NVLIST SIZE - The following = packed header nvlist data size.<br>NVLIST DATA - Packed nvlist header = data.<br><br>4KB should be enough for the HEADER to keep basic = information about<br>Sections. However, it can<br>be enlarged lately, = without breaking backward = compatibility. <br><br></blockquote></blockquote><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">I can't see an advantage of using a fixed = sized header of 4KB. You have</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">to = parse the offset and size of every section anyways. If it's = for</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">alignment requirements you can still align all sections on = save and set</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">the = offset accordingly. So, why complicating things by using a = fixed</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">header = size?</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"></div></blockquote><div><br></div><div>You are = right about 4KB restriction. I will correct it in updated format = proposal. Idea is</div><div>to reserve enough space for HEADER and write = it after all finished stages at the beginning </div><div>of a = snapshot file.</div><div><br></div><div>Implementation (snapshot path) = should know estimated maximum size of the header and can</div><div>use = the possible maximum. Currently 4KB is enough and easily can = be</div><div>increased in the bhyve=E2=80=99s code without any = problem. </div><div><br></div>Alignment is useful to debug and = looking into snapshot image file.</div><div><br><blockquote = type=3D"cite"><div><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">The = IDENT STRING seems to be very large. Even a GUID which should be = a</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">global unique identifier = uses just 16 Bytes. Additionally, it might be</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">worth using a dedicated ident and version = field for an easier version</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">parsing. E.g.:</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"></div></blockquote><div><br></div><div>Intention = is to add enough space for the future version (as reservation) and other = producers</div><div>and companies to specify it=E2=80=99s own ID string = with possible add-on information. So adding 64 bytes</div><div>for = the future is not so huge pay, but can be very = useful.</div><div><br></div><div>During resume, if IDENT string is not = the same as in bhyve, resume can fail before parsing</div><div>other = data, because it could be that internal format is not as = expected.</div><div><br></div><div>I would not to fix IDENT string = format and just apply = rule:</div><div><br></div></div></div></div><blockquote style=3D"margin: = 0 0 0 40px; border: none; padding: 0px;"><div><div>During resume, bhyve = compares its own IDENT string and IDENT string from = an</div><div>Snapshot image. If it is not the same, further assumption = about format cannot be done,</div><div>and resume should = fail.</div></div></blockquote><div><div><div><br></div><blockquote = type=3D"cite"><div><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">+------------------+-------------------+</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">| IDENT - 56 BYTES | VERSION - 8 BYTES = |</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline = !important;">+------------------+-------------------+</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">IDENT - "BHYVE CHECKPOINT IMAGE"</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">VERSION - 1 (as uint64_t)</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">Btw: I don't care but here we could leave = some free space for possible</span><br style=3D"caret-color: rgb(0, 0, = 0); font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">forward = compatibility. E.g.:</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">+------------------+-------------------+---------------------= ----+</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">| IDENT = - 16 BYTES | VERSION - 8 BYTES | _FREE_SPACE_ - 40 BYTES |</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline = !important;">+------------------+-------------------+=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94+</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"></div></blockquote>...<br><blockquote = type=3D"cite"><div><blockquote type=3D"cite" style=3D"font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;">4. = EXAMPLE:<br><br><br> IDENT STRING:<br><br> = "BHYVE CHECKPOINT IMAGE VERSION 1"<br><br> NVLIST = HEADER: <br><br> <span = class=3D"Apple-converted-space"> </span>[config]<br> = <span = class=3D"Apple-converted-space"> </span>config.offset =3D 0x1000 = (4096)<br> <span = class=3D"Apple-converted-space"> </span>config.size =3D 0x1f6 = (502)<br> <span = class=3D"Apple-converted-space"> </span>config.type =3D = "text"<br><br></blockquote><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">Not = sure if it's just an example for the "text" type. bhyve converts = it</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;"><span style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 13.999999px; font-style: normal; font-variant-caps: normal; = font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none; float: none; display: inline !important;">into a nvlist, so it = could be saved directly as nvlist.</span><br style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13.999999px; = font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: = none;"></div></blockquote><blockquote type=3D"cite"><div><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">Btw: I would only implement the "text" type = if there's an usecase that</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">can't = be solved by one of the other types.</span><br style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 13.999999px; = font-style: normal; font-variant-caps: normal; font-weight: 400; = letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: = none;"></div></blockquote><div><br></div><div><br></div><div><div>Intentio= n is to use current engine to dump bhyve=E2=80=99s config and read = config</div><div>from a file (-k = option).</div><div><br></div><div>Advantage of using =E2=80=9Ctext=E2=80=9D= type - simple implementation and as an example</div><div>of flexibility = of proposed image format. Image file can keep any types that</div><div>a = producer would like to use: text, nvlist, binary, diff-pages, = etc.</div></div><br><blockquote type=3D"cite"><div><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">All in all, it looks good. Keep on your = work!</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">Regards = checksum feature:</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">We = should focus on enabling this feature by default before adding</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">advanced features. So, keep it simple and = small.</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"></div></blockquote><div><br></div><div>Could you = give a more example what you meant about =E2=80=9Cchecksum=E2=80=9D = feature? Did you mean as</div><div>TAR=E2=80=99s checksum, i.e. only = header?</div><div><br></div><br><blockquote type=3D"cite"><div><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">Regards forward compatibility:</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;">Backward compatibility is way more = important than forward</span><br style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline = !important;">compatibility. Nevertheless, forward compatibility would be = nice to</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">have. = So, we should keep it in mind when modifying the layout. For = the</span><br style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 13.999999px; font-style: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;">moment, = just focus on a format which is backward compatible.</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;"><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 13.999999px; font-style: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: = none;"></div></blockquote><div><br></div><div>It seems that having = information about forward compatibility could be very</div><div>useful, = at least to get it in advance if it is impossible to restore. I will add = it during</div><div>implementing this = format.</div><div><br></div></div><div>Thanks,</div><div>Vitaliy = Gusev</div><div><br></div></div></body></html>= --Apple-Mail=_A8750554-1230-4B80-AEEC-C03D927B7A32--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8387AC83-6667-48E5-A3FA-11475EA96A5F>