Date: Thu, 28 Jan 2021 01:42:00 +0100 From: Willem Jan Withagen <wjw@digiware.nl> To: Matt Churchyard <matt.churchyard@userve.net>, "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: Re: Warm Migration feature for bhyve - review on Phabricator Message-ID: <5c6ff016-e15d-afad-e855-8cfed32276f2@digiware.nl> In-Reply-To: <20210125184204.GM31099@funkthat.com> References: <CAGOCPLhP3i4hPcKHX=RrKFbQiSenjtJrvzTx%2BARnP8=QN%2BzmqQ@mail.gmail.com> <e9a65e1ad4e0491083820fbb03d873e8@SERVER.ad.usd-group.com> <20210125062113.GL31099@funkthat.com> <7769d16522fa49558049f4a6e936982e@SERVER.ad.usd-group.com> <20210125184204.GM31099@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 25-1-2021 19:42, John-Mark Gurney wrote: > Matt Churchyard wrote this message on Mon, Jan 25, 2021 at 10:46 +0000: >> -----Original Message----- >> From: John-Mark Gurney <jmg@funkthat.com> >> Sent: 25 January 2021 06:21 >> To: Matt Churchyard <matt.churchyard@userve.net> >> Cc: Elena Mihailescu <elenamihailescu22@gmail.com>; freebsd-virtualization@freebsd.org >> Subject: Re: Warm Migration feature for bhyve - review on Phabricator >> >> Matt Churchyard wrote this message on Fri, Jan 22, 2021 at 10:09 +0000: >>>> Hello, all, >>>> We have recently opened a review on Phabricator for the warm migration code for > bhyve [1]. Please take a look and let us know if it is anything we can improve. >>>> [1] https://reviews.freebsd.org/D28270 >>>> Thank you, >>>> Elena >>> I appreciate that this isn't really related to the current review, and commend the work being put into bhyve - it's an invaluable addition to FreeBSD. I'm just wondering if any thought has been put into the future possibility for transfer of disk data during migration (i.e. the equivalent of "storage vmotion") >>> >>> The current process (from a mile high overview) effectively seems to be the following - >>> >>> * Start guest on host2 pointing at a shared disk, and halt any execution >>> * use bhyvectl to pause and begin migration on host1 >>> * Start the guest on host2 >>> >>> What would be the feasibility of being able to run a process such as the following? Obviously it would likely need to be orchestrated by an external tool, but to me it seems the main requirement is really just to be able to provide separate control over the pause and migrate steps on host1 - >>> >>> * send a ZFS snapshot of the running machine to host2 >>> * start the guest in migrate recv mode on host2 >>> * pause the guest on host1 >>> * send a new snapshot >>> * initiate the migration of memory/device data >>> * start guest on host2 >>> >>> Are there any major complications here I'm not aware of other than the requirement to pause the guest and kick off the state migration as two separate calls? >>> There's also hastd which can aid with this... >> Thanks for the reply. I've always been wary of the additional complexity of HAST and ZFS, as it doesn't seem to have widespread usage or support, and things get ugly fast when storage is involved. > Totally agree... > >> However, the idea of using HAST on top of zvols to provide network mirrored storage for a guest is interesting. It adds a lot of extra complexity, and probably performance impact though if it's just for the ability to move a guest between systems that may only happen every now and then. I'm also not sure it would help (or would at least be even more complex) if I have 4 hosts and want to be able to move guests anywhere. > gmirror + ggate is another option as well... I did try all kinds of these solutions that are based on a 2 system backend storage, and it was relatively easy for me to get the state of the storage in a split-brain situation which is not really repairable without a lot of manual user intervention. So I gave up on HAST and gmirror over ggate. --WjW
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5c6ff016-e15d-afad-e855-8cfed32276f2>