Date: Fri, 29 Jul 2016 10:29:05 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com> To: Wei Liu <wei.liu2@citrix.com> Cc: Karl Pielorz <kpielorz_lst@tdx.co.uk>, "Hoyer-Reuther, Christian" <Christian.Hoyer-Reuther@cac-chem.de>, <freebsd-xen@freebsd.org> Subject: Re: 'Live' Migrate messes up NTP on FreeBSD domU - any suggestions? Message-ID: <20160729082905.46js7o3zp6iwuibd@mac> In-Reply-To: <20160725153714.GW27082@citrix.com> References: <41E487BC91654544B2B8F31096F2D9D4D1514D1D8E@ex1> <20160714103016.4hgfzsjgkkgtkkgg@mac> <41E487BC91654544B2B8F31096F2D9D4D1514D1E88@ex1> <20160720093111.mpmp27wol7j3ge3d@mac> <41E487BC91654544B2B8F31096F2D9D4D1516490E9@ex1> <20160722115542.dopzb63dgkilqall@mac> <FA258C50EA60D4DE44D9289C@[10.12.30.106]> <20160725144314.yhggviqhsqzgux2w@mac> <20160725153714.GW27082@citrix.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jul 25, 2016 at 04:37:14PM +0100, Wei Liu wrote:
> On Mon, Jul 25, 2016 at 04:43:43PM +0200, Roger Pau Monné wrote:
> > Adding Wei to the Cc list since he added the multiqueue functionality.
> >
> > On Mon, Jul 25, 2016 at 02:59:02PM +0100, Karl Pielorz wrote:
> > >
> > > --On 22 July 2016 13:55 +0200 Roger Pau Monné <roger.pau@citrix.com> wrote:
> > >
> > > > In my environment I've migrated a FreeBSD VM with 2 cpus for > 100
> > > > consecutive times without seeing any issues (or freezes), although this
> > > > was with OSS Xen and without xe-guest-utilities. Karl, have you tested
> > > > HEAD recently?
> > >
> > > Ok, I have tested this with r303286 - it seems to work OK. The hosts gain no
> > > time that I can see while migrating, and NTP stays happy.
> > >
> > > I did get a panic after about 40 migrations - but that seems to be some
> > > network issue or something...
> > >
> > > ('panic called with 0 available queues / dbt_trace_self_wrapper / vpanic /
> > > kassert_panic / xn_txq_mq_start / ether_output / udp_send / sosend_dgram /
> > > kern_sendit / sendit / sys_sendto / amd64_syscall / Xfast_syscall)
> >
> > I haven't been able to reproduce this, but I think it's possible that if you
> > migrate an active netfront xn_txq_mq_start might be called during the
> > migration, just in the middle of the setup_device reconfiguation (while
> > info->num_queues is 0).
> >
> > Wei, I think netif_disconnect_backend should set IFF_DRV_OACTIVE in order to
> > notify the net subsystem that the queues are full, so no further calls to
> > xn_txq_mq_start happen until the resume has finished, do you agree?
> >
>
> Perhaps clear IFF_DRV_RUNNING and only set it when the device is ready?
> Looking at the manpage is seems more appropriate to me semantically.
Hello Karl and Christian, I have the following patches that solve all the
issues I've seen with live migration, with those I've been able to migrate a
VM > 100 times without seeing any issues. Could you give them a try?
BTW, I haven't been able to reproduce Karl's crash ("called with 0 available
queues"), but I've added a condition that should prevent it from triggering
anyway. Patches are here:
https://reviews.freebsd.org/D7349
https://reviews.freebsd.org/D7362
https://reviews.freebsd.org/D7363
It doesn't really matter in which order you apply them as long as both 3 are
applied. Ideally I would like to commit them on Monday, so that I can MFC
them to stable/11 before the releng/11 branch, could you please provide some
feedback before then?
Thanks, Roger.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20160729082905.46js7o3zp6iwuibd>