From owner-freebsd-xen@freebsd.org Fri Jul 29 08:29:14 2016 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4861FBA88D4 for ; Fri, 29 Jul 2016 08:29:14 +0000 (UTC) (envelope-from prvs=011b0443a=roger.pau@citrix.com) Received: from SMTP02.CITRIX.COM (smtp02.citrix.com [66.165.176.63]) (using TLSv1.2 with cipher RC4-SHA (128/128 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D07691A16 for ; Fri, 29 Jul 2016 08:29:13 +0000 (UTC) (envelope-from prvs=011b0443a=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.28,438,1464652800"; d="scan'208";a="376245728" Date: Fri, 29 Jul 2016 10:29:05 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= To: Wei Liu CC: Karl Pielorz , "Hoyer-Reuther, Christian" , Subject: Re: 'Live' Migrate messes up NTP on FreeBSD domU - any suggestions? Message-ID: <20160729082905.46js7o3zp6iwuibd@mac> References: <41E487BC91654544B2B8F31096F2D9D4D1514D1D8E@ex1> <20160714103016.4hgfzsjgkkgtkkgg@mac> <41E487BC91654544B2B8F31096F2D9D4D1514D1E88@ex1> <20160720093111.mpmp27wol7j3ge3d@mac> <41E487BC91654544B2B8F31096F2D9D4D1516490E9@ex1> <20160722115542.dopzb63dgkilqall@mac> <20160725144314.yhggviqhsqzgux2w@mac> <20160725153714.GW27082@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20160725153714.GW27082@citrix.com> User-Agent: Mutt/1.6.2-neo (2016-06-11) X-DLP: MIA1 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 08:29:14 -0000 On Mon, Jul 25, 2016 at 04:37:14PM +0100, Wei Liu wrote: > On Mon, Jul 25, 2016 at 04:43:43PM +0200, Roger Pau Monné wrote: > > Adding Wei to the Cc list since he added the multiqueue functionality. > > > > On Mon, Jul 25, 2016 at 02:59:02PM +0100, Karl Pielorz wrote: > > > > > > --On 22 July 2016 13:55 +0200 Roger Pau Monné wrote: > > > > > > > In my environment I've migrated a FreeBSD VM with 2 cpus for > 100 > > > > consecutive times without seeing any issues (or freezes), although this > > > > was with OSS Xen and without xe-guest-utilities. Karl, have you tested > > > > HEAD recently? > > > > > > Ok, I have tested this with r303286 - it seems to work OK. The hosts gain no > > > time that I can see while migrating, and NTP stays happy. > > > > > > I did get a panic after about 40 migrations - but that seems to be some > > > network issue or something... > > > > > > ('panic called with 0 available queues / dbt_trace_self_wrapper / vpanic / > > > kassert_panic / xn_txq_mq_start / ether_output / udp_send / sosend_dgram / > > > kern_sendit / sendit / sys_sendto / amd64_syscall / Xfast_syscall) > > > > I haven't been able to reproduce this, but I think it's possible that if you > > migrate an active netfront xn_txq_mq_start might be called during the > > migration, just in the middle of the setup_device reconfiguation (while > > info->num_queues is 0). > > > > Wei, I think netif_disconnect_backend should set IFF_DRV_OACTIVE in order to > > notify the net subsystem that the queues are full, so no further calls to > > xn_txq_mq_start happen until the resume has finished, do you agree? > > > > Perhaps clear IFF_DRV_RUNNING and only set it when the device is ready? > Looking at the manpage is seems more appropriate to me semantically. Hello Karl and Christian, I have the following patches that solve all the issues I've seen with live migration, with those I've been able to migrate a VM > 100 times without seeing any issues. Could you give them a try? BTW, I haven't been able to reproduce Karl's crash ("called with 0 available queues"), but I've added a condition that should prevent it from triggering anyway. Patches are here: https://reviews.freebsd.org/D7349 https://reviews.freebsd.org/D7362 https://reviews.freebsd.org/D7363 It doesn't really matter in which order you apply them as long as both 3 are applied. Ideally I would like to commit them on Monday, so that I can MFC them to stable/11 before the releng/11 branch, could you please provide some feedback before then? Thanks, Roger.