From owner-freebsd-xen@freebsd.org Wed Nov 18 19:58:43 2020 Return-Path: Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 108E747103B for ; Wed, 18 Nov 2020 19:58:43 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cbtss6FmKz4b1f for ; Wed, 18 Nov 2020 19:58:41 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (localhost [127.0.0.1]) by nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id 0AIJwR8V028850 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Nov 2020 11:58:27 -0800 (PST) Received: (from buhrow@localhost) by nfbcal.org (8.15.2/8.12.11) id 0AIJwRN1007886; Wed, 18 Nov 2020 11:58:27 -0800 (PST) Message-Id: <202011181958.0AIJwRN1007886@nfbcal.org> From: Brian Buhrow Date: Wed, 18 Nov 2020 11:58:27 -0800 In-Reply-To: <20201117092104.usulqtvcvyv7mcji@Air-de-Roger> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Roger Pau =?utf-8?B?TW9ubsOp?= Subject: Re: ZFS corruption using zvols as backingstore for hvm VM's Cc: , buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nfbcal.org [127.0.0.1]); Wed, 18 Nov 2020 11:58:28 -0800 (PST) X-Rspamd-Queue-Id: 4Cbtss6FmKz4b1f X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@nfbcal.org X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_TLS_ALL(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[157.22.230.125:from]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; ARC_NA(0.00)[]; SPAMHAUS_ZRD(0.00)[157.22.230.125:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-xen] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Nov 2020 19:58:43 -0000 hello Roger. Sorry for the confusion. I've been running NetBSD, or trying to, in both PV and HVM modes. Here is a mesage I sent to the port-xen@netbsd.org mailing list a few days ago, detailing what I found with running NetBSD in any mode under Xen as a domu. Unfortunately, I don't think I can provide a trace of a crash to the xen server when the HVM host misbehaves because I've had to move on in my work to get things working and I found a working combination of NetBSD and ZFS that let's me proceed. I've seen some discussion in the Xen documentation that suggests one can run a Xen dom0 in a domu guest, allowing for sand box testing of what you're describing without having to devote hardware to the issue. Am I correct in this and where might I find instructions on how to do it? In any case, here's what I found. -thanks -Brian From: Brian Buhrow Date: Mon, 16 Nov 2020 14:59:48 -0800 Subject: Re: Panic: HYPERVISOR_mmu_update failed on old NetBSD-5.2 domU Cc: port-xen@netbsd.org, buhrow@nfbcal.org hello Greg. thanks for the pointers on Xen changes over the years and the reasons for the changes. After a lot more testing, thought and cvs diffing, here is the state of the world, as I understand it, for the record. NetBSD-5/i386 won't run under Xen-4.12 or newer, possibly older versions of Xen, I didn't test, because it still has calls to xpq_queue_flush and the like which were used in pre-xen3 days. I believe, through code inspection, but didn't test, that this is also true of NetBSD-6/i386 and NetBSD-7/i386. My next thought was to use NetBSD-5 in HVM mode, but it turns out this is a show stopper for versions of NetBSD all the way through -current as of May 24, 2020, see the link below, due to a cascade of bugs in the piixide(4) emulator in Qemu and NetBSD-s inability to deal with partial success in writing to disks on that chip set. This failure is what causes the original disk corruption I wrote about at the beginning of this thread. As an alternative, I tried enabling the SCSI LSI emulator in Qemu, but this doesn't work because our esiop(4) driver doesn't play well with the emulated LSI chips in Qemu. No disk corruption, just timeouts when writing to attached emulated sd(4) devices. So, NetBSD as an HVM guest is pretty much a bust! Fortunately, there is a solution. NetBSD-5.2/amd64 DOMU's work fine under Xen-4.12 and above. So, I'll run the 64-bit amd-64 base, with a 32-bit pkgsrc set of packages to allow me to migrate my busy, working machine, to newere hardware as a VM, thereby giving me the ability to build a replacement installation for this machine and begin building a new environment without interrupting service on the old one. I have several instances of 64-bit kernels runing with 64-bit base and 32-bit pkgsrc packages running successfully, so, while it's not ideal, it works quite well and gives me space to think about how to move to NetBSD-9 and beyond. Thanks again for the ideas and, hopefully, someone will find this summary useful. -Brian http://mail-index.NetBSD.org/source-changes/2020/05/24/msg117668.html