From owner-freebsd-xen@freebsd.org Sun Nov 15 16:27:57 2020 Return-Path: <owner-freebsd-xen@freebsd.org> Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3FF7F2ED3AF for <freebsd-xen@mailman.nyi.freebsd.org>; Sun, 15 Nov 2020 16:27:57 +0000 (UTC) (envelope-from 3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com) Received: from mail-vk1-xa45.google.com (mail-vk1-xa45.google.com [IPv6:2607:f8b0:4864:20::a45]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CYyL362wlz4XyD for <freebsd-xen@FreeBSD.org>; Sun, 15 Nov 2020 16:27:55 +0000 (UTC) (envelope-from 3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com) Received: by mail-vk1-xa45.google.com with SMTP id x134so4792712vkd.17 for <freebsd-xen@FreeBSD.org>; Sun, 15 Nov 2020 08:27:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:reply-to:message-id:date:subject:from:to; bh=YYHGaezJbU9X6vy7XTd48O8RSc0RQUHjshkF47egn+A=; b=uQn13kUFJAKG06GpHj9uSwpNuuwikWRQSz9Lxx4uUBK5C1gotullNa3IbIVAYOlEsM GR5R/zMxtm4ZrvvBmLu7po/7S+2dZmPiFVInN+RPKtMysYfORkS1z18+cOdAy2DeLyho /miQC2GGHTixLGnMJzhpg0CANaL5KtnGab8HbLpGw0tB+yhRCLRA89MLzSGJYFwJugt5 OpOrRu/6vAs0jCFXL9ITbUQtiRsgmfYWpVVgD+Y2ZoFm8nt7fQbXMjSeApbB8W4mR4sC MsiBw4x63YFtjnOhePp9jYYj0mSQfEoelpf7+6cvO52ev1h6uwBSI5D+/+ZtIPTy2Wza sA+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:message-id:date:subject :from:to; bh=YYHGaezJbU9X6vy7XTd48O8RSc0RQUHjshkF47egn+A=; b=fBEnZiJLEZPsR4j5VNKKPndW3E9ErLV0MXZ0fEE8paW9X18QH1CY19AB6ZES4FMszZ LarnxC7OjRZRgJ2lOb6BaYl/vCNuFoCsUx+pYZnQByAcxUbRbORIoq8NTpZeMMPjfIsa OmnWpCVD5yxWykpPV/YaoWC0H22keC1roZoSKUUPUc2S4HV+GoAv79+iiMXtmwQCr6Nd hG50PqLg44G1TFQKoDo7pUy3Z2atl8r+T41rQ7WvHRvlULEndtob+J4kJDRzx4cDUYkV 0F7Ebw7uySQdW3aHe8FG7PGLg+/yYDPGUB/ur3BYwHcrKVgscfDKN0lEhKju+145hJ7Z i+LQ== X-Gm-Message-State: AOAM531kcxHCOB8E5Wb+Jwfi+7aJplPpXJ0U4qHRMehK+Eorut8NLy7T dP3WXwJle8AcdAhs0NjS5qDXy0HRXIII1I7qgcKc MIME-Version: 1.0 X-Received: by 2002:a67:ff01:: with SMTP id v1mt6722103vsp.16.1605457674095; Sun, 15 Nov 2020 08:27:54 -0800 (PST) Reply-To: 0123484893d89@gmail.com X-No-Auto-Attachment: 1 Message-ID: <0000000000005df0c905b427beb6@google.com> Date: Sun, 15 Nov 2020 16:27:54 +0000 Subject: Hi From: 0123484893d89@gmail.com To: freebsd-xen@FreeBSD.org X-Rspamd-Queue-Id: 4CYyL362wlz4XyD X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uQn13kUF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of 3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com designates 2607:f8b0:4864:20::a45 as permitted sender) smtp.mailfrom=3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com X-Spamd-Result: default: False [-2.58 / 15.00]; HAS_REPLYTO(0.00)[0123484893d89@gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FORGED_SENDER(0.30)[0123484893d89@gmail.com,3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::a45:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[0123484893d89@gmail.com,3ClexXw0JC4E56789D9DE8iDElrfnq.htrkwjjgxi-2jsKwjjGXI.twl@trix.bounces.google.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FREEMAIL_REPLYTO(0.00)[gmail.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-xen@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::a45:from:127.0.2.255]; NEURAL_SPAM_SHORT(0.12)[0.117]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::a45:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-xen] Content-Type: text/plain; charset="UTF-8"; format=flowed; delsp=yes X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage <freebsd-xen.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/> List-Post: <mailto:freebsd-xen@freebsd.org> List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sun, 15 Nov 2020 16:27:57 -0000 I've invited you to fill out the following form: Untitled form To fill it out, visit: https://docs.google.com/forms/d/e/1FAIpQLScGC-Yyj3A7IdOVYPn5A_Q90cvAmfSi-nmoK9EAE4efRQyX9Q/viewform?vc=0&c=0&w=1&flr=0&usp=mail_form_link Hi, I got your email through a data base and after going through your profile, I decided to reach out to you. I am a professional agent and my Principal is a former Top Libyan Politician, have asked me to look for an investment manager. That could take available investment fund as loan on a one digit interest rate repayable in full after 10 years. Funds readily available is over 1B Your swift response is highly needed. I look forward for your feedback Best Regards Jerry Michael. Google Forms: Create and analyze surveys. From owner-freebsd-xen@freebsd.org Tue Nov 17 09:21:16 2020 Return-Path: <owner-freebsd-xen@freebsd.org> Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9A33A2D5F09 for <freebsd-xen@mailman.nyi.freebsd.org>; Tue, 17 Nov 2020 09:21:16 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa4.hc3370-68.iphmx.com (esa4.hc3370-68.iphmx.com [216.71.155.144]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "esa4.hc3370-68.iphmx.com", Issuer "HydrantID SSL ICA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cb0mp69byz3tkn for <freebsd-xen@freebsd.org>; Tue, 17 Nov 2020 09:21:14 +0000 (UTC) (envelope-from roger.pau@citrix.com) IronPort-SDR: pEeNhIHZAsZJqxQDhrhSpu25RZbhBB+9z1WJdn6cz7BS7AU5kAYyLMIp2miSguF8aiBqvN351n XGV7H3NJK4lrQoJfh8f3K9FoxEdNRFk9zeTRne+Ha13XC/PoqmzceiG7EPZBGJ8SIXrJ0+Q7IC vDGLOVcWA918abOhrO0vcz9oplkzKm7ERrz9a9yoFg65fTXJ7ul9B+MJFZcn0P0a0nFCn1Civi KO6bh6LY+tUlujR0sMIc2GTntdIlAdx+Q4lGM0qP2DTMiLmF8gJiXn8KQp24CBHL5BFX2/klRp bjE= X-SBRS: None X-MesageID: 32456694 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.77,485,1596513600"; d="scan'208";a="32456694" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YXJgW42AgAmjl2leHlVeEbYr5n8o6Sod6Xe6FIDvuyB6vi3GxSzgwhoskdP3rJb99HOQXonxU5MEC9s+TCUpzSo/kx3GEyt7wDH5vtGWJJoW3IvbwAOrkYOX6sec+06MdLqQYFVrrYaTk75YiJAkErvUn6R2NSeloZCtUrG6gwa7uGTFaV62Q9YwcTlOkJsYshTPPS5oun9m8OCqme24uXSgXGOe0d7/36YYdaenMKgn8xbNQbJbMWXKBAD8qURuJRb0HXT6/rGuRSQ88WWf2krbIpZnRai+sz0zDSM1Sa2vb9AOLMhT+sw1WgsH11wHuID4+Qkxjhxe1WB8RUVBiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/gslbG8md5SpLub0LyqoxWz12rftVvyktD73oJm9OHs=; b=ViUglvVujHTOYrD4TEiWgN2k6AT6HGmkjW65gN/o+yCtoA1Y2rgnYFQuDddPGp+CqrZxBg64oMuXs3x4SDi5iQyHkb7ufgcDeOpYt++1EmjKe3s2p3GpPD6MCIshAn+FPSaSZAYxe3FWT83hFiwvFKYalTaQj0KLjwTzJZV96Oube0Q5M+Gk2NwKEJJguIzcYrg3LNhhqQ6Yn56ovHf1fyvcMLf3gnlNqXsq5tyXJMqaIuYjRKbxKw8XpUnVGLYeXMvFt/hdmO0Vd5MbP6zTyzgFaeR1s/bQdhl+479ys/kOduByXHeDkEXCu53GE0ohLf7ZA3MV2h+i9XvbinVyoQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none Date: Tue, 17 Nov 2020 10:21:04 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com> To: Brian Buhrow <buhrow@nfbcal.org> CC: <freebsd-xen@freebsd.org> Subject: Re: ZFS corruption using zvols as backingstore for hvm VM's Message-ID: <20201117092104.usulqtvcvyv7mcji@Air-de-Roger> References: <20201111095009.6lcik5y3s7wrsh5k@Air-de-Roger> <202011130518.0AD5IxtF013960@nfbcal.org> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <202011130518.0AD5IxtF013960@nfbcal.org> X-ClientProxiedBy: LO2P265CA0446.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:e::26) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 75267baf-1b09-4a42-ddee-08d88ada1df2 X-MS-TrafficTypeDiagnostic: DM5PR03MB2922: X-Microsoft-Antispam-PRVS: <DM5PR03MB292251D02AECE701AE279BA38FE20@DM5PR03MB2922.namprd03.prod.outlook.com> X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: krv3Mm4fiSeJiI67hNmS2AwR0U6XhomH4RELexT1VmY2jfFB9chMeiMrRgj/moo9J53+KcPUIXKK9hsgg2NdjpsbeJ2SVAMAl+k7M0fK2QqEwkb1XdKcExNB1GMKnAgYU47DD2wnc79L64bUEj/MMYsYk81fxuHyGr2jeNwiWVOzqFxss/cv5zy8uJMAngukjVjdbqsCeXjYDbdkDUTQ6DP+WPCQZP+1TCFMDKLbb7/15JStWZGdIXAXQWv1wFgRw2Nkv3xpBXj5bNsgmh0Zddn44dauk4bqiq0VXPabbV/LaJUD2Yc2mWs67Xpj4UgrobRb25cIqVRtqV2kTI0Ciclym8KiQDANoP9nHNoennVEZB4XDAXxQW3+D0rlsCeGnX9UTqoQ+qtUWNZPX1BuSA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS7PR03MB5608.namprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(7916004)(4636009)(376002)(39860400002)(136003)(366004)(396003)(346002)(8676002)(6916009)(33716001)(8936002)(956004)(16526019)(85182001)(4326008)(6666004)(66556008)(66946007)(5660300002)(186003)(316002)(86362001)(2906002)(1076003)(66476007)(83380400001)(26005)(478600001)(9686003)(966005)(6496006)(6486002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ABMos6djtMoQM2HGF7s2E95K/ZMwk2BRX3KvnJqscDb2EIo1l1uO28L4jJaDWjwnbIZIIOJ935UzidcMJaZdj8pvs7MKP0Dny+z0uPf055Oh8PieDj9CQHBqmSpaNrHaaF75ooRNpeBwbUtacbEhgkUL0BRKB7TfoYxsCUKWIgZc6nyVkCb0TxHBlle+IBmztT3lZz90BA2/9hJ8e5UiKGUPju2tMcHUyRAtfNl/tFQFh3RQDwfPTxeUdy2Fsl/WCfY+ctTPhrksezrtBiFSHcf3t0DyWvfPyueSG2Qo13esg2BhU0WRgnJsFRQaGt+FlEtg/ulFKswXJrdP3SbdJpHUAM6C1j5HA6ZemKF4se95yR8XQ8SjjwaSMrU2l2KtUO87pSTKYjy8DZN8xMV70L29G0OFt3PQd8e8Hb3hYae6B3sIxECtxQBHYkL7TV3PNGq9BSZj8PfxyavwubRQ+L/2jlV/OO74C+jXp3XJ6/IGO0txlxBfBYnYBisKPPv2ZHX9/dyTHca65OnE0F/Yr8PgM4+HAvdJ+cLatFKj2Vzgur3con6wX2qQtQc6xoThZ76W+W5KMyZwFxDg41vh0ZwqdgKE5c9rhYV38tqztbnPmztpIiriyl4b+pH0imXga33meQzP0dDMHG88Ve2kMEK/Puf9u5BK76mIVKhUCjYzNmvPgfYg91iDwUmMAXnG50Wc3x6bYJJVUQXTa6Yva4G1/LiHBZo+ybFMzZPI6qRfBhS0XjsziC4atjiRu4CGIl7uN844p6dGkeM9pnbqYXsmPAlSpSjCcHxIMhEzt1wfZ7HaxbCP7DCEcvkELFQvgmdr5WpgoBNgJt4M+4fb4cG9UkfRCgpMGo3b+3pJIozfMWyg4cwO/8RkV7jpHuvBs0vQc5RIjJH8211WyBO/ow== X-MS-Exchange-CrossTenant-Network-Message-Id: 75267baf-1b09-4a42-ddee-08d88ada1df2 X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2020 09:21:08.4607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: cycnSLAih7uzrJYgP+3525FcP3tVjPCyc+htBVi3HyML+ax8PduXFC7YKUI2SuUNXQEQp4A4vwhkbAP8nwhNmg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB2922 X-OriginatorOrg: citrix.com X-Rspamd-Queue-Id: 4Cb0mp69byz3tkn X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.20 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RWL_MAILSPIKE_VERYGOOD(0.00)[216.71.155.144:from]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail,citrix.onmicrosoft.com:s=selector2-citrix-onmicrosoft-com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+exists:216.71.155.144.spf.hc3370-68.iphmx.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_LOW(-1.00)[citrix.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[216.71.155.144:from]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,reject]; DKIM_TRACE(0.00)[citrix.com:+,citrix.onmicrosoft.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-xen] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage <freebsd-xen.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/> List-Post: <mailto:freebsd-xen@freebsd.org> List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=subscribe> X-List-Received-Date: Tue, 17 Nov 2020 09:21:16 -0000 On Thu, Nov 12, 2020 at 09:18:59PM -0800, Brian Buhrow wrote: > hello Roger. thanks for engaging with me on this issue. I think > I've made progress on the issue and have a better handle on what's going > wrong. There seem to be a cascade of bugs here, which I'll try to > enumerate. > > 1. The disk corruption issue seems to be a bug in qemu whereby the > emulated IDE disk controller issues partial writes instead of full writes > or no writes with appropriate failure to the disk. The IDE driver in > NetBSD-5.2 doesn't play well with this behavior, in fact, NetBSD until May > of 2020, doesn't play well with this behavior > See: > http://mail-index.NetBSD.org/source-changes/2020/05/24/msg117668.html Oh great, so it's something specific to NetBSD. This means you are running NetBSD in HVM mode? > 2. This causes memory corruption in the OS itself, which can trigger a xen > server crash! (In my view, no matter how badly behaved the guest OS is, it > shouldn't be able to bring down the xen server.) Right, we really need the trace from this crash. Do you have a serial hooked up to the box so that you can provide the panic message? > 3. Running NetBSD-5.2/i386 as a domu, which works flawlessly under xen3, gets a > panic: HYPERVISOR_mmu_update failed This would imply that you are running NetBSD in PV mode, in which case it won't be using the emulated hard disk drive, and hence the commit you referenced above would be unrelated. Can you assert whether you are running NetBSD in PV or HVM mode? > I suspect this can be worked around using some of the command line > options under Xen, xpti=true,domu=false, perhaps? > Are there others I should consider? Maybe. I think you should report this to xen-devel and NetBSD/Xen mailing lists, with a full trace of the crash and the guest config file. Roger. From owner-freebsd-xen@freebsd.org Wed Nov 18 19:58:43 2020 Return-Path: <owner-freebsd-xen@freebsd.org> Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 108E747103B for <freebsd-xen@mailman.nyi.freebsd.org>; Wed, 18 Nov 2020 19:58:43 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (ns.NFBCAL.ORG [157.22.230.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "nfbcal.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cbtss6FmKz4b1f for <freebsd-xen@freebsd.org>; Wed, 18 Nov 2020 19:58:41 +0000 (UTC) (envelope-from buhrow@nfbcal.org) Received: from nfbcal.org (localhost [127.0.0.1]) by nfbcal.org (8.15.2/8.14.1-NFBNETBSD) with ESMTPS id 0AIJwR8V028850 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 18 Nov 2020 11:58:27 -0800 (PST) Received: (from buhrow@localhost) by nfbcal.org (8.15.2/8.12.11) id 0AIJwRN1007886; Wed, 18 Nov 2020 11:58:27 -0800 (PST) Message-Id: <202011181958.0AIJwRN1007886@nfbcal.org> From: Brian Buhrow <buhrow@nfbcal.org> Date: Wed, 18 Nov 2020 11:58:27 -0800 In-Reply-To: <20201117092104.usulqtvcvyv7mcji@Air-de-Roger> X-Mailer: Mail User's Shell (7.2.6 beta(4.pl1)+dynamic 20000103) To: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com> Subject: Re: ZFS corruption using zvols as backingstore for hvm VM's Cc: <freebsd-xen@freebsd.org>, buhrow@nfbcal.org X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (nfbcal.org [127.0.0.1]); Wed, 18 Nov 2020 11:58:28 -0800 (PST) X-Rspamd-Queue-Id: 4Cbtss6FmKz4b1f X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of buhrow@nfbcal.org designates 157.22.230.125 as permitted sender) smtp.mailfrom=buhrow@nfbcal.org X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_TLS_ALL(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[157.22.230.125:from]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+a:ns.nfbcal.org]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[nfbcal.org]; ARC_NA(0.00)[]; SPAMHAUS_ZRD(0.00)[157.22.230.125:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7091, ipnet:157.22.0.0/16, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-xen] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage <freebsd-xen.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/> List-Post: <mailto:freebsd-xen@freebsd.org> List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 18 Nov 2020 19:58:43 -0000 hello Roger. Sorry for the confusion. I've been running NetBSD, or trying to, in both PV and HVM modes. Here is a mesage I sent to the port-xen@netbsd.org mailing list a few days ago, detailing what I found with running NetBSD in any mode under Xen as a domu. Unfortunately, I don't think I can provide a trace of a crash to the xen server when the HVM host misbehaves because I've had to move on in my work to get things working and I found a working combination of NetBSD and ZFS that let's me proceed. I've seen some discussion in the Xen documentation that suggests one can run a Xen dom0 in a domu guest, allowing for sand box testing of what you're describing without having to devote hardware to the issue. Am I correct in this and where might I find instructions on how to do it? In any case, here's what I found. -thanks -Brian From: Brian Buhrow <buhrow@nfbcal.org> Date: Mon, 16 Nov 2020 14:59:48 -0800 Subject: Re: Panic: HYPERVISOR_mmu_update failed on old NetBSD-5.2 domU Cc: port-xen@netbsd.org, buhrow@nfbcal.org hello Greg. thanks for the pointers on Xen changes over the years and the reasons for the changes. After a lot more testing, thought and cvs diffing, here is the state of the world, as I understand it, for the record. NetBSD-5/i386 won't run under Xen-4.12 or newer, possibly older versions of Xen, I didn't test, because it still has calls to xpq_queue_flush and the like which were used in pre-xen3 days. I believe, through code inspection, but didn't test, that this is also true of NetBSD-6/i386 and NetBSD-7/i386. My next thought was to use NetBSD-5 in HVM mode, but it turns out this is a show stopper for versions of NetBSD all the way through -current as of May 24, 2020, see the link below, due to a cascade of bugs in the piixide(4) emulator in Qemu and NetBSD-s inability to deal with partial success in writing to disks on that chip set. This failure is what causes the original disk corruption I wrote about at the beginning of this thread. As an alternative, I tried enabling the SCSI LSI emulator in Qemu, but this doesn't work because our esiop(4) driver doesn't play well with the emulated LSI chips in Qemu. No disk corruption, just timeouts when writing to attached emulated sd(4) devices. So, NetBSD as an HVM guest is pretty much a bust! Fortunately, there is a solution. NetBSD-5.2/amd64 DOMU's work fine under Xen-4.12 and above. So, I'll run the 64-bit amd-64 base, with a 32-bit pkgsrc set of packages to allow me to migrate my busy, working machine, to newere hardware as a VM, thereby giving me the ability to build a replacement installation for this machine and begin building a new environment without interrupting service on the old one. I have several instances of 64-bit kernels runing with 64-bit base and 32-bit pkgsrc packages running successfully, so, while it's not ideal, it works quite well and gives me space to think about how to move to NetBSD-9 and beyond. Thanks again for the ideas and, hopefully, someone will find this summary useful. -Brian http://mail-index.NetBSD.org/source-changes/2020/05/24/msg117668.html From owner-freebsd-xen@freebsd.org Thu Nov 19 09:07:15 2020 Return-Path: <owner-freebsd-xen@freebsd.org> Delivered-To: freebsd-xen@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8C8322EC3B0 for <freebsd-xen@mailman.nyi.freebsd.org>; Thu, 19 Nov 2020 09:07:15 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "esa5.hc3370-68.iphmx.com", Issuer "HydrantID SSL ICA G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CcDMj4dd7z3vhQ for <freebsd-xen@freebsd.org>; Thu, 19 Nov 2020 09:07:12 +0000 (UTC) (envelope-from roger.pau@citrix.com) IronPort-SDR: 0ZMp3r8EeHlC0lra6x4RgiJvgS2UP3upLJoShD1uNtkfn6ysl5fJ5vxq7s3zd4OqPN8LfENJFD rtnprQ0UZLePgZxXzPUdwtWNHo010L8sifA+XiQPXZ1V0oOfa9dQHHEPur1uICh7jajLPzafMm ic6Mvl8YH66aH1I801QQHAIWL/RA3QGEGNxATj9CWdx+E8exL+KahGHKUX0c3ucjtcV3yB/M2V bFmkaM6RCcm9ar5yScxMLL2khY0cQszBpgL8P8RVwTvjGz0F41JN563+slnoyNlcUwKFlTuyfT k2Q= X-SBRS: None X-MesageID: 31517095 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.77,490,1596513600"; d="scan'208";a="31517095" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h3nha7IEcCPOpnPu3OlmpdC6y7pDWGSVaZGSpmJNrjsWe6TAIyl+qvnwZYRE6aZ8bF5JghucZZ6srx5V515dALUqXxgSr6oVz+qfczH8JHQF8HOSIVtfSaJYG9AUQlXrSFwv3CtRdNKmcvW29NQaTvpNMGza57x3RxtM417YSl+AURV5hiUpImx/effBTn0F3hWinurbS8t48MWwGuSaHUP0AsxNE24q3YAGxZM2zkB2UN9mdNn+9hHDEKo6za35X1YFS8X7hQVBfJJr0ou3FgKaq3AvhZ0hoFyDB97AcAGK6MxbmUlQouYBRRF6dI+y+VILg5bt0sdajZmZ0xTSRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=n3sZlH8afYYRnwn0S5Zv2OcEX/fPgD+B/3tzhyZ9K50=; b=U0ScyzQm0qAqPQf61v0XiIZuAFvIwChHifZkRS/H7m3Qje0hX+TbmlVwYXMhbk6oEoEvtl/Qq+flRlxee722BliLl/9xmM7F4QxhhANlonMuBxH8xB9QRE4Hh4zyGWRBnQ+9Ix18Z5Mz1QolpnjWLNB9tKge+/8p/vGecicFt4kzrUYaTmcr74MpTHa4U3zVP30X7GC3e5tdD0QLdGquzgsF97f/GLkg2MKe//uUBKybeIUFPXxldFTnM3BM1Fh9GQd+cbuX+zI6Khx+REI/vqxFqO3Ox2WJrO3Fqbu/inYasFkiMqv2IiYtg2PkTEWysby5DMmq6Wh2ATT4TifK2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none Date: Thu, 19 Nov 2020 10:07:03 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com> To: Brian Buhrow <buhrow@nfbcal.org> CC: <freebsd-xen@freebsd.org> Subject: Re: ZFS corruption using zvols as backingstore for hvm VM's Message-ID: <20201119090703.nzkr7pvouvz4ot2k@Air-de-Roger> References: <20201117092104.usulqtvcvyv7mcji@Air-de-Roger> <202011181958.0AIJwRN1007886@nfbcal.org> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <202011181958.0AIJwRN1007886@nfbcal.org> X-ClientProxiedBy: LO2P265CA0304.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a5::28) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1df41cdd-cef1-4bee-b898-08d88c6a7ea7 X-MS-TrafficTypeDiagnostic: DM5PR03MB3209: X-Microsoft-Antispam-PRVS: <DM5PR03MB320959639F0C5DAD300BF5358FE00@DM5PR03MB3209.namprd03.prod.outlook.com> X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: yj2Z6nF69ZZoRxEMGAd779FzNgUU4licNvWRiM/+S7WcnQG3qe21g8E54EwodJpbKj4lni6qDIyDB9YI8W9G9Ftu+vWrh8dkpG6smWuDQgGk7aDDX0X9N6KJngNusd0G3jlXn7FNrSlgVjY1U0Ryt4s04yH/8yYe0KohUPlgypwHFXkULm1K2WQ6w1znnzyC51OkwsXTjmlqOPw/w+ZTAm5VXhy108KGbdVRY68Y9TFdgwVwTV6TtPAdYHcoxrMCSaKwmAa0D7k549bZxBDMWn3i/YZsS8NgGZiaT3uSmcHP+HDi5P0u6/JPYborV0JShb9qnhG8xkLmWFUKkEy+Lg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:DS7PR03MB5608.namprd03.prod.outlook.com; PTR:; CAT:NONE; SFS:(7916004)(4636009)(396003)(136003)(39860400002)(346002)(376002)(366004)(1076003)(33716001)(83380400001)(2906002)(66946007)(86362001)(85182001)(9686003)(4326008)(6916009)(478600001)(316002)(5660300002)(8676002)(16526019)(26005)(8936002)(956004)(6486002)(66556008)(186003)(6496006)(66476007)(6666004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ChUlcwRUG6ZL8UViO5T84klbZSk1Gf65uzzLYlNAUpf+xG4uv5QftG9aD/I8uYyrdh/69RRs49Wk1bRQEPslci0SLEzT0TeF3W+QpXEerJk3gMgut0elQGXrx4wMqFjYvG5qh3iygJ1dm4BePr8BPQpd4QCjNyXZt0S7Vu7B8NqR+7U5UmDr1hnMocTYwZeo4FQ6c5KvmEXL46X9VcdXtcdhVX1ixSUi8cp9cnZHOwN8mnsym61nag8DGGs2dtcDImOULrwx9KL3XALmr1HSTiSaOLlEdV8XuBSmlv+b48HOK8fBp5Bq/JhVeG1O7ONhOi/ZKd5ABnroD/FsH758uDPqinth+miUOoSocIwfHr6WXNpJ6yn/zAebQdDGpUlqDWJpx9ad7udVmhPrrpETi3WAH9e8LueaB05yntwoe8T910hcOk1wEezrJdpkUY+zm8EI5kqmgqM9L8TTzkdN77pJM12GqRIkWWEAE00jzC5jiErgumCbuuzsS0YMHb+WK142JOEiCjrEmDulNJVlNd6LuPCYHD9ynTZA5l4zb7MlB/kZMrlZ+2LlcEhxVlRYpTLZqBLpqtXsS8rNZyRHXSmJqhY66Yv/F+RR7+/eLucUir+lZcHgEQa+Hx76Y/PLj3XwoqjSgZb1PYLbTM/AUs+YkBrUUUnn1ys9IG69zoLjvKvWM5Mf9hsKJwGMffOFglZXpCAh+X8IhkwdpaZmsvtFiEfSElDwk4LhspxLcJrEwRAJtj6EAxS+713x9msk2x5gXV2UlHQpDYqlzXXLiyFOevr8gp1sF+K/AsMq2ZNhos7mwz9sZ+0tYcPolgI67xm5kJl2CY8QSdaJD0xQ0D1fQmPLfnzXVTznACwjlcG9zw1ECRACfyFrdZ+bB2jX7WZM3LnFfji1jNPHotC1TA== X-MS-Exchange-CrossTenant-Network-Message-Id: 1df41cdd-cef1-4bee-b898-08d88c6a7ea7 X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Nov 2020 09:07:09.4249 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hMAuYDEigUj//Q4Mx1Ib4NLcDoE6PLm+EtMqdfLQP3p8SnFPnb1HK9K5EF69+4E+9uTbjg/y/2NasSUASYLWRg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR03MB3209 X-OriginatorOrg: citrix.com X-Rspamd-Queue-Id: 4CcDMj4dd7z3vhQ X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.20 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; RWL_MAILSPIKE_VERYGOOD(0.00)[216.71.155.168:from]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail,citrix.onmicrosoft.com:s=selector2-citrix-onmicrosoft-com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+exists:216.71.155.168.spf.hc3370-68.iphmx.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_LOW(-1.00)[citrix.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[216.71.155.168:from]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,reject]; DKIM_TRACE(0.00)[citrix.com:+,citrix.onmicrosoft.com:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:16417, ipnet:216.71.154.0/23, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-xen] X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage <freebsd-xen.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-xen/> List-Post: <mailto:freebsd-xen@freebsd.org> List-Help: <mailto:freebsd-xen-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-xen>, <mailto:freebsd-xen-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 19 Nov 2020 09:07:15 -0000 On Wed, Nov 18, 2020 at 11:58:27AM -0800, Brian Buhrow wrote: > hello Roger. Sorry for the confusion. I've been running NetBSD, or > trying to, in both PV and HVM modes. Here is a mesage I sent to the > port-xen@netbsd.org mailing list a few days ago, detailing what I found > with running NetBSD in any mode under Xen as a domu. Unfortunately, I > don't think I can provide a trace of a crash to the xen server when the OK, I'm not really able to reproduce this myself, so if you get those crashes again can you please make sure you have a serial attached to the box so that you can provide some trace? > HVM host misbehaves because I've had to move on in my work to get things > working and I found a working combination of NetBSD and ZFS that let's me > proceed. I've seen some discussion in the Xen documentation that suggests > one can run a Xen dom0 in a domu guest, allowing for sand box testing of > what you're describing without having to devote hardware to the issue. Am > I correct in this and where might I find instructions on how to do it? You can test a nested PV dom0, but not a nested PVH dom0, which is what FreeBSD uses to run as dom0. > > In any case, here's what I found. Thanks, I'm sure this will e helpful to others. Roger.