From nobody Thu Jan 13 14:57:04 2022 X-Original-To: xen@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2D6001954741 for ; Thu, 13 Jan 2022 14:57:14 +0000 (UTC) (envelope-from roger.pau@citrix.com) Received: from esa2.hc3370-68.iphmx.com (esa2.hc3370-68.iphmx.com [216.71.145.153]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.hc3370-68.iphmx.com", Issuer "HydrantID Server CA O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JZSFj0RcXz4p58; Thu, 13 Jan 2022 14:57:12 +0000 (UTC) (envelope-from roger.pau@citrix.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1642085833; h=date:from:to:cc:subject:message-id:references: content-transfer-encoding:in-reply-to:mime-version; bh=tobwTujOKEhLI0Uv/0dnOKDEvw4wllwpIk2bVKqJ0rs=; b=GsbWg3cSYn2J/FFXpE2/SNfGj8Uby75bFpE/mrLHGluMdKeE1OQlpitQ 9NzmQuz7YGb4uipl+lmkjiVPx1LeBK+5OvhuwMSiaQ/w6Lr8jQaKSm4uq Q9lejKgV0AF94k+IE5b5W0hZzA4bUllF0k0vGsMtJzcu3wnsYmit2bbpH E=; IronPort-SDR: xUDyZVPJ6Osu4ZrNSSMVDZUjju4NtGDJlTKOnyidLbmDJApukipSRktvXRvfdqoUBIihZWp6S5 jJ8mFhK7WNg9mu6nxt8JABmTMF/CFmimkH61yY3KUTwjHSHQFf/zECVw7/mA8LtfeaEyC53Y+j 2g0Czcgaxyt9B+zBneiVZpNktuCOHLaY18vx9ZbM9vz+uZnfSs6u/Dz2LLIl0L48OfJFxn5+F3 xiio+z033FnmO3euYY4ep9ADrPw7PhUFD7iHoQSCHL7YfYWOkcPjLSsy4t57JOz2fliDosDPUk 3NhX9ZIYexHNmrAJBmzvwyNc X-SBRS: 5.2 X-MesageID: 61928788 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:HRj1EaJwAt4s7bufFE+R8JMlxSXFcZb7ZxGr2PjLsTEN9I4Qp2hZl 39JACGYeq7OMTrFz+oGa9m1/RwEuMDTmoVjQQo/rywwQntHp8GYVN/BdRavbnrPdZaeHU42t Z4VMYeQJspsRSWB+hrxauPo8SQiiarUHNIQZAK81gVZHGeIHw960Uk/8wJAvrNVvTTQ73th0 /vi+cTHOVTjwzdoOWJ8B8mrohhltaT5sj9CtVcyP/0XtgWFyygfBcgWffHgcyTzHYUPT7C0T uuSlrvopD6D9kcnAYL+nOqkKhJTH++KZFTe0BK6N0Tabj1q/0TehY5qb6p0hT5rtgi0c/BNJ PRluMK9QA53ZvbAwLhEAxIGTH4uN/AfoOSafie04JGaxEeXL3e0z/tQV0xnZodwFsSboI1tG V70DBhXM3hvUsrvmOrTptGBAq3PFeGzVG8lkigIIQrxULB2Efgvf42QvYUChG5p3ZgUdRrjT 5FxhQRHPUyojyJnYj/7OLpm9At/riCiG9HwgAv9SZsfuwA//iQouFTeGIO9luixbddUhi6lS lfupAwVNP242Ou3klJp+lr07gPGcLiStIg6TNVU/dYy6LGfK/B65LT7mjJXrNHg4nNSVe6zJ GRL2zYIhJET0nWgdcXdQx3nsVSZ5DclDo84/+0SsGlhy4LR6gedQGMFUiRAeJots8peqT4Cj wHT2YmzXHo27ePTGSn1GrS89FtePQAPKmAPf2kcRBYt6Nj/uoAjyBnIS76PFYbr14yvSW6gk 1hmqgBgi5ZOqfEt0JngxkuWrAiLvZ2ZRBEMs1C/smWNsVoiOd/Ni5aTwUjW695ONouUREKN+ n8elKC25f4BJZ+XmSGHW+lLG6umj8tpKxWF3wQpRcN4sW3wpTjzJui8/Q2SOm82E9oiUAbIR nbYgh9x+J8MLVG3N4ZONtfZ59sR8YDsEtHsV/bxZ9VIY4RseALvwByCdXJ8zEi2zhFyzPhX1 YOzNJ/1UC1EUfgPIC+eGr9FuYLH0BzS0o86qXrT6x28mYSTa3eOIVvuGAvfN7tphE9oTei8z jq+Cydo40kOOAEdSnOOmWL2EbzsBSJrbXwRg5YGHtNv2iI8RAkc5wb5mNvNgbBNkaVPjfvv9 XqgQEJew1eXrSSZdV/SMys7Nu+/BswXQZcH0coEZwbAN58LO9fH0UviX8FvIelPGBJLkJaYs MXpi+3fW68SG1wrChwWbIXnrZwKSfhYrVnmAsZRWxBmJ8QIb1WQorfMJ1KznAFTUHbfnZZg8 tWIi1OKKbJeFl8KJJuHN5qSI6aZ4CJ1dBRaBRWYe7G+uSzEreBXFsAGpqRmfJFXd0SSn2vyO sT/KU5wmNQharQdqbHhrauFs52oA615GE9bFHPc9rG4KW/R+W/L/GOKeL/gken1WDym9aO8S /9Syv2gYvQLkEwT69h3EqpxzLJ47Nzq/ucIwgNhFXTNTlKqFrI/fSXWgZgR7vVAlu1DpA+7e kOT4d0Ga7+HD9zoTQwKLw0/Y+XdifxNwmvO7e44KVnR7TNs+ObVSl1bOhSB0XQPLLZ8PI4/7 /0mvcoat162hhYwa47UhSFI7WWcaHcHVvx/5J0dBYbqjCsty01DPsOAWnOnvsnXZowVYEcwI zKSiK7TvJhmxxLPIygpCHzA/etBnpBS6hpE+0APegaSkd3fi/5pgBAIqWYrTh5Yxwls2v5oP jQ5LFV8IKiD8ms6hMVHWGzwSQhNCAfApx70wloN0mbYU1OpRirGK2hkYbSB+0UQ8mR9eDlH/ e7HlDa5AGiyJMyhjDEvXUNFquD4SY0j/wLPr8mrAsCZEsRoejHimKKvOTIFphaP7RndX6EbS T2GJNpNVJA= IronPort-HdrOrdr: A9a23:kh64n6CYDrUk3FTlHeg0sceALOsnbusQ8zAXPh9KJiC9I/b1qy nxppkmPH/P6Qr4WBkb6Le90Y27MAnhHPlOkPQs1NaZLXLbUQ6TQr2KgrGSoQEIdxeOk9K1kJ 0QD5SWa+eAfGSS7/yKmTVQeuxIqLLskNHKuQ6d9QYUcegDUdAf0+4TMHf8LqQZfngjOXJvf6 Dsmfav6gDQMkg/X4CePD0oTuLDr9rEmNbPZgMHPQcu7E2rgSmz4LD3PhCE1lNGOgk/jIsKwC zgqUjU96+ju/a0xlv10HLS1Y1fnJ/ExsFYDMKBp8AJInHHixquZq5mR7qe1QpF6t2H2RIPqp 3hsh0gN8N85zf4eXy0mwLk303a3DMn+xbZuCmlqEqmhfa8aCMxCsJHi44cWADe8VAcsNZ117 8O936FtrJMZCmw0hjV1pztbVVHh0C0qX0tnao4lHpES7YTb7dXsMg24F5VKpEdByj3gbpXX9 WGNPuspMq+TGnqLEww5gJUsZ6RtzUIb1u7q3E5y42oO2M8pgE986MarPZv6UvouqhND6Ws3N 60QZiAoos+OvP+XZgNdNvpfvHHeFAlYSi8eV56cm6XXJ3uBRr22uvKCfMOlaaXRKA= X-IronPort-AV: E=Sophos;i="5.88,286,1635220800"; d="scan'208,223";a="61928788" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gZXKMgWJznN/cxEnznxnPBf1ankpxMafFiYFzbZwPzi8JLf+TudYOHNTe0t1LSD+3hkiSMZbCaunbdap3YTkKCkTTYNoNHbca/cvBIKLK8rvHXLiC8lVm0/XuDKM+EX4zJGjJLcL+mLZw9a+E+XONzchY1wdZiHw1b3RzxvyRubVWwfCZvfQjPzlz0wwDycIIDj8EoeSMcWOp64Cm3/IAW0cdGdrApGrQnlrHYZcJ6AvyOFI1EUVEy3rlTrceFRxnAWrVIsqHQp5WpZeyy0BJi3Tfvg3HIfy6aBsx9wU9PPS4pcCFyEMkKRkZPxg65hCybGnd97YPT5ibADXneCmJg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5Aoxz1FY4TaTV68Q3ZBG9OEXDSCDXA3UJmQhLn5NC9g=; b=APLvho+NzwB6NaESuY84sUTZkcMqQDDI1V1wWXqmNc4aakwx/5SUp0SHupZXD2Dl+dFALX+Tnx0izvOddOoJExsuq72J7nt6Am4j8nm54rP/WtG0whwjYzhTx7XFa7wDavpFF+9o2jhlTZcg3X3iHdKRPwVxmm7QxdHe+tdRRjCQ31XgBnPNkW5z2J+2APzgXwB8MjlctLzGIvobj4rUNmBHuZES3dFYBCqWqvfp46LQwUeEllbx93pxQ96TqbVaGpy3H3njzMXpn4fZkHWU6P7Xi9vmdmdc8aFbTSDTXPu7c9KV1dF6HAm0MpWkTI6TPAl+iGDPJgqINhkFkElNYw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=citrix.com; dmarc=pass action=none header.from=citrix.com; dkim=pass header.d=citrix.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.onmicrosoft.com; s=selector2-citrix-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5Aoxz1FY4TaTV68Q3ZBG9OEXDSCDXA3UJmQhLn5NC9g=; b=R5CAe9yQTNldU0MBBPNCGbRKQiCoiMCKqsDykorSgeTwl7T0H/WZ4th5cfqjSMpg7m5RsURbzb9ZA6vpGifkyZ6BOf75n8EEnFjcYrh9jPGH0NkaHKr9nzCZUFKssW16yuoF9/ZBICfdBOQRqAvBq0YVvkmbkNq203BD6nhcClk= Date: Thu, 13 Jan 2022 15:57:04 +0100 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Andriy Gapon CC: Subject: Re: VCPUOP_send_nmi returns -38 Message-ID: References: <3f9f173b-40b9-0180-404d-52fa56dde45f@FreeBSD.org> <787faaf6-c639-0f6a-1a16-c4fb7a5506e5@FreeBSD.org> Content-Type: multipart/mixed; boundary="s7dTevK8xyWDIcXS" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <787faaf6-c639-0f6a-1a16-c4fb7a5506e5@FreeBSD.org> X-ClientProxiedBy: MR1P264CA0143.FRAP264.PROD.OUTLOOK.COM (2603:10a6:501:51::22) To DS7PR03MB5608.namprd03.prod.outlook.com (2603:10b6:5:2c9::18) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-xen List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-xen@freebsd.org X-BeenThere: freebsd-xen@freebsd.org MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 36ad8cc4-4ed4-4956-3ed7-08d9d6a4f8d3 X-MS-TrafficTypeDiagnostic: SA0PR03MB5467:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iYUwijimJjxgG2TKWimqVthslyAwZ8O2mDOFpuFv/unDCoNRjJT/8D6SpeJfENlqMcY2uhZ3mvke8Gpq7tv+FCoFM5IZ86QMTMo11L/dTAt9Kkm4s9KmcGDXgQx4AiG/pCMhZbEtSUlPt49syldpNYV9ssmw2HKgYSVGo6WpZ7uDAfVr/uoyFD+nJw0krAVMlIMjhgsh4cJF0TmnTukV8h3BJM0KhXcq7D+HeeX0A7Lqo6rtbeGawqPuELAJAU3UzHR6itbNKE1HlyAQCtkgO2Kuk4EmKI80g0eKbBB+71JH3HUXOcg0Fuysq8H4FR4Ta0tUFVwKb9FYIRgvg0Yc5b7a9MK6o8LY3xG0Pw3eQEDNueZ+w7jaZKdabYEhrCfHHJjwTCTpeujcPqzefo/cWykRPlf1haG8Q8g21A3RvN7aOUx2rDDGZGB9FJ+4u/i+t5+Axekln57IVb3VdKpPAWjbtlckJr4rQmzidPDG+ypPxq0Dd8ePGZARnqIaZsoDDLSdfbU4wL+hTm1eGOZvfkIzfV/o8QYG6AUh2ZKpkmTs7FbuUNMkh8/AV6BhSmqQH/vbOrBnQyrQVisbBBLIjAer7LIp0z/kjULabJtNISXUIV7I2rraSFU7dvQlqkCE0YWu7mN//9Df7tOjZ1eihTXQRNl11+wRP5qHFP2ZfdPsoFEwvuq6iNASp9e7gxHzdJWMPzM7dyolQ2BlgFeUweGn32LZCuqb/z+kINcTnog= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR03MB5608.namprd03.prod.outlook.com;PTR:;CAT:NONE;SFS:(7916004)(4636009)(366004)(508600001)(66556008)(7116003)(33964004)(6506007)(66476007)(53546011)(6666004)(66946007)(44144004)(38100700002)(316002)(8676002)(86362001)(235185007)(8936002)(26005)(83380400001)(6486002)(82960400001)(33716001)(4326008)(85182001)(9686003)(2906002)(6512007)(186003)(450100002)(5660300002)(6916009)(67856001)(2700100001);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dXJvWDNBZWYydHYrbXgxZTlFYjNNYi9qYlU0NHU5S0VoZ2lLbHpLYTNwaS9n?= =?utf-8?B?VkVDTi9jbnB2YUNsZDBEcXdBNHkvZVczUVAwKzBWNlNoT1RzUnVISENpQkJC?= =?utf-8?B?b09GL0RCV05yNEZFNmNVY3Jmb1JCVXo0TGxQNEJ3Mkd1VzEweHJRZE05Tmsx?= =?utf-8?B?UldQNmNsQ3RGaWM0RXhhdWwzcTRscm9xR0tFcjcyanNQVmZ5dDRzRFFCTWpo?= =?utf-8?B?YWN5c0hjV0cxQWphaVBQbkFKZ1NicytJd2FiWm9IR1ZDMEdKa3NzR1lsTWZN?= =?utf-8?B?VlptY0FYdjBaQTUyZzNPNWdkNENqWFFmWmdRdDlTL1Q0NFFvRkZRcnZld0Vk?= =?utf-8?B?ak00NG9sSDBQY0F3aG5oRXY2UlZBYUhzVGVIeXJXdFJ1aFZ6cGRqVmZBeHVu?= =?utf-8?B?RjVHdE5QS3pMSTVIVVZWcHU0U0hFeDVPTG9pRmJmaE5MRytLdFBFZmxUM3N6?= =?utf-8?B?MEtWY0l0cVcyK1ZmYTVkMGd6NFdVZlJtRU1kOWV2V0RUeG0zdlBrbElTTnZH?= =?utf-8?B?VmNOYjNYUnBEeHJQODFodnBYR2VhbGhGaHAwNXNjR0ZGeWJITlcvbmJ5Z1Fv?= =?utf-8?B?a2F2c3B4RGp3Zyt2MDVBeTlLSXRrdFkyd2FoTWRSUk1NVEwvaWc3UnFMSlNE?= =?utf-8?B?cUc5ak9FUERtK3hScFNtTC9uUWI5ZFVmbFZLZUt6QlcveldkdkdaRTRieG5K?= =?utf-8?B?RERhSVFrR3lmZFByU0REU1lOVlA4RXEzd24vMUlmNlVYM3E1OVhUTTdIQ3R2?= =?utf-8?B?UnZ0eDdFQVFTMVcrYXU4a2xEeDJmOWdpV1JSdUdKd0ZNM1ZoN1BNeFZ4dUNU?= =?utf-8?B?dUorSzFUeFdxRk5xVUN3ZjUzU0ZSQUxZUE1rOWptUDh1YVFKYW5sRHB3MDNj?= =?utf-8?B?K3Y5Y1NGUHAyc1BJRzgzUkhFTVBWOEJMTVBhaWFwbWhrcmNNVU52V3E3YXJD?= =?utf-8?B?NnQyQkhKOElQWUw5c2l2Vmh4U2Y4MCtaNkpJOVhaTWFrcDczTG83L3E0Y0ZO?= =?utf-8?B?TVNZNUxzZmtuRGxOS1N1Ym14V0Qvb0liN1dUV3l5TkhCbzJRQnhQL1RFT0NW?= =?utf-8?B?ZFFnRmNYaVNNcFE5Z213TjV4a3B1bFZHYXlNUHN4d1RWODVkejZCRlEvSVZI?= =?utf-8?B?ZW5ycW51T1cwcEozWFVOUVIzY2JHWE9vcS90dDJTR0g2RWVPUVNiNUsycEdF?= =?utf-8?B?K2c2TXNTdHhkanRVWHpyeFRtdG1DelIrUnh0VEpyNDN0UDBvQ2pVd1VXcDdX?= =?utf-8?B?c0gxUVA3ZDNWL0E4czZ6ZVpXTXdTYk9kVXpuTkh4Qk1ZTldqSHR4UDlaM0JM?= =?utf-8?B?VEwvTklNZGxvc3dsZjZYQ1lBQXFjVHBsSndYVlpXSzdwUElCelg1czRyK01n?= =?utf-8?B?ZGQ3RlpKZ3FnY0hlaXY5Z1BBS05IMzNOeWFTOWIyc2ZUdzdVM29xRmt5SFpp?= =?utf-8?B?Y3ZSODB3WWFqakZFRnR0WlN4LzVRdC9yVytBQkk2OXM2MWZ5Mlp6T1VUNU9H?= =?utf-8?B?em9JbHkvc09RelNTMWZ0dDVtOXJPblB4V1FhYU1RSnRmd2hqYkorVEw3VGlT?= =?utf-8?B?L1pncVhFU3dYanJ4Sm1INFk1UmJXbE0xeDBtY3prNEZGN3NOQVhYbEZGUitq?= =?utf-8?B?a0xsRFRqbkE5UWkvanU2VCt3T2VLQ3NXa1hUV2EyZ1FQZ0d1dUJLaFpJemFp?= =?utf-8?B?dWIyb1RYb2luQ1RITG9nTHJSSU9LdCtINFJrTFdvT0p2MVlHcEtjL0tGTHVk?= =?utf-8?B?RjNRMGoyNlVsSk4xTll1a2Z4dTN6Tno3VGl3K3BMK2pXd1c3a3dLUURodUNo?= =?utf-8?B?aFpoMTBES21GOUlGc0Rvb3FXenR0Y0NVSnF4bTFORk5FaGFhcU1MZnl5N3Bh?= =?utf-8?B?bnoyb3lMaUxOM2hUeDVxQ2NvZktNZlJHY1VrTUkyYmhlSFRkMGN6S0pJL3lT?= =?utf-8?B?VC8zemJlMnp3TE9Ba0hxNW5uNHFHR2k2Z3hXWE9FVVdDaVVKNzZNR1dPbHVn?= =?utf-8?B?WVNvNlNYb0NyVXd6V0hZWGZvMStrUFZuRUNHcURXUUJVS1FvZkFVQXh6NDY3?= =?utf-8?B?bVVQemY1QkgzTHJIb0IvWGRUMk9YanZFRW9MOG02VHdyeFF6cGszWEhFVzIy?= =?utf-8?B?RGpnSFFLNFNNaVZDbUZ6eW1OZGM2MGhEVmlVeTRSZWVvSUdaUE1TK2dMUTFm?= =?utf-8?Q?i3tW6yvnOa4Xt5GOWtqqHcs=3D?= X-MS-Exchange-CrossTenant-Network-Message-Id: 36ad8cc4-4ed4-4956-3ed7-08d9d6a4f8d3 X-MS-Exchange-CrossTenant-AuthSource: DS7PR03MB5608.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jan 2022 14:57:09.1740 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: o+qohI/yGt7mQyWpJOCok2gX4hBouN5Vqfp2uIFKGil48EJsALMwm1wGHriG7asbytXVeeChMGfBAUMdYdin3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR03MB5467 X-OriginatorOrg: citrix.com X-Rspamd-Queue-Id: 4JZSFj0RcXz4p58 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=citrix.com header.s=securemail header.b=GsbWg3cS; dkim=pass header.d=citrix.onmicrosoft.com header.s=selector2-citrix-onmicrosoft-com header.b=R5CAe9yQ; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=reject) header.from=citrix.com; spf=pass (mx1.freebsd.org: domain of roger.pau@citrix.com designates 216.71.145.153 as permitted sender) smtp.mailfrom=roger.pau@citrix.com X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[citrix.com:s=securemail,citrix.onmicrosoft.com:s=selector2-citrix-onmicrosoft-com]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+exists:216.71.145.153.spf.hc3370-68.iphmx.com]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; HAS_ATTACHMENT(0.00)[]; RCVD_DKIM_ARC_DNSWL_MED(-0.50)[]; DWL_DNSWL_LOW(-1.00)[citrix.com:dkim]; RCVD_IN_DNSWL_MED(-0.20)[216.71.145.153:from]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; DKIM_TRACE(0.00)[citrix.com:+,citrix.onmicrosoft.com:+]; DMARC_POLICY_ALLOW(-0.50)[citrix.com,reject]; MLMMJ_DEST(0.00)[xen]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+]; R_MIXED_CHARSET(0.71)[subject]; ASN(0.00)[asn:16417, ipnet:216.71.145.0/24, country:US]; MID_RHS_NOT_FQDN(0.50)[]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-ThisMailContainsUnwantedMimeParts: N --s7dTevK8xyWDIcXS Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit On Tue, Jan 11, 2022 at 12:28:01PM +0200, Andriy Gapon wrote: > On 11/01/2022 12:26, Roger Pau Monné wrote: > > On Tue, Jan 11, 2022 at 12:07:28PM +0200, Andriy Gapon wrote: > > > On 11/01/2022 11:50, Andriy Gapon wrote: > > > > > > > > Recently I got a report of crashes related to using procstat -k on one > > > > of our systems.  The system runs FreeBSD 12.2 on an AWS Xen-based > > > > instance (can get more specifics about it later). > > > > > > The instance type is t2.large. > > > Here are all lines from verbose boot that mention Xen: > > > > > > XEN: Hypervisor version 4.2 detected. > > > Disabling MSI-X interrupt migration due to Xen hypervisor bug. > > > XEN: disabling emulated disks > > > XEN: disabling emulated nics > > > Hypervisor: Origin = "XenVMMXenVMM" > > > x2APIC available but disabled due to running under XEN > > > > Thanks. I've asked someone at AWS for comments. I guess the only way > > to workaround this is to switch back to using the emulated APIC. I > > will try to prepare a patch later. > > Thank you very much! I've got two patches for you to try, the first one is 'mni-fix.patch' and should fix the NMI sending issue. I'm also attaching a second patch, that should allow your guest to use x2APIC. FreeBSD has previously prevented Xen guests from using x2APIC because of a bug in old Xen versions, but I think this should be solved now, so if FreeBSD can successfully boot on your VM with the patch applied I think we are safe to enable x2APIC. That one is called 'x2apic.patch'. Let me know how the testing goes. Thanks, Roger. --s7dTevK8xyWDIcXS Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="mni-fix.patch" >From 8a4a13210d7e55b65c164ed5d0a9d9e4e6f9b593 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Date: Thu, 13 Jan 2022 14:48:14 +0100 Subject: [PATCH] x86/xen: workaround for VCPUOP_send_nmi failing It has been reported that on some AWS instances VCPUOP_send_nmi returns -38 (ENOSYS). Add a fallback to use the native NMI sending procedure when that happens, so that the NMI is not lost. Reported by: avg MFC after: 1 week --- sys/x86/xen/xen_apic.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/sys/x86/xen/xen_apic.c b/sys/x86/xen/xen_apic.c index b553e5248716..701598e44f4e 100644 --- a/sys/x86/xen/xen_apic.c +++ b/sys/x86/xen/xen_apic.c @@ -107,6 +107,12 @@ static struct xen_ipi_handler xen_ipis[] = }; #endif +/* + * Save old handler as a fallback. It has been reported some AWS system don't + * support VCPUOP_send_nmi and return ENOSYS. + */ +void (*native_ipi_vectored)(u_int, int); + /*------------------------------- Per-CPU Data -------------------------------*/ #ifdef SMP DPCPU_DEFINE(xen_intr_handle_t, ipi_handle[nitems(xen_ipis)]); @@ -273,10 +279,11 @@ xen_pv_lapic_ipi_raw(register_t icrlo, u_int dest) } #define PCPU_ID_GET(id, field) (pcpu_find(id)->pc_##field) -static void +static int send_nmi(int dest) { unsigned int cpu; + int rc = 0; /* * NMIs are not routed over event channels, and instead delivered as on @@ -286,24 +293,30 @@ send_nmi(int dest) */ switch(dest) { case APIC_IPI_DEST_SELF: - HYPERVISOR_vcpu_op(VCPUOP_send_nmi, PCPU_GET(vcpu_id), NULL); + rc = HYPERVISOR_vcpu_op(VCPUOP_send_nmi, PCPU_GET(vcpu_id), NULL); break; case APIC_IPI_DEST_ALL: CPU_FOREACH(cpu) - HYPERVISOR_vcpu_op(VCPUOP_send_nmi, + rc |= HYPERVISOR_vcpu_op(VCPUOP_send_nmi, PCPU_ID_GET(cpu, vcpu_id), NULL); break; case APIC_IPI_DEST_OTHERS: CPU_FOREACH(cpu) if (cpu != PCPU_GET(cpuid)) - HYPERVISOR_vcpu_op(VCPUOP_send_nmi, + rc |= HYPERVISOR_vcpu_op(VCPUOP_send_nmi, PCPU_ID_GET(cpu, vcpu_id), NULL); break; default: - HYPERVISOR_vcpu_op(VCPUOP_send_nmi, + rc = HYPERVISOR_vcpu_op(VCPUOP_send_nmi, PCPU_ID_GET(apic_cpuid(dest), vcpu_id), NULL); break; } + + /* + * NB: for the loops above it's expected that all calls to + * VCPUOP_send_nmi will either fail or succeed. + */ + return rc; } #undef PCPU_ID_GET @@ -314,7 +327,8 @@ xen_pv_lapic_ipi_vectored(u_int vector, int dest) int ipi_idx, to_cpu, self; if (vector >= IPI_NMI_FIRST) { - send_nmi(dest); + if (send_nmi(dest)) + native_ipi_vectored(vector, dest); return; } @@ -580,8 +594,8 @@ xen_setup_cpus(void) xen_cpu_ipi_init(i); /* Set the xen pv ipi ops to replace the native ones */ - if (xen_hvm_domain()) - apic_ops.ipi_vectored = xen_pv_lapic_ipi_vectored; + native_ipi_vectored = apic_ops.ipi_vectored; + apic_ops.ipi_vectored = xen_pv_lapic_ipi_vectored; } /* Switch to using PV IPIs as soon as the vcpu_id is set. */ -- 2.34.1 --s7dTevK8xyWDIcXS Content-Type: text/plain; charset=utf-8 Content-Disposition: attachment; filename="x2apic.patch" >From d2c5cbe40836f9784307ca072f030365269586f7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= Date: Thu, 13 Jan 2022 14:54:21 +0100 Subject: [PATCH] x86/madt: allow Xen guest to use x2APIC mode The old bogus Xen versions that would deliver a GPF when writing to the LAPIC MSR are likely retired, so it's safe to enable x2APIC unconditionally now if available. --- sys/x86/acpica/madt.c | 3 --- 1 file changed, 3 deletions(-) diff --git a/sys/x86/acpica/madt.c b/sys/x86/acpica/madt.c index 8e343a6619f3..880857a8cfad 100644 --- a/sys/x86/acpica/madt.c +++ b/sys/x86/acpica/madt.c @@ -168,9 +168,6 @@ madt_x2apic_disable_reason(void) return ("inside VMWare without intr redirection"); } - if (vm_guest == VM_GUEST_XEN) - return ("due to running under XEN"); - if (vm_guest == VM_GUEST_NO && CPUID_TO_FAMILY(cpu_id) == 0x6 && CPUID_TO_MODEL(cpu_id) == 0x2a) { -- 2.34.1 --s7dTevK8xyWDIcXS--