From owner-freebsd-hackers@freebsd.org Sun Sep 20 07:41:44 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B983E3FEB58 for ; Sun, 20 Sep 2020 07:41:44 +0000 (UTC) (envelope-from weh@microsoft.com) Received: from APC01-HK2-obe.outbound.protection.outlook.com (mail-hk2apc01on0701.outbound.protection.outlook.com [IPv6:2a01:111:f400:febc::701]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BvKJl0k9Mz4CT6; Sun, 20 Sep 2020 07:41:42 +0000 (UTC) (envelope-from weh@microsoft.com) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RKeJuD+T9uj7UkB5J+cW83aWORf4A59XHWMimv5WKUrsSt2iyiOHg1bCMpzt41IT3yNo62GNtt0vHnOn3XDKvZW3loYnmJW6MKlz99ZRLzERlAqP3YsBcZyxMCccyeGv1IgXzWViXAVd6cvtO+KpIXEVDYVjEul+rT+Qp3qnyTlRVrPzvldE2DLKd5AncweuTBUijGdNSjSnRfWRwNW0anEtShG2NAGJCsgvDV+8Jmm1WsajSaF2KBw2/N/ZVHR1eQZ+mQ7YGRDV2/s1toWzJroT8TFTuJutZuTE4hnvVL9ZHOVUbgABBt0ZLcYhfEcSKPgRbnq0W+K8ZvTHgLMjsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uCONOCOIXfB4KOy8uZGde43KvKuvtAx3vLe0avYEhEI=; b=Uv46fJYiuQXXnb3u3gALdlt9L/DRVv19trDVFWEpmu4RFEkWO5o/m+lDWnMNwKCocbMYVidIhGQpXoNk74hUH0q4hoI/eUJkaF/T0J2vHKQ7nXOdXLpp11nN6UFZ0X8Sp0+tVhQTBGMB3NuHlA9EsImO/eMkSfQFZNYWIEm2fkwFP02zHH/tGj29oLSBmJsu5yqf3kNhEAh7W5TeejiNzES2TbJb6IBsiCKoSP+Pk0VUOwbSfuM9CGYWr7ztUQQX5AwFSYIxAujp/ZxebhGA64Ao2FEP5kCWUjrRJNlY4WhKiMvq4E3Cg89FH9T+VVAlolERSS1CUDy7flgWnqVgog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=microsoft.com; dmarc=pass action=none header.from=microsoft.com; dkim=pass header.d=microsoft.com; arc=none Received: from SG2P153MB0352.APCP153.PROD.OUTLOOK.COM (2603:1096:4:d4::23) by SG2P153MB0215.APCP153.PROD.OUTLOOK.COM (2603:1096:4:8c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3412.0; Sun, 20 Sep 2020 07:41:13 +0000 Received: from SG2P153MB0352.APCP153.PROD.OUTLOOK.COM ([fe80::e1d4:23b8:2c19:4745]) by SG2P153MB0352.APCP153.PROD.OUTLOOK.COM ([fe80::e1d4:23b8:2c19:4745%6]) with mapi id 15.20.3412.015; Sun, 20 Sep 2020 07:41:13 +0000 From: Wei Hu To: Konstantin Belousov , "avg@freebsd.org" CC: "freebsd-hackers@freebsd.org" Subject: RE: MSR accesses that slows down the hypervisor/host Thread-Topic: MSR accesses that slows down the hypervisor/host Thread-Index: AdaL9HYbtJyLPxf3TL+CSKzmJaxdTwAPNlyAACBa5AAAG7/AgAB/4PJA Date: Sun, 20 Sep 2020 07:41:12 +0000 Message-ID: References: <20200916135727.GO94807@kib.kiev.ua> <20200917183825.GU94807@kib.kiev.ua> In-Reply-To: <20200917183825.GU94807@kib.kiev.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: msip_labels: MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ActionId=0e8925c4-303a-47db-a31e-2880a55dc723; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_ContentBits=0; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Enabled=true; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Method=Standard; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_Name=Internal; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SetDate=2020-09-20T07:39:59Z; MSIP_Label_f42aa342-8706-4288-bd11-ebb85995028c_SiteId=72f988bf-86f1-41af-91ab-2d7cd011db47; x-originating-ip: [116.233.42.161] x-ms-publictraffictype: Email x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: f6970a1e-55d8-4dd4-3845-08d85d388cd1 x-ms-traffictypediagnostic: SG2P153MB0215: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: U8k9DfWom5CSktXntYFuUr+UJwf+9auxtXwzL2bOQMaUI2gXEm0QTJHW1XeoeAW3yKyeNkzZvReJU8Z/vP+OGfzqJdqgk9u+NLb+rD4RkPHR7RSjtzlQe986VhpgC5l1llICFJdUFykM+RA6dei0wUZ3MH5eWhk70H3YMfGAv199Bu1ewVDACKk/QhSDwdu8Hnqaz2u1comdBkgwwIolgwtn7N89Rem412o37KgCe88j1nWD7/sy9qvcgESG+HscJsr9I+05BcUGZ/zTvkkAHU+jWJDESDHcxRmnBXcEWK8u7EezcLzJGfhl8M2GPPzbneQQX8Rf3mEbXJ0Y2FN4kMnXuSaNIEBALQZvgpUi5kRemislRQCR4GmLawg/Mw5vbb6lB0qk3MLkwpMRY5JU9iHaIve3HRlb2DGAII2rsIE2XJbDNDB8ojdYkNLzAjpc x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SG2P153MB0352.APCP153.PROD.OUTLOOK.COM; PTR:; CAT:NONE; SFS:(4636009)(396003)(39860400002)(136003)(346002)(366004)(376002)(4326008)(53546011)(478600001)(6506007)(76116006)(10290500003)(86362001)(186003)(7696005)(33656002)(8990500004)(66476007)(52536014)(64756008)(2906002)(66946007)(82950400001)(66556008)(82960400001)(9686003)(66446008)(8676002)(316002)(71200400001)(966005)(26005)(110136005)(8936002)(83380400001)(55016002)(5660300002)(83080400001); DIR:OUT; SFP:1102; x-ms-exchange-antispam-messagedata: 45OHpAV40pdifsVepmqpkTHXt0fnGnvyOjZKeh3XJgJ/zNGXV6Kh/5h/G/qJQXE+NAwj56GuD4HZScHLhJuDfC4T2cpL6smIpk2y4w9DeNvKgScgx0mnVwU0rH9inQJd+fNRCFhyEopyNcUtYkakAXFEXegpckRRc3x6kZL8b7o3xsm9EHGySba6j2+d5sIaYl02hOyItIRImODHPUj1ZEcxEnruFICCnJ9wO5UR6gALYn2HNV96wGcjY0irGY7+7qnPndO3sdmozZg4kpFT7Vz+98zhGwN9qKVF1eF0kbBuf37lS2LlATbKIvHyE9wwjrQczispFZMOYJCAtE/dBo5zMA6fAyw1dw6gTjxf8UJm8IazzwAlZPJK/DQGp0pXipl7oAFW+Y6gPI2Az0+r78T8Nd3z6cDgXO/MljWlf08FvYwtjn/b+mJROI86tvoT8SOplftJOTzOk6ej1lkbW1MqxD1tUTMbQd2x3CZmBVpB4roFJFCtxZmTGz9FeooQ9t+sZasfD/y/5fnplZcmcZZBUWOOuxiuMJ67V1nA5ZVqQTd7Hm6AHaxDfzLjo/NwwZb7/GFePqaVk+/gXyRes87HpaOed6fPmgeAUY2oFklAK0HTCXsWThAnenc7biwPqO0mT6hSs6BdMGMxcLt/xA== x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: microsoft.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: SG2P153MB0352.APCP153.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: f6970a1e-55d8-4dd4-3845-08d85d388cd1 X-MS-Exchange-CrossTenant-originalarrivaltime: 20 Sep 2020 07:41:13.0551 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 72f988bf-86f1-41af-91ab-2d7cd011db47 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: aHWoZQR9WKqL0F27fUepUQUfOIicHMYlJULivGl4Y+nPbZ+q9P05nZFJ7ZMsQOTjhtjTecnbchtE//Y3FWvOTQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SG2P153MB0215 X-Rspamd-Queue-Id: 4BvKJl0k9Mz4CT6 X-Spamd-Bar: ---------- X-Spamd-Result: default: False [-10.20 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.006]; R_DKIM_ALLOW(-0.20)[microsoft.com:s=selector2]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.02)[-1.021]; DWL_DNSWL_MED(-2.00)[microsoft.com:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[microsoft.com:+]; DMARC_POLICY_ALLOW(-0.50)[microsoft.com,reject]; NEURAL_HAM_SHORT(-1.17)[-1.174]; FREEMAIL_TO(0.00)[gmail.com,freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1]; MAILMAN_DEST(0.00)[freebsd-hackers]; WHITELIST_SPF_DKIM(-3.00)[microsoft.com:d:+,microsoft.com:s:+] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Sep 2020 07:41:44 -0000 > > > From: Konstantin Belousov > > > Sent: Wednesday, September 16, 2020 9:57 PM > > > To: Wei Hu > > > Cc: freebsd-hackers@freebsd.org > > > Subject: Re: MSR accesses that slows down the hypervisor/host > > > > > > Where do you see accesses to MSR_LS_CFG ? I can only find > > > manipulations of that MSR in init_amd(), and then it is all under > > > check that we are not virtualized. > > > > > Yes, it is only accessed in init_amd() at boot time. So it is less > > concerned. MSR_AMDK8_IPM is accessed in cpu_idle() all the time, so it = is > the key place to optimize. > > > > > For MSR_AMDK8_IPM access in cpu_idle(), it seems that the workaround > > > was applied too wide. It might be that we do not need to do it on > > > recent CPUs, but I need to spent more time looking at datasheets to > confirm/deny. > > > > > > But, do you (hypervisor) indeed allow guest to initiate C1 or deeper = idle > state ? > > > If not, perhaps as the first measure, we can avoid manipulating > > > MSR_AMDK8_IPM under hypervisor at all. > > > > You are right a guest cannot initiate C1 or deeper idle state when runn= ing on > Hyper-V. > > So skipping the read of MSR_AMDK8_IPM when running under this > > hypervisor would Be a viable solution. >=20 > https://nam06.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Frevie= ws > .freebsd.org%2FD26470&data=3D02%7C01%7Cweh%40microsoft.com%7C8 > 7f5f82583ee428f49c108d85b38e9c3%7C72f988bf86f141af91ab2d7cd011db4 > 7%7C1%7C0%7C637359647283571535&sdata=3DJdKxHO1sM2InD7Eo793FF > RIpj5AQmowcc%2BLGW19dlH4%3D&reserved=3D0 Thanks for the quick response. I will review the change and do some tests o= n Hyper-V. Wei