From owner-freebsd-hackers@freebsd.org Wed Sep 16 13:57:37 2020 Return-Path: Delivered-To: freebsd-hackers@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 54D893E68A0 for ; Wed, 16 Sep 2020 13:57:37 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Bs1rH6hFyz3XLk for ; Wed, 16 Sep 2020 13:57:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 08GDvRge054685 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Wed, 16 Sep 2020 16:57:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 08GDvRge054685 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 08GDvRa6054684; Wed, 16 Sep 2020 16:57:27 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 16 Sep 2020 16:57:27 +0300 From: Konstantin Belousov To: Wei Hu Cc: "freebsd-hackers@freebsd.org" Subject: Re: MSR accesses that slows down the hypervisor/host Message-ID: <20200916135727.GO94807@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Bs1rH6hFyz3XLk X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.16 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-0.78)[-0.779]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_MEDIUM(-0.10)[-0.096]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_SPAM_SHORT(0.71)[0.714]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-hackers]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Sep 2020 13:57:37 -0000 On Wed, Sep 16, 2020 at 06:52:42AM +0000, Wei Hu via freebsd-hackers wrote: > Hello, > > There are couple AMD processor related MSRs which are being accessed in FreeBSD. > > #define MSR_AMDK8_IPM 0xc0010055 > #define MSR_LS_CFG 0xc0011020 > > We are seeing a lot of CPU time being spent in the host (Hyper-V) in handling traps when accessing these MSRs. Especially the first MRS is frequently being accessed in cpu_idle() so the performance impact to host is significant. > > We noted that Linux made some code changes in the 4.10 kernel to access the first MSR much less frequently. So we are wondering if there are similar changes in FreeBSD that might be in the plan. Microsoft Hyper-V team is also planning some work to speed up the accesses to these MSRs. However any suggestions or plan to improve in the FreeBSD guest are welcome. > Where do you see accesses to MSR_LS_CFG ? I can only find manipulations of that MSR in init_amd(), and then it is all under check that we are not virtualized. For MSR_AMDK8_IPM access in cpu_idle(), it seems that the workaround was applied too wide. It might be that we do not need to do it on recent CPUs, but I need to spent more time looking at datasheets to confirm/deny. But, do you (hypervisor) indeed allow guest to initiate C1 or deeper idle state ? If not, perhaps as the first measure, we can avoid manipulating MSR_AMDK8_IPM under hypervisor at all.