Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 16 Sep 2020 23:02:27 +0300
From:      Andriy Gapon <avg@FreeBSD.org>
To:        Konstantin Belousov <kostikbel@gmail.com>, Wei Hu <weh@microsoft.com>
Cc:        "freebsd-hackers@freebsd.org" <freebsd-hackers@freebsd.org>
Subject:   Re: MSR accesses that slows down the hypervisor/host
Message-ID:  <6d17f16e-d672-11e1-eabc-e9d64bc13ff4@FreeBSD.org>
In-Reply-To: <20200916135727.GO94807@kib.kiev.ua>
References:  <SG2P153MB0352138F1A840E8663C039C2BB210@SG2P153MB0352.APCP153.PROD.OUTLOOK.COM> <20200916135727.GO94807@kib.kiev.ua>

next in thread | previous in thread | raw e-mail | index | archive | help
On 16/09/2020 16:57, Konstantin Belousov wrote:
> On Wed, Sep 16, 2020 at 06:52:42AM +0000, Wei Hu via freebsd-hackers wrote:
>> Hello,
>>
>> There are couple AMD processor related MSRs which are being accessed in FreeBSD. 
>>
>> #define MSR_AMDK8_IPM           0xc0010055
>> #define MSR_LS_CFG      0xc0011020
>>
>> We are seeing a lot of CPU time being spent in the host (Hyper-V) in handling traps when accessing these MSRs. Especially the first MRS is frequently being accessed in cpu_idle() so the performance impact to host is significant.
>>
>> We noted that Linux made some code changes in the 4.10 kernel to access the first MSR much less frequently. So we are wondering if there are similar changes in FreeBSD that might be in the plan. Microsoft Hyper-V team is also planning some work to speed up the accesses to these MSRs. However any suggestions or plan to improve in the FreeBSD guest are welcome.
>>
> 
> Where do you see accesses to MSR_LS_CFG ?  I can only find manipulations
> of that MSR in init_amd(), and then it is all under check that we are not
> virtualized.
> 
> For MSR_AMDK8_IPM access in cpu_idle(), it seems that the workaround was
> applied too wide. It might be that we do not need to do it on recent CPUs,
> but I need to spent more time looking at datasheets to confirm/deny.
> 
> But, do you (hypervisor) indeed allow guest to initiate C1 or deeper idle
> state ?  If not, perhaps as the first measure, we can avoid manipulating
> MSR_AMDK8_IPM under hypervisor at all.

I agree with these observations and suggestions.
Additionally, I am not sure why we check and clear AMDK8_CMPHALT bit on every
call of cpu_idle().

Maybe in the past we had to deal with firmware / SMM code that aggresivekly
restored that bit despite the wishes of our OS.  I would assume that "firmware"
of virtual machines does not do that.

Having said that, echoing what Kostik said, I really doubt that any hypervisor
faithfully emulates AMDK8_CMPHALT.  I do not see any reason to do that.


-- 
Andriy Gapon



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?6d17f16e-d672-11e1-eabc-e9d64bc13ff4>