From owner-freebsd-virtualization@freebsd.org Mon May 17 05:47:30 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 74D1D63D0F0 for ; Mon, 17 May 2021 05:47:30 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Fk7Sd1Y5Rz4dWx for ; Mon, 17 May 2021 05:47:28 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 14H5lBAb027500 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 17 May 2021 08:47:14 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 14H5lBAb027500 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 14H5lBdt027499; Mon, 17 May 2021 08:47:11 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 17 May 2021 08:47:11 +0300 From: Konstantin Belousov To: Wei Hu Cc: Harry Schmalzbauer , Chris Knight , "freebsd-virtualization@freebsd.org" Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4Fk7Sd1Y5Rz4dWx X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-2.91 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; FREEMAIL_FROM(0.00)[gmail.com]; NEURAL_HAM_LONG(-1.00)[-0.995]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; NEURAL_HAM_SHORT(-0.91)[-0.912]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FREEMAIL_CC(0.00)[omnilan.de,gmail.com,freebsd.org]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 May 2021 05:47:30 -0000 On Mon, May 17, 2021 at 04:12:20AM +0000, Wei Hu wrote: > Copy Kib who was the author of PTI and other relate patches. > > This looks to be caused by Meltdown and Spectre mitigation patches which were > checked into head couple years ago. See this link for details about these patches: > https://wiki.freebsd.org/SpeculativeExecutionVulnerabilities > > The amd64 patches also caused problems > on Hyper-V (triple fault) when vmbus and other HyperV drivers (netvsc, storvsc etc) > were loaded separately. The workaround for this on amd64 is either build these > Hyper-V modules into the kernel or completely disable mitigation by > setting vm.pmap.pti="0" in /boot/loader.conf in the affect FreeBSD guests. > For amd64 I posted the patch to supposedly fix the issue, you did not responded. > It looks the problem on i386 guests is different. While multiple patches have been > checked in for amd64, there was only one patch r332489 for i386. I am not sure > whether there is similar vm.pmap.pti option to disable the patch for i386 or not. > > I am not familiar to these patches. > > @ Kib, will you be able to comment on this? > > Thanks, > Wei > > > > > -----Original Message----- > > From: Harry Schmalzbauer > > Sent: Sunday, May 16, 2021 2:55 AM > > To: Wei Hu ; Chris Knight ; freebsd- > > virtualization@freebsd.org > > Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root > > > > Am 14.05.2021 um 18:44 schrieb Harry Schmalzbauer: > > > Am 16.10.2020 um 11:35 schrieb Wei Hu via freebsd-virtualization: > > >>> : > > >>> > > >> Did you try 12.0? Maybe something changed in 12.x causes failure. > > >> I386 is not well tested on Hyper-V. I will try to reproduce and take > > >> a look. > > > > > > The regression happened before 12.0. > > > Today I stumbled across this problem too and tried 11.4 (/May 8th, > > > 2020) /sucessfully, while 12.0 (/Oct 19th, 2018)/ is failing with i386 > > > releases. > > > > > > One commit which is in 12 but not in 11.4 is r332489. > > > > > : > > : > > : > > > Unfortunately beyond my skills. > > > > In sys/x86/x86/local_apic.c, I added this debug printf: > >         for (idx = IPI_DYN_FIRST; idx <= IPI_DYN_LAST; idx++) { > > printf("DEBUG: native_lapic_ipi_alloc(), idx = %i\n", idx); > >                 ip = &idt[idx]; > >                 func = (ip->gd_hioffset << 16) | ip->gd_looffset; > >                 if ((!pti && func == (uintptr_t)&IDTVEC(rsvd)) || > >                     (pti && func == (uintptr_t)&IDTVEC(rsvd_pti))) { > >                         vector = idx; > >                         setidt(vector, ipifunc, SDT_APIC, SEL_KPL, GSEL_APIC); > >                         break; > >                 } > >         } > > > > This is my latest attempt to see if at least the origin of my fault tracing matches. > > It does. > > > > From #define IPI_DYN_FIRST   (APIC_IPI_INTS + 9) in sys/x86/include/apicvar.h > > I expected to see 252, 253 and 254, which I sucessfully verified. > > > > Now comes the part I cannot read: (uintptr_t)&IDTVEC(rsvd) > > IDTVEC(rsvd) seems to be a assembler makro from sys/i386/i386/exception.s > > > > But now it doesn't make any sense for me to dig further, I simply do not > > understand it. > > > > Unfortunately having broken i386 support on Hyper-V is not a cosmetic issue. > > Removing "device hyperv" from kernel gives access to the v-hdd via legacy > > IDE/ada, but then I have no NIC. > > Choosing between missing HDD and missing network support is hard > > nowadays ;-) > > > > Any help getting i386 up and running back on Hyper-V highly appreciated!  11.4 > > is the latest working version, all i386-releases starting with 12.0 don't work! > > > > Thanks, > > -harry >