Date: Fri, 14 May 2021 18:44:34 +0200 From: Harry Schmalzbauer <freebsd@omnilan.de> To: Wei Hu <weh@microsoft.com>, Chris Knight <stryqx@gmail.com>, "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: Re: FreeBSD 12.1 i386 on Hyper-V Fails to Mount Root Message-ID: <c6013e67-4a37-43e1-4452-568d3ef6e5a7@omnilan.de> In-Reply-To: <SG2P153MB0208B4185C3EB54120F10E74BB030@SG2P153MB0208.APCP153.PROD.OUTLOOK.COM> References: <CAHgj5TTKdiOU4bRu3SCBwKqnQaMqxUFynodh5MLBE7%2BycBiPYw@mail.gmail.com> <SG2P153MB0208B4185C3EB54120F10E74BB030@SG2P153MB0208.APCP153.PROD.OUTLOOK.COM>
next in thread | previous in thread | raw e-mail | index | archive | help
Am 16.10.2020 um 11:35 schrieb Wei Hu via freebsd-virtualization:
>> Howdy,
>>
>> Just started the upgrade process on some FreeBSD 11.4 VMs to FreeBSD 12.1
>> using freebsd-update. VMs are running on multiple Hyper-V 2012 R2 and 2019
>> systems, all systems with 2020-09 Cumulative Updates installed.
>> The amd64 VMs are upgrading happily.
>> The i386 VMs die a horrible death - the kernel throws the following error:
>>
>> vmbus0: cannot find free IDT vector
>>
>> shortly followed by mountroot failure due to the lack of storage device
>> initialisation.
>> Checked the FreeBSD 12.1 i386 ISO in case of environment pollution - same
>> vmbus error, same lack of storage device initialisation failure. Ditto with
>> the 12.2-RC2 i386 ISO.
>>
> Did you try 12.0? Maybe something changed in 12.x causes failure.
> I386 is not well tested on Hyper-V. I will try to reproduce and take a look.
The regression happened before 12.0.
Today I stumbled across this problem too and tried 11.4 (/May 8th, 2020)
/sucessfully, while 12.0 (/Oct 19th, 2018)/ is failing with i386 releases.
One commit which is in 12 but not in 11.4 is r332489.
Code path is like
sys/dev/hyperv/vmbus/vmbus.c:
vmbus_intr_setup(struct vmbus_softc *sc)
{
int cpu;
CPU_FOREACH(cpu) {
char buf[MAXCOMLEN + 1];
cpuset_t cpu_mask;
/* Allocate an interrupt counter for Hyper-V interrupt */
snprintf(buf, sizeof(buf), "cpu%d:hyperv", cpu);
intrcnt_add(buf, VMBUS_PCPU_PTR(sc, intr_cnt, cpu));
/*
* Setup taskqueue to handle events. Task will be per-
* channel.
*/
VMBUS_PCPU_GET(sc, event_tq, cpu) = taskqueue_create_fast(
"hyperv event", M_WAITOK, taskqueue_thread_enqueue,
VMBUS_PCPU_PTR(sc, event_tq, cpu));
if (vmbus_pin_evttask) {
CPU_SETOF(cpu, &cpu_mask);
taskqueue_start_threads_cpuset(
VMBUS_PCPU_PTR(sc, event_tq, cpu), 1, PI_NET,
&cpu_mask, "hvevent%d", cpu);
} else {
taskqueue_start_threads(
VMBUS_PCPU_PTR(sc, event_tq, cpu), 1, PI_NET,
"hvevent%d", cpu);
}
/*
* Setup tasks and taskqueues to handle messages.
*/
VMBUS_PCPU_GET(sc, message_tq, cpu) =
taskqueue_create_fast(
"hyperv msg", M_WAITOK, taskqueue_thread_enqueue,
VMBUS_PCPU_PTR(sc, message_tq, cpu));
CPU_SETOF(cpu, &cpu_mask);
taskqueue_start_threads_cpuset(
VMBUS_PCPU_PTR(sc, message_tq, cpu), 1, PI_NET,
&cpu_mask,
"hvmsg%d", cpu);
TASK_INIT(VMBUS_PCPU_PTR(sc, message_task, cpu), 0,
vmbus_msg_task, sc);
}
/*
* All Hyper-V ISR required resources are setup, now let's find a
* free IDT vector for Hyper-V ISR and set it up.
*/
sc->vmbus_idtvec = lapic_ipi_alloc(pti ? IDTVEC(vmbus_isr_pti) :
IDTVEC(vmbus_isr));
if (sc->vmbus_idtvec < 0) {
device_printf(sc->vmbus_dev, "cannot find free IDT
vector\n");
return ENXIO;
}
if (bootverbose) {
device_printf(sc->vmbus_dev, "vmbus IDT vector %d\n",
sc->vmbus_idtvec);
}
return 0;
}
In sys/dev/hyperv/vmbus/i386/vmbus_vector.S
SUPERALIGN_TEXT
IDTVEC(vmbus_isr_pti)
IDTVEC(vmbus_isr)
PUSH_FRAME
SET_KERNEL_SREGS
cld
KENTER
FAKE_MCOUNT(TF_EIP(%esp))
pushl %esp
call vmbus_handle_intr
add $4, %esp
MEXITCOUNT
jmp doreti
amd64/i386 diff (problem doesn"t show up on amd64):
--- sys/dev/hyperv/vmbus/i386/vmbus_vector.S 2021-05-14
18:04:58.527699000 +0200
+++ sys/dev/hyperv/vmbus/amd64/vmbus_vector.S 2021-01-23
13:00:56.902717000 +0100
@@ -38,15 +38,9 @@
*/
.text
SUPERALIGN_TEXT
-IDTVEC(vmbus_isr_pti)
-IDTVEC(vmbus_isr)
- PUSH_FRAME
- SET_KERNEL_SREGS
- cld
- KENTER
- FAKE_MCOUNT(TF_EIP(%esp))
- pushl %esp
+ INTR_HANDLER vmbus_isr
+ FAKE_MCOUNT(TF_RIP(%rsp))
+ movq %rsp, %rdi
call vmbus_handle_intr
- add $4, %esp
MEXITCOUNT
jmp doreti
Unfortunately beyond my skills.
Any news/ideas?
Thanks,
-harry
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c6013e67-4a37-43e1-4452-568d3ef6e5a7>