From owner-freebsd-virtualization@freebsd.org Wed Jan 13 12:45:45 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3F64A4DBDFE for ; Wed, 13 Jan 2021 12:45:45 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.netfence.it (mailserver.netfence.it [78.134.96.152]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mailserver.netfence.it", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DG6cR5CSNz4d34 for ; Wed, 13 Jan 2021 12:45:43 +0000 (UTC) (envelope-from ml@netfence.it) Received: from alamar.ventu (alamar.local.netfence.it [10.1.2.18]) (authenticated bits=0) by soth.netfence.it (8.16.1/8.16.1) with ESMTPSA id 10DCjZQi026680 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Wed, 13 Jan 2021 13:45:35 +0100 (CET) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.netfence.it: Host alamar.local.netfence.it [10.1.2.18] claimed to be alamar.ventu To: freebsd-virtualization@freebsd.org From: Andrea Venturoli Subject: After 12.1->12.2 upgrade FreeBSD bhyve guest does not start anymore Message-ID: <0ed9c5b2-8113-02c4-ae7b-4dc41d3c0e02@netfence.it> Date: Wed, 13 Jan 2021 13:45:35 +0100 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DG6cR5CSNz4d34 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=netfence.it; spf=pass (mx1.freebsd.org: domain of ml@netfence.it designates 78.134.96.152 as permitted sender) smtp.mailfrom=ml@netfence.it X-Spamd-Result: default: False [-3.80 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[78.134.96.152:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:78.134.96.152]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; HAS_XAW(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RBL_SENDERSCORE_FAIL(0.00)[78.134.96.152:server fail]; SPAMHAUS_ZRD(0.00)[78.134.96.152:from:127.0.2.255]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[netfence.it,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:35612, ipnet:78.134.0.0/17, country:IT]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2021 12:45:45 -0000 Hello. Due to a 12.x specific bug, I run a 11.3 guest using vm-bhyve. This is its config: > loader="bhyveload" > cpu=1 > memory=512M > network0_type="virtio-net" > network0_switch="public" > disk0_type="virtio-blk" > disk0_name="disk0" > disk0_dev="sparse-zvol" > uuid="..." > network0_mac="..." On monday I upgraded the host from 12.1 to 12.2 and, since then, it won't boot automatically, but will stop at the loader prompt. Dmesg: > /boot/kernel/kernel text=0x1564b08 data=0x145330+0x4cdf30 syms=[0x8+0x16daf0+0x8+0x186a43] > /boot/entropy size=0x1000 > Booting... > Copyright (c) 1992-2019 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 11.3-RELEASE #0 r349754: Fri Jul 5 04:45:24 UTC 2019 > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on LLVM 8.0.0) > VT: init without driver. > CPU: Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz (3000.12-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x906e9 Family=0x6 Model=0x9e Stepping=9 > Features=0x9f83fbff > Features2=0xfeda7a17 > AMD Features=0x24100800 > AMD Features2=0x121 > Structured Extended Features=0x40729 > Structured Extended Features3=0x400 > XSAVE Features=0x1 > TSC: P-state invariant > Hypervisor: Origin = "bhyve bhyve " > real memory = 536870912 (512 MB) > avail memory = 474779648 (452 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: > ioapic0 irqs 0-31 on motherboard > Timecounter "TSC-low" frequency 1500060031 Hz quality 1000 > random: entropy device external interface > kbd1 at kbdmux0 > 000.000029 [4213] netmap_init netmap: loaded module > module_register_init: MOD_LOAD (vesa, 0xffffffff81005eb0, 0) error 19 > random: registering fast source Intel Secure Key RNG > random: fast provider: "Intel Secure Key RNG" > nexus0 > cryptosoft0: on motherboard > acpi0: on motherboard > acpi0: Power Button (fixed) > atrtc0: port 0x70-0x71 irq 8 on acpi0 > atrtc0: registered as a time-of-day clock, resolution 1.000000s > Event timer "RTC" frequency 32768 Hz quality 0 > attimer0: port 0x40-0x43 irq 0 on acpi0 > Timecounter "i8254" frequency 1193182 Hz quality 0 > Event timer "i8254" frequency 1193182 Hz quality 100 > hpet0: iomem 0xfed00000-0xfed003ff on acpi0 > Timecounter "HPET" frequency 16777216 Hz quality 950 > Event timer "HPET" frequency 16777216 Hz quality 550 > Event timer "HPET1" frequency 16777216 Hz quality 450 > Event timer "HPET2" frequency 16777216 Hz quality 450 > Event timer "HPET3" frequency 16777216 Hz quality 450 > Event timer "HPET4" frequency 16777216 Hz quality 450 > Event timer "HPET5" frequency 16777216 Hz quality 450 > Event timer "HPET6" frequency 16777216 Hz quality 450 > Event timer "HPET7" frequency 16777216 Hz quality 450 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pcib0: could not evaluate _ADR - AE_NOT_FOUND > pci0: on pcib0 > virtio_pci0: port 0x2000-0x207f mem 0xc0000000-0xc0001fff irq 16 at device 4.0 on pci0 > vtblk0: on virtio_pci0 > vtblk0: 10240MB (20971520 512 byte sectors) > virtio_pci1: port 0x2080-0x209f mem 0xc0002000-0xc0003fff irq 17 at device 5.0 on pci0 > vtnet0: on virtio_pci1 > vtnet0: Ethernet address: 58:9c:fc:09:e1:26 > vtnet0: netmap queues/slots: TX 1/1024, RX 1/512 > 000.000566 [ 503] vtnet_netmap_attach vtnet attached txq=1, txd=1024 rxq=1, rxd=512 > isab0: at device 31.0 on pci0 > isa0: on isab0 > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) > psm0: irq 12 on atkbdc0 > psm0: [GIANT-LOCKED] > psm0: model Generic PS/2 mouse, device ID 0 > uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 > uart0: console (9600,n,8,1) > uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 > vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0 > ppc0: cannot reserve I/O port range > Timecounters tick every 10.000 msec > usb_needs_explore_all: no devclass > > Loader variables: > > Manual root filesystem specification: > : [options] > Mount using filesystem > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:tank > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > Abort manual input > > mountroot> Then, if I just type "ufs:/dev/vtbd0a" it will start correctly. Can anyone explain why this happens and how to avoid it? bye & Thanks av.