From owner-freebsd-virtualization@freebsd.org Wed Jan 13 18:31:05 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ED93F4E4C2C for ; Wed, 13 Jan 2021 18:31:05 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [IPv6:2001:470:1:474::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DGGGx3Q7Lz3KBn; Wed, 13 Jan 2021 18:31:05 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id AEC851A96B; Wed, 13 Jan 2021 18:30:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net AEC851A96B Subject: Re: After 12.1->12.2 upgrade FreeBSD bhyve guest does not start anymore To: freebsd-virtualization@freebsd.org References: <0ed9c5b2-8113-02c4-ae7b-4dc41d3c0e02@netfence.it> Cc: Toomas Soome From: Allan Jude Message-ID: Date: Wed, 13 Jan 2021 13:30:59 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <0ed9c5b2-8113-02c4-ae7b-4dc41d3c0e02@netfence.it> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4DGGGx3Q7Lz3KBn X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Jan 2021 18:31:06 -0000 On 2021-01-13 07:45, Andrea Venturoli wrote: > Hello. > > Due to a 12.x specific bug, I run a 11.3 guest using vm-bhyve. > This is its config: >> loader="bhyveload" >> cpu=1 >> memory=512M >> network0_type="virtio-net" >> network0_switch="public" >> disk0_type="virtio-blk" >> disk0_name="disk0" >> disk0_dev="sparse-zvol" >> uuid="..." >> network0_mac="..." > > On monday I upgraded the host from 12.1 to 12.2 and, since then, it > won't boot automatically, but will stop at the loader prompt. > > Dmesg: >> /boot/kernel/kernel text=0x1564b08 data=0x145330+0x4cdf30 >> syms=[0x8+0x16daf0+0x8+0x186a43] >> /boot/entropy size=0x1000 >> Booting... >> Copyright (c) 1992-2019 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >>         The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 11.3-RELEASE #0 r349754: Fri Jul  5 04:45:24 UTC 2019 >>     root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 >> FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on >> LLVM 8.0.0) >> VT: init without driver. >> CPU: Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz (3000.12-MHz K8-class CPU) >>   Origin="GenuineIntel"  Id=0x906e9  Family=0x6  Model=0x9e  Stepping=9 >>   >> Features=0x9f83fbff >> >>   >> Features2=0xfeda7a17 >> >>   AMD Features=0x24100800 >>   AMD Features2=0x121 >>   Structured Extended >> Features=0x40729 >>   Structured Extended Features3=0x400 >>   XSAVE Features=0x1 >>   TSC: P-state invariant >> Hypervisor: Origin = "bhyve bhyve " >> real memory  = 536870912 (512 MB) >> avail memory = 474779648 (452 MB) >> Event timer "LAPIC" quality 600 >> ACPI APIC Table: >> ioapic0 irqs 0-31 on motherboard >> Timecounter "TSC-low" frequency 1500060031 Hz quality 1000 >> random: entropy device external interface >> kbd1 at kbdmux0 >> 000.000029 [4213] netmap_init               netmap: loaded module >> module_register_init: MOD_LOAD (vesa, 0xffffffff81005eb0, 0) error 19 >> random: registering fast source Intel Secure Key RNG >> random: fast provider: "Intel Secure Key RNG" >> nexus0 >> cryptosoft0: on motherboard >> acpi0: on motherboard >> acpi0: Power Button (fixed) >> atrtc0: port 0x70-0x71 irq 8 on acpi0 >> atrtc0: registered as a time-of-day clock, resolution 1.000000s >> Event timer "RTC" frequency 32768 Hz quality 0 >> attimer0: port 0x40-0x43 irq 0 on acpi0 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> Event timer "i8254" frequency 1193182 Hz quality 100 >> hpet0: iomem 0xfed00000-0xfed003ff on acpi0 >> Timecounter "HPET" frequency 16777216 Hz quality 950 >> Event timer "HPET" frequency 16777216 Hz quality 550 >> Event timer "HPET1" frequency 16777216 Hz quality 450 >> Event timer "HPET2" frequency 16777216 Hz quality 450 >> Event timer "HPET3" frequency 16777216 Hz quality 450 >> Event timer "HPET4" frequency 16777216 Hz quality 450 >> Event timer "HPET5" frequency 16777216 Hz quality 450 >> Event timer "HPET6" frequency 16777216 Hz quality 450 >> Event timer "HPET7" frequency 16777216 Hz quality 450 >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> pcib0: port 0xcf8-0xcff on acpi0 >> pcib0: could not evaluate _ADR - AE_NOT_FOUND >> pci0: on pcib0 >> virtio_pci0: port 0x2000-0x207f mem >> 0xc0000000-0xc0001fff irq 16 at device 4.0 on pci0 >> vtblk0: on virtio_pci0 >> vtblk0: 10240MB (20971520 512 byte sectors) >> virtio_pci1: port 0x2080-0x209f mem >> 0xc0002000-0xc0003fff irq 17 at device 5.0 on pci0 >> vtnet0: on virtio_pci1 >> vtnet0: Ethernet address: 58:9c:fc:09:e1:26 >> vtnet0: netmap queues/slots: TX 1/1024, RX 1/512 >> 000.000566 [ 503] vtnet_netmap_attach       vtnet attached txq=1, >> txd=1024 rxq=1, rxd=512 >> isab0: at device 31.0 on pci0 >> isa0: on isab0 >> atkbdc0: port 0x60,0x64 irq 1 on acpi0 >> atkbd0: irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> driver bug: Unable to set devclass (class: atkbdc devname: (unknown)) >> psm0: irq 12 on atkbdc0 >> psm0: [GIANT-LOCKED] >> psm0: model Generic PS/2 mouse, device ID 0 >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 >> uart0: console (9600,n,8,1) >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 >> vga0: at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0 >> ppc0: cannot reserve I/O port range >> Timecounters tick every 10.000 msec >> usb_needs_explore_all: no devclass >> >> Loader variables: >> >> Manual root filesystem specification: >>   : [options] >>       Mount using filesystem >>       and with the specified (optional) option list. >> >>     eg. ufs:/dev/da0s1a >>         zfs:tank >>         cd9660:/dev/cd0 ro >>           (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) >> >>   ?               List valid disk boot devices >>   .               Yield 1 second (for background tasks) >>       Abort manual input >> >> mountroot> > > Then, if I just type "ufs:/dev/vtbd0a" it will start correctly. > > > Can anyone explain why this happens and how to avoid it? > > >  bye & Thanks >     av. > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org" Do you happen to have a ZFS partition? I think some changes to some of the bootstrap (pre-loader) code that is shared with bhyve-load, means it always prefers ZFS over UFS (normally for booting a real machine, you use a different bootstrap for UFS than for ZFS, but bhyve-load is a special case. -- Allan Jude