Date: Fri, 26 Apr 2019 11:16:59 -0500 From: Justin Hibbits <chmeeedalf@gmail.com> To: Alexey Kardashevskiy <aik@ozlabs.ru> Cc: freebsd-ppc@freebsd.org Subject: Re: Fail to boot FreeBSD12 or 13 on POWER9 under PowerKVM Message-ID: <20190426111659.61c4a447@titan.knownspace> In-Reply-To: <a7a06170-396d-4896-3915-c8a4420aec3e@ozlabs.ru> References: <a7a06170-396d-4896-3915-c8a4420aec3e@ozlabs.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Alexey, On Fri, 5 Apr 2019 15:07:58 +1100 Alexey Kardashevskiy <aik@ozlabs.ru> wrote: > Hi! > > I am trying a freebsd guest on a POWER9 (pvr=004e1201) host with > linux+kvm (5.1.0-rc2) and qemu (upstream, 4.0) and something goes > wrong > - it crashes as (the full output is below): > > ===== > run_interrupt_driven_hooks: still waiting after 300 seconds for > xpt_config random: unblocking device. > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > time = 361 > KDB: stack backtrace: > 0xe000000000008660: at .kdb_backtrace+0x5c > 0xe000000000008790: at .vpanic+0x1b4 > 0xe000000000008850: at .panic+0x38 > 0xe0000000000088e0: at .boot_run_interrupt_driven_config_hooks+0x194 > 0xe0000000000089e0: at .mi_startup+0x1f8 > 0xe000000000008a80: at btext+0xc4 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at .kdb_enter+0x60: ld r2, r1, 0x28 > db> > ===== > > > The systems were installed from: > FreeBSD-12.0-RELEASE-powerpc-powerpc64-dvd1.iso > FreeBSD-13.0-CURRENT-powerpc-powerpc64-20190307-r344854-disc1.iso > > Everything by default (vt100 terminal, etc), IBM vio-scsi disk and > cdrom, 8GB RAM, 16MB backing huge pages, the host is running in the > hash (HPT) mode. > > However the exact same disk images + qemu/slof binary + qemu command > line + linux kernel do boot on POWER8E (pvr=004b0201) and POWER8NVL > (pvr=004c0100) to the login prompt. > > > I told QEMU to enforce XICS interrupt controller mode, POWER8 > compatibility (although it does not make sense as FreeBSD does not do > "client-architecture-support" RTAS call), what else can I try? That seems pretty bizarre. Is the peripheral list on the working and non-working the same? > > > While at it, FreeBSD is aware of 004b0201 and 004e1201 but it fails to > recognize 004c0100 (the FreeBSD guest still boots just fine): > > cpu0: Unknown PowerPC CPU revision 0x0100, 3259.00 MHz > cpu0: Features c4000000<PPC32,PPC64,MMU> > > but in fact architecturally it behaves exactly as IBMPOWER8 (004bxxxx > or 004d0000). This just needs another entry in the CPU table. I can add that tonight. It's not entirely cosmetic, because we do make certain decisions and set capabilities based on the CPU found. > > > > build/qemu-aikrhel74alt-ppc64/ppc64-softmmu/qemu-system-ppc64 \ > -nodefaults \ > -chardev stdio,id=STDIO0,signal=off,mux=on \ > -device spapr-vty,id=svty0,reg=0x71000110,chardev=STDIO0 \ > -mon id=MON0,chardev=STDIO0,mode=readline -nographic -vga none \ > img/freebsd12-64G.qcow2 -enable-kvm \ > -smp 1 -mem-prealloc -mem-path qemu_hp_16M_node0 -m 8G \ > -machine \ > pseries,cap-hpt-max-page-size=16M,cap-cfpc=broken,max-cpu-compat=power8,ic-mode=xics > \ > -snapshot -bios ./slof.bin \ > -L /home/aik/t/qemu-ppc64-bios/ \ > -trace events=qemu_trace_events -d guest_errors \ > -chardev socket,id=SOCKET0,server,nowait,path=qemu.mon.8324 \ > -mon chardev=SOCKET0,mode=control > > > SLOF > ********************************************************************** > QEMU Starting Build Date = Apr 5 2019 13:01:51 > FW Version = git-a5b428e1c1eae703 > Press "s" to enter Open Firmware. > > Populating /vdevice methods > Populating /vdevice/nvram@71000000 > Populating /vdevice/v-scsi@71000001 > SCSI: Looking for devices > 8000000000000000 DISK : "QEMU QEMU HARDDISK 2.5+" > Populating /vdevice/vty@71000110 > Populating /pci@800000020000000 > No NVRAM common partition, re-initializing... > Scanning USB > Using default console: /vdevice/vty@71000110 > > Welcome to Open Firmware > > Copyright (c) 2004, 2017 IBM Corporation All rights reserved. > This program and the accompanying materials are made available > under the terms of the BSD License available at > http://www.opensource.org/licenses/bsd-license.php > > > Trying to load: > from: /vdevice/v-scsi@71000001/disk@8000000000000000 ... Successfully > loaded > > >> FreeBSD/powerpc Open Firmware boot block > Boot path: /vdevice/v-scsi@71000001/disk@8000000000000000 > Boot loader: /boot/loader > Boot volume: /vdevice/v-scsi@71000001/disk@8000000000000000:2 > Consoles: Open Firmware console > > FreeBSD/powerpc64 Open Firmware loader, Revision 0.1 > Memory: 8388608KB > Booted from: /vdevice/v-scsi@71000001/disk@8000000000000000 > > Loading /boot/defaults/loader.conf > /boot/kernel/kernel data=0x136c550+0x4aa1f0 > syms=[0x8+0x165c78+0x8+0x1643cb] /boot/entropy size=0x1000 > > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel]... > Kernel entry at 0x1025d0 ... > ---<<BOOT>>--- > Copyright (c) 1992-2018 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, > 1994 The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 12.0-RELEASE r341666 GENERIC powerpc > gcc version 4.2.1 20070831 patched [FreeBSD] > VT: init without driver. > cpu0: IBM POWER9 revision 2.1, 2250.00 MHz > cpu0: Features > dc007182<PPC32,PPC64,ALTIVEC,FPU,MMU,SMT,ISNOOP,ARCH205,ARCH206,VSX,TRUELE> > cpu0: Features2 > eee00000<ARCH207,HTM,DSCR,ISEL,TAR,VCRYPTO,ARCH300,IEEE128,DARN> > real memory = 8544436224 (8148 MB) > avail memory = 8258940928 (7876 MB) > random: unblocking device. > random: entropy device external interface > kbd0 at kbdmux0 > random: registering fast source PowerISA DARN random number generator > random: fast provider: "PowerISA DARN random number generator" > ofwbus0: <Open Firmware Device Tree> on nexus0 > xicp0: <External Interrupt Presentation Controller> on ofwbus0 > xicp0: Handling CPUs 0-7 > vdevice0: <POWER Hypervisor Virtual Device Root> on ofwbus0 > uart0: <POWER Hypervisor Virtual Serial Port> irq 16781585 on vdevice0 > vscsi0: <POWER Hypervisor Virtual SCSI Bus> irq 16781570 on vdevice0 > vscsi0: Queue depth 22 commands > pcib0: <RTAS Host-PCI bridge> on ofwbus0 > pci0: <POWER Hypervisor PCI bus> on pcib0 > cpulist0: <Open Firmware CPU Group> on ofwbus0 > cpu0: <Open Firmware CPU> on cpulist0 > rtas0: <Run-Time Abstraction Services> on ofwbus0 > rtas0: registered as a time-of-day clock, resolution 0.002000s > Timecounter "timebase" frequency 512000000 Hz quality 0 > Event timer "decrementer" frequency 512000000 Hz quality 1000 > Timecounters tick every 1.000 msec > usb_needs_explore_all: no devclass > run_interrupt_driven_hooks: still waiting after 60 seconds for > xpt_config run_interrupt_driven_hooks: still waiting after 120 > seconds for xpt_config run_interrupt_driven_hooks: still waiting > after 180 seconds for xpt_config run_interrupt_driven_hooks: still > waiting after 240 seconds for xpt_config run_interrupt_driven_hooks: > still waiting after 300 seconds for xpt_config random: unblocking > device. panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > time = 361 > KDB: stack backtrace: > 0xe000000000008660: at .kdb_backtrace+0x5c > 0xe000000000008790: at .vpanic+0x1b4 > 0xe000000000008850: at .panic+0x38 > 0xe0000000000088e0: at .boot_run_interrupt_driven_config_hooks+0x194 > 0xe0000000000089e0: at .mi_startup+0x1f8 > 0xe000000000008a80: at btext+0xc4 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at .kdb_enter+0x60: ld r2, r1, 0x28 > db> > > > It's possible FreeBSD is making a decision based on certain assumptions of the CPU type in the VM, but I'm not sure at all. I hope someone else might know more about the pseries FreeBSD code than I do. - Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190426111659.61c4a447>