Date: Wed, 10 Oct 2018 20:59:57 -0700 From: Nathan Whitehorn <nwhitehorn@freebsd.org> To: cam@neo-zeon.de Cc: freebsd-ppc@freebsd.org Subject: Re: FreeBSD 12.0-ALPHA4 fails to boot on POWER9/KVM Message-ID: <0a970b75-160a-e40e-b360-1b73a753f701@freebsd.org> In-Reply-To: <2b4455f4-37f6-5645-dcba-2cc41c845ae8@neo-zeon.de> References: <5d84652d-8b54-19f9-1396-e7c9acbe6c03@neo-zeon.de> <CAHSQbTAgow0yESJo-oV83Oq9ePzwoOL_viUxNirYmeOXy_2zmg@mail.gmail.com> <a9aed989-9ea0-4e34-0619-7d54ddc28ea6@neo-zeon.de> <5f44c6d6-7eec-6732-6fef-123e7e0d3292@freebsd.org> <E610CDE0-7BEE-4E79-BB15-9FB023690BA5@macmic.franken.de> <ff249f1e-9c2d-013a-1a37-cc79a261fbac@freebsd.org> <cd6efedd-627c-1758-1b01-cd1547180e8b@neo-zeon.de> <2b4455f4-37f6-5645-dcba-2cc41c845ae8@neo-zeon.de>
next in thread | previous in thread | raw e-mail | index | archive | help
The first part of this (all the errors about "NOT FOUND") I just fixed and the fixes will be included in BETA1 and subsequent builds. The remaining issue is that virtio SCSI is not part of the standard kernel on PPC (there are some endian and DMA bugs), so you will need to use an alternative storage backend. The default storage backend (VSCSI) is fine, as are more PC-ish things like AHCI emulation. This command line will work and is otherwise equivalent to the below: qemu-system-ppc64 -enable-kvm -m 2048 -nographic -vga none -cdrom FreeBSD-12.0-ALPHA9-powerpc-powerpc64-20181009-r339271-disc1.iso /var/lib/libvirt/images/freebsd-ppc.qcow2 -mem-prealloc -mem-path /dev/hugepages -smp 2 -Nathan On 10/10/18 6:41 PM, Cameron Berkenpas wrote: > Made a LITTLE progress on getting FreeBSD to run as a KVM VM on > POWER9/TalosII again with the following: > > qemu-system-ppc64 -enable-kvm -m 2048 -nographic -vga none -device > virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -drive > file=FreeBSD-12.0-ALPHA9-powerpc-powerpc64-20181009-r339271-disc1.iso,format=raw,if=none,id=drive-scsi0-0-0-0,readonly=on > -device > scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 > -drive > file=/var/lib/libvirt/images/freebsd-ppc.qcow2,format=qcow2,if=none,id=drive-scsi0-0-0-1 > -device > scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi0-0-0-1,id=scsi0-0-0-1,bootindex=2 > -mem-prealloc -mem-path /dev/hugepages -smp 2 > > > > Seems like issue is that you need to use a SCSI CDROM! Unfortunately, > we are don't quite get there. Hope it's not a problem to put this > inline, I'm unable to access pastebin right now: > > qemu-system-ppc64: warning: System page size 0x1000000 is not enabled > in page_size_mask (0x11000). Performance may be slow > > > SLOF > ********************************************************************** > QEMU Starting > Build Date = Sep 24 2017 12:23:07 > FW Version = buildd@ release 20170724 > Press "s" to enter Open Firmware. > > Populating /vdevice methods > Populating /vdevice/vty@71000000 > Populating /vdevice/nvram@71000001 > Populating /vdevice/l-lan@71000002 > Populating /pci@800000020000000 > 00 3800 (D) : 1af4 1004 virtio [ scsi ] > Populating /pci@800000020000000/scsi@7 > SCSI: Looking for devices > 100000100000000 DISK : "QEMU QEMU HARDDISK 2.5+" > 100000000000000 CD-ROM : "QEMU QEMU CD-ROM 2.5+" > No NVRAM common partition, re-initializing... > Scanning USB > Using default console: /vdevice/vty@71000000 > > Welcome to Open Firmware > > Copyright (c) 2004, 2017 IBM Corporation All rights reserved. > This program and the accompanying materials are made available > under the terms of the BSD License available at > http://www.opensource.org/licenses/bsd-license.php > > > Trying to load: from: > /pci@800000020000000/scsi@7/disk@100000000000000 ... Successfully > loaded > Consoles: Open Firmware console > > FreeBSD/powerpc64 Open Firmware loader, Revision 0.1 > Memory: 2097152KB > Booted from: /pci@800000020000000/scsi@7/disk@100000000000000 > > > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > / > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > \ > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > / > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > | > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > - > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > - > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND- > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > Loading /boot/defaults/loader.conf > > block-size NOT FOUND > #blocks NOT FOUND\ > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > | > block-size NOT FOUND > #blocks NOT FOUND- > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND- > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND/ > block-size NOT FOUND > #blocks NOT FOUND- > block-size NOT FOUND > #blocks NOT FOUND\ > block-size NOT FOUND > #blocks NOT FOUND| > block-size NOT FOUND > #blocks NOT FOUND\ > block-size NOT FOUND > #blocks NOT FOUND- > block-size NOT FOUND > #blocks NOT FOUND/ > block-size NOT FOUND > #blocks NOT FOUND/boot/kernel/kernel data=0x13776e0+0x54dfe0 > syms=[0x8+0x176ef8+0x8+0x1727c1] > > block-size NOT FOUND > #blocks NOT FOUND\ > block-size NOT FOUND > #blocks NOT FOUND/ > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel]... > > block-size NOT FOUND > #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > SCSI-DISK: Access beyond end of device ! > Kernel entry at 0x1024b0 ... > KDB: debugger backends: ddb > KDB: current backend: ddb > ---<<BOOT>>--- > Copyright (c) 1992-2018 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 12.0-ALPHA9 r339271 GENERIC powerpc > gcc version 4.2.1 20070831 patched [FreeBSD] > WARNING: WITNESS option enabled, expect reduced performance. > VT: init without driver. > cpu0: IBM POWER9 revision 2.2, 2234.00 MHz > cpu0: Features > dc007182<PPC32,PPC64,ALTIVEC,FPU,MMU,SMT,ISNOOP,ARCH205,ARCH206,VSX,TRUELE> > cpu0: Features2 > eee00000<ARCH207,HTM,DSCR,ISEL,TAR,VCRYPTO,ARCH300,IEEE128,DARN> > real memory = 2101960704 (2004 MB) > avail memory = 1983193088 (1891 MB) > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > arc4random: no preloaded entropy cache > random: entropy device external interface > random: registering fast source PowerISA DARN random number generator > random: fast provider: "PowerISA DARN random number generator" > kbd0 at kbdmux0 > ofwbus0: <Open Firmware Device Tree> on nexus0 > xicp0: <External Interrupt Presentation Controller> on ofwbus0 > xicp0: Handling CPUs 0-15 > vdevice0: <POWER Hypervisor Virtual Device Root> on ofwbus0 > llan0: <POWER Hypervisor Virtual Ethernet> irq 16781321 on vdevice0 > llan0: Ethernet address: 00:12:34:56:00:00 > uart0: <POWER Hypervisor Virtual Serial Port> irq 16781315 on vdevice0 > pcib0: <RTAS Host-PCI bridge> on ofwbus0 > pci0: <POWER Hypervisor PCI bus> on pcib0 > pci0: <mass storage, SCSI> at device 7.0 (no driver attached) > cpulist0: <Open Firmware CPU Group> on ofwbus0 > cpu0: <Open Firmware CPU> on cpulist0 > cpu1: <Open Firmware CPU> on cpulist0 > rtas0: <Run-Time Abstraction Services> on ofwbus0 > rtas0: registered as a time-of-day clock, resolution 0.002000s > Timecounter "timebase" frequency 512000000 Hz quality 0 > Event timer "decrementer" frequency 512000000 Hz quality 1000 > Timecounters tick every 1.000 msec > llan0: link state changed to UP > usb_needs_explore_all: no devclass > SMP: AP CPU #1 launched > arc4random: no preloaded entropy cache > Trying to mount root from cd9660:/dev/iso9660/12_0_ALPHA9_POWERPC64_CD > [ro]... > WARNING: WITNESS option enabled, expect reduced performance. > mountroot: waiting for device /dev/iso9660/12_0_ALPHA9_POWERPC64_CD... > Mounting from cd9660:/dev/iso9660/12_0_ALPHA9_POWERPC64_CD failed with > error 19. > > Loader variables: > vfs.root.mountfrom=cd9660:/dev/iso9660/12_0_ALPHA9_POWERPC64_CD > vfs.root.mountfrom.options=ro > > Manual root filesystem specification: > <fstype>:<device> [options] > Mount <device> using filesystem <fstype> > and with the specified (optional) option list. > > eg. ufs:/dev/da0s1a > zfs:zroot/ROOT/default > cd9660:/dev/cd0 ro > (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) > > ? List valid disk boot devices > . Yield 1 second (for background tasks) > <empty line> Abort manual input > > mountroot> ? > > List of GEOM managed disk devices: > > > mountroot> > > > On 09/04/2018 12:56 PM, Cameron Berkenpas wrote: >> The last working ISO I tried was probably from July (definitely no >> earlier at least) if that helps slightly. Unfortunately, I deleted my >> older ISO's. >> >> Unfortunately, I don't see any older 12-CURRENT images archived to >> try and get some idea of when this broke. >> >> If I have some time, I'll see if it's possible/feasible for me to >> build a PPC64 image of some kind from a PC to try unless someone has >> a better idea of course. :D >> >> Thanks! >> >> On 09/04/2018 08:10 AM, Nathan Whitehorn wrote: >>> The 32 and 64-bit loaders are identical binaries, so the issue is >>> probably universal. >>> -Nathan >>> >>> On 9/4/18 12:18 AM, Michael Tuexen wrote: >>>>> On 4. Sep 2018, at 07:13, Nathan Whitehorn >>>>> <nwhitehorn@freebsd.org> wrote: >>>>> >>>>> Maybe this is Lua loader fallout? The problem seems to be entirely in >>>>> the boot loader. >>>> At least the problem I reported earlier to this list regarding booting >>>> a G4 system is related to the lua loader. >>>> >>>> When installing the fourth loader, booting works, when installing >>>> the lua loader, booting fails with the output given. >>>> >>>> However, this is on a 32-bit PPC machine. Can't test on a 64-bit PPC >>>> machine. >>>> >>>> Best regards >>>> Michael >>>>> -Nathan >>>>> >>>>> On 9/3/18 9:54 PM, Cameron Berkenpas wrote: >>>>>> Hello, >>>>>> >>>>>> Argh. I went out of my way to select the right ISO image and then >>>>>> apparently still grabbed the wrong one without noticing the name. >>>>>> >>>>>> I went and tried this image: >>>>>> FreeBSD-12.0-ALPHA4-powerpc-powerpc64-20180831-r338410-disc1.iso >>>>>> >>>>>> Still the same issue though. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> On 09/03/2018 07:42 PM, Justin Hibbits wrote: >>>>>>> Hello Cameron, >>>>>>> >>>>>>> On Sun, Sep 2, 2018 at 11:15 PM Cameron Berkenpas <cam@neo-zeon.de> >>>>>>> wrote: >>>>>>>> Hello, >>>>>>>> >>>>>>>> It seems that FreeBSD 12.0-ALPHA4 is having trouble booting on >>>>>>>> KVM on >>>>>>>> POWER9 (Talos II). >>>>>>>> >>>>>>>> Here's the image I've tried: >>>>>>>> FreeBSD-12.0-ALPHA4-powerpc-powerpcspe-20180831-r338410-disc1.iso >>>>>>>> >>>>>>> The powerpcspe ISO won't work on a Talos. I don't even expect that >>>>>>> particular ISO to work at all, given the nature of the >>>>>>> architecture. >>>>>>> For Talos, and IBM POWER* in general, you need the powerpc64 ISO. >>>>>>> >>>>>>> I don't know if that will solve your problem, given the errors >>>>>>> printed >>>>>>> below, but it's at least a first step. >>>>>>> >>>>>>> - Justin >>>>>>> >>>>>>> >>>>>>>> It fails regardless of whether I run kvm from the console or >>>>>>>> through >>>>>>>> virt-manager. >>>>>>>> >>>>>>>> Seems to be disk related... I get the messages below regardless of >>>>>>>> whether I try to boot from CD-ROM or disk. I've tried all disk >>>>>>>> types >>>>>>>> for >>>>>>>> both (usb, scsi, sata, virtio, etc). >>>>>>>> >>>>>>>> It's important to note that this worked on an older snapshot of >>>>>>>> 12.0-CURRENT, though I don't know the last specific working >>>>>>>> revision I >>>>>>>> tried. >>>>>>>> >>>>>>>> Here's the output: >>>>>>>> >>>>>>>> SLOF >>>>>>>> ********************************************************************** >>>>>>>> >>>>>>>> QEMU Starting >>>>>>>> Build Date = Sep 24 2017 12:23:07 >>>>>>>> FW Version = buildd@ release 20170724 >>>>>>>> Press "s" to enter Open Firmware. >>>>>>>> >>>>>>>> Populating /vdevice methods >>>>>>>> Populating /vdevice/vty@71000000 >>>>>>>> Populating /vdevice/nvram@71000001 >>>>>>>> Populating /vdevice/v-scsi@71000002 >>>>>>>> SCSI: Looking for devices >>>>>>>> 8200000000000000 CD-ROM : "QEMU QEMU CD-ROM 2.5+" >>>>>>>> Populating /pci@800000020000000 >>>>>>>> 00 0000 (D) : 8086 100e e1000 [ net ] >>>>>>>> No NVRAM common partition, re-initializing... >>>>>>>> Scanning USB >>>>>>>> Using default console: /vdevice/vty@71000000 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Welcome to Open Firmware >>>>>>>> >>>>>>>> Copyright (c) 2004, 2017 IBM Corporation All rights reserved. >>>>>>>> This program and the accompanying materials are made >>>>>>>> available >>>>>>>> under the terms of the BSD License available at >>>>>>>> http://www.opensource.org/licenses/bsd-license.php >>>>>>>> >>>>>>>> >>>>>>>> Trying to load: from: >>>>>>>> /vdevice/v-scsi@71000002/disk@8200000000000000 >>>>>>>> ... Successfully loaded >>>>>>>> Consoles: Open Firmware console >>>>>>>> >>>>>>>> FreeBSD/powerpcspe Open Firmware loader, Revision 0.1 >>>>>>>> (Fri Aug 31 17:53:54 UTC 2018 root@releng3.nyi.freebsd.org) >>>>>>>> Memory: 33554432KB >>>>>>>> Booted from: /vdevice/v-scsi@71000002/disk@8200000000000000 >>>>>>>> >>>>>>>> >>>>>>>> block-size NOT FOUND >>>>>>>> #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> / >>>>>>>> block-size NOT FOUND >>>>>>>> #blocks NOT FOUNDSCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> SCSI-DISK: Access beyond end of device ! >>>>>>>> \ >>>>>>>> block-size NOT FOUND >>>>>>>> #blocks NOT FOUND >>>>>>>> >>>>>>>> ( 700 ) Program Exception [ 0 ] >>>>>>>> >>>>>>>> >>>>>>>> R0 .. R7 R8 .. R15 R16 .. R23 >>>>>>>> R24 .. >>>>>>>> R31 >>>>>>>> 000000000000000b 000000000345ddd4 ffffffffffffffff >>>>>>>> 0000000002c50dd8 >>>>>>>> 0000000002c559e0 0000000028686561 0000000002c57c50 >>>>>>>> 0000000002c51d10 >>>>>>>> 0000000000000000 0000000002c68540 0000000002c57c48 >>>>>>>> 0000000002c52594 >>>>>>>> 000000000345e900 0000000002c68280 0000000002c63300 >>>>>>>> 0000000000000000 >>>>>>>> 0000000002c682c0 0000000020000048 0000000000000000 >>>>>>>> 000000000345e900 >>>>>>>> 0000000000000000 0000000000000000 0000000000000000 >>>>>>>> 0000000000000000 >>>>>>>> 0000000000000040 0000000000000000 0000000002c6eb00 >>>>>>>> 0000000000000000 >>>>>>>> 0000000000000008 000000007fffffff 0000000002c5cb04 >>>>>>>> 000000000345e900 >>>>>>>> >>>>>>>> CR / XER LR / CTR SRR0 / SRR1 >>>>>>>> DAR / >>>>>>>> DSISR >>>>>>>> 80000044 0000000002c029f0 0000000028686560 >>>>>>>> 0000000000000000 >>>>>>>> 0000000020040000 0000000028686561 0000000000083000 00000000 >>>>>> _______________________________________________ >>>>>> freebsd-ppc@freebsd.org mailing list >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-ppc >>>>>> To unsubscribe, send any mail to >>>>>> "freebsd-ppc-unsubscribe@freebsd.org" >>>>>> >>>>> _______________________________________________ >>>>> freebsd-ppc@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-ppc >>>>> To unsubscribe, send any mail to >>>>> "freebsd-ppc-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-ppc@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-ppc >>> To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org" >> > > _______________________________________________ > freebsd-ppc@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-ppc > To unsubscribe, send any mail to "freebsd-ppc-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0a970b75-160a-e40e-b360-1b73a753f701>