From owner-freebsd-ppc@freebsd.org Fri Apr 26 16:17:05 2019 Return-Path: Delivered-To: freebsd-ppc@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A74001597B84 for ; Fri, 26 Apr 2019 16:17:05 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-it1-x141.google.com (mail-it1-x141.google.com [IPv6:2607:f8b0:4864:20::141]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 38EEE72CCD for ; Fri, 26 Apr 2019 16:17:04 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-it1-x141.google.com with SMTP id w15so6669462itc.0 for ; Fri, 26 Apr 2019 09:17:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2Xw2dFwllZB6sfC+MYQQ90FGNgPhBjM06Sf4aBWvQ6c=; b=Vi6nXsAEwCN3JQZLuKJlVRBKOARqDt4/H5E8jLSybOkvD7UGDv6fPz/igwiKTncXeq YG1TEc1MFA3hfnZWEv3nxDWveGYW8jVUcX+1eL7ilCQoumwTjDWPUk4wUKF74zWRGLQz 5Iiu5K1NrpX8Hs6KU5QVIexljO7YJLsctfGGpgPSdoBjJE3524OeoCjbux7JOdldFhWV qxDjOUB7mdoFciogrFilWHiYmYs6NDiBibj43h1bJiBACsz3C3lOp+TS/da0mxJM20/Q +zazkEMLR9ReqlxRLXvR6tytCk0WpEuRBI/3Ask1mvNufrdeeZInzfHqDOTbICN1rgvp at5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2Xw2dFwllZB6sfC+MYQQ90FGNgPhBjM06Sf4aBWvQ6c=; b=Gzj+bmbaTUIkOPGyDthKhsLqXTR6TbjA+1D8js2yU4Jsz8UPnjwA4kHYt5A0gzeO8I wXEK/pax/172GMzIiXV4A0fDW371P+7kvhJ5kcl7igrqUNkwvdCTD4zaRDrEepVoKzX4 YOXWvRsEaOaDgLFMxbEnt5n/w34b00Gxm2TfoCfXNa1Hm+BreWvXTuQFDN1p0iAI6GMq dQSydFke026qV3tOEdaBdUd8R630BcP0Yqb/rjTz0++EsoVq1gdP+6d5Snghx7/rslma 8cnyB4jdbrlOF1lNRwujFD4oef4n+QHHRMqnXws+jjMYC5Bbc3cZIvbyVvDbAKKdE+N+ 029A== X-Gm-Message-State: APjAAAVyH7QOo5tVJUc6O5fBZw1Ymln11EK43aU6iXcAo9Uk10Zg4SJs SS3PuiHRLAdu7G2c3KYSXtJE2gj5 X-Google-Smtp-Source: APXvYqw7mPFslX00lXHWXULCn1rsU+g4naENojpdOZKVra42FWWY0/A2T4JI2FEeKwOtxRb/Mqd+tQ== X-Received: by 2002:a24:cf41:: with SMTP id y62mr9174010itf.106.1556295423384; Fri, 26 Apr 2019 09:17:03 -0700 (PDT) Received: from titan.knownspace (173-25-245-129.client.mchsi.com. [173.25.245.129]) by smtp.gmail.com with ESMTPSA id 130sm4553680itl.14.2019.04.26.09.17.02 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 26 Apr 2019 09:17:02 -0700 (PDT) Date: Fri, 26 Apr 2019 11:16:59 -0500 From: Justin Hibbits To: Alexey Kardashevskiy Cc: freebsd-ppc@freebsd.org Subject: Re: Fail to boot FreeBSD12 or 13 on POWER9 under PowerKVM Message-ID: <20190426111659.61c4a447@titan.knownspace> In-Reply-To: References: X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; powerpc64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 38EEE72CCD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=Vi6nXsAE; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chmeeedalf@gmail.com designates 2607:f8b0:4864:20::141 as permitted sender) smtp.mailfrom=chmeeedalf@gmail.com X-Spamd-Result: default: False [-4.72 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; RCVD_COUNT_THREE(0.00)[3]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.92)[-0.925,0]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; IP_SCORE(-0.79)[ip: (1.54), ipnet: 2607:f8b0::/32(-3.16), asn: 15169(-2.26), country: US(-0.06)]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-ppc@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.4.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Apr 2019 16:17:06 -0000 Hi Alexey, On Fri, 5 Apr 2019 15:07:58 +1100 Alexey Kardashevskiy wrote: > Hi! > > I am trying a freebsd guest on a POWER9 (pvr=004e1201) host with > linux+kvm (5.1.0-rc2) and qemu (upstream, 4.0) and something goes > wrong > - it crashes as (the full output is below): > > ===== > run_interrupt_driven_hooks: still waiting after 300 seconds for > xpt_config random: unblocking device. > panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > time = 361 > KDB: stack backtrace: > 0xe000000000008660: at .kdb_backtrace+0x5c > 0xe000000000008790: at .vpanic+0x1b4 > 0xe000000000008850: at .panic+0x38 > 0xe0000000000088e0: at .boot_run_interrupt_driven_config_hooks+0x194 > 0xe0000000000089e0: at .mi_startup+0x1f8 > 0xe000000000008a80: at btext+0xc4 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at .kdb_enter+0x60: ld r2, r1, 0x28 > db> > ===== > > > The systems were installed from: > FreeBSD-12.0-RELEASE-powerpc-powerpc64-dvd1.iso > FreeBSD-13.0-CURRENT-powerpc-powerpc64-20190307-r344854-disc1.iso > > Everything by default (vt100 terminal, etc), IBM vio-scsi disk and > cdrom, 8GB RAM, 16MB backing huge pages, the host is running in the > hash (HPT) mode. > > However the exact same disk images + qemu/slof binary + qemu command > line + linux kernel do boot on POWER8E (pvr=004b0201) and POWER8NVL > (pvr=004c0100) to the login prompt. > > > I told QEMU to enforce XICS interrupt controller mode, POWER8 > compatibility (although it does not make sense as FreeBSD does not do > "client-architecture-support" RTAS call), what else can I try? That seems pretty bizarre. Is the peripheral list on the working and non-working the same? > > > While at it, FreeBSD is aware of 004b0201 and 004e1201 but it fails to > recognize 004c0100 (the FreeBSD guest still boots just fine): > > cpu0: Unknown PowerPC CPU revision 0x0100, 3259.00 MHz > cpu0: Features c4000000 > > but in fact architecturally it behaves exactly as IBMPOWER8 (004bxxxx > or 004d0000). This just needs another entry in the CPU table. I can add that tonight. It's not entirely cosmetic, because we do make certain decisions and set capabilities based on the CPU found. > > > > build/qemu-aikrhel74alt-ppc64/ppc64-softmmu/qemu-system-ppc64 \ > -nodefaults \ > -chardev stdio,id=STDIO0,signal=off,mux=on \ > -device spapr-vty,id=svty0,reg=0x71000110,chardev=STDIO0 \ > -mon id=MON0,chardev=STDIO0,mode=readline -nographic -vga none \ > img/freebsd12-64G.qcow2 -enable-kvm \ > -smp 1 -mem-prealloc -mem-path qemu_hp_16M_node0 -m 8G \ > -machine \ > pseries,cap-hpt-max-page-size=16M,cap-cfpc=broken,max-cpu-compat=power8,ic-mode=xics > \ > -snapshot -bios ./slof.bin \ > -L /home/aik/t/qemu-ppc64-bios/ \ > -trace events=qemu_trace_events -d guest_errors \ > -chardev socket,id=SOCKET0,server,nowait,path=qemu.mon.8324 \ > -mon chardev=SOCKET0,mode=control > > > SLOF > ********************************************************************** > QEMU Starting Build Date = Apr 5 2019 13:01:51 > FW Version = git-a5b428e1c1eae703 > Press "s" to enter Open Firmware. > > Populating /vdevice methods > Populating /vdevice/nvram@71000000 > Populating /vdevice/v-scsi@71000001 > SCSI: Looking for devices > 8000000000000000 DISK : "QEMU QEMU HARDDISK 2.5+" > Populating /vdevice/vty@71000110 > Populating /pci@800000020000000 > No NVRAM common partition, re-initializing... > Scanning USB > Using default console: /vdevice/vty@71000110 > > Welcome to Open Firmware > > Copyright (c) 2004, 2017 IBM Corporation All rights reserved. > This program and the accompanying materials are made available > under the terms of the BSD License available at > http://www.opensource.org/licenses/bsd-license.php > > > Trying to load: > from: /vdevice/v-scsi@71000001/disk@8000000000000000 ... Successfully > loaded > > >> FreeBSD/powerpc Open Firmware boot block > Boot path: /vdevice/v-scsi@71000001/disk@8000000000000000 > Boot loader: /boot/loader > Boot volume: /vdevice/v-scsi@71000001/disk@8000000000000000:2 > Consoles: Open Firmware console > > FreeBSD/powerpc64 Open Firmware loader, Revision 0.1 > Memory: 8388608KB > Booted from: /vdevice/v-scsi@71000001/disk@8000000000000000 > > Loading /boot/defaults/loader.conf > /boot/kernel/kernel data=0x136c550+0x4aa1f0 > syms=[0x8+0x165c78+0x8+0x1643cb] /boot/entropy size=0x1000 > > Hit [Enter] to boot immediately, or any other key for command prompt. > Booting [/boot/kernel/kernel]... > Kernel entry at 0x1025d0 ... > ---<>--- > Copyright (c) 1992-2018 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, > 1994 The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 12.0-RELEASE r341666 GENERIC powerpc > gcc version 4.2.1 20070831 patched [FreeBSD] > VT: init without driver. > cpu0: IBM POWER9 revision 2.1, 2250.00 MHz > cpu0: Features > dc007182 > cpu0: Features2 > eee00000 > real memory = 8544436224 (8148 MB) > avail memory = 8258940928 (7876 MB) > random: unblocking device. > random: entropy device external interface > kbd0 at kbdmux0 > random: registering fast source PowerISA DARN random number generator > random: fast provider: "PowerISA DARN random number generator" > ofwbus0: on nexus0 > xicp0: on ofwbus0 > xicp0: Handling CPUs 0-7 > vdevice0: on ofwbus0 > uart0: irq 16781585 on vdevice0 > vscsi0: irq 16781570 on vdevice0 > vscsi0: Queue depth 22 commands > pcib0: on ofwbus0 > pci0: on pcib0 > cpulist0: on ofwbus0 > cpu0: on cpulist0 > rtas0: on ofwbus0 > rtas0: registered as a time-of-day clock, resolution 0.002000s > Timecounter "timebase" frequency 512000000 Hz quality 0 > Event timer "decrementer" frequency 512000000 Hz quality 1000 > Timecounters tick every 1.000 msec > usb_needs_explore_all: no devclass > run_interrupt_driven_hooks: still waiting after 60 seconds for > xpt_config run_interrupt_driven_hooks: still waiting after 120 > seconds for xpt_config run_interrupt_driven_hooks: still waiting > after 180 seconds for xpt_config run_interrupt_driven_hooks: still > waiting after 240 seconds for xpt_config run_interrupt_driven_hooks: > still waiting after 300 seconds for xpt_config random: unblocking > device. panic: run_interrupt_driven_config_hooks: waited too long > cpuid = 0 > time = 361 > KDB: stack backtrace: > 0xe000000000008660: at .kdb_backtrace+0x5c > 0xe000000000008790: at .vpanic+0x1b4 > 0xe000000000008850: at .panic+0x38 > 0xe0000000000088e0: at .boot_run_interrupt_driven_config_hooks+0x194 > 0xe0000000000089e0: at .mi_startup+0x1f8 > 0xe000000000008a80: at btext+0xc4 > KDB: enter: panic > [ thread pid 0 tid 100000 ] > Stopped at .kdb_enter+0x60: ld r2, r1, 0x28 > db> > > > It's possible FreeBSD is making a decision based on certain assumptions of the CPU type in the VM, but I'm not sure at all. I hope someone else might know more about the pseries FreeBSD code than I do. - Justin