From owner-freebsd-virtualization@freebsd.org Tue Oct 13 13:36:11 2015 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ADB74A126A5 for ; Tue, 13 Oct 2015 13:36:11 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-outbound.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 459FB10CD for ; Tue, 13 Oct 2015 13:36:10 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) by smtp-outbound.userve.net (8.15.1/8.15.1) with ESMTPS id t9DDa1NZ014677 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=FAIL); Tue, 13 Oct 2015 14:36:01 +0100 (BST) (envelope-from matt.churchyard@userve.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1444743365; bh=/eIiyZnRgLxk0tbF8rfcbWjShG1KZaerySdbK84zaQM=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=bTY3lD2vJT3KBbKJaVR4O2D+y62aEabAgI0iwRm6nEZrP2k6Z3UjB6X40CwIYbfNQ PYKqWJ3r8nVD0Wri4jThpQuizCzfJrLGVWFuo12cAF0B7FvpmgeRkx2fJJt8WGRONK YamQs4LWdz4O3u5Ay9kUNcJ/j2CeARbeETwgULiI= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Tue, 13 Oct 2015 14:35:55 +0100 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Tue, 13 Oct 2015 14:35:55 +0100 From: Matt Churchyard To: Tycho Nightingale CC: "freebsd-virtualization@freebsd.org" Subject: RE: Illumos boot Thread-Topic: Illumos boot Thread-Index: AdEFpuLuEiXyF4rFT0iOg0wo9KoWVQABU0uAAAObRxA= Date: Tue, 13 Oct 2015 13:35:55 +0000 Message-ID: <568ad0b2decf4537bff8f310a703c40b@SERVER.ad.usd-group.com> References: <17936712-FB88-4017-8C0D-BEC4926D698E@pluribusnetworks.com> In-Reply-To: <17936712-FB88-4017-8C0D-BEC4926D698E@pluribusnetworks.com> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Oct 2015 13:36:11 -0000 >Hi, >Please see inline. >On Oct 13, 2015, at 7:17 AM, Matt Churchyard via freebsd-virtualization wrote: > In my quest to continue expanding guest support in my vm-bhyve utility (S= ee https://github.com/churchers/vm-bhyve :) ), I've found the Windows suppo= rt pretty solid once I got clear on the slot requirements. I'm now trying a= n OS that requires CSM (Illumos) but unfortunately I'm currently struggling= to get it to boot up correctly. >=20 > Here's an example of the command I'm generating at the moment (This is ru= nning on an Intel Core-i3): >=20 > bhyve -c 2 -m 2G -s 0,hostbridge -s 31,lpc \ > -s 3,ahci-cd,/data/vm/.iso/smartos-latest.iso \ > -s 4:0,ahci-hd,/data/vm/smartos/disk0.img \ > -s 5:0,virtio-net,tap0 \ > -l com1,stdio -l com2,/dev/nmdm2A \ > -H -l bootrom,/data/vm/.config/BHYVE_UEFI_CSM.fd \ > smartos >=20 > I have com1 set to stdio so I can easily watch the output as it runs. > It tends to get as far as "Legacy INT19 Boot...", then fall over. > Depending on whether I put the network interface directly in the slot aft= er the HDD, I seem to get different errors - >=20 > slot 3 - cd > slot 4 - hdd > slot 5 - virtio-net >=20 > panic[cpu0]/thread=3Dffffff01457cdb40: BAD TRAP: type=3De (#pf Page fault= ) rp=3Dffffff0004a69a60 addr=3D40 occurred in module "genunix" due to a NUL= L pointer dereference >=20 > slot 3 - cd > slot 4 - hdd > slot 7 - virtio-net >=20 > panic[cpu1]/thread=3Dffffff0004002c40: BAD TRAP: type=3Dd (#gp General pr= otection) rp=3Dffffff0004002740 addr=3D0 >=20 > On com2 I see the boot menu, then one and a half lines of dots. The secon= d line of dots stops about 2/3 of the way across. >Have you tried booting illumos in verbose mode - edit the grub command lin= e and provide '-v'. That may give you a better backtrace than a >program c= ounter. This is what I get from a verbose boot: Bhyve-HandleProtocol: Copying DevPath: PciRoot(0x0)/Pci(0x3,0x0)/Sata(0x0,0= x0,0x0) [32] Legacy INT19 Boot... cpu0: x86 (chipid 0x0 GenuineIntel 206A7 family 6 model 42 step 7 clock 310= 9 MHz) cpu0: Intel(r) Core(tm) i3-2100 CPU @ 3.10GHz pseudo-device: stmf_sbd0 stmf_sbd0 is /pseudo/stmf_sbd@0 pseudo-device: lofi0 lofi0 is /pseudo/lofi@0 pseudo-device: devinfo0 devinfo0 is /pseudo/devinfo@0 acpinex0 at root acpinex0 is /fw iscsi0 at root iscsi0 is /iscsi xsvc0 at root: space 0 offset 0 xsvc0 is /xsvc@0,0 acpinex: sb@0, acpinex1 acpinex1 is /fw/sb@0 pseudo-device: pseudo1 pseudo1 is /pseudo/zconsnex@1 pseudo-device: pseudo2 pseudo2 is /pseudo/zfdnex@2 /pci@0,0/pci8086,2821@3 : SATA CD/DVD (ATAPI) device at port 0 model BHYVE SATA DVD ROM firmware 001 serial number BHYVE-EA14-A68A-54FA supported features: DMA SATA Gen3 signaling speed (6.0Gbps) pseudo-device: llc10 llc10 is /pseudo/llc1@0 pseudo-device: power0 power0 is /pseudo/power@0 pseudo-device: ramdisk1024 ramdisk1024 is /pseudo/ramdisk@1024 pseudo-device: ucode0 ucode0 is /pseudo/ucode@0 pseudo-device: zfs0 zfs0 is /pseudo/zfs@0 pseudo-device: srn0 srn0 is /pseudo/srn@0 pseudo-device: dtrace0 dtrace0 is /pseudo/dtrace@0 pseudo-device: dcpc0 dcpc0 is /pseudo/dcpc@0 pseudo-device: fasttrap0 fasttrap0 is /pseudo/fasttrap@0 pseudo-device: fbt0 fbt0 is /pseudo/fbt@0 pseudo-device: profile0 profile0 is /pseudo/profile@0 pseudo-device: lockstat0 lockstat0 is /pseudo/lockstat@0 pseudo-device: sdt0 sdt0 is /pseudo/sdt@0 pseudo-device: systrace0 systrace0 is /pseudo/systrace@0 pseudo-device: ipd0 ipd0 is /pseudo/ipd@0 pseudo-device: stmf0 stmf0 is /pseudo/stmf@0 sd0 at ahci0: target 0 lun 0 sd0 is /pci@0,0/pci8086,2821@3/cdrom@0,0 pseudo-device: fssnap0 fssnap0 is /pseudo/fssnap@0 /pci@0,0/pci8086,2821@3/cdrom@0,0 (sd0) online /pci@0,0/pci8086,2821@4 : SATA disk device at port 0 model BHYVE SATA DISK firmware 001 serial number BHYVE-3083-1AF1-1754 supported features: 48-bit LBA, DMA, Native Command Queueing SATA Gen3 signaling speed (6.0Gbps) Supported queue depth 32 capacity =3D 62914560 sectors WARNING: kvm: no hardware support pseudo-device: pool0 pool0 is /pseudo/pool@0 pseudo-device: bpf0 bpf0 is /pseudo/bpf@0 sd1 at ahci1: target 0 lun 0 sd1 is /pci@0,0/pci8086,2821@4/disk@0,0 pseudo-device: pm0 pm0 is /pseudo/pm@0 pseudo-device: nsmb0 nsmb0 is /pseudo/nsmb@0 pseudo-device: tap0 tap0 is /pseudo/tap@0 /pci@0,0/pci8086,2821@4/disk@0,0 (sd1) online NOTICE: vioif0: Got MAC address from host: e4:94:1:0:ff:ff pseudo-device: tun0 tun0 is /pseudo/tun@0 pseudo-device: lx_systrace0 lx_systrace0 is /pseudo/lx_systrace@0 panic[cpu0]/thread=3Dffffff0002566c40: BAD TRAP: type=3Dd (#gp General prot= ection) rp=3Dffffff00025664c0 addr=3D20 sched: #gp General protection addr=3D0x20 pid=3D0, pc=3D0xfffffffff80d375a, sp=3D0xffffff00025665b0, eflags=3D0x10282 cr0: 8005003b cr4: 406b8 cr2: fed3b5accr3: 1dc00000cr8: c rdi: 7f1a90ffffff00c3 rsi: ffffff00c33321f8 rdx: ffffff00c37f5828 rcx: ffffff00c3868603 r8: ffffff00ca7aa600 r9: 2ba6 rax: 0 rbx: ffffff00c30d0ef0 rbp: ffffff00025665c0 r10: fffffffffb8554c4 r11: 1 r12: 1f r13: ffffff00c319f880 r14: 10 r15: 20 fsb: 0 gsb: fffffffffbc326a0 ds: 4b es: 4b fs: 0 gs: 1c3 trp: d err: 0 rip: fffffffff80d375a cs: 30 rfl: 10282 rsp: ffffff00025665b0 ss: 38 This is the log of the bhyve options used (apart from 1 cpu, 1G ram) Oct 13 14:22:43: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,ahci-hd,= /data/vm/smartos/disk0.img -s 7:0,virtio-net,tap1] Oct 13 14:22:43: [bhyve console: -l com1,/dev/nmdm1A -l com2,/dev/nmdm2A] Oct 13 14:22:43: [bhyve options: -Hw -l bootrom,/data/vm/.config/BHYVE_UEF= I_CSM.fd] Oct 13 14:22:43: [bhyve iso device: -s 3:0,ahci-cd,/data/vm/.iso/smartos-l= atest.iso] > Interestingly, my code normally puts the CD after the HDD, which Windows = seems happy with as long as the slots are consecutive. > In SmartOS this gives me a different error: >=20 > slot 3 - hdd > slot 4 - cd > slot 5 - virtio-net >=20 > PlatformBdsBootFail > Boot Failed. Harddisk 1 > !!!! Find PE image /home/grehan/proj/stock_edk2/Build/BhyveX64/DEBUG_GCC4= 8/X64/UefiCpuPkg/CpuDxe/CpuDxe/DEBUG/CpuDxe.dll (ImageBase=3D000000007F8DC0= 00, EntryPoint=3D000000007F8DC2AF) !!!! >This error is from the UEFI code. It implies that the CSM boot failed or = was never invoked. If the HDD isn't bootable, yet the CD is, that is the >= most likely source as the CSM assumes the first block device it encounters = is the desired boot source. Ok, so the boot semantics are currently different between the CSM and non-C= SM firmware? CSM will try and boot the first device and fail if it's not bo= otable, whereas non-CSM will always boot CD if it's bootable, regardless of= order (from Windows instructions). Matt