From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:34:32 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05C95F6290C for ; Fri, 23 Mar 2018 18:34:32 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D48E85156; Fri, 23 Mar 2018 18:34:31 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f54.google.com with SMTP id x205-v6so19683841lfa.0; Fri, 23 Mar 2018 11:34:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Ls8CzWFMvNkACTEskLgjmQm8EbU1ahjznkUHSRVuVCM=; b=un5SGNTuYFhLgZBBijPCDSPb6sy6nz+f1FobRTBbTyB5pp/MfgKvCMSnHUVgE9WnPp qY1KLrpa0yz5QsYWlOK296blw7WSCfg2E4kpqj0V30AEn2ZOYPy5YsYtfSmjhPEDQSmz GSqJin94w6+BF39E/eiVPkOLSlkA3iMWiz4haFuakLtXbOTeIwBdk0gf4SK0U2HvikcQ OdaQ5fCnO5sPy2ZDGBisbrx825C4+JMPcamIs1pwLqYJgXEUDjJMxYpOmxQrmkt5fP/i Fjh+MdxPvMA5Mhjdl2w39j1OZIzoIwXU7+m8rukPcUjRx6BP/Tlhp/+I54XTVjHESel0 zf6Q== X-Gm-Message-State: AElRT7Ht6WQJoc8qWOu9RjyQl7/7qYSUprCwjjSdGCOFRh/hzTY6bSHM ZgWi2vFZa0l4jHV/Hz9VZWZKmKTUe4g= X-Google-Smtp-Source: AG47ELsyPOQ7C+dx0JPAipY0pE/fwfDU8fQ3jqDLc5rybhgBs37psLoh06AcfzSv4MhtumlOdl+7ZQ== X-Received: by 2002:a19:385e:: with SMTP id d30-v6mr19792927lfj.16.1521830063535; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com. [209.85.215.45]) by smtp.gmail.com with ESMTPSA id u84-v6sm2345925lfk.71.2018.03.23.11.34.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 11:34:23 -0700 (PDT) Received: by mail-lf0-f45.google.com with SMTP id a22-v6so19657541lfg.9; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) X-Received: by 2002:a19:e9cc:: with SMTP id j73-v6mr18144632lfk.99.1521830063066; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 11:34:02 -0700 (PDT) In-Reply-To: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 13:34:02 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: "Rodney W. Grimes" Cc: Warner Losh , "freebsd-virtualization@freebsd.org" , Peter Grehan , Marcelo Araujo Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:34:32 -0000 On Fri, Mar 23, 2018 at 1:12 PM, Rodney W. Grimes wrote: >> On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes >> wrote: >> > [ Charset UTF-8 unsupported, converting... ] >> >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes >> >> wrote: >> >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >> >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >> >> >> > the VM. >> >> >> > >> >> >> >> >> >> What exactly are the contents of this null.iso? It sounds like we're >> >> > >> >> > touch "null.iso" >> >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso >> >> > >> >> > It is litterly a 0 byte file. This is just there to appease windows >> >> > installer that you have a cd drive. >> >> > >> >> > You can test with bhyve yourself this problem by adding: >> >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso >> >> > to your bhyve command. >> >> > >> >> >> >> Thanks for that! A couple hundred printf's later, I think this might >> >> be a firmware bug or something of the sort (?). >> >> >> >> In this loop [1], the address at 'dp' is getting immediately clobbered >> >> by the efi_lookup_devpath on every iteration, so we end up looking at >> >> basically all of the partitions and this null.iso unfortunately meets >> >> this criteria [2] for removable media with no media present. >> >> Therefore, we detect absolutely no disks. >> >> >> >> The only candidate in efi_lookup_devpath [3] is the firmware itself >> >> having overwritten the devpath at that address. We can confirm this >> >> with a printf just after the efi_lookup_devpath- the pointer it's >> >> returning is in fact exactly match 'dp' every single time. >> >> >> >> Is there a newer/different firmware you can try? This seems like >> >> really peculiar behavior for a firmware to be exhibiting. >> > >> > What happened to the boot device on the hd? >> > There should be another -s x:0, ahci-hd drive that it boots from. >> > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img >> > >> >> Right, it's there but we think it's removable media with no media >> present because of the above-mentioned clobbering by the firmware. We >> could probably work around it for this specific case, but I don't know >> if that's a great idea- there be snakes here... > > And this also appers to works with the Windows boot code.. so I > do suspect something has become broken in the new loader. > Benno points out that this is due to a uefi-edk2 hack [1] that should be ripped out now. To quote him and his quote of grehan: 13:23 <@benno> https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1030 13:23 <@benno> If you short-circuit that if so it always returns everything will be happier. 13:24 <@benno> Quoting grehan: 13:24 < kevans91> Oh no 13:25 < kevans91> ew 13:25 <@benno> "The change was put in as a workaround for a bug in the FreeBSD/UEFI loader, however, that issue was fixed with r285246. 13:25 <@benno> I'd think it unlikely that anyone would want to run <= 10.1 with bhyve/UEFI, so I the intercept could safely be removed." 13:25 <@benno> So the proper fix is to remove the override entirely. 13:25 <@benno> But in my testing I only got as far as just nerfing it via sticking a 1 || in front of the if condition. I don't think we want to workaround this in loader, and I can't explain why this is functional with Windows since it's Windows- if I could review some of the relevant source, I would. Also CC'ing grehan@, since he's the one that knows the stuff. [1] https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1008