From owner-freebsd-virtualization@freebsd.org Fri Mar 23 19:36:30 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EDA4F67EEB for ; Fri, 23 Mar 2018 19:36:30 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD8D988717; Fri, 23 Mar 2018 19:36:29 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f53.google.com with SMTP id v207-v6so19908597lfa.10; Fri, 23 Mar 2018 12:36:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ubmZq/pzW1oMeMpRWCNuckOERrJYEJZGEy6ALHVtJsc=; b=EiPeyPFIyQgWEC6rq0baDKxJH1xpWhXTkThAoICmxJTNGsK8dLZ4NAvJM9JIMGxdai Ovi/EILxXHwt5WV+bDfdSdc9wIxq6RESrCJTRu0rK9LdxtuFLFUOB1XOnmCM38Iq0wDW S/82TXvt19PJwXJk8KlswoFrJCnaUwY7K1B9rVjMDh60jU9/uHlWX4zCIh2BSD5lt7Ur OivpHypteYSFkpIWdjErzXA+pZ30QktO6O9nMKGNXlx73/i6fZru8DmDim7HUpoB+x30 2VgPUOZ7r4MIj1/g5kWvZcJZERBQcJ59H6X0msey0zoduiGw4QHJRkw9d3UVGcSV+PkK ewNg== X-Gm-Message-State: AElRT7GAHi9WTBzzaKjx1sAoWJ2PCteSmQMlbnEZfCxCcMfvPI0OeBFb rneFHm6Fn6y26SasHGsvZejq4rMd X-Google-Smtp-Source: AG47ELtXzNHUr5ua+OzKgl+9KBBsF0ZQzxwVTrQoRz/e6Lwpvqw/K6IyAVfRlaZWJ247sILoe3TfqA== X-Received: by 10.46.80.71 with SMTP id v7mr8035738ljd.45.1521833761698; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) Received: from mail-lf0-f41.google.com (mail-lf0-f41.google.com. [209.85.215.41]) by smtp.gmail.com with ESMTPSA id 31-v6sm2376375lfs.57.2018.03.23.12.36.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 12:36:01 -0700 (PDT) Received: by mail-lf0-f41.google.com with SMTP id p142-v6so19902622lfd.6; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) X-Received: by 10.46.155.204 with SMTP id w12mr2212906ljj.76.1521833761318; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 12:35:40 -0700 (PDT) In-Reply-To: References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 14:35:40 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: Kyle Evans Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" , Peter Grehan , Marcelo Araujo Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 19:36:30 -0000 On Fri, Mar 23, 2018 at 1:34 PM, Kyle Evans wrote: > On Fri, Mar 23, 2018 at 1:12 PM, Rodney W. Grimes > wrote: >>> On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes >>> wrote: >>> > [ Charset UTF-8 unsupported, converting... ] >>> >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes >>> >> wrote: >>> >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >>> >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >>> >> >> > the VM. >>> >> >> > >>> >> >> >>> >> >> What exactly are the contents of this null.iso? It sounds like we're >>> >> > >>> >> > touch "null.iso" >>> >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso >>> >> > >>> >> > It is litterly a 0 byte file. This is just there to appease windows >>> >> > installer that you have a cd drive. >>> >> > >>> >> > You can test with bhyve yourself this problem by adding: >>> >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso >>> >> > to your bhyve command. >>> >> > >>> >> >>> >> Thanks for that! A couple hundred printf's later, I think this might >>> >> be a firmware bug or something of the sort (?). >>> >> >>> >> In this loop [1], the address at 'dp' is getting immediately clobbered >>> >> by the efi_lookup_devpath on every iteration, so we end up looking at >>> >> basically all of the partitions and this null.iso unfortunately meets >>> >> this criteria [2] for removable media with no media present. >>> >> Therefore, we detect absolutely no disks. >>> >> >>> >> The only candidate in efi_lookup_devpath [3] is the firmware itself >>> >> having overwritten the devpath at that address. We can confirm this >>> >> with a printf just after the efi_lookup_devpath- the pointer it's >>> >> returning is in fact exactly match 'dp' every single time. >>> >> >>> >> Is there a newer/different firmware you can try? This seems like >>> >> really peculiar behavior for a firmware to be exhibiting. >>> > >>> > What happened to the boot device on the hd? >>> > There should be another -s x:0, ahci-hd drive that it boots from. >>> > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img >>> > >>> >>> Right, it's there but we think it's removable media with no media >>> present because of the above-mentioned clobbering by the firmware. We >>> could probably work around it for this specific case, but I don't know >>> if that's a great idea- there be snakes here... >> >> And this also appers to works with the Windows boot code.. so I >> do suspect something has become broken in the new loader. >> > > Benno points out that this is due to a uefi-edk2 hack [1] that should > be ripped out now. To quote him and his quote of grehan: > > 13:23 <@benno> https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1030 > 13:23 <@benno> If you short-circuit that if so it always returns > everything will be happier. > 13:24 <@benno> Quoting grehan: > 13:24 < kevans91> Oh no > 13:25 < kevans91> ew > 13:25 <@benno> "The change was put in as a workaround for a bug in the > FreeBSD/UEFI loader, however, that issue was fixed with > r285246. > 13:25 <@benno> I'd think it unlikely that anyone would want to run <= > 10.1 with bhyve/UEFI, so I the intercept could safely be > removed." > 13:25 <@benno> So the proper fix is to remove the override entirely. > 13:25 <@benno> But in my testing I only got as far as just nerfing it > via sticking a 1 || in front of the if condition. > > I don't think we want to workaround this in loader, and I can't > explain why this is functional with Windows since it's Windows- if I > could review some of the relevant source, I would. > > Also CC'ing grehan@, since he's the one that knows the stuff. > > [1] https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1008 FYI- I've created PR #2 [1] against the freebsd/uefi-edk2 repository and have confirmed that this fixes the broken-looking firmware along with the booting problem experienced in this thread. [1] https://github.com/freebsd/uefi-edk2/pull/2