From owner-freebsd-virtualization@freebsd.org Sun Jan 24 15:57:22 2021 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 230D64D0554 for ; Sun, 24 Jan 2021 15:57:22 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [IPv6:2a00:1450:4864:20::52e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DNyLS75XCz3jgD for ; Sun, 24 Jan 2021 15:57:20 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: by mail-ed1-x52e.google.com with SMTP id b21so12280406edy.6 for ; Sun, 24 Jan 2021 07:57:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli-net.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pTcGVVx6qtfoQESAi4KvG6agl4eITKU7V2xGp9Tk6C8=; b=cbSMbDoxmQIdxP023EO9RTLBSwtC3R5Z1ZW0TLFWDv75xQLULLjs/yOayt/ogjMa+/ Y8GvuV2EgZ5zjzrY+oLo5alY31Pxv0lo5OJE8toK0zxbfaQFry/9u1ITByGbpPWsKyZW 00lrrf3M1rQ79gi4EuaghPocjHj6REqVp6lJFHIDthySRI8QDmgyc7WayZO0UTmJVuPa hvZ+yLCa/LiOTo7vbnamTHgUEcLNFwC8kW7JFvY+GaCaOdFRx47NV+QrIAgI2xmdyKmP JzLq9L8C/ZS6up6ggIhsadOqRG85mxQStKW9csMzwfFCGYmLDmnFHmFTvqe309WCRoFR t24g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pTcGVVx6qtfoQESAi4KvG6agl4eITKU7V2xGp9Tk6C8=; b=sVv62+0c2OotBfYSiREsIh/UyYSNf3f1ZZcw34BBtMVV3qKwnwYpU12+RUzOrNPr3x miKki2ljzzQ91ne94KJojwh2pYy9tr0cMCoIYcOeXZ4KDAx5+QOwOpTGVD+FR5RofiCj 7bAzcrgPsvl2vwo5tXgRZxHZNq7FsOCdH4j0Pu335Nnvn8MgLT463qM3tetnYd13Ny3G WU5qnotLqRKnhK1wlP4lxnZsc4VpxHlg5rBRpK0J6Poqpq5sKOe++m2ZUd8HCy55xeHs DD43nvscQeqNnkw7JPandxUIkMmyMJNzEcBXn1f4iIxQwA9gR5Gbnsdyu6lueHvSuo9P 7gMw== X-Gm-Message-State: AOAM532F6g8NkHqqBcqrHvhCsQ5o1KL4hwifSV4kRWCrJwRMz8gWlygr 3p/UXUkz93R6NCPt+4eD1v4RLhEL/PCvnP/Xtvq8/6Hc+TcbeQIy X-Google-Smtp-Source: ABdhPJyHD+hN8F3UnU3M7nzIwCvu/6NeKNmznDSgKK+0jMzSs4NsESv6wvZhJ/3s+Qox+q/9WrQDFQAz2QV/fbSP4Os= X-Received: by 2002:aa7:c813:: with SMTP id a19mr726044edt.136.1611503838306; Sun, 24 Jan 2021 07:57:18 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Chuck Tuffli Date: Sun, 24 Jan 2021 07:57:07 -0800 Message-ID: Subject: Re: RHEL virtualization To: John Kennedy Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4DNyLS75XCz3jgD X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=tuffli-net.20150623.gappssmtp.com header.s=20150623 header.b=cbSMbDox; dmarc=none; spf=permerror (mx1.freebsd.org: domain of chuck@tuffli.net uses mechanism not recognized by this client) smtp.mailfrom=chuck@tuffli.net X-Spamd-Result: default: False [-3.30 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[tuffli-net.20150623.gappssmtp.com:s=20150623]; FREEFALL_USER(0.00)[chuck]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tuffli.net]; SPAMHAUS_ZRD(0.00)[2a00:1450:4864:20::52e:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_PERMFAIL(0.00)[empty SPF record]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52e:from]; DKIM_TRACE(0.00)[tuffli-net.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-1.00)[-0.997]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2a00:1450:4864:20::52e:from]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jan 2021 15:57:22 -0000 On Sat, Jan 23, 2021 at 3:16 PM John Kennedy wrote: ... > That being said, RHEL on bhyve has been a pain to figure out. The best I've > done so far is using sysutils/grub2-bhyve to set up the boot CD, using > BHYVE_UEFI.fd as UEFI firmware (sysutils/bhyve-firmware I think) and then > getting at the console via net/tigervnc-viewer. > > Currently I'm fighting grub-bhyve's issue finding the kernel to load (if I'm > finding the right problem reports, it doesn't seem to like modern XFS or > ext4 partitions). I couldn't get net/ipxe to PXE boot anything, and I din't > manage to get very far with sysutils/uefi-edk2-bhyve. And of course some > of these are flagged with python2.7 isses. Hi I ran into a similar situation (RHELL VM's in bhyve) and created a version of grub-bhyve that knows how to find the grub.cfg and/or syslinux.conf files in a VM's disk image. Because of this, you don't have to maintain a copy of the grub configuration just for grub-bhyve. I implemented this on the latest version of grub which includes XFS and other goodies. The fine print: - The syntax is slightly different. To load an image use grub-bhyve --disk=/path/to/vm/disk.img --vm=vm_name note that the root, device-map, and directory options are no longer necessary - The project is at the "it works for me" stage and hasn't received much usage aside from myself. That said, feedback is welcome. - It builds on FreeBSD 12.x but 13.x has an issue Code is here: https://gitlab.com/ctuffli/grub To build: $ env PYTHON=python3.7 ./bootstrap $ env MAKE=gmake ./configure --with-platform=bhyve $ gmake The resulting binary, `grub-bhyve`, will be in the `grub-core/` directory. If you have success or troubles with it, please let me know. --chuck