From nobody Mon Feb 12 04:10:05 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TY9wv2CJZz59Py3 for ; Mon, 12 Feb 2024 04:10:39 +0000 (UTC) (envelope-from chuck@tuffli.net) Received: from wfhigh4-smtp.messagingengine.com (wfhigh4-smtp.messagingengine.com [64.147.123.155]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TY9wt35Spz529b for ; Mon, 12 Feb 2024 04:10:38 +0000 (UTC) (envelope-from chuck@tuffli.net) Authentication-Results: mx1.freebsd.org; none Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailfhigh.west.internal (Postfix) with ESMTP id 4A3461800086; Sun, 11 Feb 2024 23:10:36 -0500 (EST) Received: from imap51 ([10.202.2.101]) by compute7.internal (MEProxy); Sun, 11 Feb 2024 23:10:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tuffli.net; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm3; t=1707711035; x=1707797435; bh=O981ebh/Zg cLEj514crP1+BGzlT1GRewEisDLhIpaSM=; b=bUiU63FECuKPYDigxsqxTHj1+0 j93u5QJGkrd5YXo60Tpgq0klIlqT5nrFp1NzO/5eQ1o9/V8t6rSjVm3TcUHJBD0L YZCEAOc5u+iCFbdU3/jLSq9NKF9BtQwQJB974s+2rVCZaUB+P4hokjT0l1aZxVhU C6FxWBNDnqVX+kHE3PCtWbz+xXzZ058vaxi+0SOZJCdWnhSI34xeXsbSQL0iODNd fB0DpL9f2N0u8fo1QertPf9cHbHRFkSPcdUb4RMhr+IDWF/RFTk4dkzgf4Z4+lKK +uyvESaGor/YxVPKGxFmeTqhAom2GxRelDCvX4FK01R63FiPRQVU681MBEPQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm3; t=1707711035; x=1707797435; bh=O981ebh/ZgcLEj514crP1+BGzlT1 GRewEisDLhIpaSM=; b=LcEiheWKh4m4eHaaLkKNSMeriowmYk6HqLCavvwWL9p6 Y8qsx77DoVAa8tC0c+/lVzIgKLCD9DtvKDBeEl2zafdf6BfhERaV/l22+WbdUs92 w8uo95YBFOtK5JUSjzqdHtBZGOz5C61+icbanUHjnLL/wXctljbaBT3MeC/zeEiM SFVPZGz8KocOPOYhd9H3O5gwTtlLXjkCYS652MiG+zo5BjFKFIpCsReWXXzCEkOK iihyyJtTn2hy8AOETN6Mf8QHBpYsYgRS3098u4oIO3WhbW3e8NBjRv/3iP6fgZGX TU9jcCRERjrtAclKThCaLL8Faqb6TosYvynQmFCBrg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddvgdeikecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvufgtsegrtderreerreejnecuhfhrohhmpedfvehhuhgt khcuvfhufhhflhhifdcuoegthhhutghksehtuhhffhhlihdrnhgvtheqnecuggftrfgrth htvghrnhepveejvedttdetffekieeghfevkeejhfetuddttdeigfejgfffjeehvedttefh fedtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomheptg hhuhgtkhesthhufhhflhhirdhnvght X-ME-Proxy: Feedback-ID: ib6f94606:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 493D3B6008D; Sun, 11 Feb 2024 23:10:35 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.11.0-alpha0-144-ge5821d614e-fm-20240125.002-ge5821d61 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Message-Id: <6c9bd61c-a3ca-4470-895c-494297af0956@app.fastmail.com> In-Reply-To: <20240212012559.Horde.aH5FMCkNv4j7GfULz1fq5rn@drive.tubnor.net> References: <678ac1d2-d00c-450e-bc2e-f9d979d19340@shrew.net> <20240212012559.Horde.aH5FMCkNv4j7GfULz1fq5rn@drive.tubnor.net> Date: Sun, 11 Feb 2024 20:10:05 -0800 From: "Chuck Tuffli" To: "Jason Tubnor" , "FreeBSD virtualization" , "Matthew Grooms" Subject: Re: 14R bhyve RHEL9+NVME Content-Type: multipart/alternative; boundary=d7044c208f3a48baa3daca6471f24497 X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US] X-Rspamd-Queue-Id: 4TY9wt35Spz529b X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated --d7044c208f3a48baa3daca6471f24497 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On Sun, Feb 11, 2024, at 5:25 PM, Jason Tubnor wrote: > Hi Matthew, > =20 > There is some underlying bug post 9.1 when it comes to Anaconda and th= e presented NVMe storage. I can't reproduce it with newer version of Ana= conda that is present in Fedora. > =20 > Uplifting in place from 9.1 to 9.3, there are no issues once the opera= ting system is up and running, so it isn't a RHEL thing, just the instal= ler. > =20 > Cheers, > =20 > Jason. > =20 > "Matthew Grooms" mgrooms@shrew.net =E2=80=93 12 February 2024 11:32 AM > > Hey All, > >=20 > > Has anyone had experience installing either Rocky or Alma Linux 9.3=20 > > using virtual NVME disks on a FreeBSD 14R host? I have no problem us= ing=20 > > VirtIO block devices or if I downgrade to an 8.9 variant. However, u= sing=20 > > NVME with a 9.3 variant will cause the installer to consistently fai= l on=20 > > my system. I'm not sure how to gather more data at this point as the=20 > > installer displays a wall of text that I'm not sure how to extract. = I've=20 > > already burned a few hours on this so I thought I'd reach out to the=20 > > list to see if anyone else has already dug into this. If not, I'll s= ee=20 > > what I can do to gather more info. For reference, here are the ISOs = I've=20 > > tried installing from so far ... I have heard that anaconda has a debug mode activated by some kernel boo= t option. Maybe that could shed some light.=20 >From the NVMe side, there aren=E2=80=99t any errors or =E2=80=9Cweird=E2= =80=9D commands. My guess is the emulation is returning something anacon= da doesn=E2=80=99t like, but it=E2=80=99s hard to tell from the Python e= xception what that might be.=20 Note the text/rescue installer uses tmux which gives better visibility i= nto the logs.=20 --d7044c208f3a48baa3daca6471f24497 Content-Type: text/html;charset=utf-8 Content-Transfer-Encoding: quoted-printable

=
On Sun, Feb 11, 2024, at 5:25 PM, Jason Tubnor wrote:
=
Hi Matthew,
 
There is some underlying bug post 9.1 w= hen it comes to Anaconda and the presented NVMe storage. I can't reprodu= ce it with newer version of Anaconda that is present in Fedora.
 
Uplifting in place from 9.1 to 9.3, there are= no issues once the operating system is up and running, so it isn't a RH= EL thing, just the installer.
 
Cheers,=
 
Jason.
 
"Matthew Grooms" mgrooms@shrew.net =E2=80=93 12 February 2024 11:32 AM
> Hey All,
> 
> Has anyone = had experience installing either Rocky or Alma Linux 9.3 
=
> using virtual NVME disks on a FreeBSD 14R host? I have no prob= lem using 
> VirtIO block devices or if I downgrad= e to an 8.9 variant. However, using 
> NVME with a= 9.3 variant will cause the installer to consistently fail on 
<= /div>
> my system. I'm not sure how to gather more data at this p= oint as the 
> installer displays a wall of text t= hat I'm not sure how to extract. I've 
> already b= urned a few hours on this so I thought I'd reach out to the 
> list to see if anyone else has already dug into this. If no= t, I'll see 
> what I can do to gather more info. = For reference, here are the ISOs I've 
> tried ins= talling from so far ...

I have= heard that anaconda has a debug mode activated by some kernel boot opti= on. Maybe that could shed some light.

From= the NVMe side, there aren=E2=80=99t any errors or =E2=80=9Cweird=E2=80=9D= commands. My guess is the emulation is returning something anaconda doe= sn=E2=80=99t like, but it=E2=80=99s hard to tell from the Python excepti= on what that might be. 

Note the text/= rescue installer uses tmux which gives better visibility into the logs.&= nbsp;
--d7044c208f3a48baa3daca6471f24497--