From nobody Tue Jul 16 20:39:47 2024 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNrXx5xc4z5QC8X for ; Tue, 16 Jul 2024 20:40:01 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNrXx2Mh7z4Fbn for ; Tue, 16 Jul 2024 20:40:01 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-2c96187b3d1so3876638a91.3 for ; Tue, 16 Jul 2024 13:40:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1721162399; x=1721767199; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=BhYkpcI1gCVbBUCeG/aPIKuQ5zZUJ8jagxD4SOY3TKU=; b=C4puxin66oYFnVWU3KdDAF5SvB9GwIaH6wyn6YKRM4wHB+PEaPL2QusaUHLpyzOzpj q7nBIduE0PM7KG8qbLyo2OhIrccyhdvts3OgXrdYLEhuYFe1CCX4k1px0aWQl5spj1Fp Yg8Vf3j1Lzf5oTjNrKu6YaqdbvTJG+kxi92XHShSsgHttvPBgsoU8rc4jZI1UXtEWTr0 WxCX2pl4lKH6ixAczB36HSxYicjQoHsGg3eHCKkR3wtIi+mWyX3r+UvYo0r/4eUU78Mt PgNsD5X96CmOnbieL6MQzmvFWCxILR8RiaJ5jvnPi0nXcjATfqqD88t3/jpUFILJ8PT5 gDxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1721162399; x=1721767199; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BhYkpcI1gCVbBUCeG/aPIKuQ5zZUJ8jagxD4SOY3TKU=; b=kOa7nRjxxYNwfBHGS6qv00I2lIikZzhMe7ZY+tzZiJDIh+Tm0XvUZedjoAAlKWbY1B VmIikJrV75Jy6P+XqaCNXyKy6rTdVvj9lg76b02D5IM6OexVFzQxYhN6K/yrcgK9VzQZ Xy0mxjfmqinLE+7kKNQArcD9pEJAEREAOYRwLucUE3yvwnwNcGNSWrTGZqXzKF657wB6 p8h4Rk+93S6omk2mtmp/zvv43Abkoa8z96puP5IekwlPtRFvn3Omx+jKiA3kpBifyyoz LYkQpILWVGs1O4mPdma9KDsz9IUsRnm7Dlfk0elk1vJkqveFpHQt94O4Mo7bJKPpU9ZU Jyow== X-Gm-Message-State: AOJu0YwWphIcQDUmeAflmhz7K4iefCGAhyVNa7+5V/rC5R3/SeXsjFof p1yee/z3ZNSVmGwHXZP41VWd3gMseZaUSTQTW0ICVkYPMr0U3hsbYmI6EjhhfQ== X-Google-Smtp-Source: AGHT+IFnV3n7NUs1v/ARBr/qjJLO9Szn6IPoAxPIf7lobY17q25/105SjWIqQvT+AeQbcSoBPDY9rg== X-Received: by 2002:a17:90a:62c6:b0:2c9:358c:babf with SMTP id 98e67ed59e1d1-2cb37455deamr2609417a91.36.1721162398831; Tue, 16 Jul 2024 13:39:58 -0700 (PDT) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2cacd3e5a87sm8826794a91.14.2024.07.16.13.39.57 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jul 2024 13:39:58 -0700 (PDT) Content-Type: text/plain; charset=utf-8 List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.600.62\)) Subject: Re: nmdm issues with bhyve From: Bakul Shah In-Reply-To: <16dfe6b3-2ef1-4859-a5b0-a36efe452804@FreeBSD.org> Date: Tue, 16 Jul 2024 13:39:47 -0700 Cc: virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20240716153851.60c1ea61@PolarianBSD> <16dfe6b3-2ef1-4859-a5b0-a36efe452804@FreeBSD.org> To: Kyle Evans X-Mailer: Apple Mail (2.3774.600.62) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WNrXx2Mh7z4Fbn > On Jul 16, 2024, at 1:15=E2=80=AFPM, Kyle Evans = wrote: >=20 > On 7/16/24 09:38, Polarian wrote: >> Hello, >> Over the last few days I have been trying to get nmdm working with >> bhyve. I have discussed it within #bhyve over on libera.chat however >> none of the suggestions so far have fixed the problem, therefore I am >> bringing it to the mailing list for further support. >> A rundown what I have done so far, I have a shell script with the >> following contents: >> #!/bin/sh >> bhyve -c 1 -m 1G -u -H \ >> -s 0,amd_hostbridge \ >> -s 3,ahci-hd,/path/to/install.img \ >> -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \ >> -s 5,virtio-net,tap0 \ >> -s 31,lpc -l com1,/dev/nmdm3A \ >> -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ >> dns >> To verify the virtual machine was in fact booting, I did test stdio, >> and the install medium for OpenBSD does successfully boot. >> The problem, well there is many. Following the FAQ entry [1], the >> following flag is suggested: >> -s 31,lpc -l com1,nmdm0A >> (where 0 can be substituted to prevent collisions) >> Enter problem #1, when I try this by executing the script (with doas, >> doas sh dns.sh) I get the following: >> Unable to initialize backend 'nmdm3A' for LPC device com1 >> Device emulation initialization error: No such file or directory >> Now, both vmm and nmdm have been loaded, checked with the following >> commands: >> kldstat | grep vmm >> kldstat | grep nmdm >> And these have been entered into /boot/loader.conf see below: >> vmm_load=3D"YES" >> nmdm_load=3D"YES" >> So they have been loaded into the kernel. >> When looking into the problem I found an article on the FreeBSD = forums >> which has the same error [2], which would suggest the example in the >> FAQ is either wrong, or deprecated and no longer works. >> The solution from this article is to ensure to specify the /dev path, = as >> seen in the full script at the beginning of the email, /dev/nmdm0A, >> this DOES work and the vm does startup. >> Problem #2, the device doesn't show up, even though the vm is, in = fact, >> running. >> ls /dev | grep nmdm >> The above returns nothing, there is no nmdm device. I have tested it = to >> ensure nmdm is in fact working, using cu to open both A and B and >> verifying that the data is being passes between them. So nmdm is >> working! >=20 > We did some light debugging in #freebsd on this; bhyve(8) is opening = the nmdm pair as it should and that works fine, but I guess at some = point (perhaps right after handoff to the OpenBSD kernel) it's getting = closed so the device isn't around anymore by the time they try to = observe it above. They attempted a run with stdio in otherwise the same = configuration, and it cut off right after: >=20 > booting hd0a:/7.5/amd64/bsd.rd: 4076463+1688576+3891240+0+708608 = [109+464016+317541]=3D0xaa40e8 > entry point at 0x1001000 > wrmsr to register 0xc0011029(0x3) on vcpu 0 This is what I used for openbsd (later upgraded to 7.5): bhyve -c 1 \ -s 0,hostbridge \ -s 4,ahci-hd,install74.img \ -s 5,nvme,obsd.img \ -s 10,e1000,tap6 \ -s 11,fbuf,tcp=3D0.0.0.0:5907 \ -s 20,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -m 2G -H -w \ -l com1,/dev/nmdm4B \ obsd exit 0 Once installed, move the "-s 4,..." line below "exit 0". Move the "-s 11,..." line below exit if you don't want to connect via VNC. Never tried installing 7.5 from scratch.