From nobody Tue Jul 16 14:38:51 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WNhXH1y1Zz5RQ74 for ; Tue, 16 Jul 2024 14:38:55 +0000 (UTC) (envelope-from polarian@polarian.dev) Received: from mail.polarian.dev (mail.polarian.dev [IPv6:2001:8b0:57a:2385::8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4WNhXG06ldz4P2Y for ; Tue, 16 Jul 2024 14:38:54 +0000 (UTC) (envelope-from polarian@polarian.dev) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=polarian.dev header.s=polarian header.b=pR26ezEV; dmarc=pass (policy=quarantine) header.from=polarian.dev; spf=pass (mx1.freebsd.org: domain of polarian@polarian.dev designates 2001:8b0:57a:2385::8 as permitted sender) smtp.mailfrom=polarian@polarian.dev Received: from PolarianBSD (_gateway [192.168.2.1]) by mail.polarian.dev (Postfix) with ESMTPSA id 6232010A071E for ; Tue, 16 Jul 2024 14:38:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/relaxed; d=polarian.dev; s=polarian; t=1721140732; bh=cgLGDug0diXGZzAZwN/gSShEGJshDu6SdYcaT6umB9M=; h=Date:From:To:Subject; b=pR26ezEVWO3eIFdF8dpU0K7VPnXQ+RR6IberST6jrsqXXYJETavhPzKV9zvf6KcCp 1Zxd1YDBW0AUiZIpGkSdqchFzV7UTTA7kCUHPjlX7WTCT7a68h7Pwin+qKmDhEttM/ gBxxQcdR/qwGWXAvWmbsYZl2RgZtRXR0l9sZ/GAQ= Date: Tue, 16 Jul 2024 15:38:51 +0100 From: Polarian To: freebsd-virtualization@FreeBSD.org Subject: nmdm issues with bhyve Message-ID: <20240716153851.60c1ea61@PolarianBSD> X-Mailer: Claws Mail 3.20.0 (GTK+ 2.24.33; amd64-portbld-freebsd14.0) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[polarian.dev,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:8b0:57a:2385::8]; R_DKIM_ALLOW(-0.20)[polarian.dev:s=polarian]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[polarian.dev:+]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:20712, ipnet:2001:8b0::/32, country:GB]; ARC_NA(0.00)[]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-virtualization@FreeBSD.org]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4WNhXG06ldz4P2Y Hello, Over the last few days I have been trying to get nmdm working with bhyve. I have discussed it within #bhyve over on libera.chat however none of the suggestions so far have fixed the problem, therefore I am bringing it to the mailing list for further support. A rundown what I have done so far, I have a shell script with the following contents: #!/bin/sh bhyve -c 1 -m 1G -u -H \ -s 0,amd_hostbridge \ -s 3,ahci-hd,/path/to/install.img \ -s 4,virtio-blk,/dev/zvol/zpool-storage/dns \ -s 5,virtio-net,tap0 \ -s 31,lpc -l com1,/dev/nmdm3A \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ dns To verify the virtual machine was in fact booting, I did test stdio, and the install medium for OpenBSD does successfully boot. The problem, well there is many. Following the FAQ entry [1], the following flag is suggested: -s 31,lpc -l com1,nmdm0A (where 0 can be substituted to prevent collisions) Enter problem #1, when I try this by executing the script (with doas, doas sh dns.sh) I get the following: Unable to initialize backend 'nmdm3A' for LPC device com1 Device emulation initialization error: No such file or directory Now, both vmm and nmdm have been loaded, checked with the following commands: kldstat | grep vmm kldstat | grep nmdm And these have been entered into /boot/loader.conf see below: vmm_load="YES" nmdm_load="YES" So they have been loaded into the kernel. When looking into the problem I found an article on the FreeBSD forums which has the same error [2], which would suggest the example in the FAQ is either wrong, or deprecated and no longer works. The solution from this article is to ensure to specify the /dev path, as seen in the full script at the beginning of the email, /dev/nmdm0A, this DOES work and the vm does startup. Problem #2, the device doesn't show up, even though the vm is, in fact, running. ls /dev | grep nmdm The above returns nothing, there is no nmdm device. I have tested it to ensure nmdm is in fact working, using cu to open both A and B and verifying that the data is being passes between them. So nmdm is working! AllanJude on IRC suggested that I change from 0 to some other number in case of conflicts, and so I did (as seen in the script at the beginning of the email) changing 0 --> 3, same issue remains. I can find no further information on this issue, and I am out of ideas, so if anyone has nmdm successfully working, could you give me a hand in getting it to work on my server? The FAQ [1] also suggested pty can be used, or a terminal multiplexer such as tmux, however I am yet to find any examples on how this could be done to substitute nmdm. If anyone has some useful links to some examples for these, that would be great! Thank you, -- Polarian GPG signature: 0770E5312238C760 Jabber/XMPP: polarian@icebound.dev [1] https://wiki.freebsd.org/bhyve#Q:_Does_bhyve_have_an_.22out-of-band.22.2F.22lights-out_management.22_serial_console.3F [2] https://forums.freebsd.org/threads/unable-to-initialize-backend-nmdm0a-for-lpc-device-com1.82177/