From nobody Fri Feb 14 23:16:25 2025 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YvnxK6fhWz5npTZ for ; Fri, 14 Feb 2025 23:16:37 +0000 (UTC) (envelope-from peter.garshtja@ambient-md.com) Received: from mail-pj1-x1034.google.com (mail-pj1-x1034.google.com [IPv6:2607:f8b0:4864:20::1034]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YvnxK1nNwz3Lyp for ; Fri, 14 Feb 2025 23:16:37 +0000 (UTC) (envelope-from peter.garshtja@ambient-md.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1034.google.com with SMTP id 98e67ed59e1d1-2f833af7a09so3814947a91.2 for ; Fri, 14 Feb 2025 15:16:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ambient-md-com.20230601.gappssmtp.com; s=20230601; t=1739574996; x=1740179796; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=byPSXc7mtc5vbwbzkjYX0mM3FLuBxtOVyxuKMxMECJ0=; b=Emh84iblXHMPQ8d4g36eb6Vy6c4qhZvBmZs023NFNZmEbZiENYi5rtgilqwzq9km2N ViD87VRJMR3g+McxJ1T6BhGJubhc2KuUe5gw1kWBDXvs9qoMnsT4HgpmWCgGQuvnwXne /Zkdpti2wf9XEdwbNT3B6th6cSZEgD3KKqn+Hk0rbDdsCAFTfkJASNt/5diODafOWB/C HkXL8YujBltKGcSJEPQQTA9JAzL9e9ck5Ng4SErdvl77bMHDJhOgE4FA+wDdNL8E0F0W xloT5Y/cfwhgbnA7NpcAPQYFUyz/oF+dWHgoxeHZggAdf59ZPUhL5WqJGuOi5xWXEhFX ixlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739574996; x=1740179796; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=byPSXc7mtc5vbwbzkjYX0mM3FLuBxtOVyxuKMxMECJ0=; b=NZUg1LGPPfGpM0fT/m/eGIOC5m4EysmtvzO99INeLu3qFYmW+wq9SixPMgXXfp1o+4 gRaiX1Fjg43xgYN2KUAI8948U6T+Og+L/5CYxqxrP9bxoyStJswqGjFGRnAYAGnCxxUP wgrNNSd7e73+KAqP7zF9BFzzCsBnTq+H1fANMMo8yysYRBhKRq2b9rFZFRN/K1NwEibb eScQ90BPAwG+HN4BKj7uZuGAtx35oSqCv56WnOtc/+t6B1uLfvsLIrh0a9KSl7F1z+nL 2Vrm/fG7t/C8lIksVnNhlWbNSP5Esxjh/GBT2/Q/wy2/l5m9HaLt1W+7MqSpw+GbSizD 7m5Q== X-Forwarded-Encrypted: i=1; AJvYcCWZYi7NT3NUGDF5PIwYhyJk5cKQST+3V+dvPEZ/d1rZFXQmvUOhcU205q2R101IlmST0iMZjOIK88XV++ck3tg6pi54Gc9w@freebsd.org X-Gm-Message-State: AOJu0YyBXCOtTI+o+KgcbGKnwEixd9ZSZjVWy5XrEfuutQsiC5UoN3It Q75kbD5B60k/rInAkcxRz3m76oiOyv+BcmqWaiHOskwMki2Np8ygnp7PQof9yKF8Kw91M/Eh80I wbLzGFm4eA+N8h/7DZnSm+RkjKI60lhKQ5z1uQsNc8g/pRQPp X-Gm-Gg: ASbGnctvlfc2ZTS6IW2Lct65miXSlbg6bRkrQRMzqbeLN8Rp1DBpj0Z3dC97a0ZsYLD x0QZgutUeXzb0g48cSflA44z745VhZ3/KH98bhNPo6fVAiyx4Nv50HlrFGjY/vVBQhuvVFyU= X-Google-Smtp-Source: AGHT+IGSjiM+4jXuf05S4bbe85FoxlZtcEZabj57lyrTBiI0daxZTTxHfT5NAM3tGShkGqyBAD9s7rZV9RolG7EqS7I= X-Received: by 2002:a17:90b:2d8d:b0:2ee:f076:20fb with SMTP id 98e67ed59e1d1-2fc40f22db2mr1560462a91.17.1739574995590; Fri, 14 Feb 2025 15:16:35 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <9865e8dd-7af9-4cc6-97c7-2e3197c62e62@ambient-md.com> <899331A6-1635-4546-ADE6-FE21B52138FF@freebsd.org> In-Reply-To: <899331A6-1635-4546-ADE6-FE21B52138FF@freebsd.org> From: peter garshtja Date: Fri, 14 Feb 2025 18:16:25 -0500 X-Gm-Features: AWEUYZnh-A9d8Rpj7xuKyGvbP1JZ6gAfIdBcL4FQPgNGW41toKYvzl8nMMjbKPs Message-ID: Subject: Re: bhyve vm and virtio-console To: Michael Gmelin Cc: Paul Vixie , virtualization@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000ac3c99062e225da4" X-Rspamd-Queue-Id: 4YvnxK1nNwz3Lyp X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] --000000000000ac3c99062e225da4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Exactly, I expect to get the shell after I connect with socat, something similar to docker exec -it. On Fri, Feb 14, 2025, 18:10 Michael Gmelin wrote: > > > > On 14. Feb 2025, at 23:46, Petru Garstea > wrote: > > > > =EF=BB=BFSpinning up the vm with /dev/nmdm* and connecting with cu -l i= s not an > issue on my end. > > I am trying to create a vm using freebsd cloud image and connect to > guest OS terminal with virtio-console and socat(no X11). > > When spawning the vm the unix socket indeed is created, however when I > connect to it with socat nothing is being shown(not actions on any termin= al > commands as well). > > Also, I noticed there is a kernel module(virtio_console.ko) and > apparently I believe it has to be loaded it, not sure here. > > > > Please advise, > > > > You can connect to the device inside the vm like I wrote and pass data to > the socat connected socket on the outside (this is what I tested, please > see my previous email). > > Are you trying to get some sort of shell over this connection? > > -m > > > >> On 2/14/25 17:23, Paul Vixie wrote: > >> if you want to use cu, you have to use -l to map guest com1 to some > host nmdm > >> device. if you want to use virtio-console, it will create unix domain > sockets > >> not pseudo tty endpoints. from bhyve(8) we see: > >> > >>> Virtio console device backends: > >>> =E2=80=A2 > >>> port1=3D/path/to/port1.sock[,portN=3D/path/to/port2.= sock > ...] > >>> A maximum of 16 ports per device > can be created. Every > >>> port > >>> is named and corresponds to a Unix domain socket > created by > >>> bhyve. bhyve accepts at most one connection per por= t > at a > >>> time. > >>> Limitations: > >>> =E2=80=A2 Due to lack of destruc= tors in > bhyve, sockets on the > >>> filesystem must be cleaned up manually after bhy= ve > >>> exits. > >>> =E2=80=A2 There is no way to use= the > =E2=80=9Cconsole port=E2=80=9D feature, nor > >>> the console port resize at present. > >>> =E2=80=A2 Emergency write is adv= ertised, > but no-op at present. > >> i know of no host-side client which will connect to these unix domain > sockets > >> and offer a cu-like user interface. "unidirectional" may be the > experience > >> you'll get from "socat". > >> > >> if what you're trying to accomplish is a graphical console, > virtio-console is > >> not the way to do it. rather, you'll want "fbuf" to create a guest > VGA/similar > >> that gateways to a host VNC. > >> > >> re: > >> > >>> On Friday, February 14, 2025 9:55:23 PM UTC Petru Garstea wrote: > >>> I tried with the following command, and got the same result. > >>> > >>> */usr/sbin/bhyve -H -P -A -c 1 -m 1500M \ > >>> -s 0,hostbridge \ > >>> -s 6,virtio-console,0=3D/tmp/vtcon.0 \ > >>> -s 31,lpc \ > >>> -l bootrom,/tmp/BHYVE_UEFI.fd \ > >>> -s 1,virtio-blk,/tmp/fbsd.img \ > >>> -s 2,virtio-net,tap0 \ > >>> tmp* > >>> > >>> > >>> I also able to connect but the bidirectional connection is not > happening > >>> > >>> *socat - UNIX-CONNECT:/tmp/vtcon.0 > >>> ls > >>> * > >>> PS: if I use null modem emulation instead virtio-console, then > >>> connecting with cu is not an issue. > >>> > >>> any ideas ? > >>> > >>> Thanks > >>> > >>> On 2/14/25 14:57, Michael Gmelin wrote: > >>>> On Fri, 14 Feb 2025 14:21:07 -0500 > >>>> > >>>> Petru Garstea wrote: > >>>>> Hi, > >>>>> I am trying to configure bhyve vm with support of virtio-console, t= he > >>>>> following option is added to bhyve command line > >>>>> > >>>>> *-s 3,virtio-console,port1=3D/test/console.sock > >>>>> > >>>>> *However when trying to connect to socket with socat trere are no > >>>>> input/ouptut is observed. > >>>>> > >>>>> Has virtio-console ever worked ? > >>>> Yes it works, I just tried it using sysutils/vm-bhyve (just add > >>>> virt_console0=3D"Yes" to the configuration file). > >>>> > >>>> Based on the logs, these device settings are used by it: > >>>> bhyve devices: > >>>> -s 0,hostbridge \ > >>>> -s 31,lpc \ > >>>> -s 4:0,virtio-blk,/dev/zvol/zroot/vms/freebsd14-test/disk0 \ > >>>> -s 5:0,virtio-net,tap0,mac=3D58:9c:fc:aa:aa:aa \ > >>>> -s 6:0,virtio-console,0=3D/vms/freebsd14-test/vtcon.0 > >>>> > >>>> I could connect to /vms/freebsd14-test/vtcon.0 using socat: > >>>> socat - UNIX-CONNECT:/vms/freebsd14-test/vtcon.0 > >>>> > >>>> Inside the vm it's /dev/vtcon/0. > >>>> > >>>> -m > >> > > > > --000000000000ac3c99062e225da4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Exactly, I expect to get the shell after I connect with s= ocat, something similar to docker exec -it.=C2=A0

On F= ri, Feb 14, 2025, 18:10 Michael Gmelin <grembo@freebsd.org> wrote:


> On 14. Feb 2025, at 23:46, Petru Garstea <peter.garshtja= @ambient-md.com> wrote:
>
> =EF=BB=BFSpinning up the vm with /dev/nmdm* and connecting with cu -l = is not an issue on my end.
> I am trying to create a vm using freebsd cloud image and connect to gu= est OS terminal with virtio-console and socat(no X11).
> When spawning the vm the unix socket indeed is created, however when I= connect to it with socat nothing is being shown(not actions on any termina= l commands as well).
> Also, I noticed there is a kernel module(virtio_console.ko) and appare= ntly I believe it has to be loaded it, not sure here.
>
> Please advise,
>

You can connect to the device inside the vm like I wrote and pass data to t= he socat connected socket on the outside (this is what I tested, please see= my previous email).

Are you trying to get some sort of shell over this connection?

-m


>> On 2/14/25 17:23, Paul Vixie wrote:
>> if you want to use cu, you have to use -l to map guest com1 to som= e host nmdm
>> device. if you want to use virtio-console, it will create unix dom= ain sockets
>> not pseudo tty endpoints. from bhyve(8) we see:
>>
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = Virtio console device backends:
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2 >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = port1=3D/path/to/port1.sock[,portN=3D/path/to/port2.sock ...]
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 A maximum of= 16 ports per device can be created.=C2=A0 Every
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = port
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = is named and corresponds to a Unix domain socket created by
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = bhyve.=C2=A0 bhyve accepts at most one connection per port at a
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = time.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Limitations:=
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0Due to lack of destructors in bhyve, sockets on the
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 filesystem must be cleaned up manually after bhyve
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 exits.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0There is no way to use the =E2=80=9Cconsole port=E2=80=9D feature= , nor
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 the console port resize at present.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0Emergency write is advertised, but no-op at present.
>> i know of no host-side client which will connect to these unix dom= ain sockets
>> and offer a cu-like user interface. "unidirectional" may= be the experience
>> you'll get from "socat".
>>
>> if what you're trying to accomplish is a graphical console, vi= rtio-console is
>> not the way to do it. rather, you'll want "fbuf" to = create a guest VGA/similar
>> that gateways to a host VNC.
>>
>> re:
>>
>>> On Friday, February 14, 2025 9:55:23 PM UTC Petru Garstea wrot= e:
>>> I tried with the following command, and got the same result. >>>
>>> */usr/sbin/bhyve -H -P -A -c 1 -m 1500M \
>>> -s 0,hostbridge \
>>> -s 6,virtio-console,0=3D/tmp/vtcon.0 \
>>> -s 31,lpc \
>>> -l bootrom,/tmp/BHYVE_UEFI.fd \
>>> -s 1,virtio-blk,/tmp/fbsd.img \
>>> -s 2,virtio-net,tap0 \
>>> tmp*
>>>
>>>
>>> I also able to connect but the bidirectional connection is not= happening
>>>
>>> *socat - UNIX-CONNECT:/tmp/vtcon.0
>>> ls
>>> *
>>> PS: if I use null modem emulation instead virtio-console, then=
>>> connecting with cu is not an issue.
>>>
>>> any ideas ?
>>>
>>> Thanks
>>>
>>> On 2/14/25 14:57, Michael Gmelin wrote:
>>>> On Fri, 14 Feb 2025 14:21:07 -0500
>>>>
>>>> Petru Garstea<peter.garshtja@ambient-md.com<= /a>> wrote:
>>>>> Hi,
>>>>> I am trying to configure bhyve vm with support of virt= io-console, the
>>>>> following option is added to bhyve command line
>>>>>
>>>>> *-s 3,virtio-console,port1=3D/test/console.sock
>>>>>
>>>>> *However when trying to connect to socket with socat t= rere are no
>>>>> input/ouptut is observed.
>>>>>
>>>>> Has virtio-console ever worked ?
>>>> Yes it works, I just tried it using sysutils/vm-bhyve (jus= t add
>>>> virt_console0=3D"Yes" to the configuration file)= .
>>>>
>>>> Based on the logs, these device settings are used by it: >>>>=C2=A0 =C2=A0 =C2=A0 bhyve devices:
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 0,hostbridge \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 31,lpc \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 4:0,virtio-blk,/dev/zvol/zro= ot/vms/freebsd14-test/disk0 \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 5:0,virtio-net,tap0,mac=3D58= :9c:fc:aa:aa:aa \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 6:0,virtio-console,0=3D/vms/= freebsd14-test/vtcon.0
>>>>
>>>> I could connect to /vms/freebsd14-test/vtcon.0 using socat= :
>>>>=C2=A0 =C2=A0 =C2=A0 socat - UNIX-CONNECT:/vms/freebsd14-te= st/vtcon.0
>>>>
>>>> Inside the vm it's /dev/vtcon/0.
>>>>
>>>> -m
>>
>

--000000000000ac3c99062e225da4--