From nobody Sat Feb 15 00:36:21 2025 X-Original-To: virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YvqjZ5zRnz5ntxf for ; Sat, 15 Feb 2025 00:36:34 +0000 (UTC) (envelope-from peter.garshtja@ambient-md.com) Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YvqjY4gxfz46wF for ; Sat, 15 Feb 2025 00:36:33 +0000 (UTC) (envelope-from peter.garshtja@ambient-md.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ambient-md-com.20230601.gappssmtp.com header.s=20230601 header.b=uI8W+QVa; spf=none (mx1.freebsd.org: domain of peter.garshtja@ambient-md.com has no SPF policy when checking 2607:f8b0:4864:20::102d) smtp.mailfrom=peter.garshtja@ambient-md.com; dmarc=none Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-2fc0d44a876so3873439a91.3 for ; Fri, 14 Feb 2025 16:36:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ambient-md-com.20230601.gappssmtp.com; s=20230601; t=1739579791; x=1740184591; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=f3u+I6v5nFc7tqRe002V0lshaj0FqAzWZ/4VVj4L8b0=; b=uI8W+QVa7xepNKnqDfsPq0Di47wkqmsPe1f/DcWeZoqask87JduTo0TteI/st8VREB 4L88kF7RGMFBfNKwYjiyOTOB6I7VTd7Mj4ke4Lo/TDxGBBMdAW9FSFxzYv2Swcw5a1a4 raSJdgryf1Oo+OwlJ63EK4j00rScM9TpMDcHDMO+I2vviXtgsOvzdB8mP+Hup7FzLKzt B6Cp4jDBaxZJVdKlfzH26Iqt997saEqL2zaaWLl7TyItAkrGAhyNu2lkAvetgrXvHsud 9tpuyGQfi8AoXMrKbNlNILU3dk5NfRQ2b5mbJF2d1o+tXFXfzwjirG+P1bvLJeqRRFUV OSlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1739579791; x=1740184591; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=f3u+I6v5nFc7tqRe002V0lshaj0FqAzWZ/4VVj4L8b0=; b=ZAVmZ9OCYli/XwCMqyJZ/exuxEZ2v+FxfBU09XaBKNnatprbAY8bixnawm4HMoP16z JMaYVzItXcvdvoLDq6TuhHg13H/h8J4znAzMAuBLiV3Od8kP3Jvdi4hzoGZfxGuKKTUC JImMkSEt4YMCcqH37mto9EzVPYFUtwYUs4oPUVmY96Tk9qD/XwDbekgDB+E3+ocaTwfP bEYVdgHjQX1bm881hLrlR8CHDHmHVHdEkAZLQGeIdK0tKpqL+Z8JG1Rpi9OhjjM6RCqh WBpS1aNFAt1p5GQtPm0Hkm9OE7RZgV12eBK9QBRJlM7I3j1DdCRTNFdtGtfQLlm956ex drHg== X-Forwarded-Encrypted: i=1; AJvYcCXVC/4sHZIZMhcIIjpc5v1ARlObqhVgZsLKqKAVML5mTKjKNowm2Y5MgnRr9JN3Cl6fiwrAxWU0CHDHbtpfcQ==@freebsd.org X-Gm-Message-State: AOJu0Yxe7HpYOOOZcivnhC3EhQXWsjVo8+1IKL5E7gg0b4GqumJp6/LG XFK7ftofFSSPXDmLfRXPtQIvdyg4KbEXhnDpPsqtEt6EtYeeY/vDVCsNdDHJ91CcFUaq0J6+rLT yWoG6KdOWt5yxIz5XUoQVccwegJa/Lb5RkM2W6A== X-Gm-Gg: ASbGncvudjdHFygG5KJ49TfCUeKEa19RLJD+wxTCCjeGo2/ELy9Om3saEBV2tBDJBOK QvWAjazIbXBIIpO8SKE6Gmx7jpEAM7Uvko8wgH6aiFtBALbmdemZ9p5zNhe4pKRz8m4lfU1A= X-Google-Smtp-Source: AGHT+IGnlLSCf3F6rJjk7dLgtFniL2q241a9a2zg/qAiUJRHrQw6Lw2a3mxg24YPhkPSH7NP+L9IPP28CIKGmD9pJK8= X-Received: by 2002:a17:90b:3c0e:b0:2ee:c2df:5d30 with SMTP id 98e67ed59e1d1-2fc41049570mr1472927a91.26.1739579791500; Fri, 14 Feb 2025 16:36:31 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <9865e8dd-7af9-4cc6-97c7-2e3197c62e62@ambient-md.com> <899331A6-1635-4546-ADE6-FE21B52138FF@freebsd.org> In-Reply-To: From: peter garshtja Date: Fri, 14 Feb 2025 19:36:21 -0500 X-Gm-Features: AWEUYZk2mRHyIuc1VTQkzhK_SEVrUA-lfuugUdiGFwd7q5RjI4y-cr66FATnV0c Message-ID: Subject: Re: bhyve vm and virtio-console To: Michael Gmelin Cc: Paul Vixie , virtualization@freebsd.org, freebsd-virtualization@freebsd.org Content-Type: multipart/alternative; boundary="000000000000880367062e237b49" X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[ambient-md-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; MISSING_XM_UA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::102d:from]; MLMMJ_DEST(0.00)[virtualization@freebsd.org]; RCPT_COUNT_THREE(0.00)[4]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[ambient-md.com]; R_SPF_NA(0.00)[no SPF record]; PREVIOUSLY_DELIVERED(0.00)[virtualization@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[ambient-md-com.20230601.gappssmtp.com:+] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4YvqjY4gxfz46wF --000000000000880367062e237b49 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Alright, I was missing the point, in my case if I want to get shell access via virtio-console then I should have attached getty to that console inside the guest os, and eventually I expected to see the same behavior as I have experienced with null modem emulator. Thx On Fri, Feb 14, 2025, 18:16 peter garshtja wrote: > Exactly, I expect to get the shell after I connect with socat, something > similar to docker exec -it. > > On Fri, Feb 14, 2025, 18:10 Michael Gmelin wrote: > >> >> >> > On 14. Feb 2025, at 23:46, Petru Garstea >> wrote: >> > >> > =EF=BB=BFSpinning up the vm with /dev/nmdm* and connecting with cu -l = is not an >> issue on my end. >> > I am trying to create a vm using freebsd cloud image and connect to >> guest OS terminal with virtio-console and socat(no X11). >> > When spawning the vm the unix socket indeed is created, however when I >> connect to it with socat nothing is being shown(not actions on any termi= nal >> commands as well). >> > Also, I noticed there is a kernel module(virtio_console.ko) and >> apparently I believe it has to be loaded it, not sure here. >> > >> > Please advise, >> > >> >> You can connect to the device inside the vm like I wrote and pass data t= o >> the socat connected socket on the outside (this is what I tested, please >> see my previous email). >> >> Are you trying to get some sort of shell over this connection? >> >> -m >> >> >> >> On 2/14/25 17:23, Paul Vixie wrote: >> >> if you want to use cu, you have to use -l to map guest com1 to some >> host nmdm >> >> device. if you want to use virtio-console, it will create unix domain >> sockets >> >> not pseudo tty endpoints. from bhyve(8) we see: >> >> >> >>> Virtio console device backends: >> >>> =E2=80=A2 >> >>> port1=3D/path/to/port1.sock[,portN=3D/path/to/port2= .sock >> ...] >> >>> A maximum of 16 ports per device >> can be created. Every >> >>> port >> >>> is named and corresponds to a Unix domain socket >> created by >> >>> bhyve. bhyve accepts at most one connection per >> port at a >> >>> time. >> >>> Limitations: >> >>> =E2=80=A2 Due to lack of destru= ctors in >> bhyve, sockets on the >> >>> filesystem must be cleaned up manually after >> bhyve >> >>> exits. >> >>> =E2=80=A2 There is no way to us= e the >> =E2=80=9Cconsole port=E2=80=9D feature, nor >> >>> the console port resize at present. >> >>> =E2=80=A2 Emergency write is ad= vertised, >> but no-op at present. >> >> i know of no host-side client which will connect to these unix domain >> sockets >> >> and offer a cu-like user interface. "unidirectional" may be the >> experience >> >> you'll get from "socat". >> >> >> >> if what you're trying to accomplish is a graphical console, >> virtio-console is >> >> not the way to do it. rather, you'll want "fbuf" to create a guest >> VGA/similar >> >> that gateways to a host VNC. >> >> >> >> re: >> >> >> >>> On Friday, February 14, 2025 9:55:23 PM UTC Petru Garstea wrote: >> >>> I tried with the following command, and got the same result. >> >>> >> >>> */usr/sbin/bhyve -H -P -A -c 1 -m 1500M \ >> >>> -s 0,hostbridge \ >> >>> -s 6,virtio-console,0=3D/tmp/vtcon.0 \ >> >>> -s 31,lpc \ >> >>> -l bootrom,/tmp/BHYVE_UEFI.fd \ >> >>> -s 1,virtio-blk,/tmp/fbsd.img \ >> >>> -s 2,virtio-net,tap0 \ >> >>> tmp* >> >>> >> >>> >> >>> I also able to connect but the bidirectional connection is not >> happening >> >>> >> >>> *socat - UNIX-CONNECT:/tmp/vtcon.0 >> >>> ls >> >>> * >> >>> PS: if I use null modem emulation instead virtio-console, then >> >>> connecting with cu is not an issue. >> >>> >> >>> any ideas ? >> >>> >> >>> Thanks >> >>> >> >>> On 2/14/25 14:57, Michael Gmelin wrote: >> >>>> On Fri, 14 Feb 2025 14:21:07 -0500 >> >>>> >> >>>> Petru Garstea wrote: >> >>>>> Hi, >> >>>>> I am trying to configure bhyve vm with support of virtio-console, >> the >> >>>>> following option is added to bhyve command line >> >>>>> >> >>>>> *-s 3,virtio-console,port1=3D/test/console.sock >> >>>>> >> >>>>> *However when trying to connect to socket with socat trere are no >> >>>>> input/ouptut is observed. >> >>>>> >> >>>>> Has virtio-console ever worked ? >> >>>> Yes it works, I just tried it using sysutils/vm-bhyve (just add >> >>>> virt_console0=3D"Yes" to the configuration file). >> >>>> >> >>>> Based on the logs, these device settings are used by it: >> >>>> bhyve devices: >> >>>> -s 0,hostbridge \ >> >>>> -s 31,lpc \ >> >>>> -s 4:0,virtio-blk,/dev/zvol/zroot/vms/freebsd14-test/disk0 \ >> >>>> -s 5:0,virtio-net,tap0,mac=3D58:9c:fc:aa:aa:aa \ >> >>>> -s 6:0,virtio-console,0=3D/vms/freebsd14-test/vtcon.0 >> >>>> >> >>>> I could connect to /vms/freebsd14-test/vtcon.0 using socat: >> >>>> socat - UNIX-CONNECT:/vms/freebsd14-test/vtcon.0 >> >>>> >> >>>> Inside the vm it's /dev/vtcon/0. >> >>>> >> >>>> -m >> >> >> > >> >> --000000000000880367062e237b49 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Alright, I was missing the point, in my case if I want to= get shell access via virtio-console then I should have attached getty to t= hat console inside the guest os, and eventually I expected to see the same = behavior as I have experienced with null modem emulator.
<= br>
Thx

On Fri, Feb 14, 2= 025, 18:16 peter garshtja <peter.garshtja@ambient-md.com> wrote:
Exactly, I expect to get the shell after I = connect with socat, something similar to docker exec -it.=C2=A0

On Fri, Feb = 14, 2025, 18:10 Michael Gmelin <grembo@freebsd.org> wrote:


> On 14. Feb 2025, at 23:46, Petru Garstea <pet= er.garshtja@ambient-md.com> wrote:
>
> =EF=BB=BFSpinning up the vm with /dev/nmdm* and connecting with cu -l = is not an issue on my end.
> I am trying to create a vm using freebsd cloud image and connect to gu= est OS terminal with virtio-console and socat(no X11).
> When spawning the vm the unix socket indeed is created, however when I= connect to it with socat nothing is being shown(not actions on any termina= l commands as well).
> Also, I noticed there is a kernel module(virtio_console.ko) and appare= ntly I believe it has to be loaded it, not sure here.
>
> Please advise,
>

You can connect to the device inside the vm like I wrote and pass data to t= he socat connected socket on the outside (this is what I tested, please see= my previous email).

Are you trying to get some sort of shell over this connection?

-m


>> On 2/14/25 17:23, Paul Vixie wrote:
>> if you want to use cu, you have to use -l to map guest com1 to som= e host nmdm
>> device. if you want to use virtio-console, it will create unix dom= ain sockets
>> not pseudo tty endpoints. from bhyve(8) we see:
>>
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = Virtio console device backends:
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2 >>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = port1=3D/path/to/port1.sock[,portN=3D/path/to/port2.sock ...]
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 A maximum of= 16 ports per device can be created.=C2=A0 Every
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = port
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = is named and corresponds to a Unix domain socket created by
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = bhyve.=C2=A0 bhyve accepts at most one connection per port at a
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = time.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 Limitations:=
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0Due to lack of destructors in bhyve, sockets on the
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 filesystem must be cleaned up manually after bhyve
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 exits.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0There is no way to use the =E2=80=9Cconsole port=E2=80=9D feature= , nor
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 the console port resize at present.
>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =E2=80=A2=C2= =A0 =C2=A0Emergency write is advertised, but no-op at present.
>> i know of no host-side client which will connect to these unix dom= ain sockets
>> and offer a cu-like user interface. "unidirectional" may= be the experience
>> you'll get from "socat".
>>
>> if what you're trying to accomplish is a graphical console, vi= rtio-console is
>> not the way to do it. rather, you'll want "fbuf" to = create a guest VGA/similar
>> that gateways to a host VNC.
>>
>> re:
>>
>>> On Friday, February 14, 2025 9:55:23 PM UTC Petru Garstea wrot= e:
>>> I tried with the following command, and got the same result. >>>
>>> */usr/sbin/bhyve -H -P -A -c 1 -m 1500M \
>>> -s 0,hostbridge \
>>> -s 6,virtio-console,0=3D/tmp/vtcon.0 \
>>> -s 31,lpc \
>>> -l bootrom,/tmp/BHYVE_UEFI.fd \
>>> -s 1,virtio-blk,/tmp/fbsd.img \
>>> -s 2,virtio-net,tap0 \
>>> tmp*
>>>
>>>
>>> I also able to connect but the bidirectional connection is not= happening
>>>
>>> *socat - UNIX-CONNECT:/tmp/vtcon.0
>>> ls
>>> *
>>> PS: if I use null modem emulation instead virtio-console, then=
>>> connecting with cu is not an issue.
>>>
>>> any ideas ?
>>>
>>> Thanks
>>>
>>> On 2/14/25 14:57, Michael Gmelin wrote:
>>>> On Fri, 14 Feb 2025 14:21:07 -0500
>>>>
>>>> Petru Garstea<peter.garshtja@ambi= ent-md.com> wrote:
>>>>> Hi,
>>>>> I am trying to configure bhyve vm with support of virt= io-console, the
>>>>> following option is added to bhyve command line
>>>>>
>>>>> *-s 3,virtio-console,port1=3D/test/console.sock
>>>>>
>>>>> *However when trying to connect to socket with socat t= rere are no
>>>>> input/ouptut is observed.
>>>>>
>>>>> Has virtio-console ever worked ?
>>>> Yes it works, I just tried it using sysutils/vm-bhyve (jus= t add
>>>> virt_console0=3D"Yes" to the configuration file)= .
>>>>
>>>> Based on the logs, these device settings are used by it: >>>>=C2=A0 =C2=A0 =C2=A0 bhyve devices:
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 0,hostbridge \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 31,lpc \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 4:0,virtio-blk,/dev/zvol/zro= ot/vms/freebsd14-test/disk0 \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 5:0,virtio-net,tap0,mac=3D58= :9c:fc:aa:aa:aa \
>>>>=C2=A0 =C2=A0 =C2=A0 =C2=A0 -s 6:0,virtio-console,0=3D/vms/= freebsd14-test/vtcon.0
>>>>
>>>> I could connect to /vms/freebsd14-test/vtcon.0 using socat= :
>>>>=C2=A0 =C2=A0 =C2=A0 socat - UNIX-CONNECT:/vms/freebsd14-te= st/vtcon.0
>>>>
>>>> Inside the vm it's /dev/vtcon/0.
>>>>
>>>> -m
>>
>

--000000000000880367062e237b49--