From nobody Thu Jul 11 15:23:16 2024 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WKdmV6zV8z5Qr3g for ; Thu, 11 Jul 2024 15:23:54 +0000 (UTC) (envelope-from odhiambo@gmail.com) Received: from mail-oa1-x34.google.com (mail-oa1-x34.google.com [IPv6:2001:4860:4864:20::34]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WKdmV2JFMz553Y for ; Thu, 11 Jul 2024 15:23:54 +0000 (UTC) (envelope-from odhiambo@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20230601 header.b=DdFZWm7O; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of odhiambo@gmail.com designates 2001:4860:4864:20::34 as permitted sender) smtp.mailfrom=odhiambo@gmail.com Received: by mail-oa1-x34.google.com with SMTP id 586e51a60fabf-25e16380bc9so509099fac.1 for ; Thu, 11 Jul 2024 08:23:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720711433; x=1721316233; darn=freebsd.org; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :from:to:cc:subject:date:message-id:reply-to; bh=NMRlmPV5G3i1CbngRPpfzidfXsqV2KCiigt+va+tgG8=; b=DdFZWm7OErkErdIgM2hq6+JJCM7K4d1iHIweAHMidDxolW6583z8G/kflgPf3HVRTH hDG2bCguXPLJojiEJOOfdFSu0YrbQN4ly2nKZQYSqTleJfgTLBT67D6Tr6Rg2IS06b25 z6StlN2lLu1ImvXM6PUX6q4++wDEFWVQEOIr1m5Tnkdi64VyYB5BbmJ1JHugWNZ3e4y4 PGLE3lsI08vcuuldRodtYu6fluNsx9vvDUpH+blW9QrHV76R2K1JOg/GKvXQ6xf0SZLo ynOSwebCQCc/Jtcd1z7faD4WWXbXwdUKCzM/5hh/z/C3XkguAGU8PT1tnfB3QeuM+2Mo Sxng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720711433; x=1721316233; h=to:subject:message-id:date:from:in-reply-to:references:mime-version :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=NMRlmPV5G3i1CbngRPpfzidfXsqV2KCiigt+va+tgG8=; b=PFPtR9b4YiQFb0D4AIiYVQ9RWyniFy2G/ioybZWg3J9Pkfzfb5f5tMzb/NovjmRHPC x25ZxDKFaDn8jxcqd1ZMZzMe0SNF+7ladFSJvDkOrvGTVWEyJdSuuWOSc+yql+TcIO64 vmPxCATrcDhKfhxRzFV/dN8ZMVqLCqv0KVJRaMVkMuQQnO4vLRTjSCdwGOC6QUakjJMh glcbZhQrW/s+9jx2uEVl0jGcNOW0gxujW5uvKMB0+CFBX9XL+yOKAmw7yHixIaa1u/Y5 /TDWmZqxyIynuU9Z+vk1nfsgo+Jdr1TAi8Yu66F38fsJOFW3xF8x4IJ+b572CsxjA/m7 vkBw== X-Gm-Message-State: AOJu0YwcrfuMcI1YnfyiT4gmNdaEDAGYEUKxuiZjW7CEpT7XRSWwoQA4 HVJu29N8j5o42rgSzqo546nLq8J9QUSQlNO3Zg7R2XCgMelW8d4AoT4RhGy0eBpW29iQcLpXy78 U83v2QeUZI6aE5CLBtShv+weUnloI1nudcXiM8A== X-Google-Smtp-Source: AGHT+IF17OcCCymVuoOFHuXSjUKDZghVpIm2/ECiCSlslsLTKnHcM14M7DTNJG7wZbxvA06Rqqh4eRVGOt23B3ttuc0= X-Received: by 2002:a05:6870:ac07:b0:254:d163:c3aa with SMTP id 586e51a60fabf-25eae85063cmr7440422fac.16.1720711433062; Thu, 11 Jul 2024 08:23:53 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org MIME-Version: 1.0 References: <202407111449.46BEnLoP051380@gndrsh.dnsmgr.net> In-Reply-To: <202407111449.46BEnLoP051380@gndrsh.dnsmgr.net> From: Odhiambo Washington Date: Thu, 11 Jul 2024 18:23:16 +0300 Message-ID: Subject: Re: Suddenly unable to access VMs To: FreeBSD virtualization Content-Type: multipart/alternative; boundary="000000000000ba9175061cfa59a0" X-Spamd-Bar: --- X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20230601]; R_SPF_ALLOW(-0.20)[+ip6:2001:4860:4000::/36:c]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MISSING_XM_UA(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; RCVD_IN_DNSWL_NONE(0.00)[2001:4860:4864:20::34:from] X-Rspamd-Queue-Id: 4WKdmV2JFMz553Y --000000000000ba9175061cfa59a0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Jul 11, 2024 at 5:49=E2=80=AFPM Rodney W. Grimes < freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > My bhyve VMs have been all fine until now. > > I can't ping them and can't SSH into them. However, I can connect to th= em > > with VNCViewer from a remote host (my PC from my house) :-( > > > > I haven't done any changes on the host at all. > > dnsmasq is running, but seems like the VMs aren't getting the IPs for > some > > reason. > > > > ``` > > cloned_interfaces=3D"bridge0 tap0 tap1 tap2 tap3 tap4 tap5" > > ifconfig_bridge0_name=3D"vmbridge" > > ifconfig_vmbridge=3D"addm em1 addm tap0 addm tap1 addm tap2 addm tap3 a= ddm > > tap4 addm tap5 up" > > ifconfig_vmbridge_alias0=3D"inet 172.16.0.1 netmask 255.255.255.0" > > ``` > > What might have happened? > > > > > > root@gw:/home/wash # ifconfig vmbridge > > vmbridge: flags=3D1008843 > > metric 0 mtu 1500 > > options=3D0 > > ether 58:9c:fc:10:df:1d > > inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 > > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > > member: tap5 flags=3D143 > > ifmaxaddr 0 port 10 priority 128 path cost 2000000 > > member: tap4 flags=3D143 > > ifmaxaddr 0 port 9 priority 128 path cost 2000000 > > member: tap3 flags=3D143 > > ifmaxaddr 0 port 8 priority 128 path cost 2000000 > > member: tap2 flags=3D143 > > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > > member: tap1 flags=3D143 > > ifmaxaddr 0 port 6 priority 128 path cost 2000000 > > member: tap0 flags=3D143 > > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > > member: em1 flags=3D143 > > ifmaxaddr 0 port 2 priority 128 path cost 55 > > groups: bridge > > nd6 options=3D9 > > root@gw:/home/wash # ssh 172.16.0.99 > > ssh: connect to host 172.16.0.99 port 22: Permission denied > > root@gw:/home/wash # ssh 172.16.0.100 > > ssh: connect to host 172.16.0.100 port 22: Permission denied > > root@gw:/home/wash # ping 172.16.0.100 > > PING 172.16.0.100 (172.16.0.100): 56 data bytes > > ping: sendto: Permission denied > > ping: sendto: Permission denied > > ping: sendto: Permission denied > > ping: sendto: Permission denied > > ^C > > --- 172.16.0.100 ping statistics --- > > 4 packets transmitted, 0 packets received, 100.0% packet loss > > root@gw:/home/wash # ping 172.16.0.99 > > PING 172.16.0.99 (172.16.0.99): 56 data bytes > > ping: sendto: Permission denied > > ping: sendto: Permission denied > > ping: sendto: Permission denied > > ^C > > --- 172.16.0.99 ping statistics --- > > 3 packets transmitted, 0 packets received, 100.0% packet loss > > root@gw:/home/wash # service dnsmasq status > > dnsmasq is running as pid 4190. > > root@gw:/home/wash # > > Permission denied is almost certainly coming from firewall, > either ipfw or pf. > I haven't changed anything in my pf.conf either. What also baffles me is that the VMs are not obtaining IP addresses from dnsmasq. --=20 Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 In an Internet failure case, the #1 suspect is a constant: DNS. "Oh, the cruft.", egrep -v '^$|^.*#' =C2=AF\_(=E3=83=84)_/=C2=AF :-) [How to ask smart questions: http://www.catb.org/~esr/faqs/smart-questions.html] --000000000000ba9175061cfa59a0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Jul 11, 2024 at 5:49=E2=80=AF= PM Rodney W. Grimes <fr= eebsd-rwg@gndrsh.dnsmgr.net> wrote:
> My bhyve VMs have been all fine until now.<= br> > I can't ping them and can't SSH into them. However, I can conn= ect to them
> with VNCViewer from a remote host (my PC from my house) :-(
>
> I haven't done any changes on the host at all.
> dnsmasq is running, but seems like the VMs aren't getting the IPs = for some
> reason.
>
> ```
> cloned_interfaces=3D"bridge0 tap0 tap1 tap2 tap3 tap4 tap5"<= br> > ifconfig_bridge0_name=3D"vmbridge"
> ifconfig_vmbridge=3D"addm em1 addm tap0 addm tap1 addm tap2 addm = tap3 addm
> tap4 addm tap5 up"
> ifconfig_vmbridge_alias0=3D"inet 172.16.0.1 netmask 255.255.255.0= "
> ```
> What might have happened?
>
>
> root@gw:/home/wash # ifconfig vmbridge
> vmbridge: flags=3D1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LO= WER_UP>
> metric 0 mtu 1500
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0options=3D0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ether 58:9c:fc:10:df:1d
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet 172.16.0.1 netmask 0xffffff00 br= oadcast 172.16.0.255
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0id 00:00:00:00:00:00 priority 32768 h= ellotime 2 fwddelay 15
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0maxage 20 holdcnt 6 proto rstp maxadd= r 2000 timeout 1200
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0root id 00:00:00:00:00:00 priority 32= 768 ifcost 0 port 0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap5 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 10 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap4 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 9 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap3 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 8 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap2 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 7 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap1 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 6 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: tap0 flags=3D143<LEARNING,= DISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 5 priority 128 path cost 2000000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: em1 flags=3D143<LEARNING,D= ISCOVER,AUTOEDGE,AUTOPTP>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifmaxaddr= 0 port 2 priority 128 path cost 55
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0groups: bridge
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D9<PERFORMNUD,IFDISAB= LED>
> root@gw:/home/wash # ssh 172.16.0.99
> ssh: connect to host 172.16.0.99 port 22: Permission denied
> root@gw:/home/wash # ssh 172.16.0.100
> ssh: connect to host 172.16.0.100 port 22: Permission denied
> root@gw:/home/wash # ping 172.16.0.100
> PING 172.16.0.100 (172.16.0.100): 56 data bytes
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ^C
> --- 172.16.0.100 ping statistics ---
> 4 packets transmitted, 0 packets received, 100.0% packet loss
> root@gw:/home/wash # ping 172.16.0.99
> PING 172.16.0.99 (172.16.0.99): 56 data bytes
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ping: sendto: Permission denied
> ^C
> --- 172.16.0.99 ping statistics ---
> 3 packets transmitted, 0 packets received, 100.0% packet loss
> root@gw:/home/wash # service dnsmasq status
> dnsmasq is running as pid 4190.
> root@gw:/home/wash #

Permission denied is almost certainly coming from firewall,
either ipfw or pf.

I haven't change= d anything in my pf.conf either.
What also baffles me is that the= VMs are not obtaining IP addresses from dnsmasq.=C2=A0


--
Best= regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 = 2274 3223
=C2=A0In=C2=A0an Internet failure case, the #1 susp= ect is a constant: DNS.
"Oh= , the cruft.",=C2=A0egrep -v '^$|^.*#'=C2=A0=C2=AF\_(=E3=83=84)_/=C2=AF=C2=A0:-)
[How to as= k smart questions:=C2=A0http:= //www.catb.org/~esr/faqs/smart-questions.html]
=
--000000000000ba9175061cfa59a0--