Date: Wed, 14 Aug 2024 11:29:38 -0700 (PDT) From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> To: Alex Arslan <ararslan@comcast.net> Cc: Bakul Shah <bakul@iitbombay.org>, "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Diagnosing virtual machine network issues Message-ID: <202408141829.47EITc7B080532@gndrsh.dnsmgr.net> In-Reply-To: <B830418F-C9A4-4B59-A5E9-0A1E7BE1CC78@comcast.net>
next in thread | previous in thread | raw e-mail | index | archive | help
> > > On Aug 13, 2024, at 9:15?AM, Bakul Shah <bakul@iitbombay.org> wrote: > > > > This weird 127. address seems like a systemd feature/bug thing: https://unix.stackexchange.com/questions/612416/why-does-etc-resolv-conf-point-at-127-0-0-53 > > > > This behavior seems like some strange interaction between systemd assumptions and freebsd?s, or something not being set up quite right on the linux side when the vm is running freebsd. > > Could libvirt be a factor here, do you think? For example, perhaps the > network should be configured differently than the default when the host > is using systemd-resolved and/or when the guest is FreeBSD. In the network > XML format for libvirt (https://libvirt.org/formatnetwork.html), there is > a `domain` element with a `localOnly` attribute that I have seen set by > some virtualization projects. As far as I can tell, our setup isn't using > the `domain` element at all. Having a /etc/resolv.conf entry of 127.0.0.53 is indeed something out of the normal on a freebsd box. You need to find where that is coming from and why that value is used. > > > > >> On Aug 13, 2024, at 8:46 AM, Alex Arslan <ararslan@comcast.net> wrote: > >> > >> ? > >> Hi Rodney, > >> > >>> On Aug 10, 2024, at 9:11?AM, Rodney W. Grimes <freebsd-rwg@gndrsh.dnsmgr.net> wrote: > >>> > >>>> > >>>> > >>>>> On Aug 2, 2024, at 5:58?PM, Bakul Shah <bakul@iitbombay.org> wrote: > >>>>> > >>>>> On Aug 2, 2024, at 3:52?PM, Alex Arslan <ararslan@comcast.net> wrote: > >>>>>> > >>>>>>> Just a comment and a name server line: > >>>>>>> > >>>>>>> $ cat /etc/resolv.conf > >>>>>>> # Generated by resolvconf > >>>>>>> nameserver 192.168.122.1 > >>>>>> > >>>>>> I believe that is the host IP, so I guess the VM is using the host for DNS > >>>>>> resolution? Interestingly, if I add `nameserver 8.8.8.8` below the line > >>>>>> with the host IP, it takes 10 seconds rather than 30 to reach the expected > >>>>>> domain resolution failure. If I put 8.8.8.8 above the host IP, the domain > >>>>>> resolution failure is instantaneous. > >>>>> > >>>>> What does your host use as a namesever? > >>>> > >>>> The nameserver is 127.0.0.53. It sets options edns0 and trust-ad, and > >>>> includes a search entry as well. > >>> > >>> First, is that a typo and you mean 127.0.0.1:53? > >> > >> No, the host's /etc/resolv.conf has `nameserver 127.0.0.53`, I just went > >> back and rechecked to be sure. > >> > >>> Second, is that name server locked to 127.0.0.1, or is it > >>> actually listinging on *:53? If it is LOCKED you have no name server > >>> running on 192.168.122.1 to be reached by the VM, if it is NOT locked > >>> can the guest ping 192.168.122.1, and can it reach dns at that IP on > >>> port 53? Can the host send a packet BACK to the guest? > >> > >> I apologize but I don't really know enough about these things to know how > >> to answer your question. I did post the output of tcpdump on the VM and > >> the host a while back but that was for the invalid request, so that > >> probably doesn't capture what you're describing. > >> > >>> Third you can "fix" the "nameserver 192.168.122.1" entry in /etc/resolv.conf > >>> by configuring the DHCP server that handed out the lease to the VM to send > >>> a namserver entry of 8.8.8.8. > >> > >> If I understand correctly, that is indeed what we've done as a Band-Aid fix > >> for the time being: I added the line `prepend_nameservers=8.8.8.8` to > >> /etc/resolvconf.conf. > >> > >>>> > >>>>> > >>>>>> Not a particularly satisfying conclusion to this saga as I don't understand > >>>>>> why it's happening but at least I have a workaround that should hopefully > >>>>>> do the job. I really appreciate everyone's help and input thus far! > >>>>>> > >>>>>> What's the best way to add `nameserver 8.8.8.8` to /etc/resolv.conf as > >>>>>> part of the VM's configuration? > >>>>> > >>>>> You should diagnose the problem of the nameserver at 192.168.122.1 > >>>>> and fix it to act properly. I don't use vm (just bhyve) so can't help > >>>>> you with its config. > >>>> > >>>> I do still plan to try to figure out what the actual issue is, but I also > >>>> now have a path forward in the meantime. :) > >>>> > >>>> > >>> > >>> -- > >>> Rod Grimes rgrimes@freebsd.org <mailto:rgrimes@freebsd.org> > -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202408141829.47EITc7B080532>