Date: Sun, 15 May 2022 15:53:34 +0000 From: Benoit Chesneau <benoitc@enki-multimedia.eu> To: "freebsd-net@FreeBSD.org" <freebsd-net@FreeBSD.org>, Mario Marietto <marietto2008@gmail.com> Subject: Re: tap0 don't receive anything in a vm Message-ID: <dgDIaT5VNKKVp158NoQTBIiqppbL-reaQt3i3Aa7CCVW5G6jgiKNHCERh636tU-aRnYxh92ex49P0Bie6pRSOq807WIOefk6wL4sZrRz0dk=@enki-multimedia.eu> In-Reply-To: <CA%2B1FSih1KjvgG=kfdthYv3mFjXE0iz6Z=8C%2Bs=Bb0cVmyeJ3jA@mail.gmail.com> References: <kp7ZKmZfegE1lJU7OybWGBgBN5Oo8FkJNM25vZBEzM6hrU0km9iRKIwICKlo6Wt5e29pACahMKGWCX2fuGHTNDDZ7U3gnn6fGeZRX-12zuE=@enki-multimedia.eu> <CA%2B1FSih1KjvgG=kfdthYv3mFjXE0iz6Z=8C%2Bs=Bb0cVmyeJ3jA@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Thanks but I have already set them to 1. This is kind of unexpected :/ ``` # sysctl net.inet.ip.forwarding net.inet.ip.forwarding: 1 # sysctl net.link.tap.up_on_opennet.link.tap.up_on_open: 1 ``` in the vm, pinging the IP hosts are OK: ``` # ifconfig vtnet0 vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE> ether 58:9c:fc:02:83:8a hwaddr 58:9c:fc:02:83:8a inet 10.1.0.2 netmask 0xffffff00 broadcast 10.1.0.255 inet6 fe80::5a9c:fcff:fe02:838a%vtnet0 prefixlen 64 scopeid 0x1 inet6 2a0e:e701:110b:102::3 prefixlen 64 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> media: Ethernet 10Gbase-T <full-duplex> status: active # ping 10.1.0.1 PING 10.1.0.1 (10.1.0.1): 56 data bytes 64 bytes from 10.1.0.1: icmp_seq=0 ttl=64 time=0.192 ms 64 bytes from 10.1.0.1: icmp_seq=1 ttl=64 time=0.223 ms ^C --- 10.1.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 0.192/0.208/0.223/0.015 ms # ping 10.2.0.11 PING 10.2.0.11 (10.2.0.11): 56 data bytes 64 bytes from 10.2.0.11: icmp_seq=0 ttl=64 time=0.204 ms64 bytes from 10.2.0.11: icmp_seq=1 ttl=64 time=0.193 ms ``` But pinging the default gateway of the host doesn't work ``` ping -t5 10.2.0.1 PING 10.2.0.1 (10.2.0.1): 56 data bytes --- 10.2.0.1 ping statistics ---5 packets transmitted, 0 packets received, 100.0% packet loss ``` When I do a tcpdump on the tap0 interface on the Host I get the following result: ``` # tcpdump -i tap0 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes 17:35:21.210796 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252684480 ecr 0], length 0 17:35:23.347332 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 0, length 64 17:35:24.348907 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 1, length 64 17:35:24.410733 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252687680 ecr 0], length 0 17:35:25.350038 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 2, length 64 17:35:26.351187 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 3, length 64 17:35:27.352990 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 4, length 64 17:35:27.611391 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252690881 ecr 0], length 0 17:35:30.811451 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252694080 ecr 0], length 0 17:35:30.998410 IP 10.1.0.2.1040 > 149.202.36.10.https: Flags [S], seq 496616510, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1054061401 ecr 0], length 0 ^C 10 packets captured 10 packets received by filter0 packets dropped by kernel ``` Any idea what's going on ? could it be related to th bhyve options set by vm-bhyve starting a console ? ``` May 15 17:55:25: [bhyve options: -c 1 -m 2G -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd -U ac3dafab-bedb-11ec-b24d-1402ec690a80 -u] May 15 17:55:25: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/vms/utm/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:07:c0:67 -s 5:1,virtio-net,tap2,mac=58:9c:fc:06:05:0c] ``` Any suggestion is welcome, ------- Original Message ------- On Sunday, May 15th, 2022 at 14:47, Mario Marietto <marietto2008@gmail.com> wrote: > /etc/sysctl.conf : > > net.link.tap.up_on_open=1 > net.inet.ip.forwarding=1 > > Il giorno dom 15 mag 2022 alle ore 14:33 Benoit Chesneau <benoitc@enki-multimedia.eu> ha scritto: > >> My goal is to setup a p2p connection between the host and a guest vm and use the host as the nethop for the vm. I'm uskng bhyve for this setup: >> >> 1. on the host i create a tap tap0 interface and assign to it the IP 10.1.0,1 >> 2. i create and start the vm using vm-bhyve. >> 3. in the vm I set the vtnet0 interface to 10.1.0.2 and set the default router to 10.1.0.1. >> >> i can ping the host but nothing up. When i run tcpdump on the host for this interface i can see the ping reply coming but it doesn't go to the guest. Any idea what's wrong? could it be related to the setup of the console port in bhyve? -l com1,stdio . >> >> Thoughts? >> >> Benoît > > -- > > Mario. [-- Attachment #2 --] <div style="font-family: arial; font-size: 14px;">Thanks but I have already set them to 1. This is kind of unexpected :/</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span># sysctl net.inet.ip.forwarding</span><div><span>net.inet.ip.forwarding: 1</span></div><div><span># sysctl net.link.tap.up_on_open</span></div><span>net.link.tap.up_on_open: 1</span><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span><br></span></div><div style="font-family: arial; font-size: 14px;"><span><br></span></div><div style="font-family: arial; font-size: 14px;"><span>in the vm, pinging the IP hosts are OK:</span></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span><span># ifconfig vtnet0</span><div><span>vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500</span></div><div><span> options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE></span></div><div><span> ether 58:9c:fc:02:83:8a</span></div><div><span> hwaddr 58:9c:fc:02:83:8a</span></div><div><span> inet 10.1.0.2 netmask 0xffffff00 broadcast 10.1.0.255</span></div><div><span> inet6 fe80::5a9c:fcff:fe02:838a%vtnet0 prefixlen 64 scopeid 0x1</span></div><div><span> inet6 2a0e:e701:110b:102::3 prefixlen 64</span></div><div><span> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL></span></div><div><span> media: Ethernet 10Gbase-T <full-duplex></span></div><span> status: active</span><br></span></div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;"><span># ping 10.1.0.1</span><div><span>PING 10.1.0.1 (10.1.0.1): 56 data bytes</span></div><div><span>64 bytes from 10.1.0.1: icmp_seq=0 ttl=64 time=0.192 ms</span></div><div><span>64 bytes from 10.1.0.1: icmp_seq=1 ttl=64 time=0.223 ms</span></div><div><span>^C</span></div><div><span>--- 10.1.0.1 ping statistics ---</span></div><div><span>2 packets transmitted, 2 packets received, 0.0% packet loss</span></div><div><span>round-trip min/avg/max/stddev = 0.192/0.208/0.223/0.015 ms</span></div><div># ping 10.2.0.11</div><div><span>PING 10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0.11</span> (10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0.11</span>): 56 data bytes</span></div><div><span>64 bytes from 10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0.11</span>: icmp_seq=0 ttl=64 time=0.204 ms</span></div><span>64 bytes from 10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0.11</span>: icmp_seq=1 ttl=64 time=0.193 ms</span><br></div><div style="font-family: arial; font-size: 14px;"><span><br></span></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">But pinging the default gateway of the host doesn't work</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span>ping -t5 10.2.0.1</span><div><span>PING 10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0.</span>1 (10.<span style="caret-color:rgb(0, 0, 0);background-color:rgb(255, 255, 255);display:inline !important">2.0</span>.1): 56 data bytes</span></div><div><br></div><div><span>--- 10.2.0.1 ping statistics ---</span></div><span>5 packets transmitted, 0 packets received, 100.0% packet loss</span></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">When I do a tcpdump on the tap0 interface on the Host I get the following result:</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span># tcpdump -i tap0</span><div><span>tcpdump: verbose output suppressed, use -v or -vv for full protocol decode</span></div><div><span>listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes</span></div><div><span>17:35:21.210796 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252684480 ecr 0], length 0</span></div><div><span>17:35:23.347332 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 0, length 64</span></div><div><span>17:35:24.348907 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 1, length 64</span></div><div><span>17:35:24.410733 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252687680 ecr 0], length 0</span></div><div><span>17:35:25.350038 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 2, length 64</span></div><div><span>17:35:26.351187 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 3, length 64</span></div><div><span>17:35:27.352990 IP 10.1.0.2 > 10.2.0.1: ICMP echo request, id 5721, seq 4, length 64</span></div><div><span>17:35:27.611391 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252690881 ecr 0], length 0</span></div><div><span>17:35:30.811451 IP 10.1.0.2.6065 > vm91.jn-hebergement.com.https: Flags [S], seq 3549455562, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 2252694080 ecr 0], length 0</span></div><div><span>17:35:30.998410 IP 10.1.0.2.1040 > 149.202.36.10.https: Flags [S], seq 496616510, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 1054061401 ecr 0], length 0</span></div><div><span>^C</span></div><div><span>10 packets captured</span></div><div><span>10 packets received by filter</span></div><span>0 packets dropped by kernel</span><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">Any idea what's going on ? could it be related to th bhyve options set by vm-bhyve starting a console ?</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><span>May 15 17:55:25: [bhyve options: -c 1 -m 2G -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CSM.fd -U ac3dafab-bedb-11ec-b24d-1402ec690a80 -u]</span><br><span>May 15 17:55:25: [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/vms/utm/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:07:c0:67 -s 5:1,virtio-net,tap2,mac=58:9c:fc:06:05:0c]</span><br></div><div style="font-family: arial; font-size: 14px;">```</div><div style="font-family: arial; font-size: 14px;"><br></div><div style="font-family: arial; font-size: 14px;">Any suggestion is welcome,</div><div style="font-family: arial; font-size: 14px;"><br></div><div class="protonmail_quote"> ------- Original Message -------<br> On Sunday, May 15th, 2022 at 14:47, Mario Marietto <marietto2008@gmail.com> wrote:<br><br> <blockquote class="protonmail_quote" type="cite"> <div dir="ltr"><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">/etc/sysctl.conf :</span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"><br></span></span></div><div><span style="font-family:monospace"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)">net.link.tap.up_on_open=1 </span></span></div><div><span style="font-family:monospace"></span></div><span style="font-family:monospace">net.inet.ip.forwarding=1<br></span></div><br><div class="gmail_quote"><div class="gmail_attr" dir="ltr">Il giorno dom 15 mag 2022 alle ore 14:33 Benoit Chesneau <<a href="mailto:benoitc@enki-multimedia.eu" rel="noreferrer nofollow noopener" target="_blank">benoitc@enki-multimedia.eu</a>> ha scritto:<br></div><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><div> <div>My goal is to setup a p2p connection between the host and a guest vm and use the host as the nethop for the vm. I'm uskng bhyve for this setup:</div><div><br></div><div>1. on the host i create a tap tap0 interface and assign to it the IP 10.1.0,1</div>2. i create and start the vm using vm-bhyve.<div>3. in the vm I set the vtnet0 interface to 10.1.0.2 and set the default router to 10.1.0.1. </div><div><br></div><div>i can ping the host but nothing up. When i run tcpdump on the host for this interface i can see the ping reply coming but it doesn't go to the guest. Any idea what's wrong? could it be related to the setup of the console port in bhyve? -l com1,stdio . </div><div><br></div><div>Thoughts?</div><div><br></div><div><br></div><div> <div id="gmail-m_7879999023221865462protonmail_mobile_signature_block"><div>Benoît<u></u><u></u></div></div></div></div></blockquote></div><br clear="all"><br>-- <br><div class="gmail_signature" dir="ltr">Mario.<br></div> </blockquote><br> </div>help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?dgDIaT5VNKKVp158NoQTBIiqppbL-reaQt3i3Aa7CCVW5G6jgiKNHCERh636tU-aRnYxh92ex49P0Bie6pRSOq807WIOefk6wL4sZrRz0dk=>