Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 7 Nov 2022 20:34:57 -0500
From:      Mark Saad <nonesuch@longcount.org>
To:        freebsd-net@freebsd.org
Subject:   Re: GRE in a fib via rc.conf
Message-ID:  <CAMXt9NaOkN-gLNjhfWwzsY=70KZNUf2w9pfKgZL-VrhVZ5Hgeg@mail.gmail.com>
In-Reply-To: <5CBAA944-5122-4BA0-854F-AF7D78ACF8AE@gmail.com>
References:  <CAMXt9NbgFUiGuQNbcQ8mj5RaYw9KiW_SxccfVYvgom2%2BnBev_Q@mail.gmail.com> <5CBAA944-5122-4BA0-854F-AF7D78ACF8AE@gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000eec62505eceb8c6c
Content-Type: text/plain; charset="UTF-8"

On Mon, Nov 7, 2022 at 8:11 PM Zhenlei Huang <zlei.huang@gmail.com> wrote:

>
> On Nov 8, 2022, at 8:26 AM, Mark Saad <nonesuch@longcount.org> wrote:
>
> All
>   I am looking for some help on if my setup makes sense.
> I have a vm with two interfaces. One for access to the host , we'll call
> this mgmt . One for routing traffic, we'll call this routing . I want to
> put the routing interface into a fib and to run a gre tunnel over it.
> Sounds simple enough.  The problem I am seeing is that it looks like the
> tunneled traffic is leaked into the default fib and I don't see why. I am
> not sure if this is config nit or if this is an issue. Should the gre10
> interface be in fib 1 ?  See below.
>
>
> The fib of the tunneling interface should also be 1 IIUC your setup.
>
>
> ### RC CONF ###
> ifconfig_vmx0="inet 10.23.121.253/24 description mgmt"
> ifconfig_vmx1="inet 100.65.101.14/28 mtu 9000 description routing fib 1"
> defaultrouter="10.23.121.1"
> static_routes="ewr10gresrc"
> route_ewr10gresrc=" 192.168.255.14 100.65.101.1 -fib 1"
> cloned_interfaces="gre10"
> ifconfig_gre10=" inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252
> tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"
>
>
> Try this for the gre tunnel interface:
>
> cloned_interfaces="gre10"
> create_args_gre10="tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"
> ifconfig_gre10="inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib
> 1"
>
>
Good catch , and I confirmed it works in either format

ifconfig_gre10=" inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib
1 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"
 or
create_args_gre10="tunnel 100.65.101.14 192.168.255.14 tunnelfib 1"
ifconfig_gre10="inet 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib
1"

I don't think this is documented very well. Thanks for your help.

###############
>
> ### DEFAULT FIB ###
> ~ # netstat -nr4Wl
> Routing tables
>
> Internet:
> Destination Gateway Flags Nhop# Mtu Netif Expire
> default 10.23.121.1 UGS 6 1500 vmx0
> 10.23.121.0/24 link#1 U 2 1500 vmx0
> 10.23.121.253 link#1 UHS 3 16384 lo0
> 100.67.103.1 link#4 UH 4 1476 gre10
> 100.67.103.2 link#4 UHS 5 16384 lo0
> 127.0.0.1 link#3 UH 1 16384 lo0
>
> ### FIB 1 ###
>
> # setfib 1 netstat -nr4Wl
> Routing tables (fib: 1)
>
> Internet:
> Destination Gateway Flags Nhop# Mtu Netif Expire
> 100.65.101.0/28 link#2 U 1 9000 vmx1
> 100.65.101.14 link#2 UHS 2 16384 lo0
> 127.0.0.1 link#3 UHS 3 16384 lo0
> 192.168.255.14 100.65.101.1 UGHS 4 9000 vmx1
>
> ##### PING EXAMPLES #####
>
> # setfib 1 ping -c 1 -t 2 100.67.103.1
> PING 100.67.103.1 (100.67.103.1): 56 data bytes
> ping: sendto: No route to host
>
> --- 100.67.103.1 ping statistics ---
> 1 packets transmitted, 0 packets received, 100.0% packet loss
> # setfib 0 ping -c 1 -t 2 100.67.103.1
> PING 100.67.103.1 (100.67.103.1): 56 data bytes
> 64 bytes from 100.67.103.1: icmp_seq=0 ttl=255 time=1.528 ms
>
> --- 100.67.103.1 ping statistics ---
> 1 packets transmitted, 1 packets received, 0.0% packet loss
> round-trip min/avg/max/stddev = 1.528/1.528/1.528/0.000 ms
>
> #### TCPDUMP ####
> ICMP packets are in fact sourced from the gre10 interface.
> The GRE packets are also only going out the routing interface.
>
> See the following pastebin for details.
>
> https://pastebin.com/n3mGXGHA
>
>
>
>
> --
> mark saad | nonesuch@longcount.org
>
>
> Best regards,
> Zhenlei
>
>

-- 
mark saad | nonesuch@longcount.org

--000000000000eec62505eceb8c6c
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">=
<div dir=3D"ltr" class=3D"gmail_attr">On Mon, Nov 7, 2022 at 8:11 PM Zhenle=
i Huang &lt;<a href=3D"mailto:zlei.huang@gmail.com">zlei.huang@gmail.com</a=
>&gt; wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px=
 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><di=
v style=3D"overflow-wrap: break-word;"><br><div><blockquote type=3D"cite"><=
div>On Nov 8, 2022, at 8:26 AM, Mark Saad &lt;<a href=3D"mailto:nonesuch@lo=
ngcount.org" target=3D"_blank">nonesuch@longcount.org</a>&gt; wrote:</div><=
br><div><div dir=3D"ltr"><div>All</div><div>=C2=A0 I am looking for some he=
lp on if my setup makes sense. <br></div><div>I
 have a vm with two interfaces. One for access to the host , we&#39;ll call=
=20
this mgmt . One for routing traffic, we&#39;ll call this routing . I want t=
o
 put the routing interface into a fib and to run a gre tunnel over it.=20
Sounds simple enough.=C2=A0 The problem I am seeing is that it looks like t=
he
 tunneled traffic is leaked into the default fib and I don&#39;t see why. I=
=20
am not sure if this is config nit or if this is an issue. Should the=20
gre10 interface be in fib 1 ?=C2=A0 See below. <br></div></div></div></bloc=
kquote><div><br></div><div>The fib of the tunneling interface should also b=
e 1 IIUC your setup.</div><br><blockquote type=3D"cite"><div><div dir=3D"lt=
r"><div><br></div><div><div>### RC CONF ###
</div><div>ifconfig_vmx0=3D&quot;inet <a href=3D"http://10.23.121.253/24" t=
arget=3D"_blank">10.23.121.253/24</a> description mgmt&quot;
</div><div>ifconfig_vmx1=3D&quot;inet <a href=3D"http://100.65.101.14/28" t=
arget=3D"_blank">100.65.101.14/28</a> mtu 9000 description routing fib 1&qu=
ot;
</div><div>defaultrouter=3D&quot;10.23.121.1&quot;
</div><div>static_routes=3D&quot;ewr10gresrc&quot;
</div><div>route_ewr10gresrc=3D&quot; 192.168.255.14 100.65.101.1 -fib 1&qu=
ot;
</div><div>cloned_interfaces=3D&quot;gre10&quot;
</div><div>ifconfig_gre10=3D&quot; inet 100.67.103.2 100.67.103.1 netmask 2=
55.255.255.252 tunnel 100.65.101.14 192.168.255.14 tunnelfib 1&quot;
</div></div></div></div></blockquote><div><br></div><div>Try this for the g=
re tunnel interface:</div><div><br></div><div>cloned_interfaces=3D&quot;gre=
10&quot;</div><div>create_args_gre10=3D&quot;tunnel 100.65.101.14 192.168.2=
55.14 tunnelfib 1&quot;</div><div>ifconfig_gre10=3D&quot;inet 100.67.103.2 =
100.67.103.1 netmask 255.255.255.252 fib 1&quot;</div><br></div></div></blo=
ckquote><div><br></div><div>Good catch , and I confirmed it works in either=
 format <br></div><div><br></div><div>ifconfig_gre10=3D&quot; inet 100.67.1=
03.2 100.67.103.1 netmask 255.255.255.252 fib 1 tunnel 100.65.101.14 192.16=
8.255.14 tunnelfib 1&quot;
</div><div>=C2=A0or</div><div><div>create_args_gre10=3D&quot;tunnel 100.65.=
101.14 192.168.255.14 tunnelfib 1&quot;</div><div>ifconfig_gre10=3D&quot;in=
et 100.67.103.2 100.67.103.1 netmask 255.255.255.252 fib 1&quot;</div><div>=
<br></div><div>I don&#39;t think this is documented very well. Thanks for y=
our help.<br></div><div><br></div></div><blockquote class=3D"gmail_quote" s=
tyle=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pad=
ding-left:1ex"><div style=3D"overflow-wrap: break-word;"><div><blockquote t=
ype=3D"cite"><div><div dir=3D"ltr"><div><div>###############
</div><div>=C2=A0</div><div>### DEFAULT FIB ###
</div><div> ~ # netstat -nr4Wl
</div><div>Routing tables
</div><div>=C2=A0</div><div>Internet:
</div><div>Destination        Gateway            Flags   Nhop#    Mtu      =
Netif Expire
</div><div>default            10.23.121.1        UGS         6   1500      =
 vmx0
</div><div><a href=3D"http://10.23.121.0/24" target=3D"_blank">10.23.121.0/=
24</a>     link#1             U           2   1500       vmx0
</div><div>10.23.121.253      link#1             UHS         3  16384      =
  lo0
</div><div>100.67.103.1       link#4             UH          4   1476      =
gre10
</div><div>100.67.103.2       link#4             UHS         5  16384      =
  lo0
</div><div>127.0.0.1          link#3             UH          1  16384      =
  lo0
</div><div>=C2=A0</div><div>### FIB 1 ###
</div><div>=C2=A0</div><div> # setfib 1 netstat -nr4Wl
</div><div>Routing tables (fib: 1)
</div><div>=C2=A0</div><div>Internet:
</div><div>Destination        Gateway            Flags   Nhop#    Mtu      =
Netif Expire
</div><div><a href=3D"http://100.65.101.0/28" target=3D"_blank">100.65.101.=
0/28</a>    link#2             U           1   9000       vmx1
</div><div>100.65.101.14      link#2             UHS         2  16384      =
  lo0
</div><div>127.0.0.1          link#3             UHS         3  16384      =
  lo0
</div><div>192.168.255.14     100.65.101.1       UGHS        4   9000      =
 vmx1
</div><div>=C2=A0</div><div>##### PING EXAMPLES #####
</div><div>=C2=A0</div><div># setfib 1 ping -c 1 -t 2 100.67.103.1
</div><div>PING 100.67.103.1 (100.67.103.1): 56 data bytes
</div><div>ping: sendto: No route to host
</div><div>=C2=A0</div><div>--- 100.67.103.1 ping statistics ---
</div><div>1 packets transmitted, 0 packets received, 100.0% packet loss
</div><div># setfib 0 ping -c 1 -t 2 100.67.103.1
</div><div>PING 100.67.103.1 (100.67.103.1): 56 data bytes
</div><div>64 bytes from <a href=3D"http://100.67.103.1/" target=3D"_blank"=
>100.67.103.1</a>: icmp_seq=3D0 ttl=3D255 time=3D1.528 ms
</div><div>=C2=A0</div><div>--- 100.67.103.1 ping statistics ---
</div><div>1 packets transmitted, 1 packets received, 0.0% packet loss
</div><div>round-trip min/avg/max/stddev =3D 1.528/1.528/1.528/0.000 ms
</div><div>=C2=A0</div><div>#### TCPDUMP ####
</div><div>ICMP packets are in fact sourced from the gre10 interface.=20
</div>The GRE packets are also only going out the routing interface.</div><=
div><br></div><div>See the following pastebin for details.</div><div><br></=
div><div><a href=3D"https://pastebin.com/n3mGXGHA" target=3D"_blank">https:=
//pastebin.com/n3mGXGHA</a><font color=3D"#888888"><br></font></div><font c=
olor=3D"#888888"><div><br><br></div></font><br clear=3D"all"><br>-- <br><di=
v dir=3D"ltr">mark saad | <a href=3D"mailto:nonesuch@longcount.org" target=
=3D"_blank">nonesuch@longcount.org</a><br></div></div>
</div></blockquote><br></div><div><div style=3D"color:rgb(0,0,0)">Best rega=
rds,</div><div style=3D"color:rgb(0,0,0)">Zhenlei</div></div><br></div></bl=
ockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"gmail=
_signature">mark saad | <a href=3D"mailto:nonesuch@longcount.org" target=3D=
"_blank">nonesuch@longcount.org</a><br></div></div>

--000000000000eec62505eceb8c6c--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAMXt9NaOkN-gLNjhfWwzsY=70KZNUf2w9pfKgZL-VrhVZ5Hgeg>