Date: Fri, 19 Jul 2024 00:15:12 +0800 From: Zhenlei Huang <zlei@FreeBSD.org> To: Santiago Martinez <sm@codenetworks.net> Cc: freebsd-net@freebsd.org Subject: Re: Multiple Fibs and INET6 Message-ID: <E2F8E66A-B141-47C4-B4B8-551D22784244@FreeBSD.org> In-Reply-To: <70305bf3-cfe4-4cdd-8cb8-c03c1a9c2b8f@codenetworks.net> References: <da781b66-4ab3-426f-bf56-f453030c6e61@codenetworks.net> <F7B1F9E1-30E0-4BC2-84F8-697A655A2C44@FreeBSD.org> <70305bf3-cfe4-4cdd-8cb8-c03c1a9c2b8f@codenetworks.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_C338E674-F01E-4A6C-8A76-67DBE390042A
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=us-ascii
> On Jul 19, 2024, at 12:11 AM, Santiago Martinez <sm@codenetworks.net> =
wrote:
>=20
> Interesting, I'm running 14.1p2.
>=20
>=20
Yes, I'm running exactly the same version with you.
> how does your routing table looks for fib1 ?
>=20
>=20
```
# netstat -6rnF 1
Routing tables (fib: 1)
Internet6:
Destination Gateway Flags =
Netif Expire
fe80::%lo1/64 link#5 U =
lo1
fe80::1%lo0 link#2 UHS =
lo0
```
> Santi
>=20
>=20
>=20
> On 7/18/24 18:09, Zhenlei Huang wrote:
>>=20
>>=20
>>> On Jul 13, 2024, at 1:06 AM, Santiago Martinez <sm@codenetworks.net =
<mailto:sm@codenetworks.net>> wrote:
>>>=20
>>> Hi Everyone.
>>>=20
>>> While adding -F ( fib as used in netstat ) to ping and ping6 I have =
found something that from my understanding is not correct.
>>> Please can you advise?
>>> I have the following setup :
>>>=20
>>> -- two fibs (0 and 1)=20
>>> -- two loop-backs (lo0 and lo1).
>>> -- Lo1 has been assigned to fib1
>>> -- net.add_addr_allfibs =3D 0
>>> My interface output looks like this:
>>>=20
>>>=20
>>> ifconfig lo0 | grep inet6=20
>>> inet6 ::1 prefixlen 128=20
>>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2=20
>>>=20
>>> ifconfig lo1 | grep inet6=20
>>> inet6 fe80::1%lo1 prefixlen 64 scopeid 0x3
>>>=20
>>>=20
>>> If I do a netstat -rn -6 -F0 I get the following which is was i =
expected.
>>>=20
>>> Internet6:=20
>>> Destination Gateway =
Flags Netif Expire=20
>>> ::/96 link#2 URS =
lo0=20
>>> ::1 link#2 UHS =
lo0=20
>>> ::ffff:0.0.0.0/96 link#2 URS =
lo0=20
>>> fe80::%lo0/10 link#2 URS =
lo0=20
>>> fe80::%lo0/64 link#2 U =
lo0=20
>>> fe80::1%lo0 link#2 UHS =
lo0=20
>>> ff02::/16 link#2 URS =
lo0
>>>=20
>>>=20
>>> Now, netstat -rn -6 -F1 shows "fe80::1%lo0" which should not be =
there and "fe80::1%lo1" is missing which should be there.
>>>=20
>>> Internet6:=20
>>> Destination Gateway =
Flags Netif Expire=20
>>> fe80::%lo1/64 link#3 U =
lo1=20
>>> fe80::1%lo0 link#2 UHS =
lo0
>>>=20
>> That seems wrong from my first glance. IIRC, there's HACK ( I'd =
prefer this ) for loopback route. For example
>> ```
>> # sysctl net.fibs=3D3
>> net.fibs: 2 -> 3
>> # ifconfig epair create
>> # epair0a
>> # ifconfig epair0a fib 2
>> # ifconfig epair0a inet6 -ifdisabled up
>> # netstat -6rnF 2
>> Routing tables (fib: 2)
>>=20
>> Internet6:
>> Destination Gateway Flags =
Netif Expire
>> fe80::%epair0a/64 link#5 U =
epair0a
>> fe80::3b:b3ff:fe8f:9a0a%lo0 link#1 UHS =
lo0
>> ```
>>=20
>> The loopback route always refer the first loop interface, aka lo0. =20=
>>>=20
>>>=20
>>> What output I was expecting was:
>>> Internet6:=20
>>> Destination Gateway =
Flags Netif Expire=20
>>> fe80::%lo1/64 link#3 U =
lo1=20
>>> fe80::1%lo1 link#3 UHS =
lo1
>>>=20
>>>=20
>>>=20
>>> This makes the ping -6 -F0 fe80::1%lo0 to work but ping -6 -F1 =
fe80::1%l01 to fail which I wanted to use as test case.
>>>=20
>> That is interesting. I can ping without failure.
>>=20
>> ```
>> # setfib 1 ping6 -c3 fe80::1%lo1
>> PING(56=3D40+8+8 bytes) fe80::1%lo1 --> fe80::1%lo1
>> 16 bytes from fe80::1%lo1, icmp_seq=3D0 hlim=3D64 time=3D0.050 ms
>> 16 bytes from fe80::1%lo1, icmp_seq=3D1 hlim=3D64 time=3D0.067 ms
>> 16 bytes from fe80::1%lo1, icmp_seq=3D2 hlim=3D64 time=3D0.096 ms
>>=20
>> --- fe80::1%lo1 ping statistics ---
>> 3 packets transmitted, 3 packets received, 0.0% packet loss
>> round-trip min/avg/max/stddev =3D 0.050/0.071/0.096/0.019 ms
>> ```
>>=20
>> Best regards,
>> Zhenlei
>>=20
>>>=20
>>> Thanks in advance.
>>>=20
>>> Santiago
>>>=20
>>>=20
>>=20
>>=20
>>=20
--Apple-Mail=_C338E674-F01E-4A6C-8A76-67DBE390042A
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=us-ascii
<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br =
class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div =
class=3D"">On Jul 19, 2024, at 12:11 AM, Santiago Martinez <<a =
href=3D"mailto:sm@codenetworks.net" class=3D"">sm@codenetworks.net</a>>=
wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D"">
=20
<meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3DUTF-8" class=3D"">
=20
<div class=3D""><p class=3D"">Interesting, I'm running 14.1p2.</p><div =
class=3D""><br class=3D""></div></div></div></blockquote><div><br =
class=3D""></div>Yes, I'm running exactly the same version with you.<br =
class=3D""><blockquote type=3D"cite" class=3D""><div class=3D""><div =
class=3D""><p class=3D"">how does your routing table looks for fib1 =
?</p><div class=3D""><br =
class=3D""></div></div></div></blockquote><div><br =
class=3D""></div><div>```</div><div># netstat -6rnF 1</div><div>Routing =
tables (fib: 1)</div><div><br =
class=3D""></div><div>Internet6:</div><div>Destination =
Gateway =
=
Flags Netif Expire</div><div>fe80::%lo1/64 =
link#5 =
=
U =
lo1</div><div>fe80::1%lo0 =
link#2 =
UHS =
lo0</div><div>```</div><blockquote type=3D"cite" =
class=3D""><div class=3D""><div class=3D""><p class=3D"">Santi</p><p =
class=3D""><br class=3D"">
</p>
<div class=3D"moz-cite-prefix">On 7/18/24 18:09, Zhenlei Huang =
wrote:<br class=3D"">
</div>
<blockquote type=3D"cite" =
cite=3D"mid:F7B1F9E1-30E0-4BC2-84F8-697A655A2C44@FreeBSD.org" class=3D"">
<meta http-equiv=3D"Content-Type" content=3D"text/html; =
charset=3DUTF-8" class=3D"">
<br class=3D"">
<div class=3D""><br class=3D"">
<blockquote type=3D"cite" class=3D"">
<div class=3D"">On Jul 13, 2024, at 1:06 AM, Santiago Martinez
<<a href=3D"mailto:sm@codenetworks.net" =
class=3D"moz-txt-link-freetext" =
moz-do-not-send=3D"true">sm@codenetworks.net</a>>
wrote:</div>
<br class=3D"Apple-interchange-newline">
<div class=3D"">
<meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3DUTF-8" class=3D"">
<div class=3D""><p class=3D"">Hi Everyone.</p>
While adding -F ( fib as used in netstat ) to ping and
ping6 I have found something that from my understanding is
not correct.<br class=3D"">
Please can you advise?<br class=3D""><p class=3D"">I have =
the following setup :</p>
-- two fibs (0 and 1) <br class=3D"">
-- two loop-backs (lo0 and lo1).<br class=3D"">
-- Lo1 has been assigned to fib1<br class=3D"">
--<span style=3D"background-color: rgb(255, 255, 255);" =
class=3D""> net.add_addr_allfibs =3D 0</span><p class=3D""><span =
style=3D"background-color: rgb(255, 255, 255);" class=3D"">My
interface output looks like this:<br class=3D"">
</span><span style=3D"font-family:monospace" =
class=3D""></span></p>
<div class=3D""><span style=3D"font-family:monospace" =
class=3D""></span><br class=3D"webkit-block-placeholder">
</div><p class=3D""><span style=3D"font-family:monospace" =
class=3D""><span style=3D"background-color: rgb(255, 255, 255);" =
class=3D"">ifconfig lo0 | grep inet6 </span><br class=3D"">
inet6 ::1 =
prefixlen 128 <br class=3D"">
inet6 =
fe80::1%lo0 prefixlen 64 scopeid 0x2 <br class=3D"">
</span></p><p class=3D""><span =
style=3D"font-family:monospace" class=3D"">ifconfig
lo1 | grep inet6 <br class=3D"">
inet6 =
fe80::1%lo1 prefixlen 64 scopeid 0x3<br class=3D"">
<br class=3D"">
</span></p><p class=3D"">If I do a netstat -rn -6 =
-F0 I get the
following which is was i expected.<br class=3D"">
</p><p class=3D""><span style=3D"font-family:monospace" =
class=3D""><span style=3D"background-color: rgb(255, 255, 255);" =
class=3D"">Internet6: </span><br class=3D"">
Destination =
&n=
bsp; Gateway
=
&n=
bsp; Flags =
Netif Expire <br class=3D"">
::/96 =
&n=
bsp; &nbs=
p; link#2
=
&n=
bsp; URS =
lo0 <br class=3D"">
::1 =
&n=
bsp; &nbs=
p; link#2
=
&n=
bsp; UHS =
lo0 <br class=3D"">
::ffff:0.0.0.0/96 =
&n=
bsp; link#2
=
&n=
bsp; URS =
lo0 <br class=3D"">
fe80::%lo0/10 =
&n=
bsp; link#2
=
&n=
bsp; URS =
lo0 <br class=3D"">
fe80::%lo0/64 =
&n=
bsp; link#2
=
&n=
bsp; U =
lo0 <br =
class=3D"">
fe80::1%lo0 =
&n=
bsp; link#2
=
&n=
bsp; UHS =
lo0 <br class=3D"">
ff02::/16 =
&n=
bsp; link=
#2
=
&n=
bsp; URS =
lo0<br class=3D"">
<br class=3D"">
</span></p>
Now, netstat -rn -6 -F1 shows =
"fe80::1%lo0" which
should not be there and "fe80::1%lo1" is missing which
should be there.
<div class=3D""><br class=3D"webkit-block-placeholder">
</div><p class=3D""><span style=3D"font-family:monospace" =
class=3D""><span style=3D"background-color: rgb(255, 255, 255);" =
class=3D"">Internet6: </span><br class=3D"">
Destination =
&n=
bsp; Gateway
=
&n=
bsp; Flags =
Netif Expire <br class=3D"">
fe80::%lo1/64 =
&n=
bsp; link#3
=
&n=
bsp; U =
lo1 <br =
class=3D"">
<b class=3D"">fe80::1%lo0 =
&n=
bsp; link#2
=
&n=
bsp; UHS =
lo0</b><br class=3D"">
</span></p>
</div>
</div>
</blockquote>
That seems wrong from my first glance. IIRC, there's HACK ( I'd
prefer this ) for loopback route. For example</div>
<div class=3D"">```</div>
<div class=3D""># sysctl net.fibs=3D3</div>
<div class=3D"">net.fibs: 2 -> 3</div>
<div class=3D""># ifconfig epair create</div>
<div class=3D""># <span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D"">epair0a</span></div>
<div class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D""># ifconfig </span><span style=3D"caret-color: rgb(0, 0, =
0);" class=3D"">epair0a fib 2</span></div>
<div class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D""># ifconfig </span><span style=3D"caret-color: rgb(0, 0, =
0);" class=3D"">epair0a inet6 -ifdisabled up</span></div>
<div class=3D""><span style=3D"caret-color: rgb(0, 0, 0);" =
class=3D""># netstat -6rnF 2</span></div>
<div class=3D"">
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">Routing tables
(fib: 2)</span></font></div>
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D""><br class=3D"">
</span></font></div>
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">Internet6:</span></font></div>
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">Destination
=
Gateway =
Flags
Netif Expire</span></font></div>
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">fe80::%epair0a/64
=
link#5 =
U
epair0a</span></font></div>
<div class=3D""><font class=3D""><span style=3D"caret-color: =
rgb(0, 0, 0);" class=3D"">fe80::3b:b3ff:fe8f:9a0a%lo0
link#1 =
UHS =
lo0</span></font></div>
</div>
<div class=3D"">```</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">The loopback route always refer the first loop =
interface, aka
lo0. </div>
<div class=3D"">
<blockquote type=3D"cite" class=3D"">
<div class=3D"">
<div class=3D""><p class=3D""><span =
style=3D"font-family:monospace" class=3D"">
<br class=3D"">
</span></p>
What output I was expecting was:
<p class=3D""><span style=3D"font-family:monospace" =
class=3D""><span style=3D"background-color: rgb(255, 255, 255);" =
class=3D"">Internet6: </span><br class=3D"">
Destination =
&n=
bsp; Gateway
=
&n=
bsp; Flags =
Netif Expire <br class=3D"">
fe80::%lo1/64 =
&n=
bsp; link#3
=
&n=
bsp; U =
lo1 <br =
class=3D"">
<b class=3D"">fe80::1%lo1 =
&n=
bsp; link#3
=
&n=
bsp; UHS =
lo1</b></span></p><p =
class=3D""><br class=3D"">
</p><p class=3D"">This makes the ping -6 -F0 =
fe80::1%lo0 to
work but ping -6 -F1 fe80::1%l01 to fail which I wanted
to use as test case.<span style=3D"font-family:monospace" =
class=3D""><br class=3D"">
</span></p>
</div>
</div>
</blockquote>
<div class=3D"">That is interesting. I can ping without =
failure.</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">```</div>
<div class=3D"">
<div class=3D""># setfib 1 ping6 -c3 fe80::1%lo1</div>
<div class=3D"">PING(56=3D40+8+8 bytes) fe80::1%lo1 --> =
fe80::1%lo1</div>
<div class=3D"">16 bytes from fe80::1%lo1, icmp_seq=3D0 =
hlim=3D64 time=3D0.050
ms</div>
<div class=3D"">16 bytes from fe80::1%lo1, icmp_seq=3D1 =
hlim=3D64 time=3D0.067
ms</div>
<div class=3D"">16 bytes from fe80::1%lo1, icmp_seq=3D2 =
hlim=3D64 time=3D0.096
ms</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">--- fe80::1%lo1 ping statistics ---</div>
<div class=3D"">3 packets transmitted, 3 packets received, =
0.0% packet
loss</div>
<div class=3D"">round-trip min/avg/max/stddev =3D =
0.050/0.071/0.096/0.019
ms</div>
</div>
<div class=3D"">```</div>
<div class=3D""><br class=3D"">
</div>
<div class=3D"">
<div style=3D"caret-color: rgb(0, 0, 0);" class=3D"">Best
regards,</div>
<div style=3D"caret-color: rgb(0, 0, 0);" =
class=3D"">Zhenlei</div>
</div>
<br class=3D"">
<blockquote type=3D"cite" class=3D"">
<div class=3D"">
<div class=3D""><div class=3D""><span =
style=3D"font-family:monospace" class=3D"">
</span><br class=3D"webkit-block-placeholder"></div><p =
class=3D"">Thanks in advance.</p>
Santiago
<p class=3D""><span style=3D"font-family:monospace" =
class=3D""><br class=3D"">
</span></p>
</div>
</div>
</blockquote>
</div>
<br class=3D"">
<div class=3D"">
<div class=3D""><br class=3D"">
</div>
</div>
<br class=3D"">
</blockquote>
</div>
</div></blockquote></div><br class=3D""><div class=3D"">
<div><br class=3D""></div>
</div>
<br class=3D""></body></html>=
--Apple-Mail=_C338E674-F01E-4A6C-8A76-67DBE390042A--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E2F8E66A-B141-47C4-B4B8-551D22784244>