Date: Thu, 20 Oct 2022 02:42:42 -0400 From: Paul Procacci <pprocacci@gmail.com> To: Dan Mahoney <freebsd@gushi.org> Cc: questions@freebsd.org Subject: Re: Interface routes and multiple fibs Message-ID: <CAFbbPuhUQmiYGWJwFc0fvDtxONgCtz-nXxfCOfVHVahQUdrdCw@mail.gmail.com> In-Reply-To: <70D0A3A6-7967-4C2D-A165-BF9A7084A706@gushi.org> References: <354F1536-D803-472A-933C-8B6D9EAED1F1@gushi.org> <CAFbbPugCxXKTOyE=QbRZ5dCQLgFCW9U0MysVZQG3Kv7-znw9Xg@mail.gmail.com> <70D0A3A6-7967-4C2D-A165-BF9A7084A706@gushi.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000054422f05eb71a2aa
Content-Type: text/plain; charset="UTF-8"
When you do a `host' command .... I'm presuming it wants to contact
182.159.249.236.
If we look at your routing table, you are directing traffic for that host
over lo0.
The host/fib/whatever that's tied to fib 1 ... when it wants to respond,
it's going to want to respond to the host which generated the packet which
is some machine in the 182.159.249.232/29 subnet.
Again, looking at the routing table, it's telling the packet to be sent out
bge0.
With the above said, I don't think that's where your question lies.
Admittedly I thought it was at first.
Is your question with the prepopulation of routes in new fibs? If so, then
look towards the net.add_addr_allfibs sysctl.
A value of 0 disables this behavior.
If I'm still wrong, then I'll need to be spoken to like a 4 yr old.
Thanks,
Paul Procacci
On Thu, Oct 20, 2022 at 1:42 AM Dan Mahoney <freebsd@gushi.org> wrote:
>
>
> On Oct 18, 2022, at 15:16, Paul Procacci <pprocacci@gmail.com> wrote:
>
>
>
> On Tue, Oct 18, 2022 at 5:12 PM Dan Mahoney <freebsd@gushi.org> wrote:
>
>> All,
>>
>> Maybe a question for the -net or -rc people. If I should ask there, let
>> me know.
>>
>> I'm running with multiple fibs. One fib is just box management, ssh,
>> etc. The other fib (which takes BGP routes from peers via BIRD) does DNS
>> anycast things. The DNS server runs in fib 1. Our default route is added
>> to both fibs.
>>
>> My fib0 routing table looks like this:
>>
>> Internet:
>> Destination Gateway Flags Netif Expire
>> default 192.159.249.233 UGS bge0
>> 127.0.0.1 link#5 UH lo0
>> 182.159.249.232/29 link#1 U bge0
>> 182.159.249.236 link#1 UHS lo0
>>
>> Fib 1 is missing that final route.:
>>
>> default 182.159.249.233 UGS bge0
>> 127.0.0.1 link#5 UH lo0
>> 182.159.249.232/29 link#1 U bge0
>>
>> I've noticed that when I try to do a query (with dig) against it from fib
>> 0, it sends over lo0 to the named process, but the reply packet just gets
>> sent out ON BGE0, and is never received, since dig is listening on the
>> interface it sent the packet over (lo0) to hear the response, which, near
>> as I can tell with tcpdump -i bge0, just goes out on the wire
>>
>> Obviously, we can add the static route to that second fib with: route
>> add -host 182.159.249.236 -interface lo0 -fib 1.
>>
>> Yes, we can also make this stick useing default_routes in rc.conf.
>>
>> But it feels like we shouldn't have to. This feels like a glitch, and
>> that if all fibs get the SUBNET route , they should also get the loopback.
>>
>> -Dan
>>
>
>
> Why would you not expect to add a route for it?
> The same subnets can exist in different fibs and be part of different lan
> segments a la vlans. Routes are required.
>
>
> But...the same route is added for the SUBNET on both fibs automatically,
> even though bge0 is in fib 1. Just not for the actual host. This feels
> woefully inconsistent.
>
>
>
--
__________________
:(){ :|:& };:
--00000000000054422f05eb71a2aa
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><div><div><div>When you do a `host' command .... =
I'm presuming it wants to contact 182.159.249.236.<br></div>If we look =
at your routing table, you are directing traffic for that host over lo0.<br=
><br></div>The host/fib/whatever that's tied to fib 1 ... when it wants=
to respond, it's going to want to respond to the host which generated =
the packet which is some machine in the=20
<a href=3D"http://182.159.249.232/29" rel=3D"noreferrer" target=3D"_blank">=
182.159.249.232/29</a> subnet.<br></div>Again, looking at the routing table=
, it's telling the packet to be sent out bge0.<br><br></div><div>With t=
he above said, I don't think that's where your question lies.=C2=A0=
Admittedly I thought it was at first.<br></div><div>Is your question with =
the prepopulation of routes in new fibs?=C2=A0 If so, then look towards the=
net.add_addr_allfibs sysctl.<br><br></div><div>A value of 0 disables this =
behavior.<br></div><div><br></div><div>If I'm still wrong, then I'l=
l need to be spoken to like a 4 yr old.<br><br></div><div>Thanks,</div><div=
>Paul Procacci<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"lt=
r" class=3D"gmail_attr">On Thu, Oct 20, 2022 at 1:42 AM Dan Mahoney <<a =
href=3D"mailto:freebsd@gushi.org">freebsd@gushi.org</a>> wrote:<br></div=
><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border=
-left:1px solid rgb(204,204,204);padding-left:1ex"><div style=3D"overflow-w=
rap: break-word;"><br><div><br><blockquote type=3D"cite"><div>On Oct 18, 20=
22, at 15:16, Paul Procacci <<a href=3D"mailto:pprocacci@gmail.com" targ=
et=3D"_blank">pprocacci@gmail.com</a>> wrote:</div><br><div><div dir=3D"=
ltr"><div><div><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote"><d=
iv dir=3D"ltr" class=3D"gmail_attr">On Tue, Oct 18, 2022 at 5:12 PM Dan Mah=
oney <<a href=3D"mailto:freebsd@gushi.org" target=3D"_blank">freebsd@gus=
hi.org</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"m=
argin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left=
:1ex">All,<br>
<br>
Maybe a question for the -net or -rc people.=C2=A0 If I should ask there, l=
et me know.<br>
<br>
I'm running with multiple fibs.=C2=A0 One fib is just box management, s=
sh, etc.=C2=A0 The other fib (which takes BGP routes from peers via BIRD) d=
oes DNS anycast things.=C2=A0 The DNS server runs in fib 1.=C2=A0 Our defau=
lt route is added to both fibs.<br>
<br>
My fib0 routing table looks like this:<br>
<br>
Internet:<br>
Destination=C2=A0 =C2=A0 =C2=A0 =C2=A0 Gateway=C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 Flags=C2=A0 =C2=A0 =C2=A0Netif Expire<br>
default=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 192.159.249.233=C2=A0 =C2=
=A0 UGS=C2=A0 =C2=A0 =C2=A0 =C2=A0 bge0<br>
127.0.0.1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 link#5=C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0UH=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 lo0<br>
<a href=3D"http://182.159.249.232/29" rel=3D"noreferrer" target=3D"_blank">=
182.159.249.232/29</a> link#1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0U=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bge0<br>
182.159.249.236=C2=A0 =C2=A0 link#1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0UHS=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0lo0<br>
<br>
Fib 1 is missing that final route.:<br>
<br>
default=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 182.159.249.233=C2=A0 =C2=
=A0 UGS=C2=A0 =C2=A0 =C2=A0 =C2=A0 bge0<br>
127.0.0.1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 link#5=C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0UH=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 lo0<br>
<a href=3D"http://182.159.249.232/29" rel=3D"noreferrer" target=3D"_blank">=
182.159.249.232/29</a> link#1=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0U=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bge0<br>
<br>
I've noticed that when I try to do a query (with dig) against it from f=
ib 0, it sends over lo0 to the named process, but the reply packet just get=
s sent out ON BGE0, and is never received, since dig is listening on the in=
terface it sent the packet over (lo0) to hear the response, which, near as =
I can tell with tcpdump -i bge0, just goes out on the wire<br>
<br>
Obviously, we can add the static route to that second fib with:=C2=A0 route=
add -host 182.159.249.236 -interface lo0 -fib 1.<br>
<br>
Yes, we can also make this stick useing default_routes in rc.conf.<br>
<br>
But it feels like we shouldn't have to.=C2=A0 This feels like a glitch,=
and that if all fibs get the SUBNET route , they should also get the loopb=
ack.<br>
<br>
-Dan<br>
</blockquote></div><br clear=3D"all"><br></div>Why would you not expect to =
add a route for it?<br></div><div>The same subnets can exist in different f=
ibs and be part of different lan segments a la vlans.=C2=A0 Routes are requ=
ired.<br></div></div></div></blockquote><div><br></div><div>But...the same =
route is added for the SUBNET on both fibs automatically, even though bge0 =
is in fib 1.=C2=A0 Just not for the actual host.=C2=A0 This feels woefully =
inconsistent.</div><div><br></div><div><br></div></div></div></blockquote><=
/div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"gmail_signature=
">__________________<br><br>:(){ :|:& };:</div>
--00000000000054422f05eb71a2aa--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFbbPuhUQmiYGWJwFc0fvDtxONgCtz-nXxfCOfVHVahQUdrdCw>