Date: Wed, 19 Oct 2022 22:42:35 -0700 From: Dan Mahoney <freebsd@gushi.org> To: Paul Procacci <pprocacci@gmail.com> Cc: questions@freebsd.org Subject: Re: Interface routes and multiple fibs Message-ID: <70D0A3A6-7967-4C2D-A165-BF9A7084A706@gushi.org> In-Reply-To: <CAFbbPugCxXKTOyE=QbRZ5dCQLgFCW9U0MysVZQG3Kv7-znw9Xg@mail.gmail.com> References: <354F1536-D803-472A-933C-8B6D9EAED1F1@gushi.org> <CAFbbPugCxXKTOyE=QbRZ5dCQLgFCW9U0MysVZQG3Kv7-znw9Xg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_FD14590B-0A75-4DF3-BE1D-3D1183F3EA69 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 18, 2022, at 15:16, Paul Procacci <pprocacci@gmail.com> wrote: >=20 >=20 >=20 > On Tue, Oct 18, 2022 at 5:12 PM Dan Mahoney <freebsd@gushi.org = <mailto:freebsd@gushi.org>> wrote: > All, >=20 > Maybe a question for the -net or -rc people. If I should ask there, = let me know. >=20 > I'm running with multiple fibs. One fib is just box management, ssh, = etc. The other fib (which takes BGP routes from peers via BIRD) does = DNS anycast things. The DNS server runs in fib 1. Our default route is = added to both fibs. >=20 > My fib0 routing table looks like this: >=20 > Internet: > Destination Gateway Flags Netif Expire > default 192.159.249.233 UGS bge0 > 127.0.0.1 link#5 UH lo0 > 182.159.249.232/29 <http://182.159.249.232/29>; link#1 U = bge0 > 182.159.249.236 link#1 UHS lo0 >=20 > Fib 1 is missing that final route.: >=20 > default 182.159.249.233 UGS bge0 > 127.0.0.1 link#5 UH lo0 > 182.159.249.232/29 <http://182.159.249.232/29>; link#1 U = bge0 >=20 > I've noticed that when I try to do a query (with dig) against it from = fib 0, it sends over lo0 to the named process, but the reply packet just = gets sent out ON BGE0, and is never received, since dig is listening on = the interface it sent the packet over (lo0) to hear the response, which, = near as I can tell with tcpdump -i bge0, just goes out on the wire >=20 > Obviously, we can add the static route to that second fib with: route = add -host 182.159.249.236 -interface lo0 -fib 1. >=20 > Yes, we can also make this stick useing default_routes in rc.conf. >=20 > But it feels like we shouldn't have to. This feels like a glitch, and = that if all fibs get the SUBNET route , they should also get the = loopback. >=20 > -Dan >=20 >=20 > Why would you not expect to add a route for it? > The same subnets can exist in different fibs and be part of different = lan segments a la vlans. Routes are required. But...the same route is added for the SUBNET on both fibs automatically, = even though bge0 is in fib 1. Just not for the actual host. This feels = woefully inconsistent. --Apple-Mail=_FD14590B-0A75-4DF3-BE1D-3D1183F3EA69 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii <html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Oct 18, 2022, at 15:16, Paul Procacci <<a href="mailto:pprocacci@gmail.com" class="">pprocacci@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class=""><div dir="ltr" class=""><br class=""></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Oct 18, 2022 at 5:12 PM Dan Mahoney <<a href="mailto:freebsd@gushi.org" class="">freebsd@gushi.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">All,<br class=""> <br class=""> Maybe a question for the -net or -rc people. If I should ask there, let me know.<br class=""> <br class=""> I'm running with multiple fibs. One fib is just box management, ssh, etc. The other fib (which takes BGP routes from peers via BIRD) does DNS anycast things. The DNS server runs in fib 1. Our default route is added to both fibs.<br class=""> <br class=""> My fib0 routing table looks like this:<br class=""> <br class=""> Internet:<br class=""> Destination Gateway Flags Netif Expire<br class=""> default 192.159.249.233 UGS bge0<br class=""> 127.0.0.1 link#5 UH lo0<br class=""> <a href="http://182.159.249.232/29" rel="noreferrer" target="_blank" class="">182.159.249.232/29</a> link#1 U bge0<br class=""> 182.159.249.236 link#1 UHS lo0<br class=""> <br class=""> Fib 1 is missing that final route.:<br class=""> <br class=""> default 182.159.249.233 UGS bge0<br class=""> 127.0.0.1 link#5 UH lo0<br class=""> <a href="http://182.159.249.232/29" rel="noreferrer" target="_blank" class="">182.159.249.232/29</a> link#1 U bge0<br class=""> <br class=""> I've noticed that when I try to do a query (with dig) against it from fib 0, it sends over lo0 to the named process, but the reply packet just gets sent out ON BGE0, and is never received, since dig is listening on the interface it sent the packet over (lo0) to hear the response, which, near as I can tell with tcpdump -i bge0, just goes out on the wire<br class=""> <br class=""> Obviously, we can add the static route to that second fib with: route add -host 182.159.249.236 -interface lo0 -fib 1.<br class=""> <br class=""> Yes, we can also make this stick useing default_routes in rc.conf.<br class=""> <br class=""> But it feels like we shouldn't have to. This feels like a glitch, and that if all fibs get the SUBNET route , they should also get the loopback.<br class=""> <br class=""> -Dan<br class=""> </blockquote></div><br clear="all" class=""><br class=""></div>Why would you not expect to add a route for it?<br class=""></div><div class="">The same subnets can exist in different fibs and be part of different lan segments a la vlans. Routes are required.<br class=""></div></div></div></blockquote><div><br class=""></div><div>But...the same route is added for the SUBNET on both fibs automatically, even though bge0 is in fib 1. Just not for the actual host. This feels woefully inconsistent.</div><div><br class=""></div><div><br class=""></div></div></body></html> --Apple-Mail=_FD14590B-0A75-4DF3-BE1D-3D1183F3EA69--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?70D0A3A6-7967-4C2D-A165-BF9A7084A706>