Date: Tue, 14 Feb 2017 18:55:51 +1100 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Robert Eckardt <rol@Robert-Eckardt.de> Cc: freebsd-questions@freebsd.org Subject: Re: A simple routing question - SOLVED Message-ID: <20170214182539.U48485@sola.nimnet.asn.au> In-Reply-To: <mailman.99.1486900802.75581.freebsd-questions@freebsd.org> References: <mailman.99.1486900802.75581.freebsd-questions@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
In freebsd-questions Digest, Vol 662, Issue 7, Message: 6
On Sat, 11 Feb 2017 20:27:37 +0100 Robert Eckardt <rol@Robert-Eckardt.de> wrote:
> Hi again,
>
> thanks to all, who helped me by pointing me to some interesting
> articles on this topic.
> Eventually, I found a solution for my problem by employing
> policy based routing with layer-2 ipfw rules and multiple routing
> tables (FIBs).
> What makes things a little more complicated is the fact that both
> routers are addressed via the same interface.
>
> This summary is for the records for those who face the same problem.
>
> Using ECMP routing with kernel option RADIX_MPATH and two default
> routes proved unsuitable.
> a) since the two routers do NAT, packets must be returned through
> the same router as the connection was established and not the
> one based on destination address. (This is different from the
> situation without NAT.)
> b) After one router had failed for a short time (disconnected from
> power supply) a telnet to a host in the internet failed with
> "no route to host" while ping worked.
> I did not dig deeper into it as I concluded from several
> postings that the RADIX_MPATH feature is still unstable.
>
> Instead, (thanks to Matthew) I decided for policy based routing.
> Getting ipfw to filter the source MAC wasn't difficult.
> Understanding the behaviour of the stateful rules and the effect
> of layer-2 or layer-3 filtering was hard.
>
> Finally the following configuration works (for me):
>
> My SOLUTION:
> 1) since we need another forward information base, increase available
> FIBs
> in /boot/loader.conf
> net.fibs="2"
>
> 2) (after reboot) set alternate default routing (this needs to get set
> on every boot, so I put it in /etc/rc.local)
> setfib 1 route delete default
> setfib 1 route add default 10.0.0.2
>
> 3) Don't forget to also add routes to provider-specific infrastucture
> (e.g. name servers) via the correspondig router ('setfib 0' opt.)
> (This can go as a static route in /etc/rc.conf or in /etc/rc.local)
> setfib 0 route add 2.2.3.4 10.0.0.2
>
> 4) make sure that the logic of layer-3 rules works as before
> 2000 skipto 10000 ip from any to any not layer2
>
> 5) add layer-2 rule to ipfw
> 3000 allow tag 101 ip from any to any MAC any 08:9a:bc:de:ff:ff in recv re1
>
> 6) avoid matching of the default deny rule for layer-2 by accepting
> everything in ether_demux
> 9000 allow ip from any to any layer2
> 10000 <from here existing layer-3 rule-set>
>
> 7) switch on filtering of layer-2 packets
> (either in /etc/rc.local or in /etc/sysctl.conf)
> net.link.ether.ipfw="1"
If in /etc/rc.local, that obviously? needs preceding with 'sysctl '
> 8) use stateful information on layer-3
> 20000 check-state
> 20100 setfib 1 ip from any to any tagged 101 keep-state
> The incoming packet, identified on layer-2 by the MAC addres of
> Router2, is handled by the alternate FIB with the default route
> pointing back to Router2.
>
> 9) make sure that in the end packets get logged if some rules don't
> work as expected
> 65500 deny log ip from any to any MAC any any <-- layer-2
> 65530 deny log ip from any to any <-- layer-3
>
>
> 10) when accessing Router2, which forwards to Server, via Router1 and
> the internet, you will see working dynamic rules
> ## Dynamic rules (1 144):
> 20100 STATE tcp 1.1.1.123 51620 <-> 10.0.0.2 643
>
>
> I found (besides 'man ipfw' ;-) the following links most helpful
> http://daemonforums.org/showthread.php?t=4610
> https://forums.freebsd.org/threads/2201/
> https://blog.plitc.eu/2014/freebsd-10-1-pbr-ecmp-fib-und-ipfw-layer2-filtering/
>
> Comments welcome.
>
> Regards,
> Robert
Very elegant. I haven't done anything at layer2 for years, not have I
used tagging, but it smells good; happy to take your word it's working.
Would you care to post that - including your original question and
diagram below - to freebsd-ipfw@freebsd.org, where I think it would be
a) well appreciated and b) scrutinised by those who know ipfw best?
cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20170214182539.U48485>