Date: Sat, 05 Apr 2014 15:22:39 +1300 From: Chris Smith <chris@nevermind.co.nz> To: freebsd-net@freebsd.org Subject: Multihomed system with jails routing issues Message-ID: <533F68EF.8060607@nevermind.co.nz>
next in thread | raw e-mail | index | archive | help
Hi All, I have a system with 1 network interface with 2 extra VLANs off it and I'm having some trouble getting the routing working correctly with it and jails. bge0 - management - 10.71.100.0/24 bge0.101 - LAN - 10.71.101.0/24 bge0.103 - DMZ - 10.71.101.0/24 Here's what I want to achieve... Host: I want the host system to only listen on one interface, bge0. I want NO ip addresses of the host on the vlan interfaces. The only service it will be exposing is its sshd. The management address for this system is 10.71.100.50. Jails: The system will also host a variety of jails, each with an IP either on the LAN or DMZ. I am using ezjail to manage the jails. Router: There is a router at the .254 address of every subnet that can route between each network. I set up jail1 on bge0.101 with the IP 10.71.101.51. Since the host does not have an address configured on bge0.101, I configured the jail address as /24 instead of the default /32. My issues: * If I do not configure the jail as a /24 (e.g. /32), the LAN cannot communicate with the jail. * When the jail is up and 10.71.101.51/24 is active, SSHing from the LAN to the mgmt interface via the router fails, as the host tries to send return traffic via the bge0.101 interface, even though traffic arrived via the bge0 interface. So I did a whole lot of research for people having these apparently problems, and decided to try the multiple routing table/fib approach. So I recompiled my kernel, configured fib 1 with the LAN interface route (setfib route add 10.71.101.0/24 -iface bge0.101), set the jail fib and set the tunable net.addr_all_fibs = 0. I still can't get this working correctly. ezjail still seems to add the interface route to fib 0 by default (but it won't if i run ezjail with the setfib 1 command). Using FIB 1 and trying to ping hosts on the LAN gives an error like: sendto failed: invalid argument. Does anybody have any best practices for doing this, or anything else I can try? I'm happy to share/pastebin any configuration and I've tried most things I've found on the internet. I'm using FreeBSD 10.0 with a custom kernel for multiple routing tables. Thanks in advance! Chris.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?533F68EF.8060607>