Date: Tue, 23 Aug 2022 18:06:59 +0000 From: tt78347 <tt78347@protonmail.com> To: Milan Obuch <freebsd-net@dino.sk> Cc: freebsd-net@freebsd.org, grembo@freebsd.org Subject: Re: Tunnel interfaces and vnet boundary crossing Message-ID: <DSv5_R2RSPi3WtHuwrcckIA4cW-3SSU5nH3a_qyLoa6jItMHdYriEiOGG1q-LRyzDPz2VCqC8ZoTUEZwo1WK2QBWXQivGR3pXdORz0JFeOs=@protonmail.com> In-Reply-To: <20220823175918.4696603f@zeta.dino.sk> References: <3N4WPlYG_OY2JAm4Rg5nWFjqo650j0ccOEOUWSwvWEp9cOudWPfFQ3C54QddWJcqyKd450QB7eczvghXmHbgImkn0tk2pAg7zbn49dJlPtM=@protonmail.com> <20220823175918.4696603f@zeta.dino.sk>
next in thread | previous in thread | raw e-mail | index | archive | help
------- Original Message -------
On Tuesday, August 23rd, 2022 at 3:58 PM, Milan Obuch <freebsd-net@dino.sk>=
wrote:
> On Tue, 23 Aug 2022 15:06:00 +0000
> tt78347 tt78347@protonmail.com wrote:
>=20
> > Hello. This is my first email to this mailing list, so I hope it has
> > been sent and formatted correctly.
>=20
>=20
> Well, top posting is being frowned upon a bit on mailing list, which
> has its racio, otherwise I think everything is fine :)
>=20
> > Regarding tun interfaces, I do something similar to M. Gmelin,
> > however, I create the tunnel (tun, for openvpn) within the vnet jail.
> > I think your question is not about tun interfaces but about gre,
> > which I discuss near the end of this email.
>=20
>=20
> You are right, terminology is sometimes somewhat ambiguous. I have no
> issues with software tunnel interface (man tun). I did not find the
> right configuration for my needs for network tunnels (man gif, man gre,
> man if_ipsec, the last one could be the most desired one).
>=20
> > Regarding devfs:
> > My /etc/devfs.rules contains the following ruleset, applicable to all
> > such jails:
> >=20
> > [devfsrules_jail=3D5]
> > add include $devfsrules_hide_all
> > add include $devfsrules_unhide_basic
> > add include $devfsrules_unhide_login
> > add path bpf unhide
> > add path tun* unhide
> > add path bpf0 unhide
> >=20
> > This ruleset ID is specified during jail creation. If you are
> > modifying this on a running system, I think you will need to use the
> > devfs utility. This is how I get tun interfaces into a jail, for use
> > with openvpn or otherwise.
>=20
>=20
> I use something similar in my setup, basically with expanded includes.
>=20
> > General background:
> > All the vnet jails are to be connected to a bridge with epairs, and
> > NAT is to be performed for jail traffic exchanged with the
> > exterior.
>=20
>=20
> In general, epair is optional, in some use cases vnet jail could be
> decisively insulated from the rest of jails. But I know this one and
> use it if in accordance with my design goals.
OK. I just meant it to specify what was included in my suggested design.
>=20
> > There is an additional vnet jail which handles ipsec; the ipsec jail
> > connects the host's jail network (usually a /24) to other local
> > networks. Each external local network's route is specified in the
> > host's routing table, with a next-hop being the ipsec jail. I
> > personally don't use ipsec interfaces, so in the ipsec jail the
> > traffic is extracted by the kernel (per TSs that are configured with
> > StrongSwan), encapsulated and sent out the host's external interface.
> > I have no knowledge of whether this is better or worse than using an
> > ipsec interface.
>=20
>=20
> This one is interested for me. Could you elaborate more on its design?
> Output of 'ifconfig', 'netstat -rn', 'pkg info', 'cat /etc/rc.conf'
> commands would be really usefull. If you do concern about privacy,
> maybe edit it a bit (I am not interested in exact IPs and such info).
I think I had misunderstood your original post, but here is information abo=
ut this ipsec jail.
I have replaced certain info with variable names.
# cat /etc/rc.conf
devfs_load_rulesets=3D"YES"
hostname=3D"$HOSTNAME"
syslogd_enable=3D"YES"
syslogd_flags=3D"-s -v -v"
inetd_enable=3D"NO"
defaultrouter=3D"${HOST_NET}.1"
firewall_enable=3D"YES"
firewall_script=3D"/etc/ipfw.conf"
firewall_logif=3D"YES"
sendmail_enable=3D"NO"=20
sendmail_submit_enable=3D"NO"=20
sendmail_outbound_enable=3D"NO"=20
sendmail_msp_queue_enable=3D"NO"
# cat /etc/ipfw.conf
ipfw add allow ip from any to any
# cat /etc/rc.conf.local=20
strongswan_enable=3D"YES"
# netstat -rn
Routing tables
Internet:
Destination Gateway Flags Netif Expire
default ${HOST_NET}.1 UGS epair${JAIL_ID}b
127.0.0.1 link#1 UH lo0
${HOST_NET}.0/24 link#${JAIL_ID} U epair${JAIL_ID}b
${HOST_NET}.${JAIL_ID} link#${JAIL_ID} UHS lo0
This ipsec jail has epair IP address of ${HOST_NET}.${JAIL_ID}.=20
For example, 192.168.1.100, where $JAIL_ID=3D100 and $HOST_NET=3D192.168.1
The host's bridge is at ${HOST_NET}.1
The only relevant package in this jail is security/strongswan.
In this design, all vnet jails have addresses on $HOST_NET using epair inte=
rfaces connected to one or more virtual host bridge interfaces.
ARP takes place on this bridge, and the host forwards packets between jails=
.
The ipsec jail is just one of those jails.=20
The host also has routes for other sites' local networks. These routes are =
set to go through the ipsec jail.
The ipsec jail then encapsulates them, and from the host's perspective, whe=
n it leaves the ipsec jail, it is just ESP/UDP.
In this design, ipsec is used for site-to-site tunneling, and GRE is used f=
or use-case-specific tunneling between hosts on this WAN. A common use case=
for this is when a machine needs to set a route or a default route through=
a gateway that is not on a connected subnet. Operating ipsec and GRE indep=
endently means you can use GRE where necessary, with the whole WAN being en=
capsulated with ipsec. Traffic within a host being unencrypted.
>=20
> > Specific steps inside an openvpn jail:
> >=20
> > After an openvpn jail is started, the tunnel (tun interface) is
> > created in the jail by running the openvpn daemon. I think tunnel
> > numbers (appearing on the interface name) need to be globally
> > distinct. I'm not sure if you needed any assistance with this.
>=20
>=20
> In my scenario, I use tun devices pre-created before running OpenVPN
> process. This allows for better control and easier to understand setup.
I think I misunderstood your posts. I thought you were running openvpn clie=
nts, but I see from reading again that you are running openvpn servers. I h=
ave never run an openvpn server before.
>=20
> > Regarding gre interfaces, I also create these within the jail. I have
> > not ever had any problems with this. I don't know that a specific
> > device is needed, so I don't know if devfs is involved here.
>=20
>=20
> This setup just requires connection to outer network, which I would
> like specifically to be avoided as it makes setup a bit more
> complicated. Just a reminder, I have almost twenty of running OpenVPN
> instances in one jail, with almost twenty vnet jails handling the
> routing for each VPN.
I don't fully understand what is happening. What do you mean by outer netwo=
rk, and why would it be an issue to create the GRE tunnel inside the jail, =
instead of creating it outside the jail and moving it into the jail?
I think you have your openvpn server instances in a vnet jail, and then chi=
ld jails within that jail which are doing something related to routing. Are=
you using many fwd-type firewall rules, instead of using default routes?
Would it make sense to have each openvpn server in a separate vnet jail, ea=
ch with its own GRE interface, and do away with your child vnet jails that =
handle routing?
Sorry for my misunderstandings.
>=20
> [ snip ]
>=20
> I hope this time I made it a bit clearer what I am looking for.
>=20
> Regards,
> Milan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DSv5_R2RSPi3WtHuwrcckIA4cW-3SSU5nH3a_qyLoa6jItMHdYriEiOGG1q-LRyzDPz2VCqC8ZoTUEZwo1WK2QBWXQivGR3pXdORz0JFeOs=>