Date: Mon, 03 Aug 2020 22:12:19 +0200 From: "Ronald Klop" <ronald-lists@klop.ws> To: "Ernie Luzar" <luzar722@gmail.com> Cc: freebsd-current@freebsd.org Subject: Re: vnet/jail crashdump Message-ID: <op.0osietfvkndu52@sjakie> In-Reply-To: <5F2856FB.1010305@gmail.com> References: <op.0osa6w1ckndu52@sjakie> <5F2856FB.1010305@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, 03 Aug 2020 20:27:07 +0200, Ernie Luzar <luzar722@gmail.com> wrote: > Ronald Klop wrote: >> Hi, >> After stopping a jail I get a crashdump. >> core.txt: >> https://www.klop.ws/core_2eef39c581f90f2f0c4921e43f1998c1/core.txt.0 >> Jail.conf: >> ---------- >> exec.stop = "/bin/sh /etc/rc.shutdown"; >> exec.clean; >> exec.prestart = "ifconfig bridge0 > /dev/null 2> /dev/null || ( >> ifconfig bridge0 create && ifconfig bridge0 addm vtnet0 && ifconfig >> bridge0 up)"; >> exec.consolelog = "/var/log/jail_${name}_console.log"; >> mount.devfs; >> path = "/data/jails/$name"; >> host.hostname = "$name"; >> mount.fstab = "/data/jails/fstab.$name"; >> vnet; >> allow.mlock; >> devfs_ruleset="110"; >> freebsd12 { >> osrelease = 12.1-RELEASE-p4; >> osreldate = 1201000; >> vnet.interface = "epair0b"; >> # make sure the exec.prestart has a "+=" as we de it in the global >> definition >> # when checking for the bridge >> exec.prestart += "ifconfig epair0 create up"; >> exec.prestart += "ifconfig bridge0 addm epair0a"; >> exec.prestart += "ifconfig epair0b link 02:xxxxxx:0c"; >> exec.start = "dhclient epair0b"; >> exec.start += "/bin/sh /etc/rc"; >> exec.poststop = "ifconfig bridge0 deletem epair0a"; >> exec.poststop += "ifconfig epair0a destroy"; >> } >> freebsd13 { >> vnet.interface = "epair1b"; >> # make sure the exec.prestart has a "+=" as we de it in the global >> definition >> # when checking for the bridge >> exec.prestart += "ifconfig epair1 create up"; >> exec.prestart += "ifconfig bridge0 addm epair1a"; >> exec.prestart += "ifconfig epair1b link 02:xxxxxx:0d"; >> exec.start = "dhclient epair1b"; >> exec.start += "/bin/sh /etc/rc"; >> exec.poststop = "ifconfig bridge0 deletem epair1a"; >> exec.poststop += "ifconfig epair1a destroy"; >> } >> ---------- >> What can I do to help debug? >> > > > Don't understand why you have these 2 statements > > exec.prestart += "ifconfig epair1b link 02:xxxxxx:0d"; > exec.start = "dhclient epair1b"; Using dhcp on a fixed MAC is much faster in my network. This might be written in a better way. Please enlighten me. After a lot of twiddling with settings this worked. > There is a well known bug with bridge vnet tear down since release 9.0. > Their is a rewrite of if_bridge going on right now to fix the problem > and increase the performance of if_bridge. As of today this fix is not > in 12.2 stable or 13.0 current. Ah ok, so it is a known issue. > There also looks like a bug in jail(8) when you have both vnet jails and > non-vnet jails being started on the same host at the same time. In most > cases the host just loses internet access until all the jails are > stopped. Some times you will get a system crash. Ok. Not my use case, but good to know. > This jail.conf def seems to work around the bridge tear down problem > > # vnet jail using the bridge/epair method on 12.1 > v0jail1 { > host.hostname = "v0jail1"; > path = "/usr/jails/v0jail1"; > mount.fstab = "/usr/local/etc/fstab/v0jail1"; > exec.consolelog = "/var/log/v0jail1.console.log"; > mount.devfs; > devfs_ruleset = "4"; > vnet = "new"; > vnet.interface = "epair55b"; > exec.prestart = "ifconfig epair55 create up"; > exec.prestart += "ifconfig bridge0 addm epair55a"; > exec.prestart += "ifconfig epair55a descr vnet-v0jail1"; > exec.prestart += "ifconfig bridge0 inet 10.0.48.2 netmask 255.255.255.0 > alias"; > exec.start = "/bin/sh /etc/rc"; > exec.start += "ifconfig epair55b inet 10.0.48.1 netmask > 255.255.255.0"; > exec.start += "route add default 10.0.48.2"; > exec.prestop = "ifconfig epair55b -vnet v0jail1"; > exec.stop = "/bin/sh /etc/rc.shutdown"; > exec.poststop = "ifconfig bridge0 deletem epair55a"; > exec.poststop += "sleep 2"; > exec.poststop += "ifconfig epair55a destroy"; > exec.poststop += "ifconfig bridge0 inet 10.0.48.2 -alias"; > } > > Remember that your host firewall processes all traffic in & out of the > host including any vnet jail traffic. Yes a vnet jail has its own stack > and can have its own firewall, but the host firewall still has the last > say. The host must NAT any private ip addresses used by the vnet jails. > > jail.conf jail definitions are based on hard codded ip addresses. You > can not use the host dhcp to assign local lan private ip addresses to a > jail. > > You may find this helpful > > https://forums.freebsd.org/threads/vnet-jail-with-public-internet-access-using-the-bridge-epair-method.76071/ > Thanks for all the info. Ronald.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.0osietfvkndu52>