Date: Sat, 18 Feb 2017 12:38:17 -0500 From: Ernie Luzar <luzar722@gmail.com> To: Jeff Kletsky <freebsd@wagsky.com> Cc: freebsd-jail@freebsd.org Subject: Re: Using jail.conf array parameters in exec.* commands Message-ID: <58A88689.9030607@gmail.com> In-Reply-To: <052f45c5-a808-724c-90d5-1b7464e9a585@wagsky.com> References: <b54f6576-7627-f3c3-70b0-b2c9c54a4fd6@wagsky.com> <58A42DC7.5040702@gmail.com> <5c11e326-cd4b-73e1-a681-9d116a0c1cd3@wagsky.com> <58A780C4.6030503@gmail.com> <052f45c5-a808-724c-90d5-1b7464e9a585@wagsky.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On release 10.3 or maybe 10.2 there were Devin Teske's examples of vnet/netgraph setups in these directories. /usr/share/examples/netgraph /usr/share/examples/jails 11.0 does not have them any more. If I recall correctly, some of the netgraph commands need the jid number, which means the vnet jail has to be started. But here "started" does not have the normal meaning. A vnet jail can be created, jail -c, and as long as there is no "exec.start sh /etc/rc" command in the jail.conf definition there is nothing running in the jail. Now your script has a jid to use and can setup the netghaph world that will allow the vnet jail to commutate. After all that stuff is complete, a jexec command can be issued to start services in the jail. "sh /etc/rc" is all you need to fire up services in the vnet jail. Any userland application like apache have to be installed the normal way and have its apache24_enable="YES" statement id the vnet's jail rc.conf file. Here's a general observation. During the 10.x releases I had a netgraph vnet script that Devin Teske posted to questions or jail list that worked. I think it was the same one that was in 10.x /usr/share/examples/jails directory. But when I tried it on 11.0 it stopped functioning giving a error on some netctl command. For me netgraph is just to hard. That is why I use the bridge/epair method. Jeff Kletsky wrote: > Thanks again for your thoughts on this. > > I *am* using "pure" jail(8) and jail.conf(5) techniques and have been > for many years now. What I'd like to get to is a robust way to start > jails the way I have been > > # jail -c some_jail > > and just have it work reliably, especially when there is setup that > needs to be done on the host to enable the smooth running of the jail. > > Preferably, *all* configuration of the jail and its connectivity is > done in jail.conf, be it /etc/jail.conf or a jail-specific one. I'm > trying to avoid going back to having to define another service that > wraps a call to jail(8) which would just start the jail with persist > set, then have to set up networking in the wrapper, jexec /etc/rc, > and then deal with shutdown of a jail created with persist set. > > I agree that *usually* the reason a jail won't start is > misconfiguration. However, there are other conditions that can occur, > such as lack of resources. If you want to more gracefully respond to > this, the wrapper script would need either to parse the appropriate > jail.conf, or to have all the pertinent information available in > another form. Having individual jail.conf files for each jail at > least makes easier to parse, at the expense of not being able to > define global and regional jail properties that are inherited across > the appropriate jails. > > > The network has to be up and connected *before* /etc/rc runs, > especially where services in the jail need network interfaces present > to bind to specific addresses, to mount network file systems, or have > access to critical services, such as DNS (for example, nginx will fail > to start if it can't resolve proxy host names). > > Since there is no jail vnet or jail ID (number) available, you can't > have jail(8) run needed operations in the jail.conf-declared > exec.prestart command, including, for example: > > * ifconfig interface vnet jail > * ipfw add action proto from src to dst jail prisonID > * ipfw add lookup jail table_name > > By the time jail(8) will run the jail.conf declared exec.poststart, > exec.start has already run to completion inside the jail. > > > How do you handle getting the network up *before* /etc/rc or the > specific service is started in the jail? > > > I unfortunately suspect you're right that I can't use the existing > jail(8) and jail.conf(5) approach without wrapping the whole thing in > a script. The hooks, even for networking, don't seem to be there. > > > Jeff > > > > On 2/17/17 3:01 PM, Ernie Luzar wrote: > > >Lets make this simple. Do not use the "service jail jailname start" > >command to start / stop your jails. > > >Your mixing legacy rc.conf jail method with jail.conf method. All > >ways use the jail(8) command itself to start/stop your jails. If you > >do this in a script then you can check the jail resulting return code > >to determine if the jail start/stop failed. But there is no > >information to tell you why it failed. In all most all cases it's > >caused by jail.conf parameters syntax coding error or invalid value > >content. Really pretty simple to determine cause by looking at the > >jail.conf content for the offending vnet jail. > > >Change your mind set from thinking you have to use the exec.* hooks > >to configure the vnet jails netgraph network setup. > > >Just have individual jail.conf files for each vnet jail with no vnet > >interface defined. > > >Now you can start the jail with just the standard exec.start line and > >standard exec.stop line. Once your script has issued the jail(8) > >command to start the jail then follow it with all the netgraph > >commands to enable its network. The vnet jail it self has no > >knowledge of any network connectivity at start up, you can wrap > >either bridge/epair or netgraph around it and it don't care. > > >This was learned the hard way. > > > Jeff Kletsky wrote: > >> Thanks for the suggestion of trying to use 'ifconfig interface vnet > jail' > >> in the scripts themselves. > >> > >> I'll get my scripts up once I've got them running again confidently > >> and can get proper licensing on them. > >> > >> TL;DR > >> > >> * Is there a clean way to "catch" failures in jail(8) creation after > >> exec.prestart completes, such as vnet.interface failing? > >> > >> * Is there a good way to execute commands in the host environment once > >> jail(8) brings up the jail, but before exec.start runs? > >> [...] >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?58A88689.9030607>