Date: Sun, 15 Feb 2004 23:32:55 +0100 From: Melvyn Sopacua <freebsd-current@webteckies.org> To: current@FreeBSD.org Cc: Robert Watson <rwatson@FreeBSD.org> Subject: Re: Jails that keep hanging around Message-ID: <200402152333.00565.freebsd-current@webteckies.org> In-Reply-To: <Pine.NEB.3.96L.1040215130356.56481G-100000@fledge.watson.org> References: <Pine.NEB.3.96L.1040215130356.56481G-100000@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--Boundary-02=_cO/LACtSa0DXzlk
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
On Sunday 15 February 2004 19:18, Robert Watson wrote:
> On Sun, 15 Feb 2004, Melvyn Sopacua wrote:
> > I have yet to figure out what triggers the bug, but I end up with
> > 'running' jails, without any processes. So I thought I'd create 'jld' to
> > remove a jail. However - prison_find isn't exported to userland.
> > Probably for good reason.
>
[ snip very informative analysis ]
> The only real exception to this is the tcpcb
> -- TCP connection remenants can last for quite a time after their socket
> exits, since they follow the TCP state machine (which has long waits). So
> this could be it -- check netstat and see if there are any largely closed
> TCP sessions from the jails. FYI, this is not a complete list of
> credential references, but it accounts for most of them.
>
> Are you using any services that pass references to sockets or other file
> descriptor objects in and out of the jail using UNIX domain sockets? If
> so, that could also be it.
Mostly this happens when setting up the jails. The first one (10) with the=
=20
jail-${IP}.idg.nl hostname, is created by a script 'makejail.sh', which is=
=20
largely based upon the make steps descriped in jail(8). After the steps=20
listed there, the jail is fired up to execute 'ports-jail.sh' inside the=20
jail, which basically sets up some standard ports (portupgrade, shells, vim=
,=20
postfix). I copy over distfiles, but it is possible, that fetch(1) came int=
o=20
play. After this script, all is done - the jail should exit.
The number 15 with the correct hostname, is me logging in and doing some=20
additional work, mostly installing and configuring ports. The use of fetch(=
1)=20
is very likely, as these ports are used inside jails only and typically don=
't=20
exist in the host system's distfiles. I've logged out, without having stuff=
=20
in the background.
I've tried to get the refcount, but get stuck on the proc structure - getti=
ng=20
a bus error a few lines down. I couldn't find a faster road to this - if=20
anybody can, I'd be happy to run some diagnostics. Probably the most=20
interesting thing is to see whether the prison refcount is really 0.
My attempt below, was compiled with -D_KERNEL -lkvm.
=2D-=20
Melvyn
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
=46reeBSD sarevok.webteckies.org 5.2-CURRENT FreeBSD 5.2-CURRENT #1: Sat Fe=
b 14=20
02:48:30 CET 2004 =20
root@sarevok.webteckies.org:/usr/obj/usr/src/sys/SAREVOK_NOAPM_NODEBUG i386
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D
/* vim600: sw=3D4 ts=3D4 ai
*/
#include <sys/param.h>
#include <sys/types.h>
#include <fcntl.h>
#include <kvm.h>
#include <limits.h>
#include <sys/proc.h>
#include <sys/ucred.h>
#include <sys/jail.h>
#include <sys/sysctl.h>
#include <sys/user.h>
#include <sys/wait.h>
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static struct prison *
prison_find(int prid)
{
struct prison *pr;
pid_t pid;
char errbuf[_POSIX2_LINE_MAX];
kvm_t *kern;
struct kinfo_proc *p_procinfo;
struct proc *p_proc;
struct ucred *p_uc;
int num_procs, i, status;
pid =3D fork();
pr =3D NULL;
switch (pid)
{
case -1 :
perror("fork()");
break;
case 0 :
setsid();
jail_attach(prid);
sleep(60);
exit(0);
default:
sleep(1); /* wait for jail_attach() */
kern =3D kvm_openfiles(NULL, NULL, NULL, O_RDONLY, errbuf);
if( kern =3D=3D NULL )
{
fprintf(stderr, "Cannot open kernel memory: %s", errbuf);
break;
}
p_procinfo =3D kvm_getprocs(kern, KERN_PROC_PID, pid, &num_procs);
printf("Getting %d processes for pid %d and my pid is %d\n", num_procs,=
=20
pid, getpid());
for( i=3Dnum_procs; --i >=3D 0 ; ++p_procinfo )
{
p_proc =3D p_procinfo->ki_paddr;
if( p_proc =3D=3D NULL )
{
fprintf(stderr,
"Cannot get proc struct from proinfo %d.\n", i);
break;
}
p_uc =3D (struct ucred *)p_proc->p_ucred;
if( p_uc =3D=3D NULL )
{
fprintf(stderr,
"Cannot get ucred struct from proc (pi %d).\n",
i);
break;
}
pr =3D (struct prison *)p_uc->cr_prison;
if( pr !=3D NULL )
break;
}
kvm_close(kern);
waitpid(pid, &status, WNOHANG);
break;
}
return pr;
}
int main(int argc, char *argv[])
{
struct prison *p;
int jid;
if( argc =3D=3D 2 )
{
jid =3D (int)strtol(argv[1], NULL, 10);
if( jid > 0 )
{
p =3D prison_find(jid);
if( p =3D=3D NULL )
{
fprintf(stderr, "Cannot find the prison!");
}
else
{
printf("Refcount is: %d\n", p->pr_ref);
}
}
else
{
printf("Invalid jid: %d", jid);
}
}
else
{
printf("No jid\n");
}
=09
return 0;
}
--Boundary-02=_cO/LACtSa0DXzlk
Content-Type: application/pgp-signature
Content-Description: signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (FreeBSD)
iD8DBQBAL/OcOv9JNmfFN5URAk0lAJoCiXBcNebYhGXsBoPBc2aOuAiclwCeKWkk
IRSyFk1DVrDORoL9+73oOGI=
=Frs0
-----END PGP SIGNATURE-----
--Boundary-02=_cO/LACtSa0DXzlk--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200402152333.00565.freebsd-current>