Date: Sun, 15 Feb 2004 23:32:55 +0100 From: Melvyn Sopacua <freebsd-current@webteckies.org> To: current@FreeBSD.org Cc: Robert Watson <rwatson@FreeBSD.org> Subject: Re: Jails that keep hanging around Message-ID: <200402152333.00565.freebsd-current@webteckies.org> In-Reply-To: <Pine.NEB.3.96L.1040215130356.56481G-100000@fledge.watson.org> References: <Pine.NEB.3.96L.1040215130356.56481G-100000@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
On Sunday 15 February 2004 19:18, Robert Watson wrote:
> On Sun, 15 Feb 2004, Melvyn Sopacua wrote:
> > I have yet to figure out what triggers the bug, but I end up with
> > 'running' jails, without any processes. So I thought I'd create 'jld' to
> > remove a jail. However - prison_find isn't exported to userland.
> > Probably for good reason.
>
[ snip very informative analysis ]
> The only real exception to this is the tcpcb
> -- TCP connection remenants can last for quite a time after their socket
> exits, since they follow the TCP state machine (which has long waits). So
> this could be it -- check netstat and see if there are any largely closed
> TCP sessions from the jails. FYI, this is not a complete list of
> credential references, but it accounts for most of them.
>
> Are you using any services that pass references to sockets or other file
> descriptor objects in and out of the jail using UNIX domain sockets? If
> so, that could also be it.
Mostly this happens when setting up the jails. The first one (10) with the
jail-${IP}.idg.nl hostname, is created by a script 'makejail.sh', which is
largely based upon the make steps descriped in jail(8). After the steps
listed there, the jail is fired up to execute 'ports-jail.sh' inside the
jail, which basically sets up some standard ports (portupgrade, shells, vim,
postfix). I copy over distfiles, but it is possible, that fetch(1) came into
play. After this script, all is done - the jail should exit.
The number 15 with the correct hostname, is me logging in and doing some
additional work, mostly installing and configuring ports. The use of fetch(1)
is very likely, as these ports are used inside jails only and typically don't
exist in the host system's distfiles. I've logged out, without having stuff
in the background.
I've tried to get the refcount, but get stuck on the proc structure - getting
a bus error a few lines down. I couldn't find a faster road to this - if
anybody can, I'd be happy to run some diagnostics. Probably the most
interesting thing is to see whether the prison refcount is really 0.
My attempt below, was compiled with -D_KERNEL -lkvm.
--
Melvyn
=======================================================
FreeBSD sarevok.webteckies.org 5.2-CURRENT FreeBSD 5.2-CURRENT #1: Sat Feb 14
02:48:30 CET 2004
root@sarevok.webteckies.org:/usr/obj/usr/src/sys/SAREVOK_NOAPM_NODEBUG i386
=======================================================
/* vim600: sw=4 ts=4 ai
*/
#include <sys/param.h>
#include <sys/types.h>
#include <fcntl.h>
#include <kvm.h>
#include <limits.h>
#include <sys/proc.h>
#include <sys/ucred.h>
#include <sys/jail.h>
#include <sys/sysctl.h>
#include <sys/user.h>
#include <sys/wait.h>
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
static struct prison *
prison_find(int prid)
{
struct prison *pr;
pid_t pid;
char errbuf[_POSIX2_LINE_MAX];
kvm_t *kern;
struct kinfo_proc *p_procinfo;
struct proc *p_proc;
struct ucred *p_uc;
int num_procs, i, status;
pid = fork();
pr = NULL;
switch (pid)
{
case -1 :
perror("fork()");
break;
case 0 :
setsid();
jail_attach(prid);
sleep(60);
exit(0);
default:
sleep(1); /* wait for jail_attach() */
kern = kvm_openfiles(NULL, NULL, NULL, O_RDONLY, errbuf);
if( kern == NULL )
{
fprintf(stderr, "Cannot open kernel memory: %s", errbuf);
break;
}
p_procinfo = kvm_getprocs(kern, KERN_PROC_PID, pid, &num_procs);
printf("Getting %d processes for pid %d and my pid is %d\n", num_procs,
pid, getpid());
for( i=num_procs; --i >= 0 ; ++p_procinfo )
{
p_proc = p_procinfo->ki_paddr;
if( p_proc == NULL )
{
fprintf(stderr,
"Cannot get proc struct from proinfo %d.\n", i);
break;
}
p_uc = (struct ucred *)p_proc->p_ucred;
if( p_uc == NULL )
{
fprintf(stderr,
"Cannot get ucred struct from proc (pi %d).\n",
i);
break;
}
pr = (struct prison *)p_uc->cr_prison;
if( pr != NULL )
break;
}
kvm_close(kern);
waitpid(pid, &status, WNOHANG);
break;
}
return pr;
}
int main(int argc, char *argv[])
{
struct prison *p;
int jid;
if( argc == 2 )
{
jid = (int)strtol(argv[1], NULL, 10);
if( jid > 0 )
{
p = prison_find(jid);
if( p == NULL )
{
fprintf(stderr, "Cannot find the prison!");
}
else
{
printf("Refcount is: %d\n", p->pr_ref);
}
}
else
{
printf("Invalid jid: %d", jid);
}
}
else
{
printf("No jid\n");
}
return 0;
}
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (FreeBSD)
iD8DBQBAL/OcOv9JNmfFN5URAk0lAJoCiXBcNebYhGXsBoPBc2aOuAiclwCeKWkk
IRSyFk1DVrDORoL9+73oOGI=
=Frs0
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200402152333.00565.freebsd-current>