From owner-freebsd-hackers@FreeBSD.ORG Fri Nov 17 03:20:31 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21C6F16A416 for ; Fri, 17 Nov 2006 03:20:31 +0000 (UTC) (envelope-from erdgeist@erdgeist.org) Received: from elektropost.org (elektropost.org [80.237.196.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7375243D49 for ; Fri, 17 Nov 2006 03:20:28 +0000 (GMT) (envelope-from erdgeist@erdgeist.org) Received: (qmail 29605 invoked by uid 0); 17 Nov 2006 03:20:18 -0000 Received: from fnord.cryptophone.de (HELO ?10.1.1.71?) (erdgeist@erdgeist.org@62.220.7.20) by elektropost.org with AES256-SHA encrypted SMTP; 17 Nov 2006 03:20:18 -0000 Message-ID: <455D2A7A.90109@erdgeist.org> Date: Fri, 17 Nov 2006 04:20:26 +0100 From: Dirk Engling User-Agent: Thunderbird 1.5.0.8 (Macintosh/20061025) MIME-Version: 1.0 To: freebsd-hackers@freebsd.org X-Enigmail-Version: 0.94.1.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Phantom Jails X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2006 03:20:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Rumors went around and tales were told about jails magically booing around in prison list, even after they deceased. Most people consider this a rather aesthetical issue, however if you run your jails from directories that need to be unmounted (e.g. from md-images, on external drives, from gbde or geli images) those phantom jails become rather annoying, since you cannot umount their roots. Investigations have shown, that 1) sockets hold a lock on (increase reference counter in) the ucred structure of the calling process 2) This ucred structure in turn keeps a lock on (increases reference counter in) the prison struct representing the jail this process belongs to 3) The prison struct in turn keeps a handle to jails root directory. If a process holding a tcp connection is killed, the connection is being inherited by the kernel. It waits there for tcp tear down or tcp time out to occur. Only then socket's lock on ucred is released, which releases ucreds lock on prison struct (thus terminating phantom jails) which may, if it is the last ucred referencing the prison, release the prison and its handle to the root directory (solving my un-umount-able images). There were kinds of phantom jails being sighted, that did not vanish after tcp timeout, that might be deadlocked by open files or mmaped regions. However the above case happens regularly with my mail server jail that holds hundreds of imap-connections, one disconnected dsl-user can prevent tcp tear down to happen successfully thus forcing me to force umount the mail server. My suggestion would be (I will provide a patch, if discussion produces no major disagreement) to release ucred structs held by sockets as soon as the process dies. They are being used for accounting purposes only, anyway. The same may apply to the other types of phantom jails, as well. I could not create those deliberately and therefore can not exactly spot the proper location to fix. Comments? erdgeist P.S.: if you want to reproduce a phantom jail try the following: 1) create and start a jail 2) Start a ssh/web/whatever server within the jail 3) Connect to that server from the host system. 4) Keep this connection open while you kill the jail 5) Do a 'jls' and compare its output to "ps axuu | grep J" 6) Kill the process that connected to the service. 7) Do a 'jls' again. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.3 (Darwin) iD8DBQFFXSp5ImmQdUyYEgkRAtOAAJ4iSzyu2LOf+RBNArvYAk1Tv8cssACfRxJa 12OGEwWugcIDhlGGTHJrz0o= =gXK8 -----END PGP SIGNATURE-----