From owner-freebsd-stable@FreeBSD.ORG Wed Sep 14 12:02:20 2005 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8C2F716A41F; Wed, 14 Sep 2005 12:02:20 +0000 (GMT) (envelope-from gemini@geminix.org) Received: from gen129.n001.c02.escapebox.net (gen129.n001.c02.escapebox.net [213.73.91.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2147C43D55; Wed, 14 Sep 2005 12:02:20 +0000 (GMT) (envelope-from gemini@geminix.org) Message-ID: <43281147.6090607@geminix.org> Date: Wed, 14 Sep 2005 14:02:15 +0200 From: Uwe Doering Organization: Private UNIX Site User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.11) Gecko/20050802 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson References: <432753CF.6020001@bfoz.net> <4327CA3C.6050403@geminix.org> <20050914110102.W33820@fledge.watson.org> In-Reply-To: <20050914110102.W33820@fledge.watson.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Received: from gemini by geminix.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.52 (FreeBSD)) id 1EFVxy-0006qz-By; Wed, 14 Sep 2005 14:02:18 +0200 Cc: Brandon Fosdick , freebsd-stable@freebsd.org, Lyndon Nerenberg Subject: Re: Jail to jail network performance? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2005 12:02:20 -0000 Robert Watson wrote: > > On Wed, 14 Sep 2005, Lyndon Nerenberg wrote: > >> On Sep 13, 2005, at 11:59 PM, Uwe Doering wrote: >> >>> Now, for security reasons jails normally are confined in separate >>> filesystems, or at least in separate parts of a common one. So in >>> case of MySQL you would have to use TCP sockets to communicate >>> between jails. This socket type typically consumes more CPU because >>> of TCP's protocol overhead. However, whether you would actually >>> notice any difference in speed basically depends on how much excess >>> CPU power there is available on that server. >> >> Ignoring security (or filesystem namespace issues) I will just note >> that using named sockets for local IPC is a Good Thing. When I worked >> at Messaging Direct I taught sendmail to speak LMTP over named >> sockets, and our local delivery rate (to our IMAP server) went up by a >> factor of 10. >> >> It would be really cool if we could figure out a way to do AF_UNIX >> between jails, but I confess to not having thought about any of the >> implications ... (Maybe netgraph can help here?) > > There are several ways you can do it, but they generally fall into two > classes of activies: > > (1) Modifying the name space exclusion assumption for jails, so that the > file system name spaces overlap. One way to do this is with nullfs. > > (2) Having a daemon or tool that runs outside of the jail and brokers > communication between the jails. One example might be a daemon that > inserts a UNIX domain socket into both jails and then provides > references to shared IPC objects between the two "by request". > Another example might be a daemon or tool that responds to a request > and creates a hard link from a socket/fifo endpoint visible in one > jail to a name visible in another jail, perhaps when setting up the > jail. The former requires more infrastructure, but the latter is less > flexible. Just a kind reminder to those interested in implementing the daemon approach: Never ever create or write to an object from outside a jail that is located in a part of the filesystem that a live jail can access and modify. Otherwise you may easily fall victim to a symlink attack or similar. Remember that jails set up for security reasons generally are to be considered enemy territory. The correct approach would be to create or open such objects from a chrooted child process. There is only one exception: In the pre-boot phase of a jail you can get away with checking the file path component by component before you touch the object. But as soon as the jail runs the window between checking the path and accessing the object can be exploited from inside the jail. Hope to have helped prevent some rude awakening for some. ;-) Uwe -- Uwe Doering | EscapeBox - Managed On-Demand UNIX Servers gemini@geminix.org | http://www.escapebox.net