From owner-freebsd-jail@FreeBSD.ORG Sun Jan 25 19:41:28 2015 Return-Path: Delivered-To: jail@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 09C545BD for ; Sun, 25 Jan 2015 19:41:28 +0000 (UTC) Received: from mail-qg0-f44.google.com (na3sys010aog110.obsmtp.com [74.125.245.88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 91E23E88 for ; Sun, 25 Jan 2015 19:41:26 +0000 (UTC) Received: from mail-qg0-f44.google.com ([209.85.192.44]) (using TLSv1) by na3sys010aob110.postini.com ([74.125.244.12]) with SMTP ID DSNKVMVG5STVBcoWo8r0glshHx0n+P8F79l1@postini.com; Sun, 25 Jan 2015 11:41:27 PST Received: by mail-qg0-f44.google.com with SMTP id l89so4408443qgf.3 for ; Sun, 25 Jan 2015 11:41:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=groupon.com; s=google; h=mime-version:date:message-id:subject:from:to:cc:content-type; bh=PWEgDisvQXpm1IU70kWuMJ84qtDXN36pnliaIHXNrYI=; b=jgBEhoKQllm1yAAS8lAm0+Mdxm28R+DQEtTqLnWZyGTCd3CX/EykCjibNh1zjEfM6H L29FErNGIqVAubrH/JNfzFOJtfStayEORop/b4vW6BOtpAAkUGA6h/NqLcdDv4CQSrvS JFiwV9bh8usy3jm0jQ3nc5Qw7NMiI/1GL62ZA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to:cc :content-type; bh=PWEgDisvQXpm1IU70kWuMJ84qtDXN36pnliaIHXNrYI=; b=QvGdc/kcDtY12sTRCPctjqPqJI2mSzkcS2TFDBa55N+v2aHn20XQHH7MJ24tv0AuoH km8Z9t5q3YHEGS/c/zLSwIWtSeo5bFS9EtQ9+HOIdJlhENnlSiL5vGr5fAmn6+6Cm+2o EVjysZOu+M+BKZA4zurOECNSMqtmw2334z2a5Z61ZSZ7FN2MUPtEMBwq/+Z5/0pB2ehr ENyj3QDL+qi0KeYputxTvKtDF67b2KTHQiZsrdCcAkofNdvj+TE1aD8fdYNqYhkcBVfu 9Tb8UrRnp4YijYKzGq6/6yv+CKBrS0y4DYzj3wuqH95mKm4GqM3eHxGyDQnIDbB5uUaa 41+g== X-Gm-Message-State: ALoCoQmbolajIjJd1DXA4YdOREh6246bgJkUiLGh1Sr0e8Fa63tbj6mQH84I8QbSz/vTPYY0c8oDhw8svpQxTisYEv3eKkTSKPDrGPiWUbYcLkf2kAE0n/TyqSaAhIQDc7jqkTVjL0+xZxPgr9goa/DUOzbRqqRidA== X-Received: by 10.224.136.135 with SMTP id r7mr17250299qat.102.1422214885414; Sun, 25 Jan 2015 11:41:25 -0800 (PST) MIME-Version: 1.0 X-Received: by 10.224.136.135 with SMTP id r7mr17250282qat.102.1422214885208; Sun, 25 Jan 2015 11:41:25 -0800 (PST) Received: by 10.96.150.3 with HTTP; Sun, 25 Jan 2015 11:41:24 -0800 (PST) Date: Sun, 25 Jan 2015 11:41:24 -0800 Message-ID: Subject: Re: preferred jail management tool From: Sean Chittenden To: jail@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "Michael W. Lucas" X-BeenThere: freebsd-jail@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Discussion about FreeBSD jail\(8\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Jan 2015 19:41:28 -0000 Well this is a rather trendy topic of late and timely. I'm very happy to see a renaissance and renewed interest in container administration for FreeBSD. Last week at work we began an evaluation of the tooling and administration of FreeBSD containers. Despite being depreciated, we're still evaluating ezjail along with bsdploy, qjail, and manually creating jails (via ansible). Ideally we're looking for something with administrative parallels between bhyve and jails, and easy to integrate in to tooling. We're settling on a technology by Wednesday this week. For years I've used and endorsed ezjail, but as stated, it is depreciated. For a book, excluding ezjail would exclude a huge portion of the user base and seems like it would hurt credibility given its dominance as the preferred tool for jail administration. Until yesterday, I'd never seen iocage but in reviewing the implementation, I really like its use of ZFS attributes as the method of persisting jail attributes and properties. This provides a really clean encapsulation mechanism that works well with `zfs send`. "Thick" containers are not opaque at rest or at runtime, are easy to reason about for new administrators on the team (not layered via nullfs at runtime, space is cheap), and the configuration file is included in the dataset itself. Administratively iocage looks simple to use and it fits in well with our configuration tooling (Ansible). I think we will write an iocage ansible module to query and set attributes, at which point iocage will be very clean for our tooling. iocage is built on top of the OS primitives and utilities, was written in shell, and looks very clean in the code's structure. Applying changes to running jails without a restart is also nice. The "feel" of the interface, control, and abstraction provided by iocage sets it apart in my mind. The examples for future administrators is also important and lend itself well to HOWTO-like guides, which adds to the pragmatism of the utility. Again, because it's a single shell script calling OS primitives, it makes it easy to version internally and provide stability guarantees going forward. Support for vnet is nice but not something we're planning on using (instead we're going to advertise container IPs via BGP to TORs). Based on some of the reasoning above and provided there aren't any unaddressable concerns by the rest of the team, I expect we will adopt iocage. My quick $0.02. -sc -- Sean Chittenden