From owner-freebsd-questions@freebsd.org Sun Feb 16 21:32:40 2020 Return-Path: Delivered-To: freebsd-questions@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BA789243912 for ; Sun, 16 Feb 2020 21:32:40 +0000 (UTC) (envelope-from ihor@antonovs.family) Received: from mail.antonovs.family (mail.antonovs.family [100.25.240.195]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48LL1g6Dn2z4Tmw for ; Sun, 16 Feb 2020 21:32:39 +0000 (UTC) (envelope-from ihor@antonovs.family) Received: from localhost (localhost [127.0.0.1]) by mail.antonovs.family (Postfix) with ESMTP id 19E78138BA0; Sun, 16 Feb 2020 21:32:33 +0000 (UTC) Received: from mail.antonovs.family ([127.0.0.1]) by localhost (mail.antonovs.family [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id HI9r7Mn4kbx8; Sun, 16 Feb 2020 21:32:32 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.antonovs.family (Postfix) with ESMTP id 9CE27138BC6; Sun, 16 Feb 2020 21:32:32 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.antonovs.family 9CE27138BC6 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=antonovs.family; s=D65AA412-CB7F-11E9-A561-802C9D403B77; t=1581888752; bh=Sg1Z/SZjAEvdlge2yPALHUREFEsMrw0tYnix3cD+hfQ=; h=Date:From:To:Message-ID:MIME-Version; b=lV761yqCOa41jItY3Ui76Ou7c4u4nOEZ9j6CpbyVIE0Dk7T8mIIbhUiuOywWMdnfY ZXdhr405fOLq/dQ2mdAQ19tUePGZCgMHY4ywfHoMzQs2YeIyS23ELw042wX1XDIBCK gZZOv3kzDm/g0HUiw037S0ZPDOESbo6GPADgPe5WFuAwCz3+oJ+4FzGTqmS5HWHQmr ohvEwL8uvLWvM/j/NI4PZepLB0C+cUjExmgDO5hIXk5UQvrkDr3HjqkKh9Zd4sC/JH EPQzXCVEVaciU8knrTZbxDn/3NhTxNv3vkY5RhsmoxvTm1vrlMr/PfqoCHEs+Ol03R Tg1nflvqPkOnA== X-Virus-Scanned: amavisd-new at antonovs.family Received: from mail.antonovs.family ([127.0.0.1]) by localhost (mail.antonovs.family [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id DtUVFA8BxMlQ; Sun, 16 Feb 2020 21:32:32 +0000 (UTC) Received: from localhost (c-73-83-210-79.hsd1.wa.comcast.net [73.83.210.79]) by mail.antonovs.family (Postfix) with ESMTPSA id 44ECC138BA0; Sun, 16 Feb 2020 21:32:32 +0000 (UTC) Date: Sun, 16 Feb 2020 13:32:29 -0800 From: Ihor Antonov To: "@lbutlr" Cc: FreeBSD Subject: Re: Technological advantages over Linux Message-ID: <20200216213229.syxeeerzcrvekj3t@sea-ll-10936> References: <20200214121620.GA80657@admin.sibptus.ru> <20200214141600.GA82559@admin.sibptus.ru> <1eb61cba-5e28-e8ea-c418-a06f0f94ec86@kicp.uchicago.edu> <1F2DC40A-8C43-43DF-9168-661FDEC32989@kreme.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <1F2DC40A-8C43-43DF-9168-661FDEC32989@kreme.com> Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 48LL1g6Dn2z4Tmw X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=antonovs.family header.s=D65AA412-CB7F-11E9-A561-802C9D403B77 header.b=lV761yqC; dmarc=pass (policy=none) header.from=antonovs.family; spf=pass (mx1.freebsd.org: domain of ihor@antonovs.family designates 100.25.240.195 as permitted sender) smtp.mailfrom=ihor@antonovs.family X-Spamd-Result: default: False [-5.78 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[antonovs.family:s=D65AA412-CB7F-11E9-A561-802C9D403B77]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(-3.28)[ip: (-9.05), ipnet: 100.24.0.0/13(-4.30), asn: 14618(-3.01), country: US(-0.05)]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[antonovs.family:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[antonovs.family,none]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:14618, ipnet:100.24.0.0/13, country:US]; RCVD_TLS_LAST(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[79.210.83.73.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.10] X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Feb 2020 21:32:40 -0000 On 2020-02-14 13:23, @lbutlr wrote: > On 14 Feb 2020, at 09:00, Valeri Galtsev wr= ote: > > In my book docker is really a disadvantage, not advantage, compared t= o FreeBSD jails >=20 > Dicker has the advantage of convenience and ease of installing/removing= dockers, but you trade that for not only poor security, but another appl= ication layer between you and the service which itself has had numerous s= ecurity issues. =20 I've been reading this tread for a while, and now I can't help but to add my 2 cents: I am long-time Linux sysadmin/devops and I work with "docker" on a daily basis. Reading this thread I got an impression that a lot of folks on BSD side have vague/wrong/incomplete understanding of Linux containers so I want to introduce more structure into this topic. First off, "docker" is really a misnomer. Nowadays linux world has a whole bunch of container tools: moby (former docker), podman, kata containers, cri-o etc. Not all of them are equal, some of them are comple= te user ecosystems, and some are just "bare" runtimes. There was a tool named "docker" once with that name and the name really stuck, so people call things "docker" left and right.=20 Second, there is no such thing as "linux containers" per se. There are 2 kernel mechanisms: namespaces(allow isolating a process from a the rest of the system, like network namespace, user namespace, pid namespace etc) and cgroups(allow limit resource usage, like cpu, ram, bandwitdh). Combing various combinations of namespaces and cgroups you get "containers". On a low level tools like docker et al do is manipulate=20 namespaces and cgroups. The design of namespaces is really the opposite to jails. With jails you start with a completely isolated environment and then you can add different capabilites if necessary. With namespaces you start with non-isolated process (process that shares namespaces with rest of the system) and you unshare namespaces one by one. (I can't compare resource limiiting part as I am not familiar with how it is done on FreeBSD) It does not mean that namespaces are less secure than jails, it is a diff= erent design, more involved, probably harder to get righ, but also more flexible.=20 Before docker it was very hard to use namespaces and cgroups for a regular linux user. There was no one "jail" command. There were only some system calls and scattered docs.(Well there was LXC, but not the point) What docker did(and was first to do it) is provided a very convenient and pretty complete ecosystem to manage namespaces and cgroups, including features like: - scripting container creation (aka Dockerfile) and sharing it as code - sharing compiled images=20 - Dockerhub is a centralized location for sharing images( it is just glorified fileserver that hosts a lot of tar.gz + some indexing ) - sharing/re-using iamges ( FROM clasue in Dockerfile ) - nice CLI tool to manage containers and images And it hid deeply notion of namespaces and cgroups, so regular joes were able to use it without learning what kernel mechanisms make it possible. Writing a dockerfile is not very different from writing a shell script really. It helped widespread adoption of the tool, but with this also created a lot of misconceptions too. One can argue that "docker" is too bloated and is not really secure. Yes, it is partially true:=20 - it makes some choices about how namespaces and cgroups are used, maybe not the way YOU want.=20 - It is also a pretty big codebase in golang, that YOU did not audit and which is not really necessary if you want to manage things manually and customize to you needs.=20 - Yes, re-using images from the internet also introduces lots of risks.=20 - And yes, big army of regular joes who don't know how the tool works allows misuse, miscofiguration etc. But if you understand how it kerlnel works and when you understand your requirements it is becomes pretty easy to find a proper solutoin.=20 Now coming to jails. jail is pretty low level tool. It should not be compared to "docker". It can be compared to namespaces though. I think it would be more productive to compare capabilities of ecosystems= .=20 - Can you securely sandbox the process with jails or namespaces? - Can you easily script sanbox creation? - Can you share/re-use recepies or built images? - What tools provides more control and what provides more productivity insread? - etc... Where FreeBSD can improve IMHO is building ecosystem tools around jails. = IOCage and Bastile are good projects, doing the right thing. But there are still little to none ways to re-use/share images and build recepies (AFAIK BasitleBSD is working in that direction). Some might argue that=20 BSD community does not need those - could be. > I use docker for things that are not very important on machines that > are (relatively) unimportant. I would never use it on something like a > mail server or web server that has other people=E2=80=99s data on it. Yes, use bubblewrap instead - really inspired by jails, minimal, oriented for maximum security. https://github.com/containers/bubblewrap=20 ------------ Ihor Antonov