From owner-freebsd-virtualization@freebsd.org Wed Jan 30 20:36:50 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62D4513196F3 for ; Wed, 30 Jan 2019 20:36:50 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [88.98.225.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E11EB856DB for ; Wed, 30 Jan 2019 20:36:48 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from crayon2.yoonka.com (crayon2.yoonka.com [10.70.7.20]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id x0UKae2r027016 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Wed, 30 Jan 2019 20:36:41 GMT (envelope-from list1@gjunka.com) Subject: Re: The status of docker To: Stefan Bethke Cc: freebsd-virtualization@freebsd.org References: <089e330d-2761-2440-3b7f-dd22e9088af5@gjunka.com> <929CF558-E8CC-4F7D-9C30-DDD63C17861D@lassitu.de> From: Grzegorz Junka Message-ID: Date: Wed, 30 Jan 2019 20:36:40 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.3.2 MIME-Version: 1.0 In-Reply-To: <929CF558-E8CC-4F7D-9C30-DDD63C17861D@lassitu.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-GB-large X-Rspamd-Queue-Id: E11EB856DB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of list1@gjunka.com designates 88.98.225.149 as permitted sender) smtp.mailfrom=list1@gjunka.com X-Spamd-Result: default: False [-6.89 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:88.98.225.149]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[gjunka.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[gjunka.com]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.94)[-0.943,0]; IP_SCORE(-3.64)[ip: (-9.53), ipnet: 88.98.192.0/18(-4.77), asn: 56478(-3.81), country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:56478, ipnet:88.98.192.0/18, country:GB]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2019 20:36:50 -0000 On 27/01/2019 13:59, Stefan Bethke wrote: > Am 19.01.2019 um 15:24 schrieb Grzegorz Junka : >> Hello, does anyone know the current status of docker on FreeBSD? Wiki https://wiki.freebsd.org/Docker states it's experimental. The last commit in https://github.com/kvasdopil/docker/tree/freebsd-compat is also from 2015. >> >> There in fact are two ports, freebsd-docker (from 2015) and docker (18.06). What's the difference between them and which one should I use to run docker images on FreeBSD host? > I believe at this point in time, all you can do is have a Linux machine somewhere (for example, in a local Bhyve VM), and use the Docker command line client from FreeBSD to manage it. That’s what sysutils/docker is for. > >> Has this project been completed and now only needs testing, or has it been abandoned, or maybe the approach has changed and I am looking in a wrong place? > It looks abandoned, unfortunately. And looking at the entire container infrastructure, reviving it would require a significant effort. For example, it most real-life usage scenarios, you want to be able to manipulate IPFW/PF entries to forward ports from a host interface into a container, which would require special plugins for Docker, or a compatibility shim that allows Docker to use iptables APIs/ABIs. > > Going even further, pretty much everything in the ecosystem (k8s, etc.) assumes it's running on top of a Linux kernel including a number of management APIs that are not (completely) emulated on FreeBSD. > > While I would love to see proper Docker support in FreeBSD, I’m not sure its even the right thing to aim for. In Docker production environments, people generally try to pare down the host OS to the absolute minimum, and at that point, what benefit would you derive from FreeBSD as a host? > > Similarly, why would you want to run FreeBSD-ABI containers, specifically? One of the benefits of the container ecosystem is that there are many ready-made images you can build on. Having to re-invent all of this seems of little benefit to me. There is nothing docker-like in FreeBSD. Sure, we have jails and bhyve but they are faaar from the convenience and support of docker containers that you can pull directly from docker hub and have a complete application running in a few lines of code. I am working on a project which uses two containers to run the back end - one is a customized container running Stellar + a DB and one running an off-the-shelf container with PostgreSQL. The middle layer is nodejs+GraphQL and front-end is React. It takes literally 30 minutes on a MacBook Pro from the moment you git clone sources from github to when you have the complete development environment running. However, I spent a day trying to make it run in FreeBSD bhyve. Half of that time took preparing the FreeBSD host and installing and configuring a Linux host for containers in bhyve. But once everything was installed, nothing actually worked because the docker containers couldn't communicate with each other for some reason. I suspected some network issues within the Linux host and spent the other half of that time trying to sort it out. Now if you consider that the cost of a MacBook Pro is a few days worth of work, why bother spending time (and money) trying develop the code on a FreeBSD? And if an AWS VM or other hardware is able to run the same already configured docker environment from a Mac, why bother spending time (and money) trying to run it on a FreeBSD server instead of a Linux server? Currently the only reasons when someone might actually consider running FreeBSD server are: ZFS and/or jails (but for that you may also need to add IlluminOS and SmartOS to the list of choices) or when you already have teams/systems running FreeBSD. Ports are not a reason for choosing FreeBSD because Arch Linux also supports a rolling release model. In many ways FreeBSD reminds me Amiga - I tried to use it for development, not just as a hobby at home, but also during my studies or work, and I was very proud when I could achieve, often with great effort, what others on IBM PC (and now Linux) had for granted. But at some point the effort of going against the flow was becoming too expensive, not only in terms of money or time, but also in more difficult to measure terms of missing opportunities. I realize that running docker natively on FreeBSD is difficult and may seem pointless. But for me it's a matter of using FreeBSD or not. If I can make it working, maybe with some reasonable effort, that would be great and I would try. But if not, well, then it's hasta la vista, baby (for this project for now). GrzegorzJ