From owner-freebsd-pkgbase@freebsd.org Mon Apr 29 12:08:19 2019 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D703D158A9D6; Mon, 29 Apr 2019 12:08:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8FB774495; Mon, 29 Apr 2019 12:08:17 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x3TC88dU068353 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 29 Apr 2019 15:08:12 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x3TC88dU068353 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x3TC88XF068352; Mon, 29 Apr 2019 15:08:08 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 29 Apr 2019 15:08:08 +0300 From: Konstantin Belousov To: kris@ixsystems.com Cc: freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: CFT: FreeBSD Package Base Message-ID: <20190429120808.GI85201@kib.kiev.ua> References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED,URIBL_RED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 12:08:19 -0000 Cc: list trimmed to relevant. Very long essey below, be warned. On Sun, Apr 28, 2019 at 03:52:21PM -0400, kris@ixsystems.com wrote: > FreeBSD Community, > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and 13-current > using "TrueOS-inspired" packaged base. These are stock FreeBSD images which > will allow users to perform all updating via the 'pkg' command directly. > Rather than trying to answer all questions in this announcement, we've > created a FAQ page with more details. Please refer to this page, and let us > know if you have additional questions that we can include on that page going > forward. > > > > Additionally, I will be hosting a Package Base working group at BSDCan 2019, > and welcome user and developer attendance to discuss this and other ongoing > package work: > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > > > > > > FAQ > > ------------------------------------------------------------- > > https://trueos.github.io/pkgbase-docs/ > I do not know what are design decisions for trueos pkgbase are, but I do know something about in-tree split and why some packaging decisions where made. I cannot attend your WG, but I believe the reasoning used for the in-tree is important enough to represent it intact from the source. I have to start with some explanatory long text to put it into the proper perspective. There are two knots of interdependinces which are critical for correctness of any upgrade where the target system cannot be simply discarded on failure: 1. C runtime 2. Minimal boot path to prompt. Let me elaborate both, starting from point 1, which is typically very obscure despite having the fundamental nature for anything related to upgrades. The basic execution environment for any program executed by the FreeBSD kernel is formed by combination of kernel' syscall interface and some system userspace code which makes the expected environment over the bare-bone image state after execve. The environment is typically named C runtime environment since C language ABI is directly tied into it, and normal C programs only get whatever is provided by the C runtime unless additional libraries are linked in. Trully, it is not just C runtime, any other execution environment on top of the OS is based on this one, but since almost every 'advanced' language runtime is backed by C language and its runtime, the name stuck. FreeBSD C runtime, arguably, is provided by the following four objects: /libexec/ld-elf.so.1 /lib/libc.so.7 /lib/libthr.so.3 /lib/libm.so.5 There, we do *guarantee* that the external ABI of the whole pack of these four objects is backward compatible, i.e. if the binary was compiled against set if base libraries at earlier date (may be also on earlier branch), then the binary behaviour would be same when executed on newer C runtime pack. This is not trivial to achieve, besides technical measures that helps there, like backward-compatible syscall interface, symbol versioning, providing fall-back code for older interface, a lot of overhead in the development is enforced, like carefull reviews of the changes, the policy and related discipline of versioning, following published ABI standards, and so on. But, internal ABI of the C runtime pack, i.e. interfaces which make rtld work with libc and libthr, or way by which libthr, when loaded, makes libc thread-aware, are not stable, and more, they are often changed in backward-incompatible way. Requiring backward-compatibility there would stop our ability to evolve the system. Answering some questions in advance, yes, rtld delves into libc, libthr patches libc on load, libc has hooks to control some libthr behaviour. The only provision that we make is that ld-elf.so.1 is required to work with older libc/libthr combination, but even then libc and libthr must be built from the same sources with the same options set. Now, returning to pkgbase, if you look at what libs are packed into clibs, you see: ld-elf.so.1 libc.so.7 (and modules like iconv tables or nss, if any) libthr.so.3 libdl.so.1 libgcc{, _eh, _s}.so.1 libm.so.5 libedit.so.7 libncurses{, w}.so.8 libc++.so.1 It adds very popular libs like libncurses/libedit, and C++ runtime. The basic reasoning is that this package is small and chances of something going wrong while installing it are small as result. Put it other way, the small clibs package organization makes it highly probable that system is left in the consistent state (either all new libs, or all old libs) after the upgrade, whetever the outcome is. If the C runtime pack is not split from the whole 700MB+ update blob, libthr update has almost certain chance to occur long after or before libc update, so failures do tend to leave inconsistent rtld/libc/libthr set. At best, it gives you strange glitches, at worst you get unusable system that cannot be repaired without external media. Now, the second item, the minimal boot path. By definition, it consists of everything that is required to get bare-bone shell prompt in single user mode, and where user can repair failed upgrade. Arguably, it should also include the tools to configure the network and fix filesystems. So it should consists of loader (including forth/lua scripts) kernel C runtime /sbin/init /bin/sh newfs/fsck/tunefs for UFS zfs/zfspool and libs for ZFS ifconfig/route/ping In this set, zfs and network management tools must be synced with the kernel, since ABI of the management syscalls is not guaranteed to be stable even on stable branches. The above brain dump is at least partial enumeration of things that were discussed between me and Glen when Glen created the current in-tree packaging code.