From owner-freebsd-hackers@freebsd.org Mon Apr 29 13:25:15 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87D16158CDB2 for ; Mon, 29 Apr 2019 13:25:15 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-it1-x12f.google.com (mail-it1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 72E607785B for ; Mon, 29 Apr 2019 13:25:14 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-it1-x12f.google.com with SMTP id r85so7527702itc.2 for ; Mon, 29 Apr 2019 06:25:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=K5ww+uvGAcRKntVX3CuKClDjGquFkWxKL+ayep4zX10=; b=18l2jegDtSE6M/3b/cI37vftQhp1g1wk9v2IzVOXTKgaosGWmiuSFAbvGdThL6KVIA EKIv560wjXu/OgSXqXmtR53wM3jehutRugi/uhQVZUazZQ18FUVNCNmeroJw0LNyopZC tDS6mVrPnaZDxWHsltKW7CDfGcSRyxcd14goOGSsXsqyfVg1/+oARd6BjdpyedNgA2b+ 7V8kBfiwsaHCw5x8JZf1xCg2M39zu1hMabIfBMYquNQwMhFMC3/CX3rFTZyh1xBGvGUQ pwU/efzGr4jFpPwjwMLuUE2q3aVerLLxxGXU/YLYDrRYr+Rzzu5BtpKvVl6YEic8jGYL DMXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=K5ww+uvGAcRKntVX3CuKClDjGquFkWxKL+ayep4zX10=; b=tNHV2jTHl27Y0IXQmGpkggp41s2W6wq6kiMWtYwlioBBcP2BCxfoEAoDrgE3SoiZvl Bi+E5GbP0vclCrtV7NbzVY5AV6fFIS+boYZ/Lc6FFVS89K4RRe5IYhIfvYYhRVUjJnAY qrzVRBLu9sGojj1Psrc2revtZ3BGJDroEEpCUDoFLM/SyhcHYgSFoMzJ5p7W0pvODIxt 96kxIvw1buVrbe6X7VuckyzuGuIe1ZDyl9g+SGeP3Lj0xNNi34V/VK+AB10ekuuAdoe0 MrDg8azA10JvHpp+XVfu860hqNkm23dkcrvh9+PqNBCS7rgQ1ZfKBc8r52FcecTTNRHS cYVw== X-Gm-Message-State: APjAAAWra6TB5XaoxagDgxZn3ZeGlSvwx9YamEP784khfdwnUj8bvvub D6wVhRqYvQfwFfO5rJTckeBDfh3fs/KtbeSVU9AJxQ== X-Google-Smtp-Source: APXvYqwm14h2MgICKuqcdrdQJ/8RW3/UZp/daTTGMbYrmHnfG1Ujj5fda1MZPlsSOZzj6zfkimSVCHWyeAgGN3FbvDM= X-Received: by 2002:a24:4682:: with SMTP id j124mr18995293itb.90.1556544313611; Mon, 29 Apr 2019 06:25:13 -0700 (PDT) MIME-Version: 1.0 References: <002901d4fdfb$e52eb890$af8c29b0$@ixsystems.com> <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> In-Reply-To: <20190429141239.f9de62f6b6721112d7143cf6@bidouilliste.com> From: Kris Moore Date: Mon, 29 Apr 2019 09:25:05 -0400 Message-ID: Subject: Re: CFT: FreeBSD Package Base To: Emmanuel Vadot Cc: FreeBSD Stable , FreeBSD Current , freebsd-pkgbase@freebsd.org, freebsd-pkg@freebsd.org, freebsd-hackers@freebsd.org, freebsd-ports@freebsd.org X-Rspamd-Queue-Id: 72E607785B X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=18l2jegD; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::12f as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-5.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; RCVD_IN_DNSWL_NONE(0.00)[f.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_SEVEN(0.00)[7]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; IP_SCORE(-2.91)[ip: (-9.09), ipnet: 2607:f8b0::/32(-3.17), asn: 15169(-2.24), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.94)[-0.938,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; SUSPICIOUS_RECIPS(1.50)[] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Apr 2019 13:25:15 -0000 On Mon, Apr 29, 2019 at 8:12 AM Emmanuel Vadot wrote: > > Hi Kris, > > On Sun, 28 Apr 2019 15:52:21 -0400 > wrote: > > > FreeBSD Community, > > > > > > > > I'm pleased to announce a CFT for builds of FreeBSD 12-stable and > 13-current > > using "TrueOS-inspired" packaged base. These are stock FreeBSD images > which > > will allow users to perform all updating via the 'pkg' command directly. > > Rather than trying to answer all questions in this announcement, we've > > created a FAQ page with more details. Please refer to this page, and let > us > > know if you have additional questions that we can include on that page > going > > forward. > > > > While I appreciate the effort I have some doubt about your > "re-implementation" of pkgbase. I don't see any improvement compared to > what is in base currently, I even see downside of your implementation. > > - How do you plan with the need of updating kernel first, reboot and > updating the rest of the userland after ? (Needed for major and minor > upgrade, 12.0 to 12.1 for example, and simple update in -STABLE and > -HEAD branch). This is still a problem with the base pkgbase. > We've written our own tool "sysutils/sysup" in GO which handles this. It performs updates using Boot-Environments to ensure that kernel/world are updated at same time. > - This is even worse because you are using the same repository for > base and pkg so if a user pkg update and both kernel and pkg(8) needs > to be updated and pkg use a new syscall or capsicum thing it will be > updated first and couldn't proceed with the rest of the update (this is > a supposition, I haven't personally tested). > See above. > - It seems that multiple kernels isn't supported in your > implementation, this is already supported in pkgbase but still need > some love. This is an important point as it will allow user to choose > easily the kernel that they want to use and will also allow us > developper to push kernels with new features to help testing. > Incorrect, on the 13-CURRENT build if you install kernel-debug, you'll get the Witness-enabled kernel installed alongside non-debugging one. > - Since you reduced the granularity on the userland bits it would mean > that if we use your implementation for -p updates we would download the > whole userland packages instead of just updating the package that was > patched. For example with pkgbase, updating from 12.0 to 12.0p1 will > only update the FreeBSD-runtime package. Yes this package is still big > to download when you compare to what have changed but until pkg(8) have > delta pkg supports (and if it will have support, I don't know if > this is a wish or not) this is the best way to go. > Correct, this is by design. We used the in-tree pkg base for nearly a year, and found that the granularity didn't really offer any savings from a download or time perspective. Updating 100+ packages took far longer than a single one, due to all the meta operations. Additionally in real-world usage, we found that base packages tended to all get updated at the same time, which took far longer via pkg, since it had to go and perform 100+ fetch operations just to download the base system bits. > - I see that you are sorting the plist for kernel and userland based > on the line length [1], why is that ? Whoops! I'll fix :) > > I think that the only advantage that your solution offers is that if > we remove a componant of base (rcmds for example in 12-CURRENT) those > files would be removed as they are in the userland-base package while > for pkgbase the FreeBSD-rcmd package will be deleted in the repo and > will not be deleted in the user computer. > Correct, this is one of the things which prompted us to go this direction. Being able to handle crazy mixed WITH/WITHOUT flags was important to us, current pkg base did not handle that so gracefully. Additionally we've added some additional features, such as being able to 'pkg install os/src' to get system sources used in exact build, as well as being able to rebuild your local world / kernel packages using ports "make config" framework is super handy. > > > > > Additionally, I will be hosting a Package Base working group at BSDCan > 2019, > > and welcome user and developer attendance to discuss this and other > ongoing > > package work: > > > > > > > > https://wiki.freebsd.org/DevSummit/201905/PackageBase > > > > I will be present and looking forward to work with you on this. > > Cheers, > > P.S. : FYI I'm working on pkgbase currently and I will have some > patches to commit soon (bsdinstall support, memstick creation that > install a pkgbase aware installaton etc ...). > Great! Looking forward to discussion then! > > [1] : > > https://github.com/trueos/trueos-ports/blob/trueos-master/os/userland-base/Makefile#L35 > > -- > Emmanuel Vadot >