From owner-freebsd-pkgbase@freebsd.org Wed Jan 27 22:33:25 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93047A6FA76; Wed, 27 Jan 2016 22:33:25 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 874F71AB5; Wed, 27 Jan 2016 22:33:25 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id DD1DC1562; Wed, 27 Jan 2016 22:33:24 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Wed, 27 Jan 2016 22:33:23 +0000 From: Glen Barber To: freebsd-pkgbase@FreeBSD.org Cc: freebsd-current@FreeBSD.org Subject: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160127223323.GG98557@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="5CUMAwwhRxlRszMD" Content-Disposition: inline X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2016 22:33:25 -0000 --5CUMAwwhRxlRszMD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline As many know, work has been in progress for quite some time to provide the ability to package and upgrade the FreeBSD base system using pkg(8). The majority of the initial implementation has provided much of the core functionality to make this possible, however much work still needs to be done. Over the past few weeks, there have been several inquiries on if this work is still targeted for the 11.0-RELEASE, as well as the status of the project branch (base/projects/release-pkg). The answer to the first question is: Yes. This is still targeted for 11.0-RELEASE, which was one of the requirements during discussion of the new support model announced early last year [1]. The status of the in progress work is a bit more complex to answer in a short email, but work on packaging the FreeBSD base system is indeed ongoing, and has been my primary focus over the past several weeks. I am finishing an initial list of outstanding items that need to be resolved before the project branch can feasibly merged back to head, which I will send to the new freebsd-pkgbase@ mailing list. People interested in discussion surrounding this topic are urged to subscribe: https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase Finally, I want to personally thank Baptiste Daroussin for all of his tireless efforts to get us to the point we are at now. Without his ideas and insights, as well as ensuring pkg(8) contained the necessary functionality, we would not be anywhere close to completing this work for the 11.0-RELEASE. [1] https://lists.freebsd.org/pipermail/freebsd-announce/2015-February/001624.html Thanks. Glen --5CUMAwwhRxlRszMD Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWqUWzAAoJEAMUWKVHj+KTF4IP/28/Racnu0THqf3d5SSCxjqS rODeF1EpM2LSTBYn1G9u4CncOpubp4GiBq4zp/VhON/3NFMdfHLFGJ/ZdJ3F9K+v hart/ydihGy+Hw3VVjWmzFd35iokUXKoTKgNtZWkunXGBO3Il5w3y+Pn0FtMZNcw rcMoxWQfIHWd0/KXyuSpvqrEeomLz51c9DxO3OQh5z8kkntLlSAQAZIvsMyVHpJh kgs8fwS/2pO+n5GvNKukFnZKVTxURxMQF/e8BmJ8lqUEB6tSrxTiTqqubWH4hYBK x4MPiHuIDtkGqw7rVQyr9cH75lbCFVmvHq6nwCMFNOHAY1iB6P9MVd2dBdxmwR+e IJfaXbU6XbMve3YMPucXhLoYBlb2SrodPyEbnU65VXweQg93EEQnIZmLdXMeeUYl anzYkP39NL0dzRTrrfTGxipAaqGHXQFC1QocN+v9aOE/Mdcgqa66EAFTLw+Sahov viDNs4bqsI4K1kcj95W3KK/l1Xomz6ySKzsvWf52Fc7QrMV8yzRukKQQLVi+z2/4 tPh+40rHp9pxPlERxMATV1Sy/Ya7gH2ceTiYiAzuogjWDcfNvzP518cIOu6/yR9V cIUtLbwGbuT6ogzWwR38cxipTP4mkcX3FSnfVUG10UJtQAXrDZEQavmxYl+dSk9c NAG//EtgP8gaHJNxnw1d =xWDr -----END PGP SIGNATURE----- --5CUMAwwhRxlRszMD-- From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 12:55:34 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2C64A71F4F for ; Thu, 28 Jan 2016 12:55:34 +0000 (UTC) (envelope-from mueller6724@bellsouth.net) Received: from nm12-vm2.access.bullet.mail.bf1.yahoo.com (nm12-vm2.access.bullet.mail.bf1.yahoo.com [216.109.114.241]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8DFAE10B8 for ; Thu, 28 Jan 2016 12:55:31 +0000 (UTC) (envelope-from mueller6724@bellsouth.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bellsouth.net; s=s2048; t=1453985279; bh=o+aK6PVmuDuiSY4CVsxvm9XWT9HpRY+ZmPLEuvXmBpk=; h=Date:From:To:CC:Subject:References:From:Subject; b=ucH12+dmxk0p5t4ZbpXV4TMsQiEXslm5V6objQ/FUjRNzk17CTH1TQJITEN0SyIlALkXt1l5962Uv/mT2rumw/n5vYztv9p/OSz+5I4mdRJWCpr3MsNUUcscPj5fQEmItEmoN+9ef2Z4qRCWjfTsd/dlWZbY6XD95yBliS/d09aq+v1i/LGqs6X5f4BjyYCYjyH+4ndBIGKKERomf55Bx/IyhXy0CRavAa5j80W+MF+N8cOLp7k1Vv6vqqlT+2HzQq4oq8SQhL36w1eVKUUwA1+EG2zIg/C7SbV2Rknxbpt/KaFYL4/5dfNyxHL8e8OF0YPTtNToWsYn7HrOyb5Hlw== Received: from [66.196.81.164] by nm12.access.bullet.mail.bf1.yahoo.com with NNFMP; 28 Jan 2016 12:47:59 -0000 Received: from [98.138.226.240] by tm10.access.bullet.mail.bf1.yahoo.com with NNFMP; 28 Jan 2016 12:47:59 -0000 Received: from [127.0.0.1] by smtp111.sbc.mail.ne1.yahoo.com with NNFMP; 28 Jan 2016 12:47:59 -0000 X-Yahoo-Newman-Id: 183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com Message-ID: <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: HGYhNOoVM1mM0NQtlXRzpPlnUD6N4qV4Exd0m4o0xmLxn2F 5TLGktc_83HOidf6uU_Bq7D3JhLbT8TlAAI7qqPJtVGPB0W8o0VKUw.fedFr MZFykLO9KHn_S8iStSLhQph58sBSD2ljmjMUjzYXHHL1twDi.tS878P66tjt i8kPQvF6BIKxyfObvUuLdo5VYw3EQca7POh4FRmyOvgTRT1zXwZBBmfr.tF1 XBnqTzLRFKg67hHJicfZC3fM7CBedp8pXTOsJfpjOoqe2foGPBm4hl2UNarf 74JPpq7uTsxQIzAmEmIiLh2GhChlI4bRdC8wNK2NbpO9.PW84oRjgwTrm0kJ 95_Ewq5rSbil3kO5w4CuFWDtqxwEGjwQQP4VRkICtc1AMb9eA8NZ_D..nzxS ZM2LFtRuvRkhRof9w.H21QnEclflGL3Z47u4RXpCnmxN6iqOs9jBjDVrOy1j b1fN2fd5aeJFRIxTsPP9w6QCgj4ItUjCL44mSJ7UBhrobqM783HfozwqRbpo Fz8jByx_ENjeT6mPKlUxKPVLPker0h7SaZslCQ1f91ajy6Hnh1qS25kw- X-Yahoo-SMTP: Kz_aW1.swBBYof3zAD7.RWzXz9ZAQVDMml1VADsbgPT4Kq79LC0- Date: Thu, 28 Jan 2016 12:46:39 +0000 From: "Thomas Mueller" To: freebsd-current@freebsd.org CC: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) References: <20160127223323.GG98557@FreeBSD.org> X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 12:55:34 -0000 from Glen Barber: > As many know, work has been in progress for quite some time to provide > the ability to package and upgrade the FreeBSD base system using pkg(8). > The majority of the initial implementation has provided much of the core > functionality to make this possible, however much work still needs to be > done. (snip) Would the base system all be one package? In Linux, everything is part of a package, even the kernel, but something comparable to FreeBSD or NetBSD base system would have many packages. Will it be possible to upgrade base system with portmaster or portupgrade, and would that be better than the current procedure in UPDATING? Would pkg then be able to show a package's required shared libraries including shared libraries from the base system? I was recently stung by pkg not showing required shared libraries from the base system. I just subscribed to freebsd-pkgbase@freebsd.org . Tom From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 13:18:10 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6F1BA70784; Thu, 28 Jan 2016 13:18:10 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E04A1CA9; Thu, 28 Jan 2016 13:18:10 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id l66so24960589wml.0; Thu, 28 Jan 2016 05:18:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=s7D9zP+x8fz/dk5f1thDnLyVWmy74hzFy5EphGHocBk=; b=irVm1jZ/GeGgFEU83SUjYBklD2w1UyrNEGZloElkGl/+1anj9e0xqO9/M8eS/0B7qV /SBGIQ+2XjVZxB/B6EET0TV588rZUnmxzaM7s6wWcU28aj4Xxa8oEloliUXV9CxWlHcL r/8n1u3I1zrfDIRQKXT5X3o4ztMbZend9okTACWfGs0i+32svNoK7sv8RoXswryXLzdn PO18Yiq4mlk51zty3ByZ0h2ewL/OItKqDJ0splnDLLGIHZqHBf7JKIaoAJka5/XAbhjX EZ36s5+VCqc8Ri/70CNq6GwaESySaDCiDrov+mdqvse84yxEdT7S2JoTBDlIH3/HH1kI j6NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=s7D9zP+x8fz/dk5f1thDnLyVWmy74hzFy5EphGHocBk=; b=AdzsyQ3XceLAV+W5hi9gn0YWYMS8sEovPacOgrHGrtkcuPXw2u5xSV5avOnjyFQF+X TGeqgN3va9bKNKgi7uMByAkF3nIyShSVL23l15ol1aETtgSUGVkP8ltUE820cTaLMSoX Um23C0smPxqyhDHl8/Rthm9o+YwcNXAS/Zv4layo8TMpUGQtJ1J1ZenxIONBmgtOux6a 2oSDYfvTix1LCZN9Rzh3umlcu0Y2+30kynlAdTZQ2nwRjHrlXi/lUktbUflpzPokKcrR hPJyRZIQPYAeqbUDd36ittiRE0SXHmLcRB6XFsJjTPLYtSB4nhvNfBcbZvB4JARBfxlL ElUQ== X-Gm-Message-State: AG10YOQwwhUHGh9J5YKB6UyWeSsnap6sMLGRgMEvsKRVbtI1HdwKmd0SRR9CWuXbuWsGkw== X-Received: by 10.28.177.85 with SMTP id a82mr3000062wmf.57.1453987088908; Thu, 28 Jan 2016 05:18:08 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id e194sm2751659wma.19.2016.01.28.05.18.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Jan 2016 05:18:08 -0800 (PST) Sender: Baptiste Daroussin Date: Thu, 28 Jan 2016 14:18:06 +0100 From: Baptiste Daroussin To: Thomas Mueller Cc: freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128131806.GB67200@ivaldir.etoilebsd.net> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="61jdw2sOBCFtR2d/" Content-Disposition: inline In-Reply-To: <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 13:18:10 -0000 --61jdw2sOBCFtR2d/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 28, 2016 at 12:46:39PM +0000, Thomas Mueller wrote: > from Glen Barber: >=20 > > As many know, work has been in progress for quite some time to provide > > the ability to package and upgrade the FreeBSD base system using pkg(8). > > The majority of the initial implementation has provided much of the core > > functionality to make this possible, however much work still needs to be > > done. >=20 > (snip) >=20 > Would the base system all be one package? multiple packages with meta packages to represent the whole base so you hav= e the best of both world :) >=20 > In Linux, everything is part of a package, even the kernel, but something= comparable to FreeBSD or NetBSD base system would have many packages. >=20 > Will it be possible to upgrade base system with portmaster or portupgrade= , and would that be better than the current procedure in UPDATING? No but one will be able to simply run pkg upgrade (and built himself the packages) >=20 > Would pkg then be able to show a package's required shared libraries incl= uding shared libraries from the base system? I was recently stung by pkg n= ot showing required shared libraries from the base system. Yes, but but real usage of it would happen in a second step because of many rought edges to be deal with. but yes the information would be available see: https://www.youtube.com/watch?v=3DBr6izhH5P1I and https://www.youtube.com/watch?v=3Dv7px6ktoDAI for a bigger view of what happened (note that some detail my have change a = bit, the overall remains the same) Best regards, Bapt >=20 > I just subscribed to freebsd-pkgbase@freebsd.org . >=20 > Tom >=20 > _______________________________________________ > freebsd-pkgbase@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase > To unsubscribe, send any mail to "freebsd-pkgbase-unsubscribe@freebsd.org" --61jdw2sOBCFtR2d/ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWqhUNAAoJEGOJi9zxtz5aMC8P/0SG5rBDifKrPfjLoZLk397e Q36vApnvh8+6VR7MaWckIUyBjFb02p8/UjvgbPxC47s+lpAtSTL51PB6BCm3ZmZA 3+vFGzJkTZtaO3r+ahAPUNFfyaMqutzoxDJzCKiMfIrI3twrhzUmFzVCEvZyz3jm 9pSaijsUeyXfouX8dv/eZPwWiVW9jElMI48DrMIDfZ3PmD5ZjlbnZSVoH5uvAUmQ nKlUzaW/MWgmykmf4gA/+1MVfGAPjzFKn6brbIi/2Cq0gbXp55uiksrmtKG1pdd8 yUiRCt4y+MrQoimjHhrjrEEWhhqosnx1j2Mtek1GyRlMLIva703er2tRmFY2vxpH HgeISezllA5SpYdZ+/vZkbTGqFmIubJDhEMXFEgRnDB5JwXEZrt7KwQdeFSv1Pkz SzuzaPkAKqqFkJ4MM4TpFLZ/ZTOgvpQ0E+hyjvx1pHRcUHGgrxDOoLuYXSZPx81F b3YkcH27mcN3PqxQ0xGlEqpLR/K9ffvHMCTQOWwW1nmYESukFrCtTm8qpgBopSms vyCzzlzcljA1p5d2MHjRdFvc/I9EqGedMWR1S7LSAQEPce34a9PeRNsxvvNJNxxT Crzdy8XsqERy5uv1XHWkTZO/k275BWodMOwMPXx2TZZtML1l04KxXfNhLo4LtxeZ 1YTT49c6+RqGxwqonT6H =ddhB -----END PGP SIGNATURE----- --61jdw2sOBCFtR2d/-- From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 16:06:33 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 042C3A70926; Thu, 28 Jan 2016 16:06:33 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B896F1B04; Thu, 28 Jan 2016 16:06:32 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOp5S-000Ck5-DT; Thu, 28 Jan 2016 19:06:22 +0300 Date: Thu, 28 Jan 2016 19:06:22 +0300 From: Slawa Olhovchenkov To: Baptiste Daroussin Cc: Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128160622.GB88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160128131806.GB67200@ivaldir.etoilebsd.net> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 16:06:33 -0000 On Thu, Jan 28, 2016 at 02:18:06PM +0100, Baptiste Daroussin wrote: > On Thu, Jan 28, 2016 at 12:46:39PM +0000, Thomas Mueller wrote: > > from Glen Barber: > > > > > As many know, work has been in progress for quite some time to provide > > > the ability to package and upgrade the FreeBSD base system using pkg(8). > > > The majority of the initial implementation has provided much of the core > > > functionality to make this possible, however much work still needs to be > > > done. > > > > (snip) > > > > Would the base system all be one package? > > multiple packages with meta packages to represent the whole base so you have the > best of both world :) > > > > In Linux, everything is part of a package, even the kernel, but something comparable to FreeBSD or NetBSD base system would have many packages. > > > > Will it be possible to upgrade base system with portmaster or portupgrade, and would that be better than the current procedure in UPDATING? > > No but one will be able to simply run pkg upgrade (and built himself the > packages) > > > > Would pkg then be able to show a package's required shared libraries including shared libraries from the base system? I was recently stung by pkg not showing required shared libraries from the base system. > > Yes, but but real usage of it would happen in a second step because of many > rought edges to be deal with. but yes the information would be available > > see: > https://www.youtube.com/watch?v=Br6izhH5P1I > and > https://www.youtube.com/watch?v=v7px6ktoDAI > > for a bigger view of what happened (note that some detail my have change a bit, > the overall remains the same) What about upgrade strongly outdated system? For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, pkg from 11.0 don't undertund package base from 18.0 and etc. From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:23:17 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98347A70893; Thu, 28 Jan 2016 17:23:17 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 605AF1A30; Thu, 28 Jan 2016 17:23:14 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8001:cee1:6400:c953:ac01:a8ff]) by elvis.mu.org (Postfix) with ESMTPSA id 6918D345A92E; Thu, 28 Jan 2016 09:23:08 -0800 (PST) Subject: Re: Packaging the FreeBSD base system with pkg(8) To: Glen Barber , freebsd-pkgbase@FreeBSD.org References: <20160127223323.GG98557@FreeBSD.org> Cc: freebsd-current@FreeBSD.org From: Alfred Perlstein Organization: FreeBSD Message-ID: <56AA4E7C.3070408@freebsd.org> Date: Thu, 28 Jan 2016 09:23:08 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <20160127223323.GG98557@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:23:17 -0000 This is going to be huge for FreeBSD. Thank you Glen, Bapt and I believe Peter Wemm as well. Having been engineering lead on multiple appliances based on FreeBSD this is going to revolutionize and make life so much easier for future appliance endeavors and general manageability of FreeBSD. Thanks again! On 1/27/16 2:33 PM, Glen Barber wrote: > As many know, work has been in progress for quite some time to provide > the ability to package and upgrade the FreeBSD base system using pkg(8). > The majority of the initial implementation has provided much of the core > functionality to make this possible, however much work still needs to be > done. > > Over the past few weeks, there have been several inquiries on if this > work is still targeted for the 11.0-RELEASE, as well as the status of > the project branch (base/projects/release-pkg). > > The answer to the first question is: Yes. This is still targeted for > 11.0-RELEASE, which was one of the requirements during discussion of the > new support model announced early last year [1]. > > The status of the in progress work is a bit more complex to answer in > a short email, but work on packaging the FreeBSD base system is indeed > ongoing, and has been my primary focus over the past several weeks. > > I am finishing an initial list of outstanding items that need to be > resolved before the project branch can feasibly merged back to head, > which I will send to the new freebsd-pkgbase@ mailing list. People > interested in discussion surrounding this topic are urged to subscribe: > > https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase > > Finally, I want to personally thank Baptiste Daroussin for all of his > tireless efforts to get us to the point we are at now. Without his > ideas and insights, as well as ensuring pkg(8) contained the necessary > functionality, we would not be anywhere close to completing this work > for the 11.0-RELEASE. > > [1] > https://lists.freebsd.org/pipermail/freebsd-announce/2015-February/001624.html > > Thanks. > > Glen > From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:28:35 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E96E6A70B1C; Thu, 28 Jan 2016 17:28:35 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pa0-x22e.google.com (mail-pa0-x22e.google.com [IPv6:2607:f8b0:400e:c03::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAFD31E91; Thu, 28 Jan 2016 17:28:35 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pa0-x22e.google.com with SMTP id uo6so26772823pac.1; Thu, 28 Jan 2016 09:28:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=yQpMa7oONmmsb8OFgjGg2Ymu70h5oHAEfnqSpiPqIhs=; b=cMNwLdSuQ1pYE9eJdvW37gPF447pL/a9lw/eJIEYe8jFPp9i+y4oqLCRDDkK84kjTe IxL1A9UXMiiJH14vUkDX7o0jikv8UEviEZQOff10SygU0pVryDFFFKfijyvCAHqwxRNl WI0pMzwdfD/c+bp599sD2m6EpL8UbrrikpKdGtNnvNU9/VwzkMTDUIDsH1dL/9ht1JFp eJjUcRO49qzf0LDqSxIMC1NZHTZ59HiteYfy5XvW6PBFr3KBvUPc23hWiqJraOMMLUhO iZAHqOpDA9+hqNIAwpb80pnEXhr+lR+RB1rs3NWTLh5TefdPTyXjW9kzFkLde13i8fh0 beCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=yQpMa7oONmmsb8OFgjGg2Ymu70h5oHAEfnqSpiPqIhs=; b=LooJ2qJKB+WO8Ls3cE86D/pVAogYRMnZjljEujijaBH6oX5c6LVuQLVsKjT+2Y8QJ/ GDDPmMGIontU7HCXvFXfCecOxY36EMnUCDGtuAvJZcCHVA3+eKArrRVKLjCkUCdUFo25 LnMjRMi1yZPP61FQ8yuDfkiamIA/6MBRwwn4mziVYB/op2NCAxxTGuVBMRKiuUAKMP/2 4lCotH653IhA2pVBosRQKrQCcg2soUDwX47hMkVeIxW7TbnKfPSASV6s+Ns9443SB5Ya rKGnP3YCh2nd+ByQ3nBI8ZWUaDFVo2hNdjdoOmz7jkovA8JiwS2vwUNa0/+PlsgcQAzZ GH2A== X-Gm-Message-State: AG10YOQf/GjTm30hpHjGwUnB7HrCP0KkRmB5nJjSspj9lvI2mv38+BrQH+dhry6zxoFKuQ== X-Received: by 10.66.216.69 with SMTP id oo5mr6157951pac.126.1454002115354; Thu, 28 Jan 2016 09:28:35 -0800 (PST) Received: from [22.160.77.234] ([172.56.38.103]) by smtp.gmail.com with ESMTPSA id f12sm17809236pat.20.2016.01.28.09.28.33 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 28 Jan 2016 09:28:33 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Packaging the FreeBSD base system with pkg(8) From: NGie Cooper X-Mailer: iPhone Mail (13D15) In-Reply-To: <20160128160622.GB88527@zxy.spb.ru> Date: Thu, 28 Jan 2016 09:28:32 -0800 Cc: Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> To: Slawa Olhovchenkov X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:28:36 -0000 > On Jan 28, 2016, at 08:06, Slawa Olhovchenkov wrote: ... > What about upgrade strongly outdated system? > For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, > pkg from 11.0 don't undertund package base from 18.0 and etc. This is an important question to ask and solve. There might be points in tim= e where seamless upgrades are not possible, and instead you need to hop from= release to release (this is not ideal, but it could happen). For instance, at $work we're allowing upgrades from version X to Y, and Y to= Z, but not direct upgrades from X to Z. Part of the rationale behind this c= hange is, deprecation of platforms and the change in upgrade framework, whic= h requires upgrading from blessed releases proven to work with the new frame= work. Thanks, -NGie= From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:38:40 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6FDDDA70F21; Thu, 28 Jan 2016 17:38:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2F98B1551; Thu, 28 Jan 2016 17:38:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOqWj-000F9Y-1j; Thu, 28 Jan 2016 20:38:37 +0300 Date: Thu, 28 Jan 2016 20:38:37 +0300 From: Slawa Olhovchenkov To: NGie Cooper Cc: Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128173836.GD88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:38:40 -0000 On Thu, Jan 28, 2016 at 09:28:32AM -0800, NGie Cooper wrote: > > > On Jan 28, 2016, at 08:06, Slawa Olhovchenkov wrote: > > > ... > > > What about upgrade strongly outdated system? > > For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, > > pkg from 11.0 don't undertund package base from 18.0 and etc. > > This is an important question to ask and solve. There might be > points in time where seamless upgrades are not possible, and instead > you need to hop from release to release (this is not ideal, but it > could happen). I see two side of this problem: support in sofware and support in infrastructure (ftp.freebsd.org and etc.). Because pkg is not part of base FreeBSD and live in ports -- this hops need to preserve (and testing?) packages collections for all past releases and don't move it to archive. And regular resigning package databases. And I miss somewere. > For instance, at $work we're allowing upgrades from version X to Y, > and Y to Z, but not direct upgrades from X to Z. Part of the > rationale behind this change is, deprecation of platforms and the > change in upgrade framework, which requires upgrading from blessed > releases proven to work with the new framework. This is common practic, yes. This is acceptably if possible got all necessary in time 18.0 for upgrade from 11.0. From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:43:51 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 372F1A711FD; Thu, 28 Jan 2016 17:43:51 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 282391ABF; Thu, 28 Jan 2016 17:43:51 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 9D12816AE; Thu, 28 Jan 2016 17:43:50 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Thu, 28 Jan 2016 17:43:49 +0000 From: Glen Barber To: Allan Jude Cc: freebsd-current@freebsd.org, freebsd-pkgbase@FreeBSD.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128174349.GO98557@FreeBSD.org> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <56AA3D54.5020502@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="8SdtHY/0P4yzaavF" Content-Disposition: inline In-Reply-To: <56AA3D54.5020502@freebsd.org> X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:43:51 -0000 --8SdtHY/0P4yzaavF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 28, 2016 at 11:09:56AM -0500, Allan Jude wrote: > The approach for offering an upgrade from 10.x to 11.0 will be the more > interesting endeavour. >=20 At present, the plan is to provide supported 10.x releases with the knowledge of what is added/removed in 11.0-RELEASE, be it through a mtree(8) metalog, or something else. Glen --8SdtHY/0P4yzaavF Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWqlNVAAoJEAMUWKVHj+KT2xcQAJ1AF0UoDwR+bP3JQe/jkE47 vhOqH3r2Xb3/C4+K3RGkvZB0vwVZ5CSany8rkeoC6qylKXpuBXLolM9UfvIcst45 oFDsSaAzv9adUVDgTx5pk1yN9bfj4EqfXW+tdOMJoFLCKc75VukiPI5ZZ+p8N5zP hy7gOHEdPoRZLSm0GMgSTxqPsC5Gr8ACwWOfdg++bnb/p1mIpJPBEbPxlvXowVme HfIKGBm0ulBYdn8MGewYG3FTNL/DdIOxaNs0Qo9vnIDEFdcdpH5GK/iUhtM3BFQl gwfV+z/xLsg8xJ/A3ND8FRV3OwM9tJLjC6E872ePWDzQ4blE073VzKzFgr/qIWjk zr1OKCsDutzTKrZnFhkAJuc3n2e0pa4soRke1LoqkN8xaL2dGbJXEjhgiQkqK27f tU60vsaLVlwFKp3Z1NywoUjjlnmw1BVQqBNV/NOiEbhpNb2cCgQT+7GWw4exq1os MIG9pwj8GYJzpm8LtICwLo5vCNxsEJcP3clAA8fjjDfYLxhPO/IcuM3KOs7Er8M9 yj48LL7eRCjEjVBhPSVzFLNYDND6zzUiwOzvIKFglgNEIN/rmPUULIrF7dBYiRXy BWrykt93IyH0NSklRO8yjxdTnn2QfDAQocEiKtQtAEzJrjHCdqK2XlRwgTog51Il GQscYiPyBfkbFHF4EiaM =r/c9 -----END PGP SIGNATURE----- --8SdtHY/0P4yzaavF-- From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:45:20 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1288A7131C; Thu, 28 Jan 2016 17:45:19 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: from mail-pa0-x234.google.com (mail-pa0-x234.google.com [IPv6:2607:f8b0:400e:c03::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B24C81D45; Thu, 28 Jan 2016 17:45:19 +0000 (UTC) (envelope-from yaneurabeya@gmail.com) Received: by mail-pa0-x234.google.com with SMTP id ho8so26449572pac.2; Thu, 28 Jan 2016 09:45:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=RtTFKHM58c3Th5W+5608AsI6zB/rj6YKo3m8/KIUbfA=; b=IUsjzwQiuxA2XlF91X0SBOv9A5gdb7KS+PoddgAiWNMMe2J7+Rh+vXDPYJla5vCleq 2wlmcATetUROtMq7RM7FfAQcvStC3adRjYgXOVI4jm7JS9hO8gPqLT43du6DiSPrygo8 IKBE3e5/mjENc/s8du58DAKBXeeTLhAR0L4DR3KH8zEd4joroE1vLF0LAnXO32xMp64U 4UKLcCLhepUHgMMuqKsSiPq3x9clgUYI35lf1b5Q4Z100v8INk/UDmKZRltzhv3XvBPW U8MQ3VymVAhgFI9Eb9IaDZedQj9RwzKq6Rw/NlMb1Lv/iDRAafOYHDeffOvIUwpz0k+K sAOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=RtTFKHM58c3Th5W+5608AsI6zB/rj6YKo3m8/KIUbfA=; b=VzkrnWdZD51tukfnM51yn9hXf+1vA2fwdbnFKLtyKgjJF5p5niZPoYlWdRlMhcpjD5 hsgOLPRJSkF1pu7LWC2AgThIKYhrwEfdiCth9uuZNZp8kz3LWpPjzzuESK3e2bGWEpO9 on6BcrgpkqlTaB7Wfka9TXK9wuC/iWkWU2/xApwhevRZDJvgxjZiTnpGRj/mNC0rlaBC 7QtN76WBANJn2y+6LfPqopcDctcsT56QkImNJ66m7UAEISp/vke1xo33+f8EjB5IgFHe jzntKrwsnWMxzwZ+qrtZMX5L8x7ysCdE4UCr37Cpvh4rRxPGJ0DKpqF2SKIFph2XkQ6A 3VWQ== X-Gm-Message-State: AG10YOTy1uYo7JAdMR8IWTYpqAx9fc1qZUzEMdj65cdbeD7ezOxWdKfTMAD3NcK7Jb8hPQ== X-Received: by 10.67.24.104 with SMTP id ih8mr6392707pad.124.1454003119350; Thu, 28 Jan 2016 09:45:19 -0800 (PST) Received: from [22.160.77.234] ([172.56.38.103]) by smtp.gmail.com with ESMTPSA id 70sm17832950pfc.69.2016.01.28.09.45.17 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 28 Jan 2016 09:45:18 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (1.0) Subject: Re: Packaging the FreeBSD base system with pkg(8) From: NGie Cooper X-Mailer: iPhone Mail (13D15) In-Reply-To: <20160128173836.GD88527@zxy.spb.ru> Date: Thu, 28 Jan 2016 09:45:16 -0800 Cc: Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> To: Slawa Olhovchenkov X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:45:20 -0000 > On Jan 28, 2016, at 09:38, Slawa Olhovchenkov wrote: >=20 >> On Thu, Jan 28, 2016 at 09:28:32AM -0800, NGie Cooper wrote: >>=20 >>=20 >>> On Jan 28, 2016, at 08:06, Slawa Olhovchenkov wrote: >>=20 >>=20 >> ... >>=20 >>> What about upgrade strongly outdated system? >>> For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, >>> pkg from 11.0 don't undertund package base from 18.0 and etc. >>=20 >> This is an important question to ask and solve. There might be >> points in time where seamless upgrades are not possible, and instead >> you need to hop from release to release (this is not ideal, but it >> could happen). >=20 > I see two side of this problem: support in sofware and support in > infrastructure (ftp.freebsd.org and etc.). Because pkg is not part of > base FreeBSD and live in ports -- this hops need to preserve (and > testing?) packages collections for all past releases and don't move it > to archive. And regular resigning package databases. And I miss > somewere. >=20 >> For instance, at $work we're allowing upgrades from version X to Y, >> and Y to Z, but not direct upgrades from X to Z. Part of the >> rationale behind this change is, deprecation of platforms and the >> change in upgrade framework, which requires upgrading from blessed >> releases proven to work with the new framework. >=20 > This is common practic, yes. > This is acceptably if possible got all necessary in time 18.0 for > upgrade from 11.0. Yes. Given the hiccups going from pkg 1.4 to 1.6 with the plist stuff, this i= s going to be more entertaining across interface breaks; it might be that pk= gng is at the point where it's stable enough that interfaces won't change (u= nlikely, software tends to be fluid), or backwards compatibility will need t= o be maintained for older versions of pkgng. Also, consider that you're going to be allowing upgrades from older RELEASE v= ersions of the OS which might be using a fixed copy of pkgng -- how are you g= oing to support that? Thanks! -NGIE= From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 17:57:00 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57D1FA716AD; Thu, 28 Jan 2016 17:57:00 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from theravensnest.org (theraven.freebsd.your.org [216.14.102.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "cloud.theravensnest.org", Issuer "StartCom Class 1 Primary Intermediate Server CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0352312D9; Thu, 28 Jan 2016 17:56:59 +0000 (UTC) (envelope-from theraven@FreeBSD.org) Received: from [192.168.0.7] (cpc91230-cmbg18-2-0-cust661.5-4.cable.virginm.net [82.1.230.150]) (authenticated bits=0) by theravensnest.org (8.15.2/8.15.2) with ESMTPSA id u0SHubAs008955 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 28 Jan 2016 17:56:42 GMT (envelope-from theraven@FreeBSD.org) X-Authentication-Warning: theravensnest.org: Host cpc91230-cmbg18-2-0-cust661.5-4.cable.virginm.net [82.1.230.150] claimed to be [192.168.0.7] Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2104\)) Subject: Re: Packaging the FreeBSD base system with pkg(8) From: David Chisnall In-Reply-To: <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> Date: Thu, 28 Jan 2016 17:56:31 +0000 Cc: Slawa Olhovchenkov , Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <5F8AC959-2B81-4787-AEE3-50B9650B1672@FreeBSD.org> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> To: NGie Cooper X-Mailer: Apple Mail (2.2104) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 17:57:00 -0000 On 28 Jan 2016, at 17:45, NGie Cooper wrote: >=20 > Also, consider that you're going to be allowing upgrades from older = RELEASE versions of the OS which might be using a fixed copy of pkgng -- = how are you going to support that? I believe that the plan is to promote the pkg tool somewhat closer to = the base system. Upgrades will do the same sort of thing that they do = currently for ports: 1. First check if the version of pkg is the latest 2. If not, upgrade it 3. Do the real upgrade The package for package is simply a tarball. It may be advantageous to = separate the pkg and pkg-static binaries into different packages, so = that pkg can always install pkg-static and pkg-static can always update = pkg. There is no guarantee that the pkg tool from X.Y can install any = packages from X+n.Y.m other than the pkg-static binary, which can then = upgrade the rest of the system. The provision of pkg-static prevents us from being in the situation that = I encountered trying to upgrade a Debian system (and ending up with a = mess requiring a full reinstall) where apt needed a newer glibc and the = glibc package needed a newer apt to install it. We will always provide = a pkg-static for every supported branch that can be installed by any = earlier version of pkg (because it=E2=80=99s just extracting a = single-file archive - and in the absolute worst case you can do this by = hand) and can install newer packages. David From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 18:12:56 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 499A5A71C88; Thu, 28 Jan 2016 18:12:56 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 084181C94; Thu, 28 Jan 2016 18:12:56 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOr3t-000Fwh-3B; Thu, 28 Jan 2016 21:12:53 +0300 Date: Thu, 28 Jan 2016 21:12:53 +0300 From: Slawa Olhovchenkov To: NGie Cooper Cc: Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128181253.GE88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 18:12:56 -0000 On Thu, Jan 28, 2016 at 09:45:16AM -0800, NGie Cooper wrote: > > > On Jan 28, 2016, at 09:38, Slawa Olhovchenkov wrote: > > > >> On Thu, Jan 28, 2016 at 09:28:32AM -0800, NGie Cooper wrote: > >> > >> > >>> On Jan 28, 2016, at 08:06, Slawa Olhovchenkov wrote: > >> > >> > >> ... > >> > >>> What about upgrade strongly outdated system? > >>> For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, > >>> pkg from 11.0 don't undertund package base from 18.0 and etc. > >> > >> This is an important question to ask and solve. There might be > >> points in time where seamless upgrades are not possible, and instead > >> you need to hop from release to release (this is not ideal, but it > >> could happen). > > > > I see two side of this problem: support in sofware and support in > > infrastructure (ftp.freebsd.org and etc.). Because pkg is not part of > > base FreeBSD and live in ports -- this hops need to preserve (and > > testing?) packages collections for all past releases and don't move it > > to archive. And regular resigning package databases. And I miss > > somewere. > > > >> For instance, at $work we're allowing upgrades from version X to Y, > >> and Y to Z, but not direct upgrades from X to Z. Part of the > >> rationale behind this change is, deprecation of platforms and the > >> change in upgrade framework, which requires upgrading from blessed > >> releases proven to work with the new framework. > > > > This is common practic, yes. > > This is acceptably if possible got all necessary in time 18.0 for > > upgrade from 11.0. > > Yes. Given the hiccups going from pkg 1.4 to 1.6 with the plist > stuff, this is going to be more entertaining across interface > breaks; it might be that pkgng is at the point where it's stable > enough that interfaces won't change (unlikely, software tends to be > fluid), or backwards compatibility will need to be maintained for > older versions of pkgng. > Also, consider that you're going to be allowing upgrades from older > RELEASE versions of the OS which might be using a fixed copy of > pkgng -- how are you going to support that? I see two hudge problem for support upgrades from older RELEASE versions (supported too): key (used for repo signing) change and need to access ports packages to install (installed outdated release can't find pkg packet for install). Can you more detailed describe current propolsal of new packaging system? pkg is part of base installed packages? Or after installing base packages pkg installed from `ports`? For disc1.iso case. How handled boot/device.hints and var/db/etcupdate? How handled boot block updates? From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 18:17:30 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E8CB0A71DF7; Thu, 28 Jan 2016 18:17:30 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A6D8D1E14; Thu, 28 Jan 2016 18:17:30 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOr8J-000G4T-My; Thu, 28 Jan 2016 21:17:27 +0300 Date: Thu, 28 Jan 2016 21:17:27 +0300 From: Slawa Olhovchenkov To: David Chisnall Cc: NGie Cooper , Baptiste Daroussin , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128181727.GF88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <5F8AC959-2B81-4787-AEE3-50B9650B1672@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5F8AC959-2B81-4787-AEE3-50B9650B1672@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 18:17:31 -0000 On Thu, Jan 28, 2016 at 05:56:31PM +0000, David Chisnall wrote: > On 28 Jan 2016, at 17:45, NGie Cooper wrote: > > > > Also, consider that you're going to be allowing upgrades from older RELEASE versions of the OS which might be using a fixed copy of pkgng -- how are you going to support that? > > I believe that the plan is to promote the pkg tool somewhat closer to the base system. Upgrades will do the same sort of thing that they do currently for ports: > > 1. First check if the version of pkg is the latest > 2. If not, upgrade it upgrade to latest version build for older release? > 3. Do the real upgrade > > The package for package is simply a tarball. It may be advantageous to separate the pkg and pkg-static binaries into different packages, so that pkg can always install pkg-static and pkg-static can always update pkg. > > There is no guarantee that the pkg tool from X.Y can install any packages from X+n.Y.m other than the pkg-static binary, which can then upgrade the rest of the system. > > The provision of pkg-static prevents us from being in the situation > that I encountered trying to upgrade a Debian system (and ending up > with a mess requiring a full reinstall) where apt needed a newer > glibc and the glibc package needed a newer apt to install it. We > will always provide a pkg-static for every supported branch that can > be installed by any earlier version of pkg (because it’s just > extracting a single-file archive - and in the absolute worst case > you can do this by hand) and can install newer packages. Even pkg-static build for 10.x can be fail to run on previos release. Builded svn-static on 9.x fail to run on 6.x, as result I build svn-static on 6.x (and run on 6.x-10.x). From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 18:23:25 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F308A72000; Thu, 28 Jan 2016 18:23:25 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 2E27911B1; Thu, 28 Jan 2016 18:23:25 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 6ED6F1088; Thu, 28 Jan 2016 18:23:24 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Thu, 28 Jan 2016 18:23:22 +0000 From: Glen Barber To: Slawa Olhovchenkov Cc: NGie Cooper , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128182322.GP98557@FreeBSD.org> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <20160128181253.GE88527@zxy.spb.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="L/bWm/e7/ricERqM" Content-Disposition: inline In-Reply-To: <20160128181253.GE88527@zxy.spb.ru> X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 18:23:25 -0000 --L/bWm/e7/ricERqM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 28, 2016 at 09:12:53PM +0300, Slawa Olhovchenkov wrote: > I see two hudge problem for support upgrades from older RELEASE > versions (supported too): key (used for repo signing) change and need > to access ports packages to install (installed outdated release can't > find pkg packet for install). >=20 > Can you more detailed describe current propolsal of new packaging > system? pkg is part of base installed packages? Or after installing > base packages pkg installed from `ports`? For disc1.iso case. >=20 > How handled boot/device.hints and var/db/etcupdate? >=20 > How handled boot block updates? These are all valid questions, but let's take a step back for a bit, and not put the carriage in front of the horse. The initial email in this thread was intended to provide an update on the status. Some things, such as file merging, is already in place in a few of the packages. Not all, yet. Unexpected things are going to happen. That is the only thing that is a guarantee right now. In other words, implementation (from what is now in the project branch) may change. And yes, there needs to be a way to upgrade from older releases. But let's not get too far ahead of ourselves. Glen --L/bWm/e7/ricERqM Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWqlyaAAoJEAMUWKVHj+KTVicP/0qKQKXQ0o7/X8AHAxBLJX47 NgMm0Gc5Qznl6kWuN9uLBkFZAc14K7JffH6B90zSH2ShmLjVfQGOdpSfw8N9rK8z nwvegj5oHce6bii+DDB3vmp19BdKMDHXEIRZpev7lpZSm52d2rVUNuJXiy9tB/6o 2otfpm3uivsrherebUFvfcjLFIWpzXK9UXMMA1cR9WBAxlpQEgsaMYexIavngDbT kycH4+QG/AbIQ/Seyl5IgGN9HFYbamzdU2VIhoRCcHEqWMefU3xqOWMm2xYvNDZU 6pkx7gfwld1NQFJYdqrwFftvehUktUASw9MtNuBUJ+fqpMATNIQUWr1sOw8UGDYk wfdX4GO8F8O669eTdDnfWnsfl1Et2MECE8mZNUdlgXFxmPAfTRSGr47qpBVgzTS9 OozClYj/dDQ6wIcXksYBFFTetnJOiPb9sa06lCV6qFnAlMcKF39iIEjkV4xu1yAJ 6M+dlZSTZlXPubrCPrcbw+ug+6alK+FymwgQWBMFikU8WTfq74M/KoQbGNFUKYkh rvR6g1F5pN3VnjMl1GD28TYQfI22ZUPg385/HS9pV6vXQhRQJ2egrbsrgkmVwTTT zdS+wFDOSagtPy1g5tsTuK+9j8VlmAePQYKTU/znJ6UOVZtwkoWmvkU3OpgX72lI aQPTUtqZ6KT4TVdNh+me =hmjF -----END PGP SIGNATURE----- --L/bWm/e7/ricERqM-- From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 18:25:08 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B5FDA700BA for ; Thu, 28 Jan 2016 18:25:08 +0000 (UTC) (envelope-from jmaloney@pcbsd.org) Received: from barracuda.ixsystems.com (mail.ixsystems.com [69.198.165.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certificate Authority - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 121A51331 for ; Thu, 28 Jan 2016 18:25:07 +0000 (UTC) (envelope-from jmaloney@pcbsd.org) X-ASG-Debug-ID: 1454005505-08ca04766a02ff0002-wjM0Vu Received: from [10.0.1.13] (ip72-209-162-1.ks.ks.cox.net [72.209.162.1]) by barracuda.ixsystems.com with ESMTP id 8a3QJXiZxfl7SHhZ (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Thu, 28 Jan 2016 10:25:06 -0800 (PST) X-Barracuda-Envelope-From: jmaloney@pcbsd.org X-Barracuda-AUTH-User: jmaloney@pcbsd.org X-Barracuda-Apparent-Source-IP: 72.209.162.1 Subject: Re: Packaging the FreeBSD base system with pkg(8) To: freebsd-pkgbase@freebsd.org X-ASG-Orig-Subj: Re: Packaging the FreeBSD base system with pkg(8) References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <20160128181253.GE88527@zxy.spb.ru> From: Joe Maloney Message-ID: <56AA5D52.8040203@pcbsd.org> Date: Thu, 28 Jan 2016 12:26:26 -0600 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <20160128181253.GE88527@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Barracuda-Connect: ip72-209-162-1.ks.ks.cox.net[72.209.162.1] X-Barracuda-Start-Time: 1454005505 X-Barracuda-Encrypted: ECDHE-RSA-AES128-GCM-SHA256 X-Barracuda-URL: https://10.2.0.41:443/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at ixsystems.com X-Barracuda-BRTS-Status: 1 X-Barracuda-Spam-Score: 0.00 X-Barracuda-Spam-Status: No, SCORE=0.00 using global scores of TAG_LEVEL=1000.0 QUARANTINE_LEVEL=1000.0 KILL_LEVEL=8.0 tests= X-Barracuda-Spam-Report: Code version 3.2, rules version 3.2.3.26539 Rule breakdown below pts rule name description ---- ---------------------- -------------------------------------------------- X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 18:25:08 -0000 I have been planning for some months to integrate this technology into what will be the future offering of something I have been working on. :) http://skynet.desktopbsd.net/packages/ Thus far I had only been able to build the world for those curious what the layout looks like an older version can be shown at the site above. I've been digging into the code, and I had noticed the stagekernel was missing which is why the kernel was not built in that repo. However now I see that gjb has committed that. So I will begin re-attempting builds, and storing them at the above location. I am curious. How will updates to or even the installation of items into /etc be handled? Will there be a special tool for that? After 11-RELEASE will there soon be a binary updating 11-STABLE, or 12-CURRENT when this is all said, and done? I am excited for this! Joe Maloney On 01/28/16 12:12, Slawa Olhovchenkov wrote: > On Thu, Jan 28, 2016 at 09:45:16AM -0800, NGie Cooper wrote: > >>> On Jan 28, 2016, at 09:38, Slawa Olhovchenkov wrote: >>> >>>> On Thu, Jan 28, 2016 at 09:28:32AM -0800, NGie Cooper wrote: >>>> >>>> >>>>> On Jan 28, 2016, at 08:06, Slawa Olhovchenkov wrote: >>>> >>>> ... >>>> >>>>> What about upgrade strongly outdated system? >>>>> For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, >>>>> pkg from 11.0 don't undertund package base from 18.0 and etc. >>>> This is an important question to ask and solve. There might be >>>> points in time where seamless upgrades are not possible, and instead >>>> you need to hop from release to release (this is not ideal, but it >>>> could happen). >>> I see two side of this problem: support in sofware and support in >>> infrastructure (ftp.freebsd.org and etc.). Because pkg is not part of >>> base FreeBSD and live in ports -- this hops need to preserve (and >>> testing?) packages collections for all past releases and don't move it >>> to archive. And regular resigning package databases. And I miss >>> somewere. >>> >>>> For instance, at $work we're allowing upgrades from version X to Y, >>>> and Y to Z, but not direct upgrades from X to Z. Part of the >>>> rationale behind this change is, deprecation of platforms and the >>>> change in upgrade framework, which requires upgrading from blessed >>>> releases proven to work with the new framework. >>> This is common practic, yes. >>> This is acceptably if possible got all necessary in time 18.0 for >>> upgrade from 11.0. >> Yes. Given the hiccups going from pkg 1.4 to 1.6 with the plist >> stuff, this is going to be more entertaining across interface >> breaks; it might be that pkgng is at the point where it's stable >> enough that interfaces won't change (unlikely, software tends to be >> fluid), or backwards compatibility will need to be maintained for >> older versions of pkgng. > >> Also, consider that you're going to be allowing upgrades from older >> RELEASE versions of the OS which might be using a fixed copy of >> pkgng -- how are you going to support that? > I see two hudge problem for support upgrades from older RELEASE > versions (supported too): key (used for repo signing) change and need > to access ports packages to install (installed outdated release can't > find pkg packet for install). > > Can you more detailed describe current propolsal of new packaging > system? pkg is part of base installed packages? Or after installing > base packages pkg installed from `ports`? For disc1.iso case. > > How handled boot/device.hints and var/db/etcupdate? > > How handled boot block updates? > _______________________________________________ > freebsd-pkgbase@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase > To unsubscribe, send any mail to "freebsd-pkgbase-unsubscribe@freebsd.org" From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 18:57:05 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B89BA70D10; Thu, 28 Jan 2016 18:57:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0BABB17A8; Thu, 28 Jan 2016 18:57:05 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOrka-000H4e-JU; Thu, 28 Jan 2016 21:57:00 +0300 Date: Thu, 28 Jan 2016 21:57:00 +0300 From: Slawa Olhovchenkov To: Glen Barber Cc: NGie Cooper , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128185700.GG88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <20160128181253.GE88527@zxy.spb.ru> <20160128182322.GP98557@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160128182322.GP98557@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 18:57:05 -0000 On Thu, Jan 28, 2016 at 06:23:22PM +0000, Glen Barber wrote: > On Thu, Jan 28, 2016 at 09:12:53PM +0300, Slawa Olhovchenkov wrote: > > I see two hudge problem for support upgrades from older RELEASE > > versions (supported too): key (used for repo signing) change and need > > to access ports packages to install (installed outdated release can't > > find pkg packet for install). > > > > Can you more detailed describe current propolsal of new packaging > > system? pkg is part of base installed packages? Or after installing > > base packages pkg installed from `ports`? For disc1.iso case. > > > > How handled boot/device.hints and var/db/etcupdate? > > > > How handled boot block updates? > > These are all valid questions, but let's take a step back for a bit, and > not put the carriage in front of the horse. > > The initial email in this thread was intended to provide an update on > the status. Some things, such as file merging, is already in place in > a few of the packages. Not all, yet. Initial email in this thread will about problems at upgrades, from my point of view and my expirence. I am currently try to upgrade systems by untar base.txz and see some inconsistence of packaging (boot/device.hints, var/db, var/log/sendmail.st, var/db/etcupdate) and etc. Sorry if this a different problem. For me -- all of this -- question of global design. Need administrativa about some questions: - how divide - how fresh install - how upgrade (on runnig system, from install media). - what packed to disk always - what got from internet - what about custom releases and src.conf I am don't know how currenly resolved this questions. Bundled with upgraded files fresh pkg (building for LAST-2 release) and preserving for 10+ years this files can resolve many issuse of upgrading outdated systems (by hop-to-hop or hop to hop+2). I am don't know about complexity this solution. > Unexpected things are going to happen. That is the only thing that is > a guarantee right now. In other words, implementation (from what is now > in the project branch) may change. And yes, there needs to be a way to > upgrade from older releases. I am ask for you check. This very complex task and need good planing. > But let's not get too far ahead of ourselves. > > Glen > From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 19:58:59 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C3A7A3C321; Thu, 28 Jan 2016 19:58:59 +0000 (UTC) (envelope-from lifanov@mail.lifanov.com) Received: from mail.lifanov.com (mail.lifanov.com [206.125.175.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD96AB36; Thu, 28 Jan 2016 19:58:58 +0000 (UTC) (envelope-from lifanov@mail.lifanov.com) Received: by mail.lifanov.com (Postfix, from userid 58) id 4590C1533; Thu, 28 Jan 2016 14:58:57 -0500 (EST) X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.lifanov.com X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT shortcircuit=ham autolearn=disabled version=3.4.1 Received: from [10.100.100.58] (unknown [107.15.95.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.lifanov.com (Postfix) with ESMTPSA id EBFC714F6; Thu, 28 Jan 2016 14:58:55 -0500 (EST) User-Agent: K-9 Mail for Android In-Reply-To: <20160128182322.GP98557@FreeBSD.org> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <20160128181253.GE88527@zxy.spb.ru> <20160128182322.GP98557@FreeBSD.org> MIME-Version: 1.0 Subject: Re: Packaging the FreeBSD base system with pkg(8) From: Nikolai Lifanov Date: Thu, 28 Jan 2016 14:58:52 -0500 To: Glen Barber ,Slawa Olhovchenkov CC: Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org, NGie Cooper Message-ID: <8CFEB1BA-1957-4BB4-980A-76199D3B831C@mail.lifanov.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 19:58:59 -0000 On January 28, 2016 1:23:22 PM EST, Glen Barber wrote: >On Thu, Jan 28, 2016 at 09:12:53PM +0300, Slawa Olhovchenkov wrote: >> I see two hudge problem for support upgrades from older RELEASE >> versions (supported too): key (used for repo signing) change and need >> to access ports packages to install (installed outdated release can't >> find pkg packet for install). >> >> Can you more detailed describe current propolsal of new packaging >> system? pkg is part of base installed packages? Or after installing >> base packages pkg installed from `ports`? For disc1.iso case. >> >> How handled boot/device.hints and var/db/etcupdate? >> >> How handled boot block updates? > >These are all valid questions, but let's take a step back for a bit, >and >not put the carriage in front of the horse. > >The initial email in this thread was intended to provide an update on >the status. Some things, such as file merging, is already in place in >a few of the packages. Not all, yet. > >Unexpected things are going to happen. That is the only thing that is >a guarantee right now. In other words, implementation (from what is >now >in the project branch) may change. And yes, there needs to be a way to >upgrade from older releases. > >But let's not get too far ahead of ourselves. > >Glen Can we have verbose and/or semi-interactive configuration merge, as offered by etcupdate or mergemaster? - Nikolai Lifanov From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 22:36:02 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4403DA7149A; Thu, 28 Jan 2016 22:36:02 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C9F931626; Thu, 28 Jan 2016 22:36:01 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id l66so31243519wml.0; Thu, 28 Jan 2016 14:36:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=PKFy29M6JCi1j4W1EfuVzHMLEy42M3TVXOP+q/tInF4=; b=NEx+mSnR1CArEOoNo/++In6+h6tK1ZGkeYfPVGc+goF83WfpMXxR2heHY2Tt0l6n+c AuQ0udaVR40UIkVOnZ9Y2HDlhuEc1k/1C/nYFY1ifi8DpgoccgvfKhNKU7ZnLWPJfbiJ t4sP619ylTMa/KlN2Vbip/m3YW2wSWhysKHp0WuYo+kw9YltXqd6ssveOxBwSwogqoqj MzojM5UO74l8EY/0q2D8xflE7cb3lMiuMfeBZQpMbkDT3eaFvPlFEigy3xqAO7oVGUvQ JhZPzCSzuw4tBkyoytlGCWhyepXKMfKXBxZXSIB11I8p0kCHgT/EzxqJ3HYED3v2aDoe 8Kow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-type:content-disposition :in-reply-to:user-agent; bh=PKFy29M6JCi1j4W1EfuVzHMLEy42M3TVXOP+q/tInF4=; b=ALqDL8boTODtVTsA50jZ05fc3ROkitl+pzHhHQvg/Y0O1sBEbWtaMUgtmHlcfFJdox UPg9U1ssjU/yDootkFLP4OmHfOFMvoYA37VW7jzjuD0Vibtt15qaWz6rzct5Uuprdwpo 23C4RY9z6Clmuk9hJXZgO/FLx0eUesbMJFgWHnlA8Yq6B0qyFzchX20Kjz7nAvEL0ayp p0hEa4h/eZld4tN5U9Gnkl4lYEp5xNCryji5Kuyv6vrhQ6YnU7gNWAX6MpBMu524sJ6C 4cMaiT8MlrXHwUt1TZfOjpJ19e7DZQwuQIl/TFP6KqVd7mDJaA05lh/s5ho/BqLoHv/l pXiA== X-Gm-Message-State: AG10YOTk0cq5LL7OFrsjknmkYA9Q5twM+eCzQeiOZZy9IXwxpXMtZpzoRidg1SzDTbTPPw== X-Received: by 10.195.17.198 with SMTP id gg6mr5342576wjd.15.1454020560407; Thu, 28 Jan 2016 14:36:00 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id az10sm12980009wjc.28.2016.01.28.14.35.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 Jan 2016 14:35:59 -0800 (PST) Sender: Baptiste Daroussin Date: Thu, 28 Jan 2016 23:35:54 +0100 From: Baptiste Daroussin To: Nikolai Lifanov Cc: Glen Barber , Slawa Olhovchenkov , Thomas Mueller , freebsd-current@freebsd.org, freebsd-pkgbase@freebsd.org, NGie Cooper Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160128223553.GD67200@ivaldir.etoilebsd.net> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <20160128173836.GD88527@zxy.spb.ru> <3A5AAF2B-F9CF-41B3-A97C-FF208803428A@gmail.com> <20160128181253.GE88527@zxy.spb.ru> <20160128182322.GP98557@FreeBSD.org> <8CFEB1BA-1957-4BB4-980A-76199D3B831C@mail.lifanov.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="RYJh/3oyKhIjGcML" Content-Disposition: inline In-Reply-To: <8CFEB1BA-1957-4BB4-980A-76199D3B831C@mail.lifanov.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 22:36:02 -0000 --RYJh/3oyKhIjGcML Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Jan 28, 2016 at 02:58:52PM -0500, Nikolai Lifanov wrote: >=20 >=20 > On January 28, 2016 1:23:22 PM EST, Glen Barber wrote: > >On Thu, Jan 28, 2016 at 09:12:53PM +0300, Slawa Olhovchenkov wrote: > >> I see two hudge problem for support upgrades from older RELEASE > >> versions (supported too): key (used for repo signing) change and need > >> to access ports packages to install (installed outdated release can't > >> find pkg packet for install). > >>=20 > >> Can you more detailed describe current propolsal of new packaging > >> system? pkg is part of base installed packages? Or after installing > >> base packages pkg installed from `ports`? For disc1.iso case. > >>=20 > >> How handled boot/device.hints and var/db/etcupdate? > >>=20 > >> How handled boot block updates? > > > >These are all valid questions, but let's take a step back for a bit, > >and > >not put the carriage in front of the horse. > > > >The initial email in this thread was intended to provide an update on > >the status. Some things, such as file merging, is already in place in > >a few of the packages. Not all, yet. > > > >Unexpected things are going to happen. That is the only thing that is > >a guarantee right now. In other words, implementation (from what is > >now > >in the project branch) may change. And yes, there needs to be a way to > >upgrade from older releases. > > > >But let's not get too far ahead of ourselves. > > > >Glen >=20 > Can we have verbose and/or semi-interactive configuration merge, as offer= ed by etcupdate or mergemaster? pkg has a merging mercanism (which one can disable if scared) which will automatically merge everything it can and live a file .new if a conflict ha= ppen (or merging disabled) Would be easy to write an etcupdate like that walk through /etc and propose= a semi-interactive merging mechanism. Best regards, Bapt --RYJh/3oyKhIjGcML Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWqpfJAAoJEGOJi9zxtz5aPOcQAKFW3eRlq8VkFpwc4tpgjduk FhFQ+NJ4+LYoOszZWjEEoo/gaCe6e6umrQeH6i5uQ2y6sxJCeKmFZ6bJ4YUPx7eE FdwBRCu4iT35eqMWJtJU5pBw2FpvMhbdI+2M4UWJelS43TQAjMAz5Hb1Ae0OInOx iLwRyfOYpKXUw4K8d6EmJbxfck3gUjUs3PB229p6dOahxTvVUJimEzIZknmyYZ+N NqBURVmsHDVZVVDBnepQ/be8o+CDpxpXXoKvkIg8Lo+jNyXZw++BVWRSNDnSdrcz LcWxYTc8wy8NqMPxUorw/VYwkOm5HkMV3qOPOoQJFnxEfmRDWQpCqJKimibhYHfM yhCYgPeJxmBQtGOrOsDFN9slmqz9mxuMhIj42mEwtd/oyqrjUzvoh7IiKSTm3oK4 U8NQsGffMn4r8EXmZRqQTlL/HAx9qelVXOXJWBysaZt/S5DUTFcMaEDCwXbZFzdr FPul01jdUb2MHNPjjTxChGEmMK8clzwBIRhexCtG7gRk/JYkHOVDSEw7cwryL8Gr EThb4qv/Wz1o5HdXhtbUZquqRFjY9nV2ZcVZtMYJFC59LaxEeVqcjQcYijRWrPuS vRAX3T/o2nGo3Ls+evDNKtggGaBUMMNYEfMDGi9RpChb7oAOHZLYrLRvntH3luu6 Q/Xl6C6ugbey0BnTnL93 =oPPu -----END PGP SIGNATURE----- --RYJh/3oyKhIjGcML-- From owner-freebsd-pkgbase@freebsd.org Thu Jan 28 23:48:16 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78C139D9F2E for ; Thu, 28 Jan 2016 23:48:16 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3859B1CA1 for ; Thu, 28 Jan 2016 23:48:16 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id u0SNmDNJ067378 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Thu, 28 Jan 2016 18:48:13 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id u0SNmCSX067374; Thu, 28 Jan 2016 18:48:12 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <22186.43196.374050.48765@khavrinen.csail.mit.edu> Date: Thu, 28 Jan 2016 18:48:12 -0500 From: Garrett Wollman To: Slawa Olhovchenkov Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) In-Reply-To: <20160128160622.GB88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Thu, 28 Jan 2016 18:48:13 -0500 (EST) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2016 23:48:16 -0000 < said: > What about upgrade strongly outdated system? > For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, > pkg from 11.0 don't undertund package base from 18.0 and etc. We've never supported this for source upgrades; I don't see any reason to expect it to work for binary upgrades either. At a minimum you should expect to have to upgrade 11.0 -> latest 11.x -> latest 12.x -> ... -> latest 17.x -> 18.0. -GAWollman From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 00:15:29 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FC3BA70BE0 for ; Fri, 29 Jan 2016 00:15:29 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 21DB0E35 for ; Fri, 29 Jan 2016 00:15:29 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOwij-000O0U-IF; Fri, 29 Jan 2016 03:15:25 +0300 Date: Fri, 29 Jan 2016 03:15:25 +0300 From: Slawa Olhovchenkov To: Garrett Wollman Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160129001525.GH88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <22186.43196.374050.48765@khavrinen.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22186.43196.374050.48765@khavrinen.csail.mit.edu> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 00:15:29 -0000 On Thu, Jan 28, 2016 at 06:48:12PM -0500, Garrett Wollman wrote: > < said: > > > What about upgrade strongly outdated system? > > For example 11.0 at time 18.0? I.e. packages for 11.0 don't available, > > pkg from 11.0 don't undertund package base from 18.0 and etc. > > We've never supported this for source upgrades; I don't see any reason > to expect it to work for binary upgrades either. At a minimum you > should expect to have to upgrade 11.0 -> latest 11.x -> latest 12.x -> > ... -> latest 17.x -> 18.0. You miss some points. Source upgrades suported over-release upgrade (currently 10-STABLE can be upgraded from 7.0, see Makefile.inc1). Source upgrade is self-contained and don't depend from environment. Contrary to binary upgrade may depend of environment: presents pkg.txz on ftp.freebsd.org, freebsd.org ovned by project and etc. As result, in case of bad design, you can't upgrade from 11.0 nowhere (ex: packages for 11.x outdated and deleted, pkg.txz from latest suported release can't be run on 11.0, pkg from 11.0 can't be parsed latest format). I am not averse to binary upgrade path like 11.0-12.last-14.last-15.last-16.last-18.0. I am disagree case when impossible to upgrade outdated install (because url is now incorrect, packages for outdated release removed or lost and etc). From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 00:34:45 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 97AEBA71BB9 for ; Fri, 29 Jan 2016 00:34:45 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 57BCA19B7 for ; Fri, 29 Jan 2016 00:34:45 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id u0T0YhXA068093 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Thu, 28 Jan 2016 19:34:43 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id u0T0YhPq068090; Thu, 28 Jan 2016 19:34:43 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <22186.45987.44063.515924@khavrinen.csail.mit.edu> Date: Thu, 28 Jan 2016 19:34:43 -0500 From: Garrett Wollman To: Slawa Olhovchenkov Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) In-Reply-To: <20160129001525.GH88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <22186.43196.374050.48765@khavrinen.csail.mit.edu> <20160129001525.GH88527@zxy.spb.ru> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Thu, 28 Jan 2016 19:34:43 -0500 (EST) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 00:34:45 -0000 < said: > You miss some points. Source upgrades suported over-release upgrade > (currently 10-STABLE can be upgraded from 7.0, see Makefile.inc1). I think this is an unreasonable expectation and an error to depend on this. > Source upgrade is self-contained and don't depend from environment. > Contrary to binary upgrade may depend of environment: presents pkg.txz > on ftp.freebsd.org, freebsd.org ovned by project and etc. As result, > in case of bad design, you can't upgrade from 11.0 nowhere (ex: packages > for 11.x outdated and deleted, pkg.txz from latest suported release > can't be run on 11.0, pkg from 11.0 can't be parsed latest format). You can always build your own release-package repository, stepping through the intermediate source releases just as you would do now. All you need to do is preserve the last release package set you build from each stable branch. -GAWollman From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 01:05:27 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE1C7A71ACA for ; Fri, 29 Jan 2016 01:05:27 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE5771AFD for ; Fri, 29 Jan 2016 01:05:27 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aOxV7-000PKq-19; Fri, 29 Jan 2016 04:05:25 +0300 Date: Fri, 29 Jan 2016 04:05:24 +0300 From: Slawa Olhovchenkov To: Garrett Wollman Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160129010524.GI88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <22186.43196.374050.48765@khavrinen.csail.mit.edu> <20160129001525.GH88527@zxy.spb.ru> <22186.45987.44063.515924@khavrinen.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22186.45987.44063.515924@khavrinen.csail.mit.edu> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 01:05:28 -0000 On Thu, Jan 28, 2016 at 07:34:43PM -0500, Garrett Wollman wrote: > > Source upgrade is self-contained and don't depend from environment. > > Contrary to binary upgrade may depend of environment: presents pkg.txz > > on ftp.freebsd.org, freebsd.org ovned by project and etc. As result, > > in case of bad design, you can't upgrade from 11.0 nowhere (ex: packages > > for 11.x outdated and deleted, pkg.txz from latest suported release > > can't be run on 11.0, pkg from 11.0 can't be parsed latest format). > > You can always build your own release-package repository, stepping Another question is about complexity own release repository. For example, own freebsd-update server is too complex and incorrectly build. > through the intermediate source releases just as you would do now. > All you need to do is preserve the last release package set you build > from each stable branch. I am talk about SUDDENLY discovered outdated setup. From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 16:33:47 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65675A71CBE for ; Fri, 29 Jan 2016 16:33:47 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2483E17D3 for ; Fri, 29 Jan 2016 16:33:46 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id u0TGXhkf073020 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Fri, 29 Jan 2016 11:33:44 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id u0TGXgMw073017; Fri, 29 Jan 2016 11:33:42 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <22187.37990.812269.553074@khavrinen.csail.mit.edu> Date: Fri, 29 Jan 2016 11:33:42 -0500 From: Garrett Wollman To: Slawa Olhovchenkov Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) In-Reply-To: <20160129010524.GI88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <22186.43196.374050.48765@khavrinen.csail.mit.edu> <20160129001525.GH88527@zxy.spb.ru> <22186.45987.44063.515924@khavrinen.csail.mit.edu> <20160129010524.GI88527@zxy.spb.ru> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Fri, 29 Jan 2016 11:33:44 -0500 (EST) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 16:33:47 -0000 < said: >> [I wrote:] >> through the intermediate source releases just as you would do now. >> All you need to do is preserve the last release package set you build >> from each stable branch. > I am talk about SUDDENLY discovered outdated setup. Well, modern best practice says "burn them to the ground". That's an unsatisfying answer, but it is widely accepted that this is not a situation worth turning somersaults for. -GAWollman From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 16:42:59 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CEB08A7204C for ; Fri, 29 Jan 2016 16:42:59 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8FC771E3D for ; Fri, 29 Jan 2016 16:42:59 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aPC8I-000GUF-HK; Fri, 29 Jan 2016 19:42:50 +0300 Date: Fri, 29 Jan 2016 19:42:50 +0300 From: Slawa Olhovchenkov To: Garrett Wollman Cc: freebsd-pkgbase@freebsd.org Subject: Re: Packaging the FreeBSD base system with pkg(8) Message-ID: <20160129164250.GJ88527@zxy.spb.ru> References: <20160127223323.GG98557@FreeBSD.org> <183431.62409.bm@smtp111.sbc.mail.ne1.yahoo.com> <20160128131806.GB67200@ivaldir.etoilebsd.net> <20160128160622.GB88527@zxy.spb.ru> <22186.43196.374050.48765@khavrinen.csail.mit.edu> <20160129001525.GH88527@zxy.spb.ru> <22186.45987.44063.515924@khavrinen.csail.mit.edu> <20160129010524.GI88527@zxy.spb.ru> <22187.37990.812269.553074@khavrinen.csail.mit.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <22187.37990.812269.553074@khavrinen.csail.mit.edu> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 16:42:59 -0000 On Fri, Jan 29, 2016 at 11:33:42AM -0500, Garrett Wollman wrote: > < said: > > >> [I wrote:] > >> through the intermediate source releases just as you would do now. > >> All you need to do is preserve the last release package set you build > >> from each stable branch. > > > I am talk about SUDDENLY discovered outdated setup. > > Well, modern best practice says "burn them to the ground". That's an > unsatisfying answer, but it is widely accepted that this is not a > situation worth turning somersaults for. I am understund. But not always. From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 17:35:08 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED3C4A72417 for ; Fri, 29 Jan 2016 17:35:07 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E06251FF1; Fri, 29 Jan 2016 17:35:07 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 82AA01623; Fri, 29 Jan 2016 17:35:07 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Fri, 29 Jan 2016 17:35:06 +0000 From: Glen Barber To: freebsd-pkgbase@FreeBSD.org Subject: TODO/wish list for packaging the FreeBSD base system Message-ID: <20160129173506.GG1727@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="1giRMj6yz/+FOIRq" Content-Disposition: inline X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 17:35:08 -0000 --1giRMj6yz/+FOIRq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable As previously mentioned, packaging the base system is a requirement of the 11.0-RELEASE. First, a point of clarification on the end goals that surround this. At present, there is *no* intent to connect pkg(8) to source-based upgrades and/or 'installworld/installkernel'. There are a few reasons for this, the most notable reason is the implicit dependency on an active network link and ability to bootstrap pkg(8) if it is not installed. Since the pkg(8) utility is not included in the base system, we cannot depend on the binary to exist on a system, and enforcing network connectivity as a requirement is at best a POLA violation. The goal of this is to provide the ability to manage system installation with pkg(8), not enforce its use as a requirement of FreeBSD systems. The primary target audience of this work is mainly binary upgrade consumers, while providing the scalability for everyone to be able to manage their systems with pkg(8) without overhead and maintenance of homemade tools. Historically we have provided binary upgrade paths for the "default" FreeBSD installation. However, the current binary upgrade tools only provide an upgrade path for x86-based systems (amd64 and i386). Using pkg(8) as for binary upgrades, we can expand beyond this, and provide an upgrade path for non-x86 systems (arm, mips, powerpc). Below follows a braindump of known issues, TODO tasks, wish list items, etc. Known issues: - There are likely a number of other issues I have not yet encountered, so consider this list far from complete. - plist prioritization: pkg(8) needs to be aware of package installation prioritization (ordering), which is one of the items bapt@ is aware of. In particular, one of the major requirements of this is the ordering of rtld, libc, and libthr, which must be handled in that order. - Conflicting packages are non-obvious until attempting to install or upgrade. In a best case scenario, something like the FreeBSD-acct package may fail to install. In a worst case scenario, this can be fatal to the system. When FreeBSD-runtime fails to install, pkg(8) will rollback the changes, which behaviorally leads to removing all files included in the package - a dead system. =20 - Several files/directories do not have an associated package. The default package in this case is supposed to be 'FreeBSD-runtime', however there is a non-zero number of files/directories that have no package assignment. A few examples are /etc/login.conf, /etc/rc, /etc/network.subr. These can be found in the METALOG file, by looking for lines without 'tags=3D.*package=3D'. - Files flagged for automatic merge that would normally be handled by tools like etcupdate(8) or mergemaster(8) do not appear to be handled properly. In a recent build, I noticed /etc/devd/asus.conf.pkgnew was created (flagged as a file to handle merges in r279661), while the /etc/devd/asus.conf file did not have any local changes. I suspect this is a bug, and still investigating. - Some files installed during 'installworld' have the immutable file flag set as result of an 'afterinstall' target, which is not flagged as schg in the mtree(8) METALOG file. This is one of the failure cases that can lead to a dead system, as mentioned above. An ugly fix was committed as r294966 and r294972, however I am unhappy with this solution, since the intent of honoring MK_FSCHG=3Dyes for the installed files is lost, as well as the installed files now being symbolic links instead of hard links. I have tried fixing this with a pre-install script in the runtime.ucl file, but these attempts failed because test(1) does not recognize hard links. TODO List / Wish List: - Further separation of the base system (FreeBSD-runtime package) into more granular package sets. Extreme caution needs to be taken when testing updates to do this, so using virtualization is encouraged. I have spent countless hours over the past few weeks recovering failed installations and/or dead systems as noted above. - Creating meta-packages for logically-related utilities, and making the packages created more granular. One example that was provided in private email was the 'FreeBSD-rcmds' package, where the use case given had no need for tools like ruptime(1), rwho(1), and the like were useful to have installed, but rcp(1), rsh(1), etc., were not necessary. - Unless there is a specific reason to have an in-tree .ucl file in src/release/packages/, I would like to see a template used instead. Use cases where an in-tree .ucl file is needed are runtime.ucl and kernel.ucl, where post-install scripts are defined. For the rest of the currently-existing package definitions, it is not necessary, and adds to the complexity of breaking down the base system into more granular package sets. I tried a few tests yesterday using a file called '_template_.ucl', which if the release/packages/foo.ucl did not exist, the '_template_.ucl' file would be copied to the stage directory. This did not work, however, because there can be several '.ucl' files for any given 'package', notably runtime.plist, debug.plist (which should be runtime-debug.plist), runtime-lib32.plist and so on. Glen --1giRMj6yz/+FOIRq Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWq6LKAAoJEAMUWKVHj+KT5zQQAKQ94EkguInVyUX1J/vA1lMv TCHLYKq0Vfi/FtZHlaWcWz2SuVqwtXbRV1u6kXRfIaRcUZ6CcMOnshylJBRV1d9E NzlqNvNvzsjHqqbpU5PEJcZ3aavwGUHtGmMHl4KWrMvdJvE9fOeEs0BGt+5TnfTQ aSMTXABbCqn9+pFcGdmKdSMOvkLVDCv3cnden9nSjkV8J1NyjDwZiinPGk3djjFV zyE6A4HSgYCLmS+6yYX69HjSzMMUKLSjZ1lV8Occ/SqsbdEbkUbb1vX4iNRfaJY9 Q+vbixfop6/IFvpPXG28Li0Dn5yfdbznj49PfRSJRyeDjFirmBOqL0ESRWXXWhQl TlohNIMoQgbql7tPoHdsxSpaQLdNd0rPVL49Ejd93lHuN2/a7D+u2JoiWjgeBJnl rVmhA+oDHHsR3eMHYPkIq2P2TsHY0+yz9a2l7P+3QwA07TQAucpMIwTr4u/obNNx I1++bqg2e5e3mGxC989SGYijnf3SMNZDyvwadgbEs1lolmVimLtEPhVL5z/EnRjE 1CXLbFgY3LtOKrNL0iEWKfl4Ipd8Ka/VIKa22EqQELQuwVGyusPYTObcTN7WW5Yq +fR2T4fR/QqJCAZgBGK6ziwg2XM0aIInhrM4SrnaNc9LOu6mKEh5yTXEtbmiuCbr Jrchcvmo2tTQP4rj+vTC =EDSh -----END PGP SIGNATURE----- --1giRMj6yz/+FOIRq-- From owner-freebsd-pkgbase@freebsd.org Fri Jan 29 22:03:57 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 29472A73F47 for ; Fri, 29 Jan 2016 22:03:57 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EF2CB1606; Fri, 29 Jan 2016 22:03:56 +0000 (UTC) (envelope-from wollman@khavrinen.csail.mit.edu) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.9/8.14.9) with ESMTP id u0TM3s0P075427 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Fri, 29 Jan 2016 17:03:54 -0500 (EST) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.9/8.14.9/Submit) id u0TM3ra8075424; Fri, 29 Jan 2016 17:03:53 -0500 (EST) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <22187.57801.906140.971947@khavrinen.csail.mit.edu> Date: Fri, 29 Jan 2016 17:03:53 -0500 From: Garrett Wollman To: Glen Barber Cc: freebsd-pkgbase@FreeBSD.org Subject: TODO/wish list for packaging the FreeBSD base system In-Reply-To: <20160129173506.GG1727@FreeBSD.org> References: <20160129173506.GG1727@FreeBSD.org> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Fri, 29 Jan 2016 17:03:54 -0500 (EST) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jan 2016 22:03:57 -0000 < said: > - plist prioritization: pkg(8) needs to be aware of package installation > prioritization (ordering), which is one of the items bapt@ is aware > of. In particular, one of the major requirements of this is the > ordering of rtld, libc, and libthr, which must be handled in that > order. There's also the related issue of start/stop ordering for services. Right now (when HANDLE_RC_SCRIPTS is enabled), every service in every package is considered independent, so it can't handle some fairly common cases like: a) Service B depends on data files installed by package A. Upgrading A should stop (or signal in some other way) service B before ripping the data out from under it, and restart B only after B and all of its dependencies have been upgraded. b) Package X declares two services, A and B, which must be started in that order (and stopped in the reverse order). The rcorder constraints in the scripts ensure that this happens correctly at boot, but when upgrading the package, there's no guarantee of the ordering. (Or maybe there is, but it's not the right one as expressed in the rcorder constraints.) [NB: I haven't actually looked at the code -- does HANDLE_RC_SCRIPTS actually parse the rcorder constraints and handle this already?] c) Services A, B, C, and D all depend on library installed by package L. A security update to L requires that all of those services be restarted. Presently, if updates are applied to the package(s) containing those services, they will get restarted automatically, but there is no way to ensure that every service gets restarted short of making a trivial change to downstream packages to force all to be updated. > - Conflicting packages are non-obvious until attempting to install or > upgrade. In a best case scenario, something like the FreeBSD-acct > package may fail to install. In a worst case scenario, this can be > fatal to the system. When FreeBSD-runtime fails to install, pkg(8) > will rollback the changes, which behaviorally leads to removing all > files included in the package - a dead system. While ZFS is not something we can require, it would be good to have hooks in the system that would allow for, e.g., automatically creating a ZFS snapshot before/after updating certain packages. > TODO List / Wish List: FreeBSD-update as it stands goes to significant lengths to limit the amount of data that must be transferred to install a patch. In particular, it uses bsdiff(1)/bspatch(1) to generate binary patches to avoid transferring complete files. It would be good if pkg(8) had a concept of a "patch package" that can be reversibly applied to an existing package (as stored in the /var/cache/pkg) to transform it into an updated package, which can then be installed. Obviously, reproducible builds would be required to accomplish this. I don't know if pkg(8) can do this already, but having a "quick upgrade" mode that does not update files known (by comparing hashes in the package database) to be unchanged may be worthwhile -- but obviously would need to be optional. -GAWollman From owner-freebsd-pkgbase@freebsd.org Sat Jan 30 02:15:41 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4C5BA727D7 for ; Sat, 30 Jan 2016 02:15:41 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 85C351AF3; Sat, 30 Jan 2016 02:15:41 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by freefall.freebsd.org (Postfix) with ESMTP id 263C2108E; Sat, 30 Jan 2016 02:15:41 +0000 (UTC) (envelope-from gjb@FreeBSD.org) Date: Sat, 30 Jan 2016 02:15:39 +0000 From: Glen Barber To: freebsd-pkgbase@FreeBSD.org Subject: Re: TODO/wish list for packaging the FreeBSD base system Message-ID: <20160130021539.GM1727@FreeBSD.org> References: <20160129173506.GG1727@FreeBSD.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="/TUrtqMIkCP4YtJm" Content-Disposition: inline In-Reply-To: <20160129173506.GG1727@FreeBSD.org> X-Operating-System: FreeBSD 11.0-CURRENT amd64 X-SCUD-Definition: Sudden Completely Unexpected Dataloss X-SULE-Definition: Sudden Unexpected Learning Event X-PEKBAC-Definition: Problem Exists, Keyboard Between Admin/Computer User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jan 2016 02:15:41 -0000 --/TUrtqMIkCP4YtJm Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Note, I did see Garrett's email. Garrett, I'll reply to your questions but following up on one specific point in this list.] On Fri, Jan 29, 2016 at 05:35:06PM +0000, Glen Barber wrote: > TODO List / Wish List: >=20 > [...] >=20 > - Unless there is a specific reason to have an in-tree .ucl > file in src/release/packages/, I would like to see a template used > instead. Use cases where an in-tree .ucl file is needed are > runtime.ucl and kernel.ucl, where post-install scripts are defined. > For the rest of the currently-existing package definitions, it is not > necessary, and adds to the complexity of breaking down the base system > into more granular package sets. I tried a few tests yesterday using > a file called '_template_.ucl', which if the release/packages/foo.ucl > did not exist, the '_template_.ucl' file would be copied to the stage > directory. This did not work, however, because there can be several > '.ucl' files for any given 'package', notably runtime.plist, > debug.plist (which should be runtime-debug.plist), runtime-lib32.plist > and so on. It occurred to me that there is a use case for in-tree .ucl files, specifically when an update is necessary. The package versions come from the PKG_VERSION make(1) environment, and if not set, will default to the epoch timestamp. This was a design decision that was put in place to handle SAs/ENs, where we could trivially bump the package version. Specifically, for an SA/EN, a commit to the tree that would trigger a rebuild of an affected package would look similar to this: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Index: release/packages/rcmds.ucl =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- release/packages/rcmds.ucl (revision 295058) +++ release/packages/rcmds.ucl (working copy) @@ -1,6 +1,6 @@ name =3D "FreeBSD-%PKGNAME%" origin =3D "base" -version =3D "%VERSION%" +version =3D "%VERSION%,1" comment =3D "Remote Command Tools" categories =3D [ base ] maintainer =3D "re@FreeBSD.org" %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% So, while I do not necessarily like relying on in-tree .ucl files, as right now it makes it difficult to package components not "normally" included in the base system, I now remember the necessity of this. It might be a better idea to add a TAGS or PACKAGE_VERSION value to the relevant makefiles, which could be included in the mtree(8) tag when creating the METALOG files. Glen --/TUrtqMIkCP4YtJm Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJWrBzGAAoJEAMUWKVHj+KTFGwQAIQNNYi5pe0nR/TOgFnzZ54e tWTy1R/FumG7Nth9zUyF7+jOqHs1MQIqb6nNh3Z/5miX1TpDUaYljZwNFFbx5iv5 2/mPvpuyxumPVjsn+Kuz6ti845YEQOnpgufgxSq7saNHsMqSTy6FiaPtj6Zor5Of +jvxAUPQeITnBo4O7hFZcsKn3SlDOxuTHwBCeFre+KhsaIuV04hMhkdt7jU9R+br 2cpwGk+yRfM7rp8cqpl0PDF60E0vuvv0GuV3f+MlESgnH+ebMcGJYJkCbfoRN1og 9EUeibz3VEm+X2RaaGPDKwucCDP44cyQ+FhcsPxA2OM8zyAqCLwbgJ1WzWCjeENP ehoXNvrjlFhOpQ11N8ppX/dVuJcO8PgsXCDOidCCtCNgnIypc9qBwUJY2iLcTzkL 9Tw5jfLGtYKcLh1YaC/dvIfjAQbZidqyJAWM/IcErrjT68jnV+gj1TezJ/1H7Fhu hry204jj92AeHfW4hKGjuLMMDn1U9qqg5ZsCOHREKGIATiKSEwPwAnQjRjStxx/n mkUfxocRSdwSzM4UUqCORXDrBgYrGi238UcxsH34mf+OXYROdIQUgjZbqNlQ9inf Cq2eKDdYd9vgaPVAKdjfTOYx7ZTjA0fTyKk/5vSTNmwGZZ6EZbaqp0Fk83+fFx6J IPltYmfroQd0zSaqrs20 =gwAE -----END PGP SIGNATURE----- --/TUrtqMIkCP4YtJm-- From owner-freebsd-pkgbase@freebsd.org Sat Jan 30 09:10:04 2016 Return-Path: Delivered-To: freebsd-pkgbase@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1FA1A72EEC for ; Sat, 30 Jan 2016 09:10:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 836541A07; Sat, 30 Jan 2016 09:10:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1aPRXc-000BA2-Ox; Sat, 30 Jan 2016 12:10:00 +0300 Date: Sat, 30 Jan 2016 12:10:00 +0300 From: Slawa Olhovchenkov To: Glen Barber Cc: freebsd-pkgbase@FreeBSD.org Subject: Re: TODO/wish list for packaging the FreeBSD base system Message-ID: <20160130091000.GL88527@zxy.spb.ru> References: <20160129173506.GG1727@FreeBSD.org> <20160130021539.GM1727@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160130021539.GM1727@FreeBSD.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-pkgbase@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Packaging the FreeBSD base system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Jan 2016 09:10:04 -0000 On Sat, Jan 30, 2016 at 02:15:39AM +0000, Glen Barber wrote: > [Note, I did see Garrett's email. Garrett, I'll reply to your questions > but following up on one specific point in this list.] > > On Fri, Jan 29, 2016 at 05:35:06PM +0000, Glen Barber wrote: > > TODO List / Wish List: > > > > [...] > > > > - Unless there is a specific reason to have an in-tree .ucl > > file in src/release/packages/, I would like to see a template used > > instead. Use cases where an in-tree .ucl file is needed are > > runtime.ucl and kernel.ucl, where post-install scripts are defined. > > For the rest of the currently-existing package definitions, it is not > > necessary, and adds to the complexity of breaking down the base system > > into more granular package sets. I tried a few tests yesterday using > > a file called '_template_.ucl', which if the release/packages/foo.ucl > > did not exist, the '_template_.ucl' file would be copied to the stage > > directory. This did not work, however, because there can be several > > '.ucl' files for any given 'package', notably runtime.plist, > > debug.plist (which should be runtime-debug.plist), runtime-lib32.plist > > and so on. > > It occurred to me that there is a use case for in-tree .ucl > files, specifically when an update is necessary. The package versions > come from the PKG_VERSION make(1) environment, and if not set, will > default to the epoch timestamp. This was a design decision that was put > in place to handle SAs/ENs, where we could trivially bump the package > version. Specifically, for an SA/EN, a commit to the tree that would > trigger a rebuild of an affected package would look similar to this: How this will interaction with customs builds/rebuild/packaging with or w/o source updating? > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > Index: release/packages/rcmds.ucl > =================================================================== > --- release/packages/rcmds.ucl (revision 295058) > +++ release/packages/rcmds.ucl (working copy) > @@ -1,6 +1,6 @@ > name = "FreeBSD-%PKGNAME%" > origin = "base" > -version = "%VERSION%" > +version = "%VERSION%,1" > comment = "Remote Command Tools" > categories = [ base ] > maintainer = "re@FreeBSD.org" > %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% > > So, while I do not necessarily like relying on in-tree .ucl > files, as right now it makes it difficult to package components not > "normally" included in the base system, I now remember the necessity of > this. > > It might be a better idea to add a TAGS or PACKAGE_VERSION value to the > relevant makefiles, which could be included in the mtree(8) tag when > creating the METALOG files. > > Glen >