From owner-freebsd-ports-announce@FreeBSD.ORG Mon Jan 30 12:39:35 2012 Return-Path: Delivered-To: ports-announce@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0CD65106564A; Mon, 30 Jan 2012 12:39:35 +0000 (UTC) (envelope-from bapt@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E95B98FC13; Mon, 30 Jan 2012 12:39:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q0UCdYM9095231; Mon, 30 Jan 2012 12:39:34 GMT (envelope-from bapt@FreeBSD.org) Received: (from bapt@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q0UCdYdf095230; Mon, 30 Jan 2012 12:39:34 GMT (envelope-from bapt@FreeBSD.org) X-Authentication-Warning: freefall.freebsd.org: bapt set sender to bapt@FreeBSD.org using -f Date: Mon, 30 Jan 2012 13:39:30 +0100 From: Baptiste Daroussin To: current@FreeBSD.org, ports@FreeBSD.org, ports-announce@FreeBSD.org Message-ID: <20120130123930.GB40244@azathoth.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="3lcZGd9BuhuYXNfi" Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) X-Mailman-Approved-At: Mon, 30 Jan 2012 12:44:26 +0000 Cc: Subject: [FreeBSD-Ports-Announce] [HEADSUP][CFT] pkgng beta1 is out X-BeenThere: freebsd-ports-announce@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Important news and instructions about the FreeBSD " ports collection" \[moderated\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 12:39:35 -0000 --3lcZGd9BuhuYXNfi Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, pkgng has just reached the beta phase, and has now found its way to the ports tree (disabled by default). 1/ Why pkgng? ------------ Our current pkg_install tools are showing their age, are hard to maintain, and they lack features: - missing metadata - no upgrade support - no repository support - no fine dependency tracking - no modern binary package management - and many others Having old tools makes it hard to improve the ports infrastructure, as a result lots of hacks have found their way into the different Mk/bsd.*.mk files to work around pkg_install limitations plus there are lots of hacks in the packages metadata itself such as @comment which are not comments, and so forth. We have people writing tools to improve the situation (portmaster and portupgrade to name two), but they are limited by and can become quite complicated to maintain because of the pkg_install limitations. 2/ What it is? -------------- It is a tool that is designed to replace pkg_install and provide modern features to advance package management on FreeBSD. It has been done with compatibility in mind. Most of the ports tree are able to build on pkgng without modification (21500 successful packages is the highest pkgng score so far). The missing ones will be easily fixed with pkgng in ports. It has been done with ease of migration in mind. It is easy to migrate =66rom pkg_install to pkgng. (Please note that going backwards is not possible.) It has been done with FreeBSD features in mind: it supports chroot, jails, rcng, etc. It has been done with scripting features in mind: 'pkg query' will allow you to query almost everything from the pkgng database in a script friendly way. It has been done with improvement in mind: it doesn't require a privileged account to create packages with root files in it; it is already able to package from a stage/fakeroot/name_it_like_you_want directory; it is also able to fake the package creation to directly install the package from that fake/stage/whatever directory. It has been done with human readability in mind: the new metadata is stored in YAML format; the plist keywords can be extended with YAML (for the ports). It has been our thinking that the pkg binary is not able to please everyone's needs, so it has been written on top of a library which can be used by any other third party tools. (Think about packagekit, or ruby binding for portupgrade for example, or any other usage like these). pkgng is the result of my long studies and reflection about packaging (studying what is done elsewhere: apt/dpkg, yum/rpm, pacman, aix, solaris, netbsd, openbsd) and how to have something that tries to take the good ideas from them, but tries not to take the *over engineered* complicated parts. And most importantly, tries to do it the FreeBSD way: which means it should work with the ports tree as-is (and help improve it in the future= ). 3/ Roadmap ---------- We plan a very long beta phase with lots of beta versions, released as often as possible to ease testing and help improve the tool as much as possible. The goal, now that we are in beta is to not break anything for users, which means that pkgng will be able to safely upgrade itself. (No real breakage occurred during the alpha phase; expect even less in beta.) Most of the big features are implemented, so now if you have a revolutionary idea that breaks everything, it won't find its way into pkgng 1.0. You can still provide it for pkgng 2.0. 1.0 is not revolutionary because of the way that it is full of workarounds to allow compatibility with the current ports tree. At some future time (TBD), once we have dropped pkg_install support, things will be able to move forward faster. pkgng will live in the ports tree, so it will evolve with the infrastructure, allowing us not to have to wait for the EOL of a release to be able to move forward to new features. The library API is currently not considered stable; it will be designated stable as of pkgng 2.0. Therefore, if you are going to use the library in a third party project, you can expect some breakage from time to time. Of course, we will avoid breakage as much as possible. The plan is to have pkgng 1.0 ready and rock solid for 10.0-RELEASE and 9.1-RELEASE. The more testers/contributors we have, the faster we can go, and the faster we go, the faster we can drop pkg_install and improve our port infrastructure. (Note: due to limitations in FreeBSD 7.x, we do not plan to backport there.) 4/ pkgng itself ---------------- pkg add: add packages the old way (should be avoided by users) pkg audit: audit the installed packages for vulnerabilities pkg autoremove: interactively propose packages to be removed that were installed automatically (as a dependency) and not depended on anymore pkg check: check the installed packages database, prompting for inconsistency and proposing to try to fix it pkg clean: cleanup the package cache from binary installation (from repositories) pkg delete: deinstall packages pkg info: query information from the installed packages in a user-friendly way pkg install: install packages from a remote repository pkg query: query information from the installed packages in a script-friendly way pkg register: register packages in the database, synchronising the files =66rom a fake/stage directory, or with already installed files (like currently with ports) pkg search: search the remote database for packages pkg update: update the remote repositories databases pkg updating: scan the installed ports and show all UPDATING entries that affect one of the installed ports (same as old pkg_updating) pkg upgrade: perform a full binary upgrade of the installed packages pkg version: determine whether package(s) need to be updated (same as old pkg_version) pkg which: determine which package owns a file pkg2ng: convert a pkg_install installed database to a pkgng installed database (you would need the ports tree for pkg2ng to be able to gather missing information about packages) Sample output of pkg info: $ pkg info -f libreoffice: Name : libreoffice Version : 3.4.4 Origin : editors/libreoffice Prefix : /usr/local Categories : editors Licenses : MPL & LGPL3 Maintainer : office@FreeBSD.org WWW : http://www.libreoffice.org/ Comment : Full integrated office productivity suite Options : DEBUG: off GNOME: off GTK: on JAVA: off KDE4: off MMEDIA: off PYUNO: off SDK: off SYSTRAY: off WEBDAV: off Flat size : 319 MB Description : LibreOffice is the free power-packed Open Source personal productivity suit= e for Windows, Macintosh and Linux, that gives you six feature-rich applications = for all your document production and data processing needs: Writer, Calc, Impre= ss, Draw, Math and Base. WWW: http://www.libreoffice.org/ 5/ what is missing ------------------ - for 1.0: * currently the user handling is done using the @exec/@unexec scripts from the ports; we need to finish the switch to the pw_/gr_util API to have cleaner handling. * better error reporting; lots of error/warning messages are currently technical and need to be improved to become more user-friendly. * better documentation; pkgng does a lot of things (more than what is described here) and needs to be documented. We lack enough native English speakers that are able to correctly document everything. * bug hunting and fixing. - for future: * capsicum: during EuroBSDCon some ideas were shared on how we can capsicumize pkgng, and more generally, increase security in package management; we need to have more thought about this subject. * improved arch support; currently pkgng is able to prevent installing amd64 packages into an arm box for examples, but we can go further and have real arch handling -- packaging noarch packages only once for all architectures (shell scripts, data, etc) and share them between the repository. This would reduce the size of the repositories and speed up the packages building process. * incrementally updated packages (diff packages): this is a really complicated task but could be done, would need a clean design. * abstract/alternative packages (e.g. "provide http_server") * having a real sat solver for the dependency tree. Currently we have a really simple and minimalistic solver which works well but if we can to go to an even finer package management we would need a real solver. * many more to join the project to share your ideas. Keep in mind that in pkgng, every thing should be and will remain simple, so when you come up with ideas, try providing a simple and clean design :) to use pkgng: echo WITH_PKGNG=3Dyes >> /etc/make.conf make -C /usr/ports/ports-mgmt/pkg install clean Some links:=20 http://wiki.FreeBSD.org/pkgng http://github.com/pkgng/pkgng Note that on github you can find a patch for portmaster (against 3.10) On behalf of the pkgng team Bapt --3lcZGd9BuhuYXNfi Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAk8mj4IACgkQ8kTtMUmk6Exq2ACffqfOUVo/pz4MpmTAjsjMDdHr q28AoJQ7sBHyafRxjQXC4VD3XM7smTDk =nTG+ -----END PGP SIGNATURE----- --3lcZGd9BuhuYXNfi-- From owner-freebsd-ports-announce@FreeBSD.ORG Mon Jan 30 17:11:41 2012 Return-Path: Delivered-To: ports-announce@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20AD71065673; Mon, 30 Jan 2012 17:11:41 +0000 (UTC) (envelope-from jos@catnook.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id B6BC18FC1B; Mon, 30 Jan 2012 17:11:40 +0000 (UTC) Received: by ghbg15 with SMTP id g15so2062287ghb.13 for ; Mon, 30 Jan 2012 09:11:40 -0800 (PST) MIME-Version: 1.0 Received: by 10.50.202.97 with SMTP id kh1mr17789218igc.19.1327941838476; Mon, 30 Jan 2012 08:43:58 -0800 (PST) Received: by 10.42.218.7 with HTTP; Mon, 30 Jan 2012 08:43:58 -0800 (PST) In-Reply-To: <20120130123930.GB40244@azathoth.lan> References: <20120130123930.GB40244@azathoth.lan> Date: Mon, 30 Jan 2012 08:43:58 -0800 Message-ID: From: Jos Backus To: Baptiste Daroussin X-Mailman-Approved-At: Mon, 30 Jan 2012 17:33:50 +0000 Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: ports@freebsd.org, ports-announce@freebsd.org, current@freebsd.org Subject: [FreeBSD-Ports-Announce] Re: [HEADSUP][CFT] pkgng beta1 is out X-BeenThere: freebsd-ports-announce@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Important news and instructions about the FreeBSD " ports collection" \[moderated\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2012 17:11:41 -0000 Hi Baptiste, This looks great! On Mon, Jan 30, 2012 at 4:39 AM, Baptiste Daroussin wrote: > > Sample output of pkg info: > > $ pkg info -f libreoffice: > Name : libreoffice > Version : 3.4.4 > Origin : editors/libreoffice > Prefix : /usr/local > Categories : editors > Licenses : MPL & LGPL3 > Maintainer : office@FreeBSD.org > WWW : http://www.libreoffice.org/ > Comment : Full integrated office productivity suite > Options : > DEBUG: off > GNOME: off > GTK: on > JAVA: off > KDE4: off > MMEDIA: off > PYUNO: off > SDK: off > SYSTRAY: off > WEBDAV: off > Flat size : 319 MB > Description : > LibreOffice is the free power-packed Open Source personal productivity > suite for > Windows, Macintosh and Linux, that gives you six feature-rich applications > for > all your document production and data processing needs: Writer, Calc, > Impress, > Draw, Math and Base. > > WWW: http://www.libreoffice.org/ > > I haven't checked if `pkg query' can do this yet, but how about emitting the above output in YAML as well? It would still be very readable for humans. Anyway, I'll try this out. Jos -- Jos Backus jos at catnook.com