From owner-freebsd-ports@FreeBSD.ORG Tue Jan 13 08:39:17 2015 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B2BB8599 for ; Tue, 13 Jan 2015 08:39:17 +0000 (UTC) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 406B3AC6 for ; Tue, 13 Jan 2015 08:39:17 +0000 (UTC) Received: by mail-wi0-f170.google.com with SMTP id bs8so18611571wib.1 for ; Tue, 13 Jan 2015 00:39:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=iqxw2N/yWmOgeLbxMbX/xWqgRPT2U/h2hN+b8y9F6S4=; b=veGLXwOjdcHbiCeYbsGOUgWFkLgM2rQ0v+hS+I5v96AO2TwWBM8Ark6RipfrS6TFEp 821oUw7VaURK7cwegQ6DijE621u/BDhUEb3uBg/L5diNrZRdqal0ueDMrVD1FDUHy0Dw xYzPMzYnwngxO3JayYNKTT11FSomFxH9Xn6dRnUFGjhc2dW/93zG6/72XeQVqPpkrmzE wOoyYBQH5r9QQRloaA0gkAONhGxCto41GiFlWadgvFsQC+5olpVVzldKc3ODKQqiIVHy H9mgqSXtL+gbRt5Z+UKXUwJ92Fo9qhduTSkhnGWTlfbLve5vNAZF+owXf9z9N6FK5CaI gj8g== X-Received: by 10.180.11.140 with SMTP id q12mr38138116wib.45.1421138355285; Tue, 13 Jan 2015 00:39:15 -0800 (PST) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by mx.google.com with ESMTPSA id x2sm24764003wjx.9.2015.01.13.00.39.13 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 13 Jan 2015 00:39:14 -0800 (PST) Sender: Baptiste Daroussin Date: Tue, 13 Jan 2015 09:39:12 +0100 From: Baptiste Daroussin To: Stefan Ehmann Subject: Re: poudriere: reduce the number of rebuilt packages? Message-ID: <20150113083911.GG17273@ivaldir.etoilebsd.net> References: <54A67B1A.5060007@gmx.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ulDeV4rPMk/y39in" Content-Disposition: inline In-Reply-To: <54A67B1A.5060007@gmx.net> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Jan 2015 08:39:17 -0000 --ulDeV4rPMk/y39in Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Jan 02, 2015 at 12:03:54PM +0100, Stefan Ehmann wrote: > I've recently switched from portmaster to poudriere/'pkg upgrade' to=20 > manage my port updates. Basically it works fine, but incremental builds= =20 > don't quite work as I expected. >=20 > poudriere rebuilds all packages if any dependency has changed. If there= =20 > are only some ports with new versions, possibly hundreds of packages are= =20 > rebuilt. So far it looks like I'll end up rebuilding packages like=20 > libreoffice/KDE/chromium several times a week. The rebuilt packages=20 > won't even be installed by 'pkg upgrade' because their version number=20 > has not changed. >=20 > That's a huge waste of resources. With portmaster only ports with=20 > increased version numbers are rebuilt. >=20 > Can I use poudriere to rebuild only packages where the version number=20 > changed? >=20 The problem here in consistency while in theory we can cherry-pick what we really want to rebuild based on libraries provided/required in binary packa= ges poudriere has to deal with the ports tree and compatibility. The ports tree was a heavy user of pkg_add which became pkg add, this tool = was relying on the version of dependencies as registered in the package creatio= n: - A-v1 was depending on B-v2 if B-v2 is bumped to B-v3 then A-v1 dependency chain is broken in regard pkg_add. Just for that we have no choices but rebuilding everything that depends on B This can now be "easily" fixed because pkg_install is gone and we do not ha= ve to rely on compatiblity with it anymore, the problem is people willing to work= on that (flexible dependencies and smart dependencies) have been mostly killed= by the nightmare this compatibility has introduced into pkg. Lots of scripts still rely on the pkg_add behaviour and until all of them a= re killed I'm afraid we won't be able to prevent those massive rebuilds. That is if you are doing the things correctly. Now there is an alternative. Introduce a new repository format for file:// kind of url (like Zypper) whi= ch will not need all the metadata and be blazing fast to produce. Use pkg inst= all instead of pkg add in ports and then we can reduce the massive rebuilds to = only rebuild things that really requires a library and only a library being removed/upgraded. Any volunteers? regards, Bapt --ulDeV4rPMk/y39in Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEABECAAYFAlS02a0ACgkQ8kTtMUmk6EyU5QCfR9JlnMtjZe/kOOVhUh/2SPWL /tsAn1QmvtBmwHwEHfguO11/KG7g+hTF =Hjbk -----END PGP SIGNATURE----- --ulDeV4rPMk/y39in--