From owner-freebsd-ports@freebsd.org Wed Oct 28 07:22:39 2015 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4AF72A1F4B4 for ; Wed, 28 Oct 2015 07:22:39 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 27C6A12F3 for ; Wed, 28 Oct 2015 07:22:39 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 274B3A1F4B3; Wed, 28 Oct 2015 07:22:39 +0000 (UTC) Delivered-To: ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C267A1F4B2 for ; Wed, 28 Oct 2015 07:22:39 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: from mail-wi0-x22a.google.com (mail-wi0-x22a.google.com [IPv6:2a00:1450:400c:c05::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A263B12F2 for ; Wed, 28 Oct 2015 07:22:38 +0000 (UTC) (envelope-from baptiste.daroussin@gmail.com) Received: by wicll6 with SMTP id ll6so187868852wic.0 for ; Wed, 28 Oct 2015 00:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=f7oJbHH80gkDfBA9Nn4JXmOGZwpKI0utWKwiVMRJpt0=; b=wdqKzfSTJlmnOzeNJxyImEXw4YPW2yXrD3bwFNGuix8Gr9pz5lzfdAzz5MVgcbcg4Y wRRNCRJawcSxb3tyFUnQGoUeuqjidhw9MemM7XCoQFi+jXZXMyV6b2fB5xlv2VNHiQCO UNyS7qV/bPsNwEQoHseHahq+UHGpKoAweH/0b/vHVFZTDsTaeCsZC3b3e1gBqMtOQZTZ /z8vUXkWlvh++NQi0r1lfpnT5nAtVUilXlHExBHKuarz6AD/lcFeS50sDBI5NcGdUdk2 Eb1xIzgK48ioyrPVn5soNznSGtaRHHmAn6KpxitPXyJUvDIUip+DeXQYbfAsGm9Kj/T5 OJ2A== X-Received: by 10.194.186.238 with SMTP id fn14mr34341639wjc.128.1446016956784; Wed, 28 Oct 2015 00:22:36 -0700 (PDT) Received: from ivaldir.etoilebsd.net ([2001:41d0:8:db4c::1]) by smtp.gmail.com with ESMTPSA id q1sm48581369wje.39.2015.10.28.00.22.35 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Oct 2015 00:22:35 -0700 (PDT) Sender: Baptiste Daroussin Date: Wed, 28 Oct 2015 08:22:34 +0100 From: Baptiste Daroussin To: soralx@cydem.org Cc: ports@FreeBSD.ORG Subject: Re: Speedup ports install Message-ID: <20151028072233.GM17025@ivaldir.etoilebsd.net> References: <20151027220549.2bb7a660@sol> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EOHJn1TVIJfeVXv2" Content-Disposition: inline In-Reply-To: <20151027220549.2bb7a660@sol> User-Agent: Mutt/1.5.24 (2015-08-30) X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 07:22:39 -0000 --EOHJn1TVIJfeVXv2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Oct 27, 2015 at 10:05:49PM -0700, soralx@cydem.org wrote: >=20 > Howdy! >=20 > I have a suggestion: to significantly speedup the packaging phase > when installing ports, pass '-f tgz' flag to `pkg create`: > --- Mk/bsd.commands.mk (revision 400371) > +++ Mk/bsd.commands.mk (working copy) > @@ -126,7 +126,7 @@ > PKG_DELETE?=3D ${PKG_BIN} delete -y > PKG_INFO?=3D ${PKG_BIN} info -g > PKG_VERSION?=3D ${PKG_BIN} version > -PKG_CREATE?=3D ${PKG_BIN} create > +PKG_CREATE?=3D ${PKG_BIN} create -f tgz > PKG_ADD?=3D ${PKG_BIN} add > PKG_QUERY?=3D ${PKG_BIN} query >=20 > This way, gzip will be used instead of default xz (which > is very slow). The extra compression of xz is not needed, > as the package file will be deleted shortly anyway. >=20 > Same idea applies to portmaster. Change=20 > pkg_create=3D"pkg create " > to > pkg_create=3D"pkg create -f tgz " > on line 1916 of 'portmaster' script. >=20 > Ports install a lot faster this way, even on a machine with a fast CPU > (especially noticeable when doing portupgrade with lots of small ports). >=20 > Example. [root@soralx /usr/ports/science/paraview]# time make package > With xz: > =3D=3D=3D> Building package for paraview-4.3.1_1 > real 1m41.120s > user 1m40.070s > sys 0m1.089s > With gzip: > =3D=3D=3D> Building package for paraview-4.3.1_1 > real 0m15.931s > user 0m15.010s > sys 0m0.925s > Note that `make package` will still produce xz-compressed file after > the flag is added to "PKG_CREATE"; I changed the behaviour only for > this example, to demonstrate the time difference (factor of ~6.7). >=20 When installing a port there is no packaging phase. pkg register which is u= sed during make install uses the stage directory "as an archive" directly so th= ere is no compression at all. portupgrade/portmaster is another storry, IIRC portmaster only creates pack= ages as backup of the installed version just in case the new package will fail i= t can reinstall the old one in that case, yes a tgz would be way faster, you can = even chose plain tar to even more reduce the over head. for portupgrade I do not remember what it does. Note that the backup thing of portmaster could now possibly be removed since the ports tree support staging because now the po= rts tree checks that everything is ok before installing on the system, but that would require even more refactoring. Bapt --EOHJn1TVIJfeVXv2 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlYwd7kACgkQ8kTtMUmk6Exr5wCdGcwoY1XD1uIAiI6IzV9PubKp bvkAn0kAc1aK27Ncc6/NrwEKAd08zw3H =wnzA -----END PGP SIGNATURE----- --EOHJn1TVIJfeVXv2--