From owner-freebsd-ports@freebsd.org Fri Apr 8 20:22:43 2016 Return-Path: Delivered-To: freebsd-ports@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CED64B0992D for ; Fri, 8 Apr 2016 20:22:43 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx1.enfer-du-nord.net (mx1.enfer-du-nord.net [IPv6:2001:41d0:1008:bcb:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9CD6216AA; Fri, 8 Apr 2016 20:22:43 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e04:5601:ede3:9a2:4cc5:697a] (p2003008C2E045601EDE309A24CC5697A.dip0.t-ipconnect.de [IPv6:2003:8c:2e04:5601:ede3:9a2:4cc5:697a]) by mx1.enfer-du-nord.net (Postfix) with ESMTPSA id 3qhW9D4XXMzTWt; Fri, 8 Apr 2016 22:22:40 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: pkg-1.7.0 is an order of magnitude slower than pkg-1.6.4 From: Michael Grimm In-Reply-To: <20160407064336.GR49864@ivaldir.etoilebsd.net> Date: Fri, 8 Apr 2016 22:22:39 +0200 Cc: freebsd-ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <7A204031-8AD1-4988-ACAD-988386AFF92D@ellael.org> References: <6DD156C6-C3E1-43BC-8EC5-1ACB16EBFC3E@ellael.org> <20160402124845.GM1128@ivaldir.etoilebsd.net> <05DDCF70-8865-4F1F-A264-3000B8EF5244@ellael.org> <20160407064336.GR49864@ivaldir.etoilebsd.net> To: Baptiste Daroussin X-Virus-Scanned: clamav-milter 0.99.1 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Apr 2016 20:22:43 -0000 Baptiste Daroussin wrote: >=20 > On Sat, Apr 02, 2016 at 02:59:06PM +0200, Michael Grimm wrote: >> Baptiste Daroussin wrote: >>> On Sat, Apr 02, 2016 at 02:42:06PM +0200, Michael Grimm wrote: >>>> 26 seconds for 74 ports within a jail and pkg-1.6.4: >> [=E2=80=A6] >>>> 309 seconds for the very same 74 ports within the very same jail = and pkg-1.7.0: >> [=E2=80=A6] >>>> Is this an expected slow-down? /usr/ports/UPGRADE and = https://svnweb.freebsd.org/ports/head/ports-mgmt/pkg/?view=3Dlog are not = indicating that behavior. >>>> But I might have missed something. >>>>=20 >>>> Any feedback is highly appriciated, thanks, and regards, >>>=20 >>> pkg 1.7 is IO intensive that may explain. >>=20 >> Ok, understood. >>=20 >> JFTR: perl (24s), python27 (44s), and ruby (125s) take the longest = time to reinstall. >>=20 >>> I plan to readd some improvements on this side before 1.8 >>=20 >> Good to know, thanks for your feedback. >=20 > Just to follow up on the performance issue, there is a regression that = happened > on FreeBSD 10.3-RELEASE (also HEAD) that causes pkg extraction process = to be 10 > times slower as it should. r297626 fixes it in head. We are working on = bringing > that into the 10 branch: > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D297626 Thanks. I added the relevant patch [1] to 10.3-STABLE (r297721) and I = can confirm that that fixed the performance issue.=20 Now, with pkg-1.7.2 for the very same 74 ports and jail reinstallation = is back to 26 seconds: Apr 8 20:04:42 pkg: indexinfo reinstalled: 0.2.4 = -> 0.2.4=20 ... Apr 8 20:05:08 pkg: pkg reinstalled: 1.7.2 -> = 1.7.2=20 Up to now, 10.3-STABLE runs rock solid. I will report if that might = change. Thanks for your engagement and with kind regards, Michael [1] = https://svnweb.freebsd.org/base/head/lib/libc/db/hash/hash.c?view=3Dpatch&= r1=3D297626&r2=3D297625&pathrev=3D297626=20