From owner-freebsd-ports@FreeBSD.ORG Wed Mar 26 16:18:52 2008 Return-Path: Delivered-To: freebsd-ports@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCE86106566C for ; Wed, 26 Mar 2008 16:18:52 +0000 (UTC) (envelope-from pav@FreeBSD.org) Received: from nat-application.b1.lan.prg.vol.cz (nat-application.b1.lan.prg.vol.cz [195.122.204.152]) by mx1.freebsd.org (Postfix) with ESMTP id 1A8328FC26 for ; Wed, 26 Mar 2008 16:18:51 +0000 (UTC) (envelope-from pav@FreeBSD.org) Received: from pav.hide.vol.cz (localhost [127.0.0.1]) by nat-application.b1.lan.prg.vol.cz (8.14.2/8.14.2) with ESMTP id m2QGIUYV093914; Wed, 26 Mar 2008 17:18:30 +0100 (CET) (envelope-from pav@FreeBSD.org) Received: (from pav@localhost) by pav.hide.vol.cz (8.14.2/8.14.2/Submit) id m2QGITVk093913; Wed, 26 Mar 2008 17:18:29 +0100 (CET) (envelope-from pav@FreeBSD.org) X-Authentication-Warning: pav.hide.vol.cz: pav set sender to pav@FreeBSD.org using -f From: Pav Lucistnik To: freebsd-ports@FreeBSD.org Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-vC/DqVTgUoEBp98Zlv9h" Date: Wed, 26 Mar 2008 17:18:29 +0100 Message-Id: <1206548309.78560.69.camel@pav.hide.vol.cz> Mime-Version: 1.0 X-Mailer: Evolution 2.12.3 FreeBSD GNOME Team Port Cc: rdivacky@FreeBSD.org, soralx@cydem.org Subject: [patch] pkg_delete(1) speedup X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pav@FreeBSD.org List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2008 16:18:52 -0000 --=-vC/DqVTgUoEBp98Zlv9h Content-Type: multipart/mixed; boundary="=-gy6w4Z4DMzOlLo/sf16H" --=-gy6w4Z4DMzOlLo/sf16H Content-Type: text/plain Content-Transfer-Encoding: quoted-printable You might have noticed a thread on the mailing list called "ports system woes". The submitter pointed out an inefficiency in pkg_delete routine, that parses the whole /var/db/pkg over and over again for every dependency of a package being removed. Attached is a patch by rdivacky that implements the idea of looking up all the values in a single pass over /var/db/pkg content. A trivial benchmark, wall times: port with two dependencies (comms/obexapp) before 0.083s after 0.049s port with 172 dependencies (graphics/agave) before 8.404s 8.955s 11.734s after 2.816s 2.690s 3.195s The patch is not WARNS clean, that needs to be fixed. And I'd like a review by someone with src commit bit. --=20 Pav Lucistnik 42.7 percent of all statistics are made up on the spot. --=-gy6w4Z4DMzOlLo/sf16H Content-Disposition: attachment; filename=pkg_delete.patch Content-Type: text/x-patch; name=pkg_delete.patch; charset=ISO8859-2 Content-Transfer-Encoding: base64 SW5kZXg6IGRlbGV0ZS9wZXJmb3JtLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvaG9tZS9uY3Zz L3NyYy91c3Iuc2Jpbi9wa2dfaW5zdGFsbC9kZWxldGUvcGVyZm9ybS5jLHYNCnJldHJpZXZpbmcg cmV2aXNpb24gMS40MQ0KZGlmZiAtYSAtdSAtcjEuNDEgcGVyZm9ybS5jDQotLS0gZGVsZXRlL3Bl cmZvcm0uYwkyOSBKdW4gMjAwNCAxOTowNjo0MSAtMDAwMAkxLjQxDQorKysgZGVsZXRlL3BlcmZv cm0uYwkyNiBNYXIgMjAwOCAxNTo0NzoyOSAtMDAwMA0KQEAgLTEyMywxNiArMTIzLDIwIEBADQog ew0KICAgICBGSUxFICpjZmlsZTsNCiAgICAgY2hhciAqZGVwb3JpZ2luLCAqKmRlcG5hbWVzLCBo b21lW0ZJTEVOQU1FX01BWF07DQorICAgIGNoYXIgKip1bmRpcmVjdF9kZXBzOw0KICAgICBQYWNr aW5nTGlzdCBwOw0KICAgICBpbnQgaSwgbGVuOw0KICAgICBpbnQgaXNpbnN0YWxsZWQ7DQogICAg IC8qIHN1cHBvcnQgZm9yIHNlcGFyYXRlIHByZS9wb3N0IGluc3RhbGwgc2NyaXB0cyAqLw0KLSAg ICBpbnQgbmV3X20gPSAwOw0KKyAgICBpbnQgbmV3X20gPSAwLCB1ZF9jb3VudDsNCiAgICAgY29u c3QgY2hhciAqcHJlX3NjcmlwdCA9IERFSU5TVEFMTF9GTkFNRTsNCiAgICAgY29uc3QgY2hhciAq cG9zdF9zY3JpcHQsICpwcmVfYXJnLCAqcG9zdF9hcmc7DQogICAgIHN0cnVjdCByZXFyX2J5X2Vu dHJ5ICpyYl9lbnRyeTsNCiAgICAgc3RydWN0IHJlcXJfYnlfaGVhZCAqcmJfbGlzdDsNCiANCisg ICAgZGVwbmFtZXMgPSBOVUxMOw0KKyAgICB1bmRpcmVjdF9kZXBzID0gTlVMTDsNCisNCiAgICAg aWYgKCFwa2cgfHwgIShsZW4gPSBzdHJsZW4ocGtnKSkpDQogCXJldHVybiAxOw0KICAgICBpZiAo cGtnW2xlbiAtIDFdID09ICcvJykNCkBAIC0yNjMsNiArMjY3LDcgQEANCiAJfQ0KICAgICB9DQog DQorICAgIHVkX2NvdW50ID0gMDsNCiAgICAgZm9yIChwID0gUGxpc3QuaGVhZDsgcCA7IHAgPSBw LT5uZXh0KSB7DQogCWlmIChwLT50eXBlICE9IFBMSVNUX1BLR0RFUCkNCiAJICAgIGNvbnRpbnVl Ow0KQEAgLTI3NSwxOCArMjgwLDI4IEBADQogCSAgICBwcmludGYoIi5cbiIpOw0KIAl9DQogCWlm ICghRmFrZSkgew0KLQkgICAgZGVwbmFtZXMgPSAoZGVwb3JpZ2luICE9IE5VTEwpID8gbWF0Y2hi eW9yaWdpbihkZXBvcmlnaW4sIE5VTEwpIDoNCi0JCQkJCSAgICAgTlVMTDsNCi0JICAgIGlmIChk ZXBuYW1lcyA9PSBOVUxMKSB7DQotCQlkZXBuYW1lcyA9IGFsbG9jYShzaXplb2YoKmRlcG5hbWVz KSAqIDIpOw0KLQkJZGVwbmFtZXNbMF0gPSBwLT5uYW1lOw0KLQkJZGVwbmFtZXNbMV0gPSBOVUxM Ow0KKwkgICAgaWYgKGRlcG9yaWdpbiA9PSBOVUxMKSB7DQorCQl1bmRlcGVuZChwLT5uYW1lLCBw a2cpOw0KKwkgICAgfSBlbHNlIHsNCisJICAgICAgIHVuZGlyZWN0X2RlcHMgPSByZWFsbG9jKHVu ZGlyZWN0X2RlcHMsICh1ZF9jb3VudCArIDIpICogc2l6ZW9mKCp1bmRpcmVjdF9kZXBzKSk7DQor CSAgICAgICB1bmRpcmVjdF9kZXBzW3VkX2NvdW50XSA9IGRlcG9yaWdpbjsNCisJICAgICAgIHVu ZGlyZWN0X2RlcHNbdWRfY291bnQgKyAxXSA9IE5VTEw7DQorCSAgICAgICB1ZF9jb3VudCsrOw0K IAkgICAgfQ0KLQkgICAgZm9yIChpID0gMDsgZGVwbmFtZXNbaV0gIT0gTlVMTDsgaSsrKQ0KLQkJ dW5kZXBlbmQoZGVwbmFtZXNbaV0sIHBrZyk7DQogCX0NCiAgICAgfQ0KIA0KKyAgICBpZiAodW5k aXJlY3RfZGVwcyAhPSBOVUxMKSB7DQorCWRlcG5hbWVzID0gbWF0Y2hhbGxieW9yaWdpbih1bmRp cmVjdF9kZXBzLCBOVUxMKTsNCisNCisJZnJlZSh1bmRpcmVjdF9kZXBzKTsNCisNCisJLyogVW5k ZXBlbmQgYWxsIHRoZSBkZXBlbmRhbmNpZXMgYXQgb25jZSAqLw0KKwlmb3IgKGkgPSAwOyBkZXBu YW1lc1tpXSAhPSBOVUxMOyBpKyspDQorCSAgIHVuZGVwZW5kKGRlcG5hbWVzW2ldLCBwa2cpOw0K Kw0KKyAgICB9DQorDQogICAgIGlmIChjaGRpcihob21lKSA9PSBGQUlMKSB7DQogCWNsZWFudXAo MCk7DQogCWVycngoMiwgIiVzOiB1bmFibGUgdG8gcmV0dXJuIHRvIHdvcmtpbmcgZGlyZWN0b3J5 ICVzISIsIF9fZnVuY19fLA0KSW5kZXg6IGxpYi9saWIuaA0KPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZpbGU6 IC9ob21lL25jdnMvc3JjL3Vzci5zYmluL3BrZ19pbnN0YWxsL2xpYi9saWIuaCx2DQpyZXRyaWV2 aW5nIHJldmlzaW9uIDEuNTYuMi4yDQpkaWZmIC1hIC11IC1yMS41Ni4yLjIgbGliLmgNCi0tLSBs aWIvbGliLmgJMTQgTWF5IDIwMDYgMDc6MDY6MzcgLTAwMDAJMS41Ni4yLjINCisrKyBsaWIvbGli LmgJMjYgTWFyIDIwMDggMTU6NDc6MjkgLTAwMDANCkBAIC0yMTYsNiArMjE2LDcgQEANCiAvKiBR dWVyeSBpbnN0YWxsZWQgcGFja2FnZXMgKi8NCiBjaGFyCQkqKm1hdGNoaW5zdGFsbGVkKG1hdGNo X3QsIGNoYXIgKiosIGludCAqKTsNCiBjaGFyCQkqKm1hdGNoYnlvcmlnaW4oY29uc3QgY2hhciAq LCBpbnQgKik7DQorY2hhcgkJKiptYXRjaGFsbGJ5b3JpZ2luKGNvbnN0IGNoYXIgKiosIGludCAq KTsNCiBpbnQJCWlzaW5zdGFsbGVkcGtnKGNvbnN0IGNoYXIgKm5hbWUpOw0KIGludAkJcGF0dGVy bl9tYXRjaChtYXRjaF90IE1hdGNoVHlwZSwgY2hhciAqcGF0dGVybiwgY29uc3QgY2hhciAqcGtn bmFtZSk7DQogDQpJbmRleDogbGliL21hdGNoLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvaG9t ZS9uY3ZzL3NyYy91c3Iuc2Jpbi9wa2dfaW5zdGFsbC9saWIvbWF0Y2guYyx2DQpyZXRyaWV2aW5n IHJldmlzaW9uIDEuMTkuOC4yDQpkaWZmIC1hIC11IC1yMS4xOS44LjIgbWF0Y2guYw0KLS0tIGxp Yi9tYXRjaC5jCTEwIE5vdiAyMDA3IDIyOjA0OjMxIC0wMDAwCTEuMTkuOC4yDQorKysgbGliL21h dGNoLmMJMjYgTWFyIDIwMDggMTU6NDc6MjkgLTAwMDANCkBAIC0yMzgsMTAgKzIzOCwxMCBAQA0K ICAqIGFzIGEga2V5IGZvciBtYXRjaGluZyBwYWNrYWdlcy4NCiAgKi8NCiBjaGFyICoqDQotbWF0 Y2hieW9yaWdpbihjb25zdCBjaGFyICpvcmlnaW4sIGludCAqcmV0dmFsKQ0KK21hdGNoYWxsYnlv cmlnaW4oY29uc3QgY2hhciAqKm9yaWdpbnMsIGludCAqcmV0dmFsKQ0KIHsNCiAgICAgY2hhciAq Kmluc3RhbGxlZDsNCi0gICAgaW50IGk7DQorICAgIGludCBpLCBqOw0KICAgICBzdGF0aWMgc3Ry dWN0IHN0b3JlICpzdG9yZSA9IE5VTEw7DQogDQogICAgIHN0b3JlID0gc3RvcmVjcmVhdGUoc3Rv cmUpOw0KQEAgLTI5MCw4ICsyOTAsOSBAQA0KIAkJY29udGludWU7DQogCSAgICBjbWQgPSBwbGlz dF9jbWQodG1wICsgMSwgJmNwKTsNCiAJICAgIGlmIChjbWQgPT0gUExJU1RfT1JJR0lOKSB7DQot CQlpZiAoY3NoX21hdGNoKG9yaWdpbiwgY3AsIEZOTV9QQVRITkFNRSkgPT0gMCkNCi0JCSAgICBz dG9yZWFwcGVuZChzdG9yZSwgaW5zdGFsbGVkW2ldKTsNCisJCWZvciAoaiA9IDA7IG9yaWdpbnNb al0gIT0gTlVMTDsgaisrKQ0KKwkJICAgaWYgKGNzaF9tYXRjaChvcmlnaW5zW2pdLCBjcCwgRk5N X1BBVEhOQU1FKSA9PSAwKQ0KKwkJICAgICAgc3RvcmVhcHBlbmQoc3RvcmUsIGluc3RhbGxlZFtp XSk7DQogCQlicmVhazsNCiAJICAgIH0NCiAJfQ0KQEAgLTMwNyw2ICszMDgsMjUgQEANCiB9DQog DQogLyoNCisgKiBTeW5vcHNpcyBpcyBzaW1pbGFyIHRvIG1hdGNoaW5zdGFsbGVkKCksIGJ1dCB1 c2Ugb3JpZ2luDQorICogYXMgYSBrZXkgZm9yIG1hdGNoaW5nIHBhY2thZ2VzLg0KKyAqLw0KK2No YXIgKioNCittYXRjaGJ5b3JpZ2luKGNvbnN0IGNoYXIgKm9yaWdpbiwgaW50ICpyZXR2YWwpDQor ew0KKyAgIGNoYXIgKipvcmlnaW5zOw0KKyAgIGNoYXIgKipkZXBzOw0KKw0KKyAgIG9yaWdpbnMg PSBtYWxsb2Moc2l6ZW9mKCpvcmlnaW5zKSAqIDIpOw0KKyAgIG9yaWdpbnNbMF0gPSBvcmlnaW47 DQorICAgb3JpZ2luc1sxXSA9IE5VTEw7DQorDQorICAgZGVwcyA9IG1hdGNoYWxsYnlvcmlnaW4o b3JpZ2lucywgcmV0dmFsKTsNCisgICBmcmVlKG9yaWdpbnMpOw0KKyAgIHJldHVybiBkZXBzOw0K K30NCisNCisvKg0KICAqIFNtYWxsIGxpbmtlZCBsaXN0IHRvIG1lbW9pemUgcmVzdWx0cyBvZiBp c2luc3RhbGxlZHBrZygpLiAgQSBoYXNoIHRhYmxlDQogICogd291bGQgYmUgZmFzdGVyIGJ1dCBm b3IgbiB+PSAxMDAwIG1heSBiZSBvdmVya2lsbC4NCiAgKi8NCg== --=-gy6w4Z4DMzOlLo/sf16H-- --=-vC/DqVTgUoEBp98Zlv9h Content-Type: application/pgp-signature; name=signature.asc Content-Description: Toto je =?UTF-8?Q?digit=C3=A1ln=C4=9B?= =?ISO-8859-1?Q?_podepsan=E1?= =?UTF-8?Q?_=C4=8D=C3=A1st?= =?ISO-8859-1?Q?_zpr=E1vy?= -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iEYEABECAAYFAkfqd1UACgkQntdYP8FOsoIrOwCfextuhs/04NJx/c1Pp8zJ/Ueb pzQAoJTDF3DtyR4dBOHT4Mw6EmgrmM0T =cYb2 -----END PGP SIGNATURE----- --=-vC/DqVTgUoEBp98Zlv9h--