Date: Wed, 26 Mar 2008 17:18:29 +0100 From: Pav Lucistnik <pav@FreeBSD.org> To: freebsd-ports@FreeBSD.org Cc: rdivacky@FreeBSD.org, soralx@cydem.org Subject: [patch] pkg_delete(1) speedup Message-ID: <1206548309.78560.69.camel@pav.hide.vol.cz>
next in thread | raw e-mail | index | archive | help
--=-vC/DqVTgUoEBp98Zlv9h
Content-Type: multipart/mixed; boundary="=-gy6w4Z4DMzOlLo/sf16H"
--=-gy6w4Z4DMzOlLo/sf16H
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable
You might have noticed a thread on the mailing list called "ports system
woes". The submitter pointed out an inefficiency in pkg_delete routine,
that parses the whole /var/db/pkg over and over again for every
dependency of a package being removed.
Attached is a patch by rdivacky that implements the idea of looking up
all the values in a single pass over /var/db/pkg content.
A trivial benchmark, wall times:
port with two dependencies (comms/obexapp)
before
0.083s
after
0.049s
port with 172 dependencies (graphics/agave)
before
8.404s
8.955s
11.734s
after
2.816s
2.690s
3.195s
The patch is not WARNS clean, that needs to be fixed. And I'd like a
review by someone with src commit bit.
--=20
Pav Lucistnik <pav@oook.cz>
<pav@FreeBSD.org>
42.7 percent of all statistics are made up on the spot.
--=-gy6w4Z4DMzOlLo/sf16H
Content-Disposition: attachment; filename=pkg_delete.patch
Content-Type: text/x-patch; name=pkg_delete.patch; charset=ISO8859-2
Content-Transfer-Encoding: base64
SW5kZXg6IGRlbGV0ZS9wZXJmb3JtLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvaG9tZS9uY3Zz
L3NyYy91c3Iuc2Jpbi9wa2dfaW5zdGFsbC9kZWxldGUvcGVyZm9ybS5jLHYNCnJldHJpZXZpbmcg
cmV2aXNpb24gMS40MQ0KZGlmZiAtYSAtdSAtcjEuNDEgcGVyZm9ybS5jDQotLS0gZGVsZXRlL3Bl
cmZvcm0uYwkyOSBKdW4gMjAwNCAxOTowNjo0MSAtMDAwMAkxLjQxDQorKysgZGVsZXRlL3BlcmZv
cm0uYwkyNiBNYXIgMjAwOCAxNTo0NzoyOSAtMDAwMA0KQEAgLTEyMywxNiArMTIzLDIwIEBADQog
ew0KICAgICBGSUxFICpjZmlsZTsNCiAgICAgY2hhciAqZGVwb3JpZ2luLCAqKmRlcG5hbWVzLCBo
b21lW0ZJTEVOQU1FX01BWF07DQorICAgIGNoYXIgKip1bmRpcmVjdF9kZXBzOw0KICAgICBQYWNr
aW5nTGlzdCBwOw0KICAgICBpbnQgaSwgbGVuOw0KICAgICBpbnQgaXNpbnN0YWxsZWQ7DQogICAg
IC8qIHN1cHBvcnQgZm9yIHNlcGFyYXRlIHByZS9wb3N0IGluc3RhbGwgc2NyaXB0cyAqLw0KLSAg
ICBpbnQgbmV3X20gPSAwOw0KKyAgICBpbnQgbmV3X20gPSAwLCB1ZF9jb3VudDsNCiAgICAgY29u
c3QgY2hhciAqcHJlX3NjcmlwdCA9IERFSU5TVEFMTF9GTkFNRTsNCiAgICAgY29uc3QgY2hhciAq
cG9zdF9zY3JpcHQsICpwcmVfYXJnLCAqcG9zdF9hcmc7DQogICAgIHN0cnVjdCByZXFyX2J5X2Vu
dHJ5ICpyYl9lbnRyeTsNCiAgICAgc3RydWN0IHJlcXJfYnlfaGVhZCAqcmJfbGlzdDsNCiANCisg
ICAgZGVwbmFtZXMgPSBOVUxMOw0KKyAgICB1bmRpcmVjdF9kZXBzID0gTlVMTDsNCisNCiAgICAg
aWYgKCFwa2cgfHwgIShsZW4gPSBzdHJsZW4ocGtnKSkpDQogCXJldHVybiAxOw0KICAgICBpZiAo
cGtnW2xlbiAtIDFdID09ICcvJykNCkBAIC0yNjMsNiArMjY3LDcgQEANCiAJfQ0KICAgICB9DQog
DQorICAgIHVkX2NvdW50ID0gMDsNCiAgICAgZm9yIChwID0gUGxpc3QuaGVhZDsgcCA7IHAgPSBw
LT5uZXh0KSB7DQogCWlmIChwLT50eXBlICE9IFBMSVNUX1BLR0RFUCkNCiAJICAgIGNvbnRpbnVl
Ow0KQEAgLTI3NSwxOCArMjgwLDI4IEBADQogCSAgICBwcmludGYoIi5cbiIpOw0KIAl9DQogCWlm
ICghRmFrZSkgew0KLQkgICAgZGVwbmFtZXMgPSAoZGVwb3JpZ2luICE9IE5VTEwpID8gbWF0Y2hi
eW9yaWdpbihkZXBvcmlnaW4sIE5VTEwpIDoNCi0JCQkJCSAgICAgTlVMTDsNCi0JICAgIGlmIChk
ZXBuYW1lcyA9PSBOVUxMKSB7DQotCQlkZXBuYW1lcyA9IGFsbG9jYShzaXplb2YoKmRlcG5hbWVz
KSAqIDIpOw0KLQkJZGVwbmFtZXNbMF0gPSBwLT5uYW1lOw0KLQkJZGVwbmFtZXNbMV0gPSBOVUxM
Ow0KKwkgICAgaWYgKGRlcG9yaWdpbiA9PSBOVUxMKSB7DQorCQl1bmRlcGVuZChwLT5uYW1lLCBw
a2cpOw0KKwkgICAgfSBlbHNlIHsNCisJICAgICAgIHVuZGlyZWN0X2RlcHMgPSByZWFsbG9jKHVu
ZGlyZWN0X2RlcHMsICh1ZF9jb3VudCArIDIpICogc2l6ZW9mKCp1bmRpcmVjdF9kZXBzKSk7DQor
CSAgICAgICB1bmRpcmVjdF9kZXBzW3VkX2NvdW50XSA9IGRlcG9yaWdpbjsNCisJICAgICAgIHVu
ZGlyZWN0X2RlcHNbdWRfY291bnQgKyAxXSA9IE5VTEw7DQorCSAgICAgICB1ZF9jb3VudCsrOw0K
IAkgICAgfQ0KLQkgICAgZm9yIChpID0gMDsgZGVwbmFtZXNbaV0gIT0gTlVMTDsgaSsrKQ0KLQkJ
dW5kZXBlbmQoZGVwbmFtZXNbaV0sIHBrZyk7DQogCX0NCiAgICAgfQ0KIA0KKyAgICBpZiAodW5k
aXJlY3RfZGVwcyAhPSBOVUxMKSB7DQorCWRlcG5hbWVzID0gbWF0Y2hhbGxieW9yaWdpbih1bmRp
cmVjdF9kZXBzLCBOVUxMKTsNCisNCisJZnJlZSh1bmRpcmVjdF9kZXBzKTsNCisNCisJLyogVW5k
ZXBlbmQgYWxsIHRoZSBkZXBlbmRhbmNpZXMgYXQgb25jZSAqLw0KKwlmb3IgKGkgPSAwOyBkZXBu
YW1lc1tpXSAhPSBOVUxMOyBpKyspDQorCSAgIHVuZGVwZW5kKGRlcG5hbWVzW2ldLCBwa2cpOw0K
Kw0KKyAgICB9DQorDQogICAgIGlmIChjaGRpcihob21lKSA9PSBGQUlMKSB7DQogCWNsZWFudXAo
MCk7DQogCWVycngoMiwgIiVzOiB1bmFibGUgdG8gcmV0dXJuIHRvIHdvcmtpbmcgZGlyZWN0b3J5
ICVzISIsIF9fZnVuY19fLA0KSW5kZXg6IGxpYi9saWIuaA0KPT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZpbGU6
IC9ob21lL25jdnMvc3JjL3Vzci5zYmluL3BrZ19pbnN0YWxsL2xpYi9saWIuaCx2DQpyZXRyaWV2
aW5nIHJldmlzaW9uIDEuNTYuMi4yDQpkaWZmIC1hIC11IC1yMS41Ni4yLjIgbGliLmgNCi0tLSBs
aWIvbGliLmgJMTQgTWF5IDIwMDYgMDc6MDY6MzcgLTAwMDAJMS41Ni4yLjINCisrKyBsaWIvbGli
LmgJMjYgTWFyIDIwMDggMTU6NDc6MjkgLTAwMDANCkBAIC0yMTYsNiArMjE2LDcgQEANCiAvKiBR
dWVyeSBpbnN0YWxsZWQgcGFja2FnZXMgKi8NCiBjaGFyCQkqKm1hdGNoaW5zdGFsbGVkKG1hdGNo
X3QsIGNoYXIgKiosIGludCAqKTsNCiBjaGFyCQkqKm1hdGNoYnlvcmlnaW4oY29uc3QgY2hhciAq
LCBpbnQgKik7DQorY2hhcgkJKiptYXRjaGFsbGJ5b3JpZ2luKGNvbnN0IGNoYXIgKiosIGludCAq
KTsNCiBpbnQJCWlzaW5zdGFsbGVkcGtnKGNvbnN0IGNoYXIgKm5hbWUpOw0KIGludAkJcGF0dGVy
bl9tYXRjaChtYXRjaF90IE1hdGNoVHlwZSwgY2hhciAqcGF0dGVybiwgY29uc3QgY2hhciAqcGtn
bmFtZSk7DQogDQpJbmRleDogbGliL21hdGNoLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09
PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvaG9t
ZS9uY3ZzL3NyYy91c3Iuc2Jpbi9wa2dfaW5zdGFsbC9saWIvbWF0Y2guYyx2DQpyZXRyaWV2aW5n
IHJldmlzaW9uIDEuMTkuOC4yDQpkaWZmIC1hIC11IC1yMS4xOS44LjIgbWF0Y2guYw0KLS0tIGxp
Yi9tYXRjaC5jCTEwIE5vdiAyMDA3IDIyOjA0OjMxIC0wMDAwCTEuMTkuOC4yDQorKysgbGliL21h
dGNoLmMJMjYgTWFyIDIwMDggMTU6NDc6MjkgLTAwMDANCkBAIC0yMzgsMTAgKzIzOCwxMCBAQA0K
ICAqIGFzIGEga2V5IGZvciBtYXRjaGluZyBwYWNrYWdlcy4NCiAgKi8NCiBjaGFyICoqDQotbWF0
Y2hieW9yaWdpbihjb25zdCBjaGFyICpvcmlnaW4sIGludCAqcmV0dmFsKQ0KK21hdGNoYWxsYnlv
cmlnaW4oY29uc3QgY2hhciAqKm9yaWdpbnMsIGludCAqcmV0dmFsKQ0KIHsNCiAgICAgY2hhciAq
Kmluc3RhbGxlZDsNCi0gICAgaW50IGk7DQorICAgIGludCBpLCBqOw0KICAgICBzdGF0aWMgc3Ry
dWN0IHN0b3JlICpzdG9yZSA9IE5VTEw7DQogDQogICAgIHN0b3JlID0gc3RvcmVjcmVhdGUoc3Rv
cmUpOw0KQEAgLTI5MCw4ICsyOTAsOSBAQA0KIAkJY29udGludWU7DQogCSAgICBjbWQgPSBwbGlz
dF9jbWQodG1wICsgMSwgJmNwKTsNCiAJICAgIGlmIChjbWQgPT0gUExJU1RfT1JJR0lOKSB7DQot
CQlpZiAoY3NoX21hdGNoKG9yaWdpbiwgY3AsIEZOTV9QQVRITkFNRSkgPT0gMCkNCi0JCSAgICBz
dG9yZWFwcGVuZChzdG9yZSwgaW5zdGFsbGVkW2ldKTsNCisJCWZvciAoaiA9IDA7IG9yaWdpbnNb
al0gIT0gTlVMTDsgaisrKQ0KKwkJICAgaWYgKGNzaF9tYXRjaChvcmlnaW5zW2pdLCBjcCwgRk5N
X1BBVEhOQU1FKSA9PSAwKQ0KKwkJICAgICAgc3RvcmVhcHBlbmQoc3RvcmUsIGluc3RhbGxlZFtp
XSk7DQogCQlicmVhazsNCiAJICAgIH0NCiAJfQ0KQEAgLTMwNyw2ICszMDgsMjUgQEANCiB9DQog
DQogLyoNCisgKiBTeW5vcHNpcyBpcyBzaW1pbGFyIHRvIG1hdGNoaW5zdGFsbGVkKCksIGJ1dCB1
c2Ugb3JpZ2luDQorICogYXMgYSBrZXkgZm9yIG1hdGNoaW5nIHBhY2thZ2VzLg0KKyAqLw0KK2No
YXIgKioNCittYXRjaGJ5b3JpZ2luKGNvbnN0IGNoYXIgKm9yaWdpbiwgaW50ICpyZXR2YWwpDQor
ew0KKyAgIGNoYXIgKipvcmlnaW5zOw0KKyAgIGNoYXIgKipkZXBzOw0KKw0KKyAgIG9yaWdpbnMg
PSBtYWxsb2Moc2l6ZW9mKCpvcmlnaW5zKSAqIDIpOw0KKyAgIG9yaWdpbnNbMF0gPSBvcmlnaW47
DQorICAgb3JpZ2luc1sxXSA9IE5VTEw7DQorDQorICAgZGVwcyA9IG1hdGNoYWxsYnlvcmlnaW4o
b3JpZ2lucywgcmV0dmFsKTsNCisgICBmcmVlKG9yaWdpbnMpOw0KKyAgIHJldHVybiBkZXBzOw0K
K30NCisNCisvKg0KICAqIFNtYWxsIGxpbmtlZCBsaXN0IHRvIG1lbW9pemUgcmVzdWx0cyBvZiBp
c2luc3RhbGxlZHBrZygpLiAgQSBoYXNoIHRhYmxlDQogICogd291bGQgYmUgZmFzdGVyIGJ1dCBm
b3IgbiB+PSAxMDAwIG1heSBiZSBvdmVya2lsbC4NCiAgKi8NCg==
--=-gy6w4Z4DMzOlLo/sf16H--
--=-vC/DqVTgUoEBp98Zlv9h
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: Toto je =?UTF-8?Q?digit=C3=A1ln=C4=9B?=
=?ISO-8859-1?Q?_podepsan=E1?= =?UTF-8?Q?_=C4=8D=C3=A1st?=
=?ISO-8859-1?Q?_zpr=E1vy?=
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (FreeBSD)
iEYEABECAAYFAkfqd1UACgkQntdYP8FOsoIrOwCfextuhs/04NJx/c1Pp8zJ/Ueb
pzQAoJTDF3DtyR4dBOHT4Mw6EmgrmM0T
=cYb2
-----END PGP SIGNATURE-----
--=-vC/DqVTgUoEBp98Zlv9h--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1206548309.78560.69.camel>