From owner-freebsd-toolchain@freebsd.org Sun Jun 28 14:44:18 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5682598F451 for ; Sun, 28 Jun 2015 14:44:18 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id 3D7471DCA for ; Sun, 28 Jun 2015 14:44:18 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 3A484B24F; Sun, 28 Jun 2015 14:44:18 +0000 (UTC) Date: Sun, 28 Jun 2015 14:44:18 +0000 To: freebsd-toolchain@freebsd.org From: "emaste (Ed Maste)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Request, 20 lines] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJl Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_6a703d38f4991d5770a85cd3a93d4c9e" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jun 2015 14:44:18 -0000 --b1_6a703d38f4991d5770a85cd3a93d4c9e Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit emaste created this revision. emaste added a reviewer: kib. emaste added subscribers: davide, dim, freebsd-toolchain-list, kib. REVISION SUMMARY Fault in the buffer prior to writing as a workaround for poor performance due to interaction with kernel fs deadlock avoidance code. See the comment prior to vn_io_fault_doio() in vfs_vnops.c for details of the issue. Thanks @kib for diagnosing, providing an explanation of the issue and workaround. TEST PLAN ``` % truncate -s 16M obj.o % ar r out.a obj.o ``` Total time with stock and patched ar(1): ``` x ar.r284891 + ar.patched +----------------------------------------------------------------------+ |+ | |+ x| |+ xx| |A |A| +----------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 1.307 1.321 1.315 1.3143333 0.0070237692 + 3 0.02 0.023 0.022 0.021666667 0.0015275252 Difference at 95.0% confidence -1.29267 +/- 0.0115203 -98.3515% +/- 0.876513% (Student's t, pooled s = 0.00508265) ``` REVISION DETAIL https://reviews.freebsd.org/D2933 AFFECTED FILES usr.bin/ar/write.c CHANGE DETAILS diff --git a/usr.bin/ar/write.c b/usr.bin/ar/write.c --- a/usr.bin/ar/write.c +++ b/usr.bin/ar/write.c @@ -41,6 +41,7 @@ #include #include #include +#include #include "ar.h" @@ -61,6 +62,7 @@ static void free_obj(struct bsdar *bsdar, struct ar_obj *obj); static void insert_obj(struct bsdar *bsdar, struct ar_obj *obj, struct ar_obj *pos); +static void prefault_buffer(const char *buf, size_t s); static void read_objs(struct bsdar *bsdar, const char *archive, int checkargv); static void write_archive(struct bsdar *bsdar, char mode); @@ -551,11 +553,29 @@ } /* + * Fault in the buffer prior to writing as a workaround for poor performance + * due to interaction with kernel fs deadlock avoidance code. See the comment + * prior to vn_io_fault_doio() in vfs_vnops.c for details of the issue. + */ +static void +prefault_buffer(const char *buf, size_t s) +{ + volatile const char *p; + size_t page_size; + + page_size = sysconf(_SC_PAGESIZE); + for (p = buf; p < buf + s; + p += page_size - ((uintptr_t)p & page_size - 1)) + *p; +} + +/* * Wrapper for archive_write_data(). */ static void write_data(struct bsdar *bsdar, struct archive *a, const void *buf, size_t s) { + prefault_buffer(buf, s); if (archive_write_data(a, buf, s) != (ssize_t)s) bsdar_errc(bsdar, EX_SOFTWARE, 0, "%s", archive_error_string(a)); EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, kib Cc: kib, freebsd-toolchain-list, dim, davide --b1_6a703d38f4991d5770a85cd3a93d4c9e Content-Type: text/x-patch; charset=utf-8; name="D2933.6524.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D2933.6524.patch" ZGlmZiAtLWdpdCBhL3Vzci5iaW4vYXIvd3JpdGUuYyBiL3Vzci5iaW4vYXIvd3JpdGUuYwotLS0g YS91c3IuYmluL2FyL3dyaXRlLmMKKysrIGIvdXNyLmJpbi9hci93cml0ZS5jCkBAIC00MSw2ICs0 MSw3IEBACiAjaW5jbHVkZSA8c3RkbGliLmg+CiAjaW5jbHVkZSA8c3RyaW5nLmg+CiAjaW5jbHVk ZSA8c3lzZXhpdHMuaD4KKyNpbmNsdWRlIDx1bmlzdGQuaD4KIAogI2luY2x1ZGUgImFyLmgiCiAK QEAgLTYxLDYgKzYyLDcgQEAKIHN0YXRpYyB2b2lkCWZyZWVfb2JqKHN0cnVjdCBic2RhciAqYnNk YXIsIHN0cnVjdCBhcl9vYmogKm9iaik7CiBzdGF0aWMgdm9pZAlpbnNlcnRfb2JqKHN0cnVjdCBi c2RhciAqYnNkYXIsIHN0cnVjdCBhcl9vYmogKm9iaiwKIAkJICAgIHN0cnVjdCBhcl9vYmogKnBv cyk7CitzdGF0aWMgdm9pZAlwcmVmYXVsdF9idWZmZXIoY29uc3QgY2hhciAqYnVmLCBzaXplX3Qg cyk7CiBzdGF0aWMgdm9pZAlyZWFkX29ianMoc3RydWN0IGJzZGFyICpic2RhciwgY29uc3QgY2hh ciAqYXJjaGl2ZSwKIAkJICAgIGludCBjaGVja2FyZ3YpOwogc3RhdGljIHZvaWQJd3JpdGVfYXJj aGl2ZShzdHJ1Y3QgYnNkYXIgKmJzZGFyLCBjaGFyIG1vZGUpOwpAQCAtNTUxLDExICs1NTMsMjkg QEAKIH0KIAogLyoKKyAqIEZhdWx0IGluIHRoZSBidWZmZXIgcHJpb3IgdG8gd3JpdGluZyBhcyBh IHdvcmthcm91bmQgZm9yIHBvb3IgcGVyZm9ybWFuY2UKKyAqIGR1ZSB0byBpbnRlcmFjdGlvbiB3 aXRoIGtlcm5lbCBmcyBkZWFkbG9jayBhdm9pZGFuY2UgY29kZS4gU2VlIHRoZSBjb21tZW50Cisg KiBwcmlvciB0byB2bl9pb19mYXVsdF9kb2lvKCkgaW4gdmZzX3Zub3BzLmMgZm9yIGRldGFpbHMg b2YgdGhlIGlzc3VlLgorICovCitzdGF0aWMgdm9pZAorcHJlZmF1bHRfYnVmZmVyKGNvbnN0IGNo YXIgKmJ1Ziwgc2l6ZV90IHMpCit7CisJdm9sYXRpbGUgY29uc3QgY2hhciAqcDsKKwlzaXplX3Qg cGFnZV9zaXplOworCisJcGFnZV9zaXplID0gc3lzY29uZihfU0NfUEFHRVNJWkUpOworCWZvciAo cCA9IGJ1ZjsgcCA8IGJ1ZiArIHM7CisJICAgIHAgKz0gcGFnZV9zaXplIC0gKCh1aW50cHRyX3Qp cCAmIHBhZ2Vfc2l6ZSAtIDEpKQorCQkqcDsKK30KKworLyoKICAqIFdyYXBwZXIgZm9yIGFyY2hp dmVfd3JpdGVfZGF0YSgpLgogICovCiBzdGF0aWMgdm9pZAogd3JpdGVfZGF0YShzdHJ1Y3QgYnNk YXIgKmJzZGFyLCBzdHJ1Y3QgYXJjaGl2ZSAqYSwgY29uc3Qgdm9pZCAqYnVmLCBzaXplX3QgcykK IHsKKwlwcmVmYXVsdF9idWZmZXIoYnVmLCBzKTsKIAlpZiAoYXJjaGl2ZV93cml0ZV9kYXRhKGEs IGJ1ZiwgcykgIT0gKHNzaXplX3QpcykKIAkJYnNkYXJfZXJyYyhic2RhciwgRVhfU09GVFdBUkUs IDAsICIlcyIsCiAJCSAgICBhcmNoaXZlX2Vycm9yX3N0cmluZyhhKSk7Cgo= --b1_6a703d38f4991d5770a85cd3a93d4c9e-- From owner-freebsd-toolchain@freebsd.org Sun Jun 28 15:29:47 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EE4D898FC67 for ; Sun, 28 Jun 2015 15:29:47 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id D9DA81F87 for ; Sun, 28 Jun 2015 15:29:47 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id D4B72B1F9; Sun, 28 Jun 2015 15:29:47 +0000 (UTC) Date: Sun, 28 Jun 2015 15:29:47 +0000 To: freebsd-toolchain@freebsd.org From: "eadler (Eitan Adler)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Accepted] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: <30c1e0b4bb46be3dc61103bc130f6038@localhost.localdomain> X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJlIFWQEus= Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Jun 2015 15:29:48 -0000 eadler accepted this revision. eadler added a reviewer: eadler. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D2933 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, kib, eadler Cc: kib, freebsd-toolchain-list, dim, davide From owner-freebsd-toolchain@freebsd.org Mon Jun 29 13:02:40 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE98898F47B for ; Mon, 29 Jun 2015 13:02:40 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id A27151094 for ; Mon, 29 Jun 2015 13:02:40 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 9D8A8B901; Mon, 29 Jun 2015 13:02:40 +0000 (UTC) Date: Mon, 29 Jun 2015 13:02:40 +0000 To: freebsd-toolchain@freebsd.org From: "kib (Konstantin Belousov)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Commented On] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: <477917a4a4027dd0b775cf010fc861e5@localhost.localdomain> X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJlIFWRQfA= Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 13:02:40 -0000 kib added inline comments. INLINE COMMENTS usr.bin/ar/write.c:568 It is simpler to do if (s == 0) return; for (p = buf; p < buf + s; p += page_size) *p; *(volatile const char *)(buf + s - 1); I do not insist. REVISION DETAIL https://reviews.freebsd.org/D2933 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, kib, eadler Cc: kib, freebsd-toolchain-list, dim, davide From owner-freebsd-toolchain@freebsd.org Mon Jun 29 13:14:12 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 98AA998F664 for ; Mon, 29 Jun 2015 13:14:12 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id 78DD11892 for ; Mon, 29 Jun 2015 13:14:12 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 73CB4BEDB; Mon, 29 Jun 2015 13:14:12 +0000 (UTC) Date: Mon, 29 Jun 2015 13:14:12 +0000 To: freebsd-toolchain@freebsd.org From: "emaste (Ed Maste)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Updated, 26 lines] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJlIFWRRKQ= Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , , , MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_a222931c107847e855b5d47d28ef7204" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 13:14:12 -0000 --b1_a222931c107847e855b5d47d28ef7204 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit emaste updated this revision to Diff 6543. emaste added a comment. This revision now requires review to proceed. - give full path to file with comment explaining the deadlock avoidance - avoid confusing page mask manipulation and just touch the last byte explicitly CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D2933?vs=6524&id=6543 REVISION DETAIL https://reviews.freebsd.org/D2933 AFFECTED FILES usr.bin/ar/write.c CHANGE DETAILS diff --git a/usr.bin/ar/write.c b/usr.bin/ar/write.c --- a/usr.bin/ar/write.c +++ b/usr.bin/ar/write.c @@ -41,6 +41,7 @@ #include #include #include +#include #include "ar.h" @@ -61,6 +62,7 @@ static void free_obj(struct bsdar *bsdar, struct ar_obj *obj); static void insert_obj(struct bsdar *bsdar, struct ar_obj *obj, struct ar_obj *pos); +static void prefault_buffer(const char *buf, size_t s); static void read_objs(struct bsdar *bsdar, const char *archive, int checkargv); static void write_archive(struct bsdar *bsdar, char mode); @@ -551,11 +553,35 @@ } /* + * Fault in the buffer prior to writing as a workaround for poor performance + * due to interaction with kernel fs deadlock avoidance code. See the comment + * above vn_io_fault_doio() in sys/kern/vfs_vnops.c for details of the issue. + */ +static void +prefault_buffer(const char *buf, size_t s) +{ + volatile const char *p; + size_t page_size; + + if (s == 0) + return; + page_size = sysconf(_SC_PAGESIZE); + for (p = buf; p < buf + s; p += page_size) + *p; + /* + * Ensure we touch the last page as well, in case the buffer is not + * page-aligned. + */ + *(volatile const char *)(buf + s - 1); +} + +/* * Wrapper for archive_write_data(). */ static void write_data(struct bsdar *bsdar, struct archive *a, const void *buf, size_t s) { + prefault_buffer(buf, s); if (archive_write_data(a, buf, s) != (ssize_t)s) bsdar_errc(bsdar, EX_SOFTWARE, 0, "%s", archive_error_string(a)); EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, kib, eadler Cc: kib, freebsd-toolchain-list, dim, davide --b1_a222931c107847e855b5d47d28ef7204 Content-Type: text/x-patch; charset=utf-8; name="D2933.6543.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D2933.6543.patch" ZGlmZiAtLWdpdCBhL3Vzci5iaW4vYXIvd3JpdGUuYyBiL3Vzci5iaW4vYXIvd3JpdGUuYwotLS0g YS91c3IuYmluL2FyL3dyaXRlLmMKKysrIGIvdXNyLmJpbi9hci93cml0ZS5jCkBAIC00MSw2ICs0 MSw3IEBACiAjaW5jbHVkZSA8c3RkbGliLmg+CiAjaW5jbHVkZSA8c3RyaW5nLmg+CiAjaW5jbHVk ZSA8c3lzZXhpdHMuaD4KKyNpbmNsdWRlIDx1bmlzdGQuaD4KIAogI2luY2x1ZGUgImFyLmgiCiAK QEAgLTYxLDYgKzYyLDcgQEAKIHN0YXRpYyB2b2lkCWZyZWVfb2JqKHN0cnVjdCBic2RhciAqYnNk YXIsIHN0cnVjdCBhcl9vYmogKm9iaik7CiBzdGF0aWMgdm9pZAlpbnNlcnRfb2JqKHN0cnVjdCBi c2RhciAqYnNkYXIsIHN0cnVjdCBhcl9vYmogKm9iaiwKIAkJICAgIHN0cnVjdCBhcl9vYmogKnBv cyk7CitzdGF0aWMgdm9pZAlwcmVmYXVsdF9idWZmZXIoY29uc3QgY2hhciAqYnVmLCBzaXplX3Qg cyk7CiBzdGF0aWMgdm9pZAlyZWFkX29ianMoc3RydWN0IGJzZGFyICpic2RhciwgY29uc3QgY2hh ciAqYXJjaGl2ZSwKIAkJICAgIGludCBjaGVja2FyZ3YpOwogc3RhdGljIHZvaWQJd3JpdGVfYXJj aGl2ZShzdHJ1Y3QgYnNkYXIgKmJzZGFyLCBjaGFyIG1vZGUpOwpAQCAtNTUxLDExICs1NTMsMzUg QEAKIH0KIAogLyoKKyAqIEZhdWx0IGluIHRoZSBidWZmZXIgcHJpb3IgdG8gd3JpdGluZyBhcyBh IHdvcmthcm91bmQgZm9yIHBvb3IgcGVyZm9ybWFuY2UKKyAqIGR1ZSB0byBpbnRlcmFjdGlvbiB3 aXRoIGtlcm5lbCBmcyBkZWFkbG9jayBhdm9pZGFuY2UgY29kZS4gU2VlIHRoZSBjb21tZW50Cisg KiBhYm92ZSB2bl9pb19mYXVsdF9kb2lvKCkgaW4gc3lzL2tlcm4vdmZzX3Zub3BzLmMgZm9yIGRl dGFpbHMgb2YgdGhlIGlzc3VlLgorICovCitzdGF0aWMgdm9pZAorcHJlZmF1bHRfYnVmZmVyKGNv bnN0IGNoYXIgKmJ1Ziwgc2l6ZV90IHMpCit7CisJdm9sYXRpbGUgY29uc3QgY2hhciAqcDsKKwlz aXplX3QgcGFnZV9zaXplOworCisJaWYgKHMgPT0gMCkKKwkJcmV0dXJuOworCXBhZ2Vfc2l6ZSA9 IHN5c2NvbmYoX1NDX1BBR0VTSVpFKTsKKwlmb3IgKHAgPSBidWY7IHAgPCBidWYgKyBzOyBwICs9 IHBhZ2Vfc2l6ZSkKKwkJKnA7CisJLyoKKwkgKiBFbnN1cmUgd2UgdG91Y2ggdGhlIGxhc3QgcGFn ZSBhcyB3ZWxsLCBpbiBjYXNlIHRoZSBidWZmZXIgaXMgbm90CisJICogcGFnZS1hbGlnbmVkLgor CSAqLworCSoodm9sYXRpbGUgY29uc3QgY2hhciAqKShidWYgKyBzIC0gMSk7Cit9CisKKy8qCiAg KiBXcmFwcGVyIGZvciBhcmNoaXZlX3dyaXRlX2RhdGEoKS4KICAqLwogc3RhdGljIHZvaWQKIHdy aXRlX2RhdGEoc3RydWN0IGJzZGFyICpic2Rhciwgc3RydWN0IGFyY2hpdmUgKmEsIGNvbnN0IHZv aWQgKmJ1Ziwgc2l6ZV90IHMpCiB7CisJcHJlZmF1bHRfYnVmZmVyKGJ1Ziwgcyk7CiAJaWYgKGFy Y2hpdmVfd3JpdGVfZGF0YShhLCBidWYsIHMpICE9IChzc2l6ZV90KXMpCiAJCWJzZGFyX2VycmMo YnNkYXIsIEVYX1NPRlRXQVJFLCAwLCAiJXMiLAogCQkgICAgYXJjaGl2ZV9lcnJvcl9zdHJpbmco YSkpOwoK --b1_a222931c107847e855b5d47d28ef7204-- From owner-freebsd-toolchain@freebsd.org Mon Jun 29 13:27:45 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA3CD98F837 for ; Mon, 29 Jun 2015 13:27:45 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id A5A1A1CC8 for ; Mon, 29 Jun 2015 13:27:45 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id A0D66B354; Mon, 29 Jun 2015 13:27:45 +0000 (UTC) Date: Mon, 29 Jun 2015 13:27:45 +0000 To: freebsd-toolchain@freebsd.org From: "kib (Konstantin Belousov)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Accepted] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: <1fc661db5b42e58505c78ebc4a793c23@localhost.localdomain> X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJlIFWRR9E= Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset="utf-8" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 13:27:45 -0000 kib accepted this revision. This revision has a positive review. REVISION DETAIL https://reviews.freebsd.org/D2933 EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, eadler, kib Cc: kib, freebsd-toolchain-list, dim, davide From owner-freebsd-toolchain@freebsd.org Mon Jun 29 13:48:59 2015 Return-Path: Delivered-To: freebsd-toolchain@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A61C98FBFF for ; Mon, 29 Jun 2015 13:48:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: from phabric-backend.isc.freebsd.org (phabric-backend.isc.freebsd.org [IPv6:2001:4f8:3:ffe0:406a:0:50:2]) by mx1.freebsd.org (Postfix) with ESMTP id 3FD591977 for ; Mon, 29 Jun 2015 13:48:59 +0000 (UTC) (envelope-from daemon-user@freebsd.org) Received: by phabric-backend.isc.freebsd.org (Postfix, from userid 1346) id 3AF5FBAB4; Mon, 29 Jun 2015 13:48:59 +0000 (UTC) Date: Mon, 29 Jun 2015 13:48:59 +0000 To: freebsd-toolchain@freebsd.org From: "emaste (Ed Maste)" Reply-to: D2933+334+e35df629633d6443@FreeBSD.org Subject: [Differential] [Closed] D2933: Significantly speed up ar(1) on UFS file systems Message-ID: <59d242779a924a551c3b8c777e6f2f13@localhost.localdomain> X-Priority: 3 Thread-Topic: D2933: Significantly speed up ar(1) on UFS file systems X-Herald-Rules: none X-Phabricator-To: X-Phabricator-To: X-Phabricator-To: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: X-Phabricator-Cc: In-Reply-To: References: Thread-Index: OTY3MzA4YjIwYTMzODRmYTQ0NTRmMTdjYzJlIFWRTMs= Precedence: bulk X-Phabricator-Sent-This-Message: Yes X-Mail-Transport-Agent: MetaMTA X-Auto-Response-Suppress: All X-Phabricator-Mail-Tags: , MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="b1_59d242779a924a551c3b8c777e6f2f13" X-BeenThere: freebsd-toolchain@freebsd.org X-Mailman-Version: 2.1.20 List-Id: Maintenance of FreeBSD's integrated toolchain List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2015 13:48:59 -0000 --b1_59d242779a924a551c3b8c777e6f2f13 Content-Type: text/plain; charset = "utf-8" Content-Transfer-Encoding: 8bit This revision was automatically updated to reflect the committed changes. Closed by commit rS284928: speed up ar(1) on UFS file systems (authored by emaste). CHANGED PRIOR TO COMMIT https://reviews.freebsd.org/D2933?vs=6543&id=6545#toc REPOSITORY rS FreeBSD src repository CHANGES SINCE LAST UPDATE https://reviews.freebsd.org/D2933?vs=6543&id=6545 REVISION DETAIL https://reviews.freebsd.org/D2933 AFFECTED FILES head/usr.bin/ar/write.c CHANGE DETAILS diff --git a/head/usr.bin/ar/write.c b/head/usr.bin/ar/write.c --- a/head/usr.bin/ar/write.c +++ b/head/usr.bin/ar/write.c @@ -41,6 +41,7 @@ #include #include #include +#include #include "ar.h" @@ -61,6 +62,7 @@ static void free_obj(struct bsdar *bsdar, struct ar_obj *obj); static void insert_obj(struct bsdar *bsdar, struct ar_obj *obj, struct ar_obj *pos); +static void prefault_buffer(const char *buf, size_t s); static void read_objs(struct bsdar *bsdar, const char *archive, int checkargv); static void write_archive(struct bsdar *bsdar, char mode); @@ -551,11 +553,35 @@ } /* + * Fault in the buffer prior to writing as a workaround for poor performance + * due to interaction with kernel fs deadlock avoidance code. See the comment + * above vn_io_fault_doio() in sys/kern/vfs_vnops.c for details of the issue. + */ +static void +prefault_buffer(const char *buf, size_t s) +{ + volatile const char *p; + size_t page_size; + + if (s == 0) + return; + page_size = sysconf(_SC_PAGESIZE); + for (p = buf; p < buf + s; p += page_size) + *p; + /* + * Ensure we touch the last page as well, in case the buffer is not + * page-aligned. + */ + *(volatile const char *)(buf + s - 1); +} + +/* * Wrapper for archive_write_data(). */ static void write_data(struct bsdar *bsdar, struct archive *a, const void *buf, size_t s) { + prefault_buffer(buf, s); if (archive_write_data(a, buf, s) != (ssize_t)s) bsdar_errc(bsdar, EX_SOFTWARE, 0, "%s", archive_error_string(a)); EMAIL PREFERENCES https://reviews.freebsd.org/settings/panel/emailpreferences/ To: emaste, eadler, kib Cc: kib, freebsd-toolchain-list, dim, davide --b1_59d242779a924a551c3b8c777e6f2f13 Content-Type: text/x-patch; charset=utf-8; name="D2933.6545.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="D2933.6545.patch" ZGlmZiAtLWdpdCBhL2hlYWQvdXNyLmJpbi9hci93cml0ZS5jIGIvaGVhZC91c3IuYmluL2FyL3dy aXRlLmMKLS0tIGEvaGVhZC91c3IuYmluL2FyL3dyaXRlLmMKKysrIGIvaGVhZC91c3IuYmluL2Fy L3dyaXRlLmMKQEAgLTQxLDYgKzQxLDcgQEAKICNpbmNsdWRlIDxzdGRsaWIuaD4KICNpbmNsdWRl IDxzdHJpbmcuaD4KICNpbmNsdWRlIDxzeXNleGl0cy5oPgorI2luY2x1ZGUgPHVuaXN0ZC5oPgog CiAjaW5jbHVkZSAiYXIuaCIKIApAQCAtNjEsNiArNjIsNyBAQAogc3RhdGljIHZvaWQJZnJlZV9v Ymooc3RydWN0IGJzZGFyICpic2Rhciwgc3RydWN0IGFyX29iaiAqb2JqKTsKIHN0YXRpYyB2b2lk CWluc2VydF9vYmooc3RydWN0IGJzZGFyICpic2Rhciwgc3RydWN0IGFyX29iaiAqb2JqLAogCQkg ICAgc3RydWN0IGFyX29iaiAqcG9zKTsKK3N0YXRpYyB2b2lkCXByZWZhdWx0X2J1ZmZlcihjb25z dCBjaGFyICpidWYsIHNpemVfdCBzKTsKIHN0YXRpYyB2b2lkCXJlYWRfb2JqcyhzdHJ1Y3QgYnNk YXIgKmJzZGFyLCBjb25zdCBjaGFyICphcmNoaXZlLAogCQkgICAgaW50IGNoZWNrYXJndik7CiBz dGF0aWMgdm9pZAl3cml0ZV9hcmNoaXZlKHN0cnVjdCBic2RhciAqYnNkYXIsIGNoYXIgbW9kZSk7 CkBAIC01NTEsMTEgKzU1MywzNSBAQAogfQogCiAvKgorICogRmF1bHQgaW4gdGhlIGJ1ZmZlciBw cmlvciB0byB3cml0aW5nIGFzIGEgd29ya2Fyb3VuZCBmb3IgcG9vciBwZXJmb3JtYW5jZQorICog ZHVlIHRvIGludGVyYWN0aW9uIHdpdGgga2VybmVsIGZzIGRlYWRsb2NrIGF2b2lkYW5jZSBjb2Rl LiBTZWUgdGhlIGNvbW1lbnQKKyAqIGFib3ZlIHZuX2lvX2ZhdWx0X2RvaW8oKSBpbiBzeXMva2Vy bi92ZnNfdm5vcHMuYyBmb3IgZGV0YWlscyBvZiB0aGUgaXNzdWUuCisgKi8KK3N0YXRpYyB2b2lk CitwcmVmYXVsdF9idWZmZXIoY29uc3QgY2hhciAqYnVmLCBzaXplX3QgcykKK3sKKwl2b2xhdGls ZSBjb25zdCBjaGFyICpwOworCXNpemVfdCBwYWdlX3NpemU7CisKKwlpZiAocyA9PSAwKQorCQly ZXR1cm47CisJcGFnZV9zaXplID0gc3lzY29uZihfU0NfUEFHRVNJWkUpOworCWZvciAocCA9IGJ1 ZjsgcCA8IGJ1ZiArIHM7IHAgKz0gcGFnZV9zaXplKQorCQkqcDsKKwkvKgorCSAqIEVuc3VyZSB3 ZSB0b3VjaCB0aGUgbGFzdCBwYWdlIGFzIHdlbGwsIGluIGNhc2UgdGhlIGJ1ZmZlciBpcyBub3QK KwkgKiBwYWdlLWFsaWduZWQuCisJICovCisJKih2b2xhdGlsZSBjb25zdCBjaGFyICopKGJ1ZiAr IHMgLSAxKTsKK30KKworLyoKICAqIFdyYXBwZXIgZm9yIGFyY2hpdmVfd3JpdGVfZGF0YSgpLgog ICovCiBzdGF0aWMgdm9pZAogd3JpdGVfZGF0YShzdHJ1Y3QgYnNkYXIgKmJzZGFyLCBzdHJ1Y3Qg YXJjaGl2ZSAqYSwgY29uc3Qgdm9pZCAqYnVmLCBzaXplX3QgcykKIHsKKwlwcmVmYXVsdF9idWZm ZXIoYnVmLCBzKTsKIAlpZiAoYXJjaGl2ZV93cml0ZV9kYXRhKGEsIGJ1ZiwgcykgIT0gKHNzaXpl X3QpcykKIAkJYnNkYXJfZXJyYyhic2RhciwgRVhfU09GVFdBUkUsIDAsICIlcyIsCiAJCSAgICBh cmNoaXZlX2Vycm9yX3N0cmluZyhhKSk7Cgo= --b1_59d242779a924a551c3b8c777e6f2f13--