From nobody Wed Aug 3 21:02:06 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4LyknV5P1Sz4XlJ8; Wed, 3 Aug 2022 21:02:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4LyknV4xy0z3Nqm; Wed, 3 Aug 2022 21:02:06 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1659560526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Q8bt5Z2zMj0bh+856WDCWPCTWPkCH6tydTIUvvUiVdw=; b=QhHCowD/M2HrwwqraX87+gLtC2ZSZ2io+vISihJgrv8Inel6BLiduuIbreInth2iO7uG0z CV/+RLfRhz9QAbH16oxdF5aZhGkVzxwapk5hYmcUPFW9WhYP54rfP27IaSsW8BZGVIKyJM wTPlhi7Qrok4sWWpDoqkN95zwAU2XY9A0cSqePwjULVN6eVud8769gT+JQU5Kj+xDJVZ+e 3LO09HFQ6jsNNqSw/7JjpC5vnS9ckLMDKJwXDdg/4P+BIVws71hcJ3CpVMgckno9DHAMCD 8CvecplP4MdnBfYW5BQ0BWmvievnSe3kI5eR8uH5eiKGcsNOhscDNi7ypHCWcg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4LyknV40xfz14nq; Wed, 3 Aug 2022 21:02:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 273L26tI081715; Wed, 3 Aug 2022 21:02:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 273L26U6081714; Wed, 3 Aug 2022 21:02:06 GMT (envelope-from git) Date: Wed, 3 Aug 2022 21:02:06 GMT Message-Id: <202208032102.273L26U6081714@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: =?utf-8?Q?Dag-Erling=20Sm=C3=B8rgrav?= Subject: git: 54d8d0fe12a4 - main - xinstall: use dynamic bufsize as in cat(1) / cp(1). List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: des X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 54d8d0fe12a4996427923048ab4261819774fbd4 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1659560526; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Q8bt5Z2zMj0bh+856WDCWPCTWPkCH6tydTIUvvUiVdw=; b=c+OfQlZgZ7ARK5Z6HL2NqZ5gtl968yfHtdECvnaymk68zMuDetO2lwTX0VlOWRF5XqgANI /nIUw2qmHMz2W5AxexJeWO2zsWJELkS5Lg1oCOT097hfwPKY5tlwT8LjIPrCjIP23cntla j/HjTkpZtrLljWaAtmbkvMjuhUHSYiZLphXeR/5MWatqL93SZP4OVcCrseX7DB5FXmHy+4 pf3zHRVCKwqEqyAo6oJNs8Vo6itlOYdfx+5ZI91RhtpF+LHepDZ7F4nKqgN1udTGi/HpkS dQE8tJ76Zw1Ht1UVCfYZdWL3RbczuUpYbCrFMvfvz+Jm9Xe9394lDNjL/h5MwA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1659560526; a=rsa-sha256; cv=none; b=YBD3fK9Lf/xs6KQ7N6eFkONMyYqRcnBzf822nPryISRn1f3LiSqfDoTVsffDT+9AHvcuye xfQJd44pAkPQuN34W1IgcFkR2QMXEjtQHgSVvbIPgyKmY4ek6bLUsf90jvzbKq1waHejcx 6ipcUtdlwUzbdHIz3IMpIHlLYHsv2/9/Kjmydr5MpFqQnXHI0k8o0AhniIlsSy1Lv3hUST j/7UJS39ELxh2S1+ZANav5EegUDUhIKIeq2C55suvP2YNEB6BaKCRJmTy02TNE2UcQB7cY 3cY6LYZu295FJoNPvz8dQFxpfkmd+N45i2jmjZDraXpdP/W4kLX/got4Rrh5NA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by des: URL: https://cgit.FreeBSD.org/src/commit/?id=54d8d0fe12a4996427923048ab4261819774fbd4 commit 54d8d0fe12a4996427923048ab4261819774fbd4 Author: Dag-Erling Smørgrav AuthorDate: 2022-08-03 21:00:14 +0000 Commit: Dag-Erling Smørgrav CommitDate: 2022-08-03 21:01:13 +0000 xinstall: use dynamic bufsize as in cat(1) / cp(1). Sponsored by: Klara, Inc. --- usr.bin/xinstall/xinstall.c | 58 +++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 53 insertions(+), 5 deletions(-) diff --git a/usr.bin/xinstall/xinstall.c b/usr.bin/xinstall/xinstall.c index a236838c8fd1..b0e52453ca29 100644 --- a/usr.bin/xinstall/xinstall.c +++ b/usr.bin/xinstall/xinstall.c @@ -79,6 +79,21 @@ __FBSDID("$FreeBSD$"); #include "mtree.h" +/* + * Memory strategy threshold, in pages: if physmem is larger then this, use a + * large buffer. + */ +#define PHYSPAGES_THRESHOLD (32*1024) + +/* Maximum buffer size in bytes - do not allow it to grow larger than this. */ +#define BUFSIZE_MAX (2*1024*1024) + +/* + * Small (default) buffer size in bytes. It's inefficient for this to be + * smaller than MAXPHYS. + */ +#define BUFSIZE_SMALL (MAXPHYS) + /* * We need to build xinstall during the bootstrap stage when building on a * non-FreeBSD system. Linux does not have the st_flags and st_birthtime @@ -1139,15 +1154,32 @@ compare(int from_fd, const char *from_name __unused, size_t from_len, } out: if (!done_compare) { - char buf1[MAXBSIZE]; - char buf2[MAXBSIZE]; + static char *buf, *buf1, *buf2; + static size_t bufsize; int n1, n2; + if (buf == NULL) { + /* + * Note that buf and bufsize are static. If + * malloc() fails, it will fail at the start + * and not copy only some files. + */ + if (sysconf(_SC_PHYS_PAGES) > + PHYSPAGES_THRESHOLD) + bufsize = MIN(BUFSIZE_MAX, MAXPHYS * 8); + else + bufsize = BUFSIZE_SMALL; + buf = malloc(bufsize * 2); + if (buf == NULL) + err(1, "Not enough memory"); + buf1 = buf; + buf2 = buf + bufsize; + } rv = 0; lseek(from_fd, 0, SEEK_SET); lseek(to_fd, 0, SEEK_SET); while (rv == 0) { - n1 = read(from_fd, buf1, sizeof(buf1)); + n1 = read(from_fd, buf1, bufsize); if (n1 == 0) break; /* EOF */ else if (n1 > 0) { @@ -1264,10 +1296,11 @@ static char * copy(int from_fd, const char *from_name, int to_fd, const char *to_name, off_t size) { + static char *buf = NULL; + static size_t bufsize; int nr, nw; int serrno; char *p; - char buf[MAXBSIZE]; int done_copy; DIGEST_CTX ctx; @@ -1301,7 +1334,22 @@ copy(int from_fd, const char *from_name, int to_fd, const char *to_name, done_copy = 1; } if (!done_copy) { - while ((nr = read(from_fd, buf, sizeof(buf))) > 0) { + if (buf == NULL) { + /* + * Note that buf and bufsize are static. If + * malloc() fails, it will fail at the start + * and not copy only some files. + */ + if (sysconf(_SC_PHYS_PAGES) > + PHYSPAGES_THRESHOLD) + bufsize = MIN(BUFSIZE_MAX, MAXPHYS * 8); + else + bufsize = BUFSIZE_SMALL; + buf = malloc(bufsize); + if (buf == NULL) + err(1, "Not enough memory"); + } + while ((nr = read(from_fd, buf, bufsize)) > 0) { if ((nw = write(to_fd, buf, nr)) != nr) { serrno = errno; (void)unlink(to_name);