Date: Mon, 10 Apr 2023 08:31:28 -0700 From: Rick Macklem <rick.macklem@gmail.com> To: Alexey Dokuchaev <danfe@freebsd.org> Cc: Rick Macklem <rmacklem@freebsd.org>, src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: c98a764c681f - main - cp(1): fix performance issue for large non-sparse file copies Message-ID: <CAM5tNy71DMXUMyQwYFgTwArtMhvXoC9uniE2CzC3BN0CLyx49Q@mail.gmail.com> In-Reply-To: <ZDPAyZiFtkJv79oF@FreeBSD.org> References: <202101030102.10312IAC061762@gitrepo.freebsd.org> <ZDPAyZiFtkJv79oF@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000003ba28b05f8fd12ea
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Mon, Apr 10, 2023 at 12:54=E2=80=AFAM Alexey Dokuchaev <danfe@freebsd.or=
g> wrote:
>
> CAUTION: This email originated from outside of the University of Guelph. =
Do not click links or open attachments unless you recognize the sender and =
know the content is safe. If in doubt, forward suspicious emails to IThelp@=
uoguelph.ca
>
>
> On Sun, Jan 03, 2021 at 01:02:18AM +0000, Rick Macklem wrote:
> > commit c98a764c681f8b70812a9f13a6e61c96aa1a69d2
> >
> > cp(1): fix performance issue for large non-sparse file copies
> >
> > PR252358 reported a serious performance problem when
> > copying a large non-sparse file on a UFS file system.
> > This problem seems to have been caused by a large
> > number of SEEK_HOLE operations, with one done
> > for each copy_file_range(2) call.
> >
> > This patch modifies cp(1) to use a large (SSIZE_MAX)
> > len argument, reducing the number of system calls
> > and resolving the performance issue.
> > char *p;
> >
> > @@ -236,7 +235,7 @@ copy_file(const FTSENT *entp, int dne)
> > do {
> > if (use_copy_file_range) {
> > rcount =3D copy_file_range(from_fd, NULL,
> > - to_fd, NULL, bufsize, 0);
> > + to_fd, NULL, SSIZE_MAX, 0);
>
> Hi Rick,
>
> This change unfortunately breaks copying files in resource-limited
> environments (e.g. many port builders do that to prevent runaway
> processes):
>
> ulimit -f 16384000
> cp -p packages/13.0-i386-wip/All/perl5-5.32.1_3.tbz /tmp ; echo $?
> Filesize limit exceeded
> 153
>
> Previously bufsize was 2097152 which was a lot saner than current
> 9223372036854775807. Perhaps we should set it per getrlimit(2)
> value for RLIMIT_FSIZE?
I think zfs_copy_file_range() needs to use vn_rlimit_fsizex() the
same way that vn_generic_copy_file_range() does.
I have posted the attached patch to D39419.
danfe@. Assuming you were using zfs, could you test this patch?
(You will need an up to date main kernel and, hopefully, the block
cloning stuff has not trashed your zpool.)
rick
>
> ./danfe
>
--0000000000003ba28b05f8fd12ea
Content-Type: application/octet-stream; name="zfscopyrlimit.patch"
Content-Disposition: attachment; filename="zfscopyrlimit.patch"
Content-Transfer-Encoding: base64
Content-ID: <f_lgazr7f60>
X-Attachment-Id: f_lgazr7f60
LS0tIHpmc192bm9wc19vcy5jLnNhdgkyMDIzLTA0LTEwIDA4OjAxOjA1LjkwNTkwNjAwMCAtMDcw
MAorKysgemZzX3Zub3BzX29zLmMJMjAyMy0wNC0xMCAwODoxNDo0MC41NjM3NDMwMDAgLTA3MDAK
QEAgLTYyNDIsNyArNjI0Miw4IEBAIHpmc19mcmVlYnNkX2NvcHlfZmlsZV9yYW5nZShzdHJ1Y3Qg
dm9wX2NvcHlfZmlsZV9yYW5nZQogCXN0cnVjdCBtb3VudCAqbXA7CiAJc3RydWN0IHVpbyBpbzsK
IAlpbnQgZXJyb3I7Ci0JdWludDY0X3QgbGVuID0gKmFwLT5hX2xlbnA7CisJdWludDY0X3QgbGVu
OworCXNzaXplX3QgciA9IDA7CiAKIAkvKgogCSAqIFRPRE86IElmIG9mZnNldC9sZW5ndGggaXMg
bm90IGFsaWduZWQgdG8gcmVjb3Jkc2l6ZSwgdXNlCkBAIC02MjgwLDkgKzYyODEsMTQgQEAgemZz
X2ZyZWVic2RfY29weV9maWxlX3JhbmdlKHN0cnVjdCB2b3BfY29weV9maWxlX3JhbmdlCiAKIAlp
by51aW9fb2Zmc2V0ID0gKmFwLT5hX291dG9mZnA7CiAJaW8udWlvX3Jlc2lkID0gKmFwLT5hX2xl
bnA7Ci0JZXJyb3IgPSB2bl9ybGltaXRfZnNpemUob3V0dnAsICZpbywgYXAtPmFfZnNpemV0ZCk7
CisJZXJyb3IgPSB2bl9ybGltaXRfZnNpemV4KG91dHZwLCAmaW8sIDAsICZyLCBhcC0+YV9mc2l6
ZV90ZCk7CiAJaWYgKGVycm9yICE9IDApCiAJCWdvdG8gb3V0X2xvY2tlZDsKKwlsZW4gPSBpby51
aW9fcmVzaWQ7CisJLyoKKwkgKiBObyBuZWVkIHRvIGNhbGwgdm5fcmxpbWl0X2ZzaXpleF9yZXMg
YmVmb3JlIHJldHVybiwKKwkgKiBzaW5jZSB0aGUgdWlvIGlzIGxvY2FsLgorCSAqLwogCiAJZXJy
b3IgPSB6ZnNfY2xvbmVfcmFuZ2UoVlRPWihpbnZwKSwgYXAtPmFfaW5vZmZwLCBWVE9aKG91dHZw
KSwKIAkgICAgYXAtPmFfb3V0b2ZmcCwgJmxlbiwgYXAtPmFfb3V0Y3JlZCk7Cg==
--0000000000003ba28b05f8fd12ea--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy71DMXUMyQwYFgTwArtMhvXoC9uniE2CzC3BN0CLyx49Q>