From owner-freebsd-current@freebsd.org Sat Jan 2 15:42:14 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EFDF64D413B for ; Sat, 2 Jan 2021 15:42:14 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D7R3B0VW9z4XNd for ; Sat, 2 Jan 2021 15:42:13 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f180.google.com with SMTP id s2so27084075oij.2 for ; Sat, 02 Jan 2021 07:42:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=BI25AOzT9eC7cnSMObWaaokQg9GtJL7+4IWlO1t2BFQ=; b=WtYezzb4lxn5TYBzktVmQ7ySbo7nmzz6dPnayhXKjUCUfcfqJxlDPtqkF84PwwabQb kTEKlJDdTJUj9+xo/S1CWVMfN6pdg/VT7GLTc1M+yNh+Wp8jSs13phiF/61mHh2QCUsh /Gm0bQQGw/cGU2qiLPzB5JtKLG7RuP47P5HWZEMnp4eBplaSpeWRzfNeUCQCEsqtTQNC aaiFL3CC/Xz6VkW17mruueeKsfvnER4iZYq8ayjivaFJWstLhz7kUBzPDjFVrHuauFOr bhHFzsrS27nmts/cJTKGXacqYQBIYdWx5J4YhVF1Lp7kdk2WGFxrZhIdAkKPo8+QV7Bk WDxg== X-Gm-Message-State: AOAM530dflTUD2YS2RxzEOETUQlMbk09XshMyt0dIV/6i5Nz6t4OyDmQ axDB8NrJjcGZ0++iROdWmOE14G6o/2ZW5tEDP5dbeHzcO9TaBw== X-Google-Smtp-Source: ABdhPJxd7bnnd3HLLJzF3C+uHKBtbT7+j9I/EbeTWPvxhXkEkJrFiTcxeuCWYpyB1z3F2U3j0fzhAQRbJMKTkrxLpbg= X-Received: by 2002:aca:dd09:: with SMTP id u9mr13477373oig.73.1609602132873; Sat, 02 Jan 2021 07:42:12 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Alan Somers Date: Sat, 2 Jan 2021 08:42:01 -0700 Message-ID: Subject: Re: cp(1) of large files is causing 100% CPU utilization and poor transfer To: Matthias Apitz , FreeBSD CURRENT X-Rspamd-Queue-Id: 4D7R3B0VW9z4XNd X-Spamd-Bar: -- X-Spamd-Result: default: False [-3.00 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RBL_DBL_DONT_QUERY_IPS(0.00)[209.85.167.180:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[asomers]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_DKIM_NA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; SPAMHAUS_ZRD(0.00)[209.85.167.180:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.180:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.180:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Jan 2021 15:42:15 -0000 On Sat, Jan 2, 2021 at 7:59 AM Matthias Apitz wrote: > > This is with: > > # uname -a > FreeBSD c720-r368166 13.0-CURRENT FreeBSD 13.0-CURRENT #23 r368166M: Thu > Dec 17 13:12:37 CET 2020 guru@c720-r368166:/usr/obj/usr/src/amd64.amd64/sys/GENERIC > amd64 > > I copy often large backup files to an external USB disk and hit the > following problem since updating to r368166: > > A transfer with dd(1) works fast and as expected (~70M / sec): > > # dd if=guru-20210102.tar.gz of=/mnt/AcerC720/backups/guru-20210102.tar.gz > bs=8m > 4601+1 records in > 4601+1 records out > 38603862368 bytes transferred in 506.778929 secs (76174956 bytes/sec) > # ls -lh guru-20210102.tar.gz > -r-------- 1 root wheel 36G 2 ene. 12:19 guru-20210102.tar.gz > > When I use cp(1) what I normaly do the transfer is very poor and causes > 100% CPU itilization: > > # cp -p guru-20210102.tar.gz /mnt/AcerC720/backups/xxx > ^C > > killed after 1 minute; transfered in 1 minute: > > # ls -lh /mnt/AcerC720/backups/xxx > -r-------- 1 root wheel 168M 2 ene. 15:34 /mnt/AcerC720/backups/xxx > > 168*1024*1024/60 = 2936012 bytes/sec ./. 76174956 bytes/sec > > Trussing the cp(1) process shows these sys calls: > > # truss -p 37655 > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000) > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000) > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000) > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000) > copy_file_range(0x3,0x0,0x4,0x0,0x200000,0x0) = 2097152 (0x200000) > > The problem is unrelated to the external disk, also a copy > in the local file system shows the same transfer speed of 168M per > minute. > > Is this a known issue or a regressionc ? > > I see in the man page of copy_file_range(2) that this is new with > FreeBSD 13... > > matthias > Not an issue I've heard of before. Could you please describe how your USB and local disk are formatted? Also, where is the source file stored? It could be that the source file's file system has a very slow implementation of FIOSEEKDATA/FIOSEEKHOLE. -Alan