From nobody Wed Aug 30 08:49:54 2023 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RbJ04499Jz4rtTF for ; Wed, 30 Aug 2023 08:50:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-19.consmr.mail.gq1.yahoo.com (sonic306-19.consmr.mail.gq1.yahoo.com [98.137.68.82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RbJ033mxWz3LPV for ; Wed, 30 Aug 2023 08:50:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=MFVGNVE1; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.82 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693385409; bh=zk3RAs79DTPdy8bHcuEAu1SQuwV1Ljfgf/9f7QxrcBw=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=MFVGNVE18UPSYXhwNlVdMJl0cHVXuUftDLCE/1flWwYFsPtrid0slDkKzpw/u6nKLv0zT3TVpgaBp02hpyWu5PyM3nbxG5fs1vwUshxpbyiW0oWNyOirlCxgi1ui1mg3uhtsIY18jLXh/VKL576rtfUmgmmNSet6ES084q2+O7e4/TLMZeYC+P8NjVrWjJ/7vVs4GeTnxx88aRTFfflC7SYdHPl3DUK+/NCCqts4xeRADFCK8dC0DTy39uQ4RjvE61TapSERTMHDNaZpz8KB36x8tehU5E15LJx3g/Y8ODc7NQJ4uVjmtgckAnvB+AL3cdUv2swjBJFVNQXAtHt+9g== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1693385409; bh=P5hIBoSBKIddN2V11Gd7Xs9TNWW4/1ulrXKKrF7l+xt=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Fbn5qN+6KjxjeEyoXywu9aGn3T/p4ZsWAY52Caqp1EPtaoSqeCZUvrT3HEMI3LEyKCZt29QE4BlSnZdicT3qpAFnRHSHB78MMnNI3ihenNkKDYZ4/KfbDO/mHZFuLOtvxiLc+nksHgbyBDNIsnmz+PscoL9GY3e2Zf21p3BHt+kZVw0xWCi1IZzgnNVw88ijkq5e05wP5zBd3z57sCO0XOJwgeVtQPrgbofaqGDTZpCPIQOh/QefIsAlQZ8SqZ4dX7sexzi4Y4OiTwN81Fi3oim5Ih8RQe003z4M2zcfZllt96c15BQyzAsQGe554cTECRfvUsG0doPLMftqgg5c8A== X-YMail-OSG: Ffd2lCIVM1nqcsBdg_pklrGp1wszUo0CmggcSA4ukafBEiyyvw7PVIZnw7oGs.T ViwwYH.02VFOWYRw9DsZABCi9xKFlrGWcyX7bWn.INknR92KN6lGrqchEdqeAT1VNPlxeDG0Ng5m NmUaVYUIDwJ2uXJcNTcS_KEPHRgw79A7TLUbPHCEd1c6pCIX7mRB0vdFGbrHdugpU4Kawymn.aUn 098dic1.IkCo1hQ7bfLrr0HEQdmjC2QrwSs4mwaRdjB1Ez5Ujww_bIsXzGUDHHGQ.NLCgB_5kn7S f5qF5o7XqOp7w5HilEWyQzmUFb5B9_7LlnqzW3B600eYWlPJpKV5l5R9hc35b10t10pHtqmBB_nu 38cf884OuXTqYdAAaCbv7YqUzwvx1qR5.rtiWa23Ve85TlTMjeyOFhZM76LJolSe8J9648WaMk1s .KslRXGfJTIY2ZzsX411qoxZs6sIw08G9wLoAB5zowOlqaueP_sKYBpmKazRP5nthmVSMTUea9eU TCpNtDqg0od9haUjulgpAaq5C_i6qXkEINlndF92fdVD1f1JUIMLe9TdlDq7UU1IkdYvzwyMpnAQ zY57YobqqWYfhyJ9_I6ZKlHALpccvQ5wKbACmx9xQtWcMOLGhdRql9TIo0DCNhpQj_vT7hH2pr3f x12T7ree10RGBNIxzwhbbtVDT8gHiK8_340KYHfVQJOiDaBVtzWOFnrCxmrsdvoLltlxn_glQDP1 oiWpQrYHxOTqBQ4uRhDy9JuKGN5wpqkx59TipGEF8IzKx4THSiMsk61SCtQ4JmiEuzjpgZA.XBLl lcRP08PUdsR5_PFq.DZoIwhG0jAEPAGr6uuDsYe4H6NMXCouLG7pdpf_F9PR7jP3dXy5.I8ev3JU vc78PgoBk1xXhBnCjpB6cNKyhgAuyRpAH4cS_09uMNHs.xOu5dSEYT7CEr1GjDmJYczNoDWIO7kc XE1Fw9H6WBpftUuPyZQm9f7YEiaX.X3qg9VHPv5y.H92w6Zx0V2n.1MN5oyGo4x.LHyKMS.0m5z0 6hZm_QjIJdtikA.2v1o9SCy.YErHyQ64Ewco2oy3AJkumbZGqG9jR3mOFxmK79KS1y_Pk.i8wQwY UkgtEa6PtznDBnGI5pJP_Oz3kfxMzvpqiJXLnuLYQK7iTxGnTt5G4v8.41vFPANzl37moVjQ.57t FHPWGymna_WUSBzackCeL1CErux1bNpWOCCaKQspw__Gf9tyfUjB.AfBHbNwXPHG5lzGva7rEpYj LAHncrqqn3SDyhUZlRMzjIBVHwAejydQo6Em7W1ossw09vjpOfpxtMUc30e3r0BdlbKqJ4PaIyL8 pm2jU07t7l3S9rXfKMSJCgqoUzRp6L0MZXiA4wThnkaAFLlZDUVplDATPqZyUGdAyHTkE75WK1Ek iyTt4xWE2F7twBrndVQfX29KJCB91Y3WrkmxCM0AGV3duCGXyJtuE_wqBkGJa76Czjly_gQnyCwN XIOSuSD6u_9i4.RDPBj1bVq9Bdxs4wqKZHvzJv_irQeNtV.d7BhFP_lJjfRQ7hvvrG_GJbnqgqe. ZKBESwjLkzmmsIl1midI3iE9YOpajslOXOfmMyQAX7AQ0dulNGvaymmJpawKQ1HpJ71HgvRYLTIU mKdRHDZ4Av5.P0FDJSXVLI_3YZSZlaoY6GCOXFPaTAemrH8iuYNoOfL84ujHxHvIqnwPu8Hy_r8G yvI4r86tBODsf2MSXXxc1ncbD5gUSTnvmCHpVTeyJ6fkbJEJP9UWsYth2xFnMUU6IYKLVVjQ6JHb wPWVE9U_zIpfsNOdzm2m556Yxf4F4RUA_rNINBZ6AkeYXzp1S34UmNElyzu2_r_ic_RPSJiFcd.0 JjpYvd3BJ6TGR6TFYH4ALaaYKQ015sMm6iwc6YpOBdzJ_ub_Q7qHYa2AWGwuomnQ8f47s0gdUG0L N.LFozcXh09XeQ.sA4x0FSNAWoP3WpORYTJDtonEikc206heVY0aLVbCsvY3eZR2k8l9q.qJvR.o UEJjDtfmOgUHiETZd3pOsxfPuZqOXUTp2u9QX5ko6sc8BDyO7InjDR7yCPKcjPbJXjYlB5UsEHxd aZ1HNMN5M9iWchwpRqTRT0Pt.Lrsf6SeHS067BT.ccArB5LFfAmDk0BC27J_TeMes6CxLgmJyVVF Fujhb084tmePO0uFwZSm2xa4odlPriEEEpQnfbbMZTgb1UI5bidKXAdGxVqtgCWJf0lYHxz4bvI8 HpkA2WqB.X7gTqCQekM6x.OFi9Qt87P9wfhSYycZIjq7q34gO_uvyYCWn15VwvhCxVirXhQFnUBU - X-Sonic-MF: X-Sonic-ID: cce7fe73-194f-4c38-af10-3204c3d84a2a Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Wed, 30 Aug 2023 08:50:09 +0000 Received: by hermes--production-gq1-6b7c87dcf5-cjxhs (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c44cc49087e7af26cd526d0c7f457324; Wed, 30 Aug 2023 08:50:05 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Very slow scp performance comparing to Linux Date: Wed, 30 Aug 2023 01:49:54 -0700 References: <87B95CDA-1812-44B8-9356-46631DEA9428@yahoo.com> To: weh@microsoft.com, FreeBSD Hackers In-Reply-To: Message-Id: X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.82:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.82:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4RbJ033mxWz3LPV On Aug 30, 2023, at 01:22, Mark Millard wrote: > On Aug 30, 2023, at 01:17, Mark Millard wrote: >=20 >> On Aug 29, 2023, at 12:52, Mark Millard wrote: >>=20 >>> Wei Hu wrote on >>> Date: Tue, 29 Aug 2023 12:55:35 UTC : >>>=20 >>>> Thanks for the update. Seems the numbers are the same on zfs and = ufs. That's=20 >>>> good to know.=20 >>>>=20 >>>> Yes, your numbers on ARM64 are better than mine on Intel. However, = my original >>>> intention was to find out why scp on Linux is performing much = better than FreeBSD >>>> under the same hardware env.=20 >>>>=20 >>>> Is it possible to try Linux in your ARM64 setting? I am using = Ubuntu 22.04 on ext4=20 >>>> file system. >>>=20 >>>=20 >>> I tried to use the Hyper-V Quick Create on the Windows Dev Kit 2023 >>> to install a Ubuntu 22.04 . (No clue if ext4 would result.) But the >>> Hyper-V UEFI reports for the disk created: >>>=20 >>> 1. SCSI Disk 0,0 >>> The boot loader did not load an operating system. >>>=20 >>> (It then reports the network adapter attempt found no >>> boot image, but that is expected.) >>>=20 >>> That leaves me wondering if Hyper-V Quick Create >>> established a VM file holding Intel/AMD material >>> despite the aarch64 context. >>>=20 >>> Establishing a Ubuntu more directly is not familiar and >>> will have to be a background activity and, so, likely >>> will not be timely. If I did any experiments outside >>> Hyper-V (native booting), they would be with slower >>> USB3 SSD media than I use for FreeBSD. >>>=20 >>> I did notice that Hyper-V Quick Create did not create >>> a fixed sized disk but a dynamic sized one. That is >>> different than what I did for FreeBSD. >>>=20 >>> Also, it was not obvious if you were after aarch64 >>> Hyper-V testing vs. native-boot testing vs. both. So >>> I may have gone the wrong direction from the start. >>> It is possible that I'd find establishing a native-boot >>> easier and then be able to have a VM file created from >>> the media, more like what I did with FreeBSD. >>>=20 >>> The Ubuntu activity likely would not be analogous to >>> the FreeBSD builds having -mcpu=3D optimization used. >>>=20 >>> Back to $work. >>>=20 >>=20 >> I found a sequence of UI operations that worked for >> installing Ubuntu server 22.04.3 into Hyper-V in >> Windows 11 Pro on the Windows Dev Kit 2023 via >> use of a downloaded *.iso . >>=20 >> The kernel that results predates 6.0: >>=20 >> $ uname -ap >> Linux ubwdk23s 5.15.0-82-generic #91-Ubuntu SMP Mon Aug 14 14:19:18 = UTC 2023 aarch64 aarch64 aarch64 GNU/Linux >>=20 >> Using my usual rule of rebooting before the first scp: >>=20 >> $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:FreeBSD-14-TEST.img >> . . . >> = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 431.3MB/s 00:11=20 >>=20 >> $ rm FreeBSD-14-TEST.img >> $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:FreeBSD-14-TEST.img >> . . . >> = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 482.2MB/s 00:10 >>=20 >> Definitely faster than the FreeBSD results that I reported >> earlier, including faster than the ThreadRipper 1950X with >> Optane in a PCIe slot (more like 300 MiBytes/sec). >>=20 >> I again used 6 cores, 24576 MiBytes of RAM, a fixed sized virtual = hard >> disk under Hyper-V. >>=20 >> For reference: >>=20 >> $ lsblk -f >> NAME FSTYPE FSVER LABEL UUID = FSAVAIL FSUSE% MOUNTPOINTS >> loop0 squashfs 4.0 = 0 100% /snap/core20/1977 >> loop1 squashfs 4.0 = 0 100% /snap/lxd/24326 >> loop2 squashfs 4.0 = 0 100% /snap/snapd/19459 >> sda = =E2=94=9C=E2=94=80sda1 vfat FAT32 F7E9-1344 = 1G 1% /boot/efi >> =E2=94=94=E2=94=80sda2 ext4 1.0 = 48a0dbe6-5a99-4b6e-92dc-fe6d8efc6ffe 99.3G 14% / >>=20 >>=20 >>=20 >> An experiment would be to have a small amount if RAM relative >> the file size. That would force it to actually write to media >> for some part of the file copy. >=20 > The wording was poor: "force it" here is just from the > Ubuntu viewpoint. I make no claim to know if Hyper-V > is actually writing the material out to media at the > time vs. later. >=20 >> So using 1024 MiByte of RAM assigned in Hyper-V: >>=20 >> $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:FreeBSD-14-TEST.img >> . . . >> = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 407.5MB/s 00:12 >>=20 >> $ rm FreeBSD-14-TEST.img >> $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:FreeBSD-14-TEST.img >> . . . >> = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 404.7MB/s 00:12 >>=20 >> Still definitely faster than the FreeBSD results that I >> reported earlier, including faster than the ThreadRipper >> 1950X with Optane in a PCIe slot (more like 300 MiBytes/sec). One more variation in ubuntu under Hyper-V, still with 1024 MiBytes of assigned RAM: use of localhost:/dev/null $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = =20 $ scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = markmi@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 492.9MB/s 00:10 The matching FreeBSD examples with 24576 MiBytes of RAM assigned (ZFS = context): # scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = root@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = =20 # scp = FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = root@localhost:/dev/null . . . FreeBSD-14.0-ALPHA2-arm-armv7-GENERICSD-20230818-77013f29d048-264841.img = = 100% 5120MB 198.7MB/s 00:25 Note: At most one VM running at a time, never both in overlapping times. =3D=3D=3D Mark Millard marklmi at yahoo.com