From nobody Sun Dec 17 21:00:52 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Stb2s3nvDz54FC7 for ; Sun, 17 Dec 2023 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Stb2s1TDXz3bBd for ; Sun, 17 Dec 2023 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1702846853; a=rsa-sha256; cv=none; b=qlYdp9LUx6W2XZRUSsjYemDWqxdd8le8RHRgCOupjDMFFmMHwqIPBCurLhU1prWPKpykzM n94BR/RLQ3iECH8QbDYc9bT+2FtFfOpK/Xtc272bBhU9OwDFYCQ/8k2XP4Twho8U3dq60o JCZAvdEvAUUbSzzsLOCymjUz6UIbT1y89DZkrmAHUhcnY1EmAstq/AtKeiwsRfvB3l9hu6 P55KjJU2q4awlYFqsSW5NQj6grkNGw8tlbkfh3OXLAB0VBWuH4VNpRnIxfYTLYq4IXsSHM SU4QozccS9ttI7FJCj1NB4W6H/HVebRA9lw6o0tYj5TPlyduZurR8sj8PaZtbQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1702846853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=y9izTOEkwZ7ZRD5kAhEhGiL1olHxz0ha4Eu9J29wZ+A=; b=uhMDUM1dNkMIZm3S/KRRKmIWoUPYR2HWUTr2GKw4zaBB1kPTUx6jx/CaF1A5ZXxV2CIdR6 qYo46C0Dqg8x6N45MJp1IDwe2U9Xgkkau5SvXbunIKWA6vpuGGPoJ5zaQ7AojRV5RxHj0M 90ADar05qswRPXpZKU+UlHp+FUGKGeCKEDP8g5V88+oc5hhyq64XhfN9YdUpFSLPv3Cvj3 xYoCAk2PglBbia+d6ZcSQhrzE/wNrrBmw8F81KdxAUI3c/gFgRfhf/Mm0CoWtQfdey1dmB gI5wpLArk3dimWKZbt1jXncyM1ObOLvbwxGZYjdizjQ0Ca4K9Y/39rDYpyhx1g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Stb2s0TZwzsqx for ; Sun, 17 Dec 2023 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BHL0rlF017030 for ; Sun, 17 Dec 2023 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BHL0rPF017029 for fs@FreeBSD.org; Sun, 17 Dec 2023 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202312172100.3BHL0rPF017029@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 17 Dec 2023 21:00:52 +0000 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="17028468525.C15A09C.8338" Content-Transfer-Encoding: 7bit --17028468525.C15A09C.8338 Date: Sun, 17 Dec 2023 21:00:52 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 231794 | zfs: Panic due to ARC related KVA memory exhausti Open | 237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w Open | 244692 | gjournal: Does not support TRIM Open | 251035 | ZFS: Allow 64 bit ZFS to support 32 bit ioctls (W Open | 264139 | ata: NCQ_DSM_TRIM trim method for Seagate IronWol Open | 266409 | 13.1-RELEASE amd64 crashes in: sa_handle_destroy Open | 268162 | zfskeys_enable: each successful load of a key is Open | 269503 | docs.freebsd.org: default vfs.zfs.arc.meta_limit Open | 271384 | zfs_load is not suitably documented Open | 226130 | ZFS: solaris assert: zrl->zr_refcount == 0 (0x1 = 10 problems total for which you should take action. --17028468525.C15A09C.8338 Date: Sun, 17 Dec 2023 21:00:52 +0000 MIME-Version: 1.0 Content-Type: text/html; charset="UTF-8" 
The following is a listing of current problems submitted by FreeBSD users,
which need special attention. These represent problem reports covering
all versions including experimental development code and obsolete releases.

Status      |    Bug Id | Description
------------+-----------+---------------------------------------------------
Open        |    231794 | zfs: Panic due to ARC related KVA memory exhausti
Open        |    237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w
Open        |    244692 | gjournal: Does not support TRIM
Open        |    251035 | ZFS: Allow 64 bit ZFS to support 32 bit ioctls (W
Open        |    264139 | ata: NCQ_DSM_TRIM trim method for Seagate IronWol
Open        |    266409 | 13.1-RELEASE amd64 crashes in: sa_handle_destroy 
Open        |    268162 | zfskeys_enable: each successful load of a key is 
Open        |    269503 | docs.freebsd.org: default vfs.zfs.arc.meta_limit
Open        |    271384 | zfs_load is not suitably documented
Open        |    226130 | ZFS: solaris assert: zrl->zr_refcount == 0 (0x1 =

10 problems total for which you should take action.
--17028468525.C15A09C.8338-- From nobody Mon Dec 18 01:45:12 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4StjLx3KPjz54bwm for ; Mon, 18 Dec 2023 01:45:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4StjLw3tDkz4FDZ for ; Mon, 18 Dec 2023 01:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1702863912; a=rsa-sha256; cv=none; b=gN1whW7Ol2ZwbWBgXaWCIMmk1wKT8gIvm94aDOk2GIJRkNiMvt0kzorZb2sy97V83KjK2X NVNWPQCjM7Fy/bBHGlJ6fwLj2sO6UXfj7Yf+VKG9+sI6J+7FDsBaO8sCrdjWEw1LwaGq3T sTj3p41wPrtIBP/Hz+bMP8xn6mqXm+bwt0yHjogJ+Nojnz7PagvfzfOtPO6u4c2MV+k18G 1wj4VlRb5zzL706hTVj0xOJ/rZ93RWKU0NboWNSXOkO+BZ2cGgwtn1muaKRHJl6yn0H6/e g2uO4jGr1/O9k5Ys7LWzDZHYDW3erJ/C6j8KCTUlLdCfY92seotFpE6JTshTdA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1702863912; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=62jGwCATh+a63RjHBIalShk4oyB1TwxFcb2+0AwA7gA=; b=tO11HoSVVXhcX3cTBvgxsCY3e20oNsHb8gxyH99Br5gZHslU6nmFGiGulN4LtUiu9BjKNk fJQX9S0KJfGru0H8C30Or5pFjBYsay81d2M7LwMip6z/uVr66WPLyGG7sOyIXOhFpnWLuo NbmUQ4VIhYzdnWOUwZg1jBFTTiTl7MJ2o7ABpOtED+gmkxvqqJL7oREvIl6FXXFhzfyYVj uiY0y7dFaypZPyVx5EVwKX4C6nSRsQdBXcFuq1la30yCvECyii4lODRx73TCStfldVtAGp lhXn1OqkpFGCh1EmbSFtB3a+DyzJTLJK6HgbRtqYnYTRoxa2K/mLFdzvq+79DA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4StjLw2tWJz1271 for ; Mon, 18 Dec 2023 01:45:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BI1jCOt037971 for ; Mon, 18 Dec 2023 01:45:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BI1jCgx037970 for fs@FreeBSD.org; Mon, 18 Dec 2023 01:45:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275597] Samba: smbd sometimes aborts by PANIC when 'vfs objects = cap' Date: Mon, 18 Dec 2023 01:45:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: uratan@miomio.jp X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275597 --- Comment #2 from uratan --- Created attachment 247109 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D247109&action= =3Dedit a script to make a set of test files that is used by the debugging I noticed that I could do printf-debugging by setting the debug level in /usr/local/etc/smb4.conf, ... (modification to) /usr/local/etc/smb4.conf +----------------------------------------- |; max log size =3D 50 | max log size =3D 9999 | log level =3D 6 +----------------------------------------- ... so I decided to start tracking down the cause by myself. I made samba-4.16.11 (same version as packages) from ports and gathered information with the DEBUG() macro. which are output to /var/log/samba4/log.. - * - * - I confirmed at first the place where the 'Segmentation fault' were caused at, it was in function cap_readdir() shown below. (/usr/ports/net/samba416/work/samba-4.16.11/source3/modules/vfs_cap.c) +------------------------------------------------------------------ 87 |static struct dirent *cap_readdir(vfs_handle_struct *handle, 88 | struct files_struct *dirfsp, 89 | DIR *dirp, 90 | SMB_STRUCT_STAT *sbuf) 91 |{ 92 | struct dirent *result; 93 | struct dirent *newdirent; 94 | char *newname; 95 | size_t newnamelen; 96 | DEBUG(3,("cap: cap_readdir\n")); 97 | 98 | result =3D SMB_VFS_NEXT_READDIR(handle, dirfsp, dirp, NULL); 99 | if (!result) { 100 | return NULL; 101 | } 102 | 103 | newname =3D capdecode(talloc_tos(), result->d_name); 104 | if (!newname) { 105 | return NULL; 106 | } 107 | DEBUG(3,("cap: cap_readdir: %s\n", newname)); 108 | newnamelen =3D strlen(newname)+1; 109 | newdirent =3D talloc_size( 110 | talloc_tos(), sizeof(struct dirent) + newnamelen); 111 | if (!newdirent) { 112 | return NULL; 113 | } 114 | talloc_set_name_const(newdirent, "struct dirent"); **115 | memcpy(newdirent, result, sizeof(struct dirent)); **116 | memcpy(&newdirent->d_name, newname, newnamelen); 117 | return newdirent; 118 |} +------------------------------------------------------------------ In the function cap_readdir(), there are 2 memcpy()s used just before return statement. I confirmed that the 'Segmentation fault' were caused only at first memcpy(), the second memcpy() has no problem, with DEBUG() like below. +------------------------------------------------------------------ 87 |static struct dirent *cap_readdir(vfs_handle_struct *handle, | -- omitted -- | 114 | talloc_set_name_const(newdirent, "struct dirent"); |DEBUG(3,("@@u@@ 456\n")); |memcpy(&newdirent->d_name, newname, newnamelen); //// dup of #= 116 |DEBUG(3,("@@u@@ 888\n")); **115 | memcpy(newdirent, result, sizeof(struct dirent)); |DEBUG(3,("@@u@@ 789\n")); **116 | memcpy(&newdirent->d_name, newname, newnamelen); |DEBUG(3,("@@u@@ 999\n")); 117 | return newdirent; 118 |} +------------------------------------------------------------------ | | V V (excerpt of /var/log/samba4/log.argon near PANIC abort with above codes) +------------------------------------------------------------------ | : | cap: cap_readdir | cap: cap_readdir: igfxCPL.cpl | @@u@@ 456 | @@u@@ 888 | @@u@@ 789 | @@u@@ 999 | smbd_dirptr_get_entry: dirptr 0x2af766c0 now at offset 7585 | fsp_new: allocated files structure (508 used) | fget_ea_dos_attribute: Cannot get attribute from EA on file win/tst/teHW2/N3x50B-ITX/v15.38.2.64.4189/Graphics/igfxCPL.cpl: Error =3D Attribute not found | dos_mode_debug_print: fdos_mode returning (0x80): "" | smbd_dirptr_get_entry mask=3D[*] found win/tst/teHW2/N3x50B-ITX/v15.38.2.64.4189/Graphics/igfxCPL.cpl fname=3DigfxCPL.cpl (igfxCPL.cpl) | file_free: freed files structure 0 (507 used) | cap: cap_readdir | cap: cap_readdir: igfxCUIService.exe | @@u@@ 456 | @@u@@ 888 | @@u@@ 789 | @@u@@ 999 | smbd_dirptr_get_entry: dirptr 0x2af766c0 now at offset 7681 | fsp_new: allocated files structure (508 used) | fget_ea_dos_attribute: Cannot get attribute from EA on file win/tst/teHW2/N3x50B-ITX/v15.38.2.64.4189/Graphics/igfxCUIService.exe: Erro= r =3D Attribute not found | dos_mode_debug_print: fdos_mode returning (0x80): "" | smbd_dirptr_get_entry mask=3D[*] found win/tst/teHW2/N3x50B-ITX/v15.38.2.64.4189/Graphics/igfxCUIService.exe fname=3DigfxCUIService.exe (igfxCUIService.exe) | file_free: freed files structure 0 (507 used) | cap: cap_readdir | cap: cap_readdir: igfxCUIServicePS.dll | @@u@@ 456 | @@u@@ 888 | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | INTERNAL ERROR: Signal 11: Segmentation fault in pid 16090 (4.16.11) | If you are running a recent Samba version, and if you think .... | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | PANIC (pid 16090): Signal 11: Segmentation fault in 4.16.11 | BACKTRACE: | #0 log_stack_trace + 0x43 [ip=3D0x203e8643] [sp=3D0xffbfd35c] | : +------------------------------------------------------------------ - * - * - I tried various things hoping something would change the situation for about a week (only within the cap_readdir()). Finally, after over 20 builds & tests, I reached some solution that, writing down the first memcpy() by replacing with simple assignment statements (see below) made the problem disappeared. +------------------------------------------------------------------ 87 |static struct dirent *cap_readdir(vfs_handle_struct *handle, | -- omitted -- | 114 | talloc_set_name_const(newdirent, "struct dirent"); |#define Q_HACK_U 1 |#if Q_HACK_U | newdirent->d_pad1 =3D result->d_pad1 ; | newdirent->d_namlen =3D result->d_namlen ; | newdirent->d_pad0 =3D result->d_pad0 ; | newdirent->d_type =3D result->d_type ; | newdirent->d_reclen =3D result->d_reclen ; | newdirent->d_off =3D result->d_off ; | newdirent->d_fileno =3D result->d_fileno ; |#else /* Q_HACK_U */ **115 | memcpy(newdirent, result, sizeof(struct dirent)); |#endif /* Q_HACK_U */ **116 | memcpy(&newdirent->d_name, newname, newnamelen); 117 | return newdirent; 118 |} +------------------------------------------------------------------ note: I considered that the bottom side of the memory area is more safe because the second memcpy() did not cause any problems, so it is the reason why the structure member assignments are in reverse order (to insert further DEBUG()s if needed). Also member d_name[] is not copied manually because it is done by second memcpy(). - * - * - Since there was nothing unnatural about the pointer values, etc. by the result of another research with DEBUG() like below, ... +------------------------------------------------------------------ 87 |static struct dirent *cap_readdir(vfs_handle_struct *handle, | -- omitted -- | |DEBUG(3,("@u@ %lx %lx %x\n", newdirent, result, sizeof(struct dirent= ))); **115 | memcpy(newdirent, result, sizeof(struct dirent)); +------------------------------------------------------------------ | | V V (excerpt of /var/log/samba4/log.argon near PANIC abort with above codes) +------------------------------------------------------------------ | : | cap: cap_readdir: HDXRtGi.inf | @u@ 2a7b6130 22e82e90 118 | @@u@@ 999 | cap: cap_readdir: HDXSBCH.inf | @u@ 2a7b6430 22e82eb8 118 | @@u@@ 999 | cap: cap_readdir: HDXSEDS.inf | @u@ 2a7b65b0 22e82ee0 118 | @@u@@ 999 | cap: cap_readdir: HDXSF.inf | @u@ 2a7b6730 22e82f08 118 | =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D | INTERNAL ERROR: Signal 11: Segmentation fault in pid 94386 (4.16.11) | : +------------------------------------------------------------------ ... I can only assume that there is something in the implementation of memcpy() that is incompatible with smbd's behavior... (maybe) (multi-core ? or threading ?) Well, however, I feel very very very happy right now because the diff.exe works well now. (and quiet /var/log/messages) - * - * - ADDITIONAL INFO [a] I also modified the version message to distinguish my smbd. (/usr/ports/net/samba416/work/samba-4.16.11/source3/include/smb.h) +------------------------------------------------------------------ 31 | 32 |/* logged when starting the various Samba daemons */ |#define Q_HACK_U 1 |#if Q_HACK_U |#define COPYRIGHT_STARTUP_MESSAGE "Copyright Andrew Tridgell a= nd the Samba Team 1992-2023 +u" |#else /* Q_HACK_U */ 33 |#define COPYRIGHT_STARTUP_MESSAGE "Copyright Andrew Tridgell a= nd the Samba Team 1992-2022" |#endif /* Q_HACK_U */ 34 | +------------------------------------------------------------------ [b] The subjected 14.0R uses UFS file system. (/home is shared by Samba) +----------------------------------------- | % mount |/dev/ada0p3 on / (ufs, local, noatime) |devfs on /dev (devfs) |/dev/ada0p4 on /usr (ufs, local, noatime, soft-updates) |/dev/ada0p5 on /home (ufs, NFS exported, local, noatime, soft-updates) |/dev/md1 on /mnt (ufs, local) |fdescfs on /var/run/samba4/fd (fdescfs) +----------------------------------------- [c] The depth of the directory hierarchy might have no relations to this problem. Attached sh-scripts: z2-make-test-files.sh makes a set of test files that was used by above debugging. +----------------------------------------- |% sh z2-make-test-files.sh |....... : | ...... done | |% du -d2 -m teHWs/ |620 teHWs/N3x50B-ITX/7464 |621 teHWs/N3x50B-ITX/7492 |322 teHWs/N3x50B-ITX/v15.38.2.64.4189 |261 teHWs/N3x50B-ITX/v15.40.0.4177_BSW |1822 teHWs/N3x50B-ITX |1822 teHWs/ | |% find teHWs/ -type f | wc -l | 1591 |% find teHWs/ -type d | wc -l | 135 +----------------------------------------- Since this problem may or may not appear once per comparison by diff.exe below (about 1min), so the frequency of occurrence is about less less less than 1/3182 (1591 files * 2 trials) in my environment... +----------------------------------------- |U:\> diff.exe -rq j:\win\tst\teHWs u:\tst\teHWs (u: is win7 local di= sk) +----------------------------------------- --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Mon Dec 18 07:59:20 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Stsft5cVjz546d7 for ; Mon, 18 Dec 2023 07:59:34 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Received: from umail2.aei.mpg.de (umail2.aei.mpg.de [194.94.224.8]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Stsfp26RWz3Hmc for ; Mon, 18 Dec 2023 07:59:30 +0000 (UTC) (envelope-from gerrit.kuehn@aei.mpg.de) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of gerrit.kuehn@aei.mpg.de designates 194.94.224.8 as permitted sender) smtp.mailfrom=gerrit.kuehn@aei.mpg.de; dmarc=none Received: from arc.aei.uni-hannover.de (ahgate1.aei.uni-hannover.de [130.75.117.49]) by umail2.aei.mpg.de (Postfix) with ESMTPS id 5F04B1F9123B for ; Mon, 18 Dec 2023 08:59:27 +0100 (CET) Date: Mon, 18 Dec 2023 08:59:20 +0100 From: Gerrit =?UTF-8?B?S8O8aG4=?= To: freebsd-fs@freebsd.org Subject: question on zfs canmount vs. zpool import vs. zfs mount Message-ID: <20231218085920.17247d47@arc.aei.uni-hannover.de> Organization: MPG X-Mailer: Claws Mail 3.19.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.1) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="Sig_/I6VpOGWuBjiIePrm7S6E858"; protocol="application/pkcs7-signature"; micalg=SHA384 X-Spamd-Result: default: False [-5.80 / 15.00]; SIGNED_SMIME(-2.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; RWL_MAILSPIKE_VERYGOOD(-0.20)[194.94.224.8:from]; R_SPF_ALLOW(-0.20)[+ip4:194.94.224.8]; RCVD_IN_DNSWL_MED(-0.20)[194.94.224.8:from]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:680, ipnet:194.94.0.0/15, country:DE]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[mpg.de]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4Stsfp26RWz3Hmc X-Spamd-Bar: ----- --Sig_/I6VpOGWuBjiIePrm7S6E858 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hello, I have a question about the zfs canmount property. Let's suppose a filesystem like this (simplified): pool/system1/tank/dir1/dir2 pool/system2/tank/dir1/dir2 I use something like this for backups of systems with (usually more than one) zfs filesystems. I use syncoid for these backups which recommends not mounting the filesystems on the backup server due to -F being used (and indeed this can cause trouble). So ideally I'd like to set canmount=3Dnoauto on the "system/tank" level to prevent anything beyond if from being mounted on the backup server. This would also allow to simply create new filesystems (somewhere beyond "tank") and have them backed up without any further interaction using a recursive backup strategy. However, the canmount property is not inherited, and during reboot of the backup system FreeBSD uses "zpool import -a -N" in rc.d/zpool and "zfs mount -a" in "rc.d/zfs". This will mount all filesystems at the "dir" level even when the tank above them is set to "canmount=3Dnoauto". The only way to prevent this is to set all systems on the "dir" level to "noauto". This is quite tedious and prone-to-error because it has to be done manually for every new filesystem created on the lower levels. Interestingly, "zpool import -a" appears to do exactly what I want: it stops mounting filesystems at the first level that has canmount=3Dnoauto set. It is a bit strange to see that "zfs mount -a" afterwards behaves differently. Is this intended behaviour? Is there any clean way to achieve the behaviour I need? I can remove the "-N" in rc.d/zpool and skip the "zfs mount -a" step in rc.d/zfs to do this, but patching rc.d scripts is not really a clean solution, either. cu Gerrit --Sig_/I6VpOGWuBjiIePrm7S6E858 Content-Type: application/pkcs7-signature; name=smime.p7s Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=smime.p7s MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgIFADCABgkqhkiG9w0B BwEAAKCCF/QwggQyMIIDGqADAgECAgEBMA0GCSqGSIb3DQEBBQUAMHsxCzAJBgNV BAYTAkdCMRswGQYDVQQIDBJHcmVhdGVyIE1hbmNoZXN0ZXIxEDAOBgNVBAcMB1Nh bGZvcmQxGjAYBgNVBAoMEUNvbW9kbyBDQSBMaW1pdGVkMSEwHwYDVQQDDBhBQUEg Q2VydGlmaWNhdGUgU2VydmljZXMwHhcNMDQwMTAxMDAwMDAwWhcNMjgxMjMxMjM1 OTU5WjB7MQswCQYDVQQGEwJHQjEbMBkGA1UECAwSR3JlYXRlciBNYW5jaGVzdGVy MRAwDgYDVQQHDAdTYWxmb3JkMRowGAYDVQQKDBFDb21vZG8gQ0EgTGltaXRlZDEh MB8GA1UEAwwYQUFBIENlcnRpZmljYXRlIFNlcnZpY2VzMIIBIjANBgkqhkiG9w0B AQEFAAOCAQ8AMIIBCgKCAQEAvkCd9G7h6naHHE1FRI6+RsiDBp3BKv4YH47kAvrz q11QihYxC5oG0MVwIs1JLVRjzLZuaEYLU+rLTCTAvHJO6vEVrvRUmhIKw3qyM2Di 2olV8yJY897cz++DhqKMlE+faPKYkEaEJ8d2v+PMNSyLXgdkZYLASLCokflhn3Yg UKiRx2a163hiA1bwihoT6jGjHqCZ/Tj29icyWG8H9Wu4+xQrr7eqzNZjX3OM2gWZ qDioyxd4NlGs6Z70eDqNzw/ZQuKYDKsvnw4B3u+fmUnxLd+sdE0bmLVHxeUp0fmQ GMdinL6DxyZ7Poolx8DdneY1aBAgnY/Y3tLDhJwNXugvyQIDAQABo4HAMIG9MB0G A1UdDgQWBBSgEQojPpbxB+zirynvgqV/0DCktDAOBgNVHQ8BAf8EBAMCAQYwDwYD VR0TAQH/BAUwAwEB/zB7BgNVHR8EdDByMDigNqA0hjJodHRwOi8vY3JsLmNvbW9k b2NhLmNvbS9BQUFDZXJ0aWZpY2F0ZVNlcnZpY2VzLmNybDA2oDSgMoYwaHR0cDov L2NybC5jb21vZG8ubmV0L0FBQUNlcnRpZmljYXRlU2VydmljZXMuY3JsMA0GCSqG SIb3DQEBBQUAA4IBAQAIVvwC8Jvo/6T61nvGRIDOT8TF9gBYzKa2vBRJaAR26Obu XewCD2DWjVAYTyZOAePmsKXuv7x0VEG//fwSuMdPWvSJYAV/YLcFSvP28cK/xLl0 hrYtfWvM0vNG3S/G4GrDwzQDLH2W3VrCDqcKmcEFi6sML/NcOs9sN1UJh95TQGxY 7/y2q2VuBPYb3DzgWhXGntnxWUgwIWUDbOzpIXPsmwOh4DetoBUYj/q6As6nLKkQ EyzU5QgmqyKXYPiQXnTUoppTvfKpaOCibsLXbLGjD56/62jnVvKu8uMrODoJgbVr hde+Le0/GreyY+L1YiyC1GoAQVDxOYOflek2lphuMIIFgTCCBGmgAwIBAgIQOXJE Ovkit1HX02wQ3TE1lTANBgkqhkiG9w0BAQwFADB7MQswCQYDVQQGEwJHQjEbMBkG A1UECAwSR3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHDAdTYWxmb3JkMRowGAYD VQQKDBFDb21vZG8gQ0EgTGltaXRlZDEhMB8GA1UEAwwYQUFBIENlcnRpZmljYXRl IFNlcnZpY2VzMB4XDTE5MDMxMjAwMDAwMFoXDTI4MTIzMTIzNTk1OVowgYgxCzAJ BgNVBAYTAlVTMRMwEQYDVQQIEwpOZXcgSmVyc2V5MRQwEgYDVQQHEwtKZXJzZXkg Q2l0eTEeMBwGA1UEChMVVGhlIFVTRVJUUlVTVCBOZXR3b3JrMS4wLAYDVQQDEyVV U0VSVHJ1c3QgUlNBIENlcnRpZmljYXRpb24gQXV0aG9yaXR5MIICIjANBgkqhkiG 9w0BAQEFAAOCAg8AMIICCgKCAgEAgBJlFzYOw9sIs9CsVw127c0n00ytUINh4qog TQktZAnczomfzD2p7PbPwdzx07HWezcoEStH2jnGvDoZtF+mvX2do2NCtnbyqTsr kfjib9DsFiCQCT7i6HTJGLSR1GJk23+jBvGIGGqQIjy8/hPwhxR79uQfjtTkUcYR Z0YIUcuGFFQ/vDP+fmyc/xadGL1RjjWmp2bIcmfbIWax1Jt4A8BQOujM8Ny8nkz+ rwWWNR9XWrf/zvk9tyy29lTdyOcSOk2uTIq3XJq0tyA9yn8iNK5+O2hmAUTnAU5G U5szYPeUvlM3kHND8zLDU+/bqv50TmnHa4xgk97Exwzf4TKuzJM7UXiVZ4vuPVb+ DNBpDxsP8yUmazNt925H+nND5X4OpWaxKXwyhGNVicQNwZNUMBkTrNN9N6frXTps NVzbQdcS2qlJC9/YgIoJk2KOtWbPJYjNhLixP6Q5D9kCnusSTJV882sFqV4Wg8y4 Z+LoE53MW4LTTLPtW//e5XOsIzstAL81VXQJSdhJWBp/kjbmUZIO8yZ9HE0XvMns QybQv0FfQKlERPSZ51eHnlAfV1SoPv10Yy+xUGUJ5lhCLkMaTLTwJUdZ+gQek9Qm RkpQgbLevni3/GcV4clXhB4PY9bpYrrWX1Uu6lzGKAgEJTm4Diup8kyXHAc/DVL1 7e8vgg8CAwEAAaOB8jCB7zAfBgNVHSMEGDAWgBSgEQojPpbxB+zirynvgqV/0DCk tDAdBgNVHQ4EFgQUU3m/WqorSs9UgOHYm8Cd8rIDZsswDgYDVR0PAQH/BAQDAgGG MA8GA1UdEwEB/wQFMAMBAf8wEQYDVR0gBAowCDAGBgRVHSAAMEMGA1UdHwQ8MDow OKA2oDSGMmh0dHA6Ly9jcmwuY29tb2RvY2EuY29tL0FBQUNlcnRpZmljYXRlU2Vy dmljZXMuY3JsMDQGCCsGAQUFBwEBBCgwJjAkBggrBgEFBQcwAYYYaHR0cDovL29j c3AuY29tb2RvY2EuY29tMA0GCSqGSIb3DQEBDAUAA4IBAQAYh1HcdCE9nIrgJ7cz 0C7M7PDmy14R3iJvm3WOnnL+5Nb+qh+cli3vA0p+rvSNb3I8QzvAP+u431yqqcau 8vzY7qN7Q/aGNnwU4M309z/+3ri0ivCRlv79Q2R+/czSAaF9ffgZGclCKxO/WIu6 pKJmBHaIkU4MiRTOok3JMrO66BQavHHxW/BBC5gACiIDEOUMsfnNkjcZ7Tvx5Dq2 +UUTJnWvu6rvP3t3O9LEApE9GQDTF1w52z97GA1FzZOFli9d31kWTz9RvdVFGD/t So7oBmF0Ixa1DVBzJ0RHfxBdiSprhTEUxOipakyAvGp4z7h/jnZymQyd/teRCBah o1+VMIIG5jCCBM6gAwIBAgIQMQJw1DW+mySa+FbQ4eKFSTANBgkqhkiG9w0BAQwF ADCBiDELMAkGA1UEBhMCVVMxEzARBgNVBAgTCk5ldyBKZXJzZXkxFDASBgNVBAcT C0plcnNleSBDaXR5MR4wHAYDVQQKExVUaGUgVVNFUlRSVVNUIE5ldHdvcmsxLjAs BgNVBAMTJVVTRVJUcnVzdCBSU0EgQ2VydGlmaWNhdGlvbiBBdXRob3JpdHkwHhcN MjAwMjE4MDAwMDAwWhcNMzMwNTAxMjM1OTU5WjBGMQswCQYDVQQGEwJOTDEZMBcG A1UEChMQR0VBTlQgVmVyZW5pZ2luZzEcMBoGA1UEAxMTR0VBTlQgUGVyc29uYWwg Q0EgNDCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBALNK4iJeJ1vpBFsU BDUyIBSutNIxQMbNUMAeoUTKr55KYX8tkN5imzNqLaRCypYBPP9wED2AaO6e8njk bjzJwLgPqDBkW9sG3kmi3GW6cF4Hwr5ysZqve/5EJDhV+9OhfTu/4dMnoR4Q41Hc jMk9MzLOADAQ0awBZ/29r0d49AUmIKELNeqEqmnTN6fndL7x/2K0TLToZLxqS7sy /Jvi0wEFr0CfdjcAsioh7KaD+Jizyb1aRKQzJ6Q20VEHX7UqWc1SkzTkbz6xj0S5 ydBBFQh0fNiy+qM/deVpK4HgmPSJrrpQZ+LlbHfWabmwoDPxF71QZVYiqrrAoUrG RJ+47iLBiIg8miIYS7Hd2ppvAUt24CugMXUjETjQ+oYh09fNi5n/AvoER8UBvTHL xt+blL0bvL+2z2YiUWk+2Qtn+dD+JU5Z2y71qV7+cr+4YXjvGzF5bYsi8HiwflTb 4Php3y+k1twKtchdcq2QGc0eDG6Y01nRHUiyr8/PtMAsLHEPNZ2wzsA7fb8mftHi V20ZFmYqknJ8AIOfwdTVA+E62JayOJ+sxadqcmFDorsz/mrPwGZ8+txr4xSuvVjg 0dlv0yuA+1YpBDIYNfL4bkX+IcZ1mTstL4Xw0f4N2iW3bBmnPnYmoYxMM8gflCiT gss73nBvG2f7v1PD7BDGYNO4iD4vAgMBAAGjggGLMIIBhzAfBgNVHSMEGDAWgBRT eb9aqitKz1SA4dibwJ3ysgNmyzAdBgNVHQ4EFgQUaQChxyFY+ODFGyCwCt2nUb8T 2eQwDgYDVR0PAQH/BAQDAgGGMBIGA1UdEwEB/wQIMAYBAf8CAQAwHQYDVR0lBBYw FAYIKwYBBQUHAwIGCCsGAQUFBwMEMDgGA1UdIAQxMC8wLQYEVR0gADAlMCMGCCsG AQUFBwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzBQBgNVHR8ESTBHMEWgQ6BB hj9odHRwOi8vY3JsLnVzZXJ0cnVzdC5jb20vVVNFUlRydXN0UlNBQ2VydGlmaWNh dGlvbkF1dGhvcml0eS5jcmwwdgYIKwYBBQUHAQEEajBoMD8GCCsGAQUFBzAChjNo dHRwOi8vY3J0LnVzZXJ0cnVzdC5jb20vVVNFUlRydXN0UlNBQWRkVHJ1c3RDQS5j cnQwJQYIKwYBBQUHMAGGGWh0dHA6Ly9vY3NwLnVzZXJ0cnVzdC5jb20wDQYJKoZI hvcNAQEMBQADggIBAAoFTnsNjx8TOQD9b+xixsPt7Req4wHMeNw/R5dddEPgQAQA YJZKz5BEv1cjGbH7nbPH3AxrxhN6OVH40p6OLIo9MXSrrfMzGs7/P+FTCjwgNxFE tLQ1KC9NboA3asJcl7mIs3l8h9iAgEH1zLUvq2s+5n++NQmbzudDsTFDMapY3kX1 TwyUCTRzmItqcbsYIyg2MeIXWfRtqPqC5R4bufmpzA5BPINLX340Sp/CNQ9QZqw3 VkfyHWwTo+vO9Gm2L6srNamJT6Lb+TeXZvl8UPL5a72O/pH0GgGHjt6z9QzPARna RKshVWviNK6ST4WmZHllu3CJg0BXqx1vWyswawgvNeWt1qxITacYe9mSWTbNR2Cf tvTUwerruDSY2jMaZPoNqbjUpuG/blYwWzzvVerBUhviAahPXJF/9V48ybWPBq6q KOEokW+s3B4ad5sY96KlovEijaIQDip1HO0SD+rLNYaiBcr9MV2aK+DfbZ8w9BaN CQyFEYwzxIKOVk3bYvzHRk5ihUDascmbk/bkiNl74c/KfuKQmJImaqWoWZR6jBcX cPV0WUIKz/nILTpFhGojZEQW77by3aezAi9jrEIUBHRG1LwzPbJc2V3SOzYyaJFQ atzuKZbN1Q9s9y/2x1QXtKwREY8jNgvx0iIfOK35gKgYJJcyDql4XfuEc2nVMIIH SzCCBTOgAwIBAgIRAMCEqCZW/bEp9AgcdlGEWuEwDQYJKoZIhvcNAQEMBQAwRjEL MAkGA1UEBhMCTkwxGTAXBgNVBAoTEEdFQU5UIFZlcmVuaWdpbmcxHDAaBgNVBAMT E0dFQU5UIFBlcnNvbmFsIENBIDQwHhcNMjMwODE1MDAwMDAwWhcNMjYwODE0MjM1 OTU5WjCB0zEOMAwGA1UEERMFODA1MzkxRzBFBgNVBAoMPk1heC1QbGFuY2stR2Vz ZWxsc2NoYWZ0IHp1ciBGw7ZyZGVydW5nIGRlciBXaXNzZW5zY2hhZnRlbiBlLlYu MRswGQYDVQQJDBJIb2ZnYXJ0ZW5zdHJhw59lIDgxDzANBgNVBAgTBkJheWVybjEL MAkGA1UEBhMCREUxFTATBgNVBAMTDEdlcnJpdCBLdWVobjEmMCQGCSqGSIb3DQEJ ARYXZ2Vycml0Lmt1ZWhuQGFlaS5tcGcuZGUwggIiMA0GCSqGSIb3DQEBAQUAA4IC DwAwggIKAoICAQCg7n7fRC0hIeomyBYF0RZ0L/jKjURwqPL3vBN+HvDxzp+Wcn0a Voeia3LPeXvf18d7BeIQ2SVFXWnWzVpVKzv7VUg4OD424GmcQrFXkChSvOc/rLaA FmNIaKWgYwUOAqmDh3t9JzQTVj6FrAeJwzXmnv42msNUfnhA2dRllOCmilLUqm/5 nOgrImuiA3R1S0CcljAmEr5PnUmKJaanbaq74Jb54gf622cRyWwylMJijMGboDYw uaGynrLgfo+rWbXc2TASO6pjSQDKAAfXO/NzLgp+BmneN1II9alVUAJRUpFDkgx9 peM+qUJryLtO+veOKElsOe2S4qvk0PaE/MVAcIJiThdY7qde8Q9FyOJsDN5kiX4g fsKmtF7EdB71Uc8N78L62r7/7Y5WL8gRxXCN8BsmLXSiCylvtIYsbJMDhK6C+37w 9Cg1A8AWeksg1TmCcvolEJy3+bfPx7NlmEfRdkdzuVb1KxfB0z4SbhSwOAR1WYVg mEAQuj1l9k7suUtdUY4ZeMnRLVPtmQh+bxcJPaRllpHSTYbYVQlSNXkP0al2/J8d jJHhulOsCX8oYfyQ9a33jHsKUf632Lpg8446ym19UrNPh9pntXRXVhhkw+/tPE8G BxH81BCvvSUhVu0Nckx8zOWiI1+6Z5t71udnXOEv9lJFvqDlY71lkiu+jQIDAQAB o4IBpDCCAaAwHwYDVR0jBBgwFoAUaQChxyFY+ODFGyCwCt2nUb8T2eQwHQYDVR0O BBYEFOsacOXMtCWA1hWcY7A/tb8e/tTSMA4GA1UdDwEB/wQEAwIFoDAMBgNVHRMB Af8EAjAAMB0GA1UdJQQWMBQGCCsGAQUFBwMEBggrBgEFBQcDAjA/BgNVHSAEODA2 MDQGCysGAQQBsjEBAgJPMCUwIwYIKwYBBQUHAgEWF2h0dHBzOi8vc2VjdGlnby5j b20vQ1BTMEIGA1UdHwQ7MDkwN6A1oDOGMWh0dHA6Ly9HRUFOVC5jcmwuc2VjdGln by5jb20vR0VBTlRQZXJzb25hbENBNC5jcmwweAYIKwYBBQUHAQEEbDBqMD0GCCsG AQUFBzAChjFodHRwOi8vR0VBTlQuY3J0LnNlY3RpZ28uY29tL0dFQU5UUGVyc29u YWxDQTQuY3J0MCkGCCsGAQUFBzABhh1odHRwOi8vR0VBTlQub2NzcC5zZWN0aWdv LmNvbTAiBgNVHREEGzAZgRdnZXJyaXQua3VlaG5AYWVpLm1wZy5kZTANBgkqhkiG 9w0BAQwFAAOCAgEAbUB7zWvNZ98vh3u7hzpnbA1K4U9bga1YkpVbOgv7/UY5RiZP Rk06O18f5TnRSWiiF3XImBG1uVjbcwVKIemliCQRQzVVt2JXOJVT1EafDDe9DK5o QaXGHY7NAT1lPLEwtgv8hxBBvthMaMa6lpibT/IUi83jHPZUgsGajCgPXd05Bh/L jCzWDOmHuwFdjRAMQs1VsPYx+OVcRvS1jmw0bT6o5/nruRwF5brxUK39Mftj3sIN b+UvVkXdAGw5iQWFwllGpwBgo3iESa1R72qkBMWph8D6Jbg795WBgjMULCPTiZkq eOif9sW1/37AoutSh7VMh7WMrEW9QURVWYR1hYjS0/TMo8aXfPOLtLYoSg/R6i+j eXqREsJQxMAl0e/JJej1TAFCsWg0r6Dg4mYq636plAr6pu7pJATNVPT0HrsBMYWu PV2WRH8Obs+n1xe4ftGxE4yDWiL56lnp6tnfVR8qinEqpGBfj7BAwEcO/Na9b+oK tDEmWHzupKkdmoOWktURY+Q/5RVWoiozNujYljc9iaK3agqBbJ5ZzRyrCKOPLnw4 9b8koO03WkXPqlm59nxAOdJE6ZQ2aQ8ev6ji+UlGnlIvgk70MsRukY2shpAiowb6 bKjKyK3QGNnT4zmL6ixSRmnYhC95U923Yf+hy+6jqS1Ec6kgpREYG53Qv5IxggMs MIIDKAIBATBbMEYxCzAJBgNVBAYTAk5MMRkwFwYDVQQKExBHRUFOVCBWZXJlbmln aW5nMRwwGgYDVQQDExNHRUFOVCBQZXJzb25hbCBDQSA0AhEAwISoJlb9sSn0CBx2 UYRa4TANBglghkgBZQMEAgIFAKCBozAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcB MBwGCSqGSIb3DQEJBTEPFw0yMzEyMTgwNzU5MjBaMCgGCSqGSIb3DQEJDzEbMBkw CwYJYIZIAWUDBAECMAoGCCqGSIb3DQMHMD8GCSqGSIb3DQEJBDEyBDANf+l+Zn+H CwH3dAngruICwNC16BqXIgfQ8Dt3qVFZlaEL4F/a7xNzlWT7dDULD34wDQYJKoZI hvcNAQEBBQAEggIAfu2wvLyozHFISTVZtF3md/qxMeeqBqN0Vrvo2xHxdzvPwA8d sRGNwRf6Mn7vEF8sGXjqbtmvYIzYMW5Bbkg7EdBMNlF408JOGZN2VrVev4F7CJPV jBHa8C4yawo3a9Lt8TuyJHHbDmuFJ+mMgrVgzffwpJFLWEyU7ytQGz3CK2w95d1u QpO0jPZ3Er4PZ0A9bsNOI/NbAMUBnAglmfCnQnp3ckYRDbFHeE7fat5I11mR4/0y XIhB0d14akFJObE4LlFI7kduUORfCCvYttgap7DRTz+xb5TjeR7pGr7lzEBYBljk VVdQG3fuxfHDd21dg69Ep2KD/se7KEvIrD9KPTQtDUhhLJQ+31MJtdbdMJwTZPNc JcBwoGHTc+G9j4TAiNGF9C3GtfR4dFXPM45MpgxxNGLgza1RQnn4bGTBDHICem04 TQknBgfJCmyTXebtpGhmGAWl2Q1pxz/vrKLEH77AF/y+CyTku527tPOsn5/FFAQr gjeWfuxBDYWCWA5x1kMmPRLDUwfspO+bw+tSQVLzjrwbed6EDdqHKhEVHwVwQ7P/ FnNQVzesB7AkxFhXcv48JYD3h+QjG4MmUx03Sc6XZ06BGhwU1GIgFe5/7Tmw6hcc uYSikJBD23Fc2tMaXdj9Ey3ShmzrSrmZq4DrhyEHpMz2tfhsAvV7rQIeR28AAAAA AAA= --Sig_/I6VpOGWuBjiIePrm7S6E858-- From nobody Tue Dec 19 13:23:31 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SvcpK5K4xz54Dx0 for ; Tue, 19 Dec 2023 13:23:37 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com [64.147.123.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SvcpJ2qLvz4TSX for ; Tue, 19 Dec 2023 13:23:36 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=CSLaXqi5; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=qkCFQ7AG; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.24 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 49EC33200A35 for ; Tue, 19 Dec 2023 08:23:34 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Tue, 19 Dec 2023 08:23:34 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1702992213; x=1703078613; bh=K8Gr8IxX1X fS/4skJhGb32TXLGnquMYAc7xKmM82low=; b=CSLaXqi5vO76Gkji7coOR0Demo yVQmLfGJlpzwahNzKB/m1q0vv6z1y/rXqrfEOg2RjVVSgPoR5bRh+3YX7RRt9zmr 4I5CbP//lb7Otp60UTEVFy7Ldrl0OB1CWN9r7/U3Zbb3A1Wu6jeOueialXdq1zel 9YCFpYCHw7VxlpDxLwKn9ZZDe3d9VNFVBgZNP8OHIENyVAhcJoVshzZ5RzzXss59 l2wACIuojc6XHrxls0pu76xZXSOlifYYLkIFMrZ+fiECPoaGdo2hgA4iSY+XHVoy 6I18LIl8s30ya6PuhENlQrE3W0Ogz34lG4PuLnKsIxFhYsZvqWtxMnqH0LQw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1702992213; x=1703078613; bh=K8Gr8IxX1XfS/4skJhGb32TXLGnq uMYAc7xKmM82low=; b=qkCFQ7AGj+NaA9giU8cpTgWgalWvwxv5i9LirKomjSCQ BmwUOQia485VjTJum8C8pT7SCA1dHHfy4IgvpuLROqbBrTRaVZN81qZ9PmW0tE98 EEULh8ilX1vNlyy9dgevQhWjsa7ptthy4Rk4BEqlJBRW6TZX0E/M5HpJyUovp4oT rDs1d2TgTaQ5UUz/tMze1cl1UZw3XrcwGSuM+u5ipKiprFnyriP+20Nfjy+WwaBj EYNpICmTVZ2MoB8AmrPvYhwy83ICJSCUg6QA91E9cVNMNfP6pZhZtTuwflP1Qtk+ LM8P1PdT3tPdCURLWTbaBkL8m5J0lG8DgwpFZi9eYw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddutddghedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepkeeluddvlefhieelfefggffhffektdehleelgfdugfdvgeekjeejuddthe ehgfeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep vhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Tue, 19 Dec 2023 08:23:33 -0500 (EST) Date: Tue, 19 Dec 2023 13:23:31 +0000 From: void To: freebsd-fs@freebsd.org Subject: Re: measuring swap partition speed Message-ID: Mail-Followup-To: freebsd-fs@freebsd.org References: List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.24:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.24:from]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SvcpJ2qLvz4TSX X-Spamd-Bar: --- On Fri, Dec 15, 2023 at 08:41:10AM -0700, Warner Losh wrote: >benchmarking of storage. Do a random workload with a 8k write size (since >that's the page >size of aarch64) on one of the swap partitions when it's not in active use. I've not got my head around benchmarks/fio yet. But I've done the following tests which seem to show abysmal swap speed, not linked to the hardware : # dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s 1494+0 records in 1493+0 records out 12230656 bytes transferred in 41.495178 secs (294749 bytes/sec) # dd if=/dev/urandom of=/dev/da0p4 bs=128k count=250000 conv=sync status=progress ^C88080384 bytes (88 MB, 84 MiB) transferred 47.309s, 1862 kB/s 674+0 records in 673+0 records out 88211456 bytes transferred in 48.677246 secs (1812170 bytes/sec) But when the same partition is formatted as ufs: # newfs -U /dev/da0p4 /dev/da0p4: 2048.0MB (4194304 sectors) block size 32768, fragment size 4096 using 4 cylinder groups of 512.03MB, 16385 blks, 65664 inodes. with soft updates super-block backups (for fsck_ffs -b #) at: 192, 1048832, 2097472, 3146112 # mount /dev/da0p4 /mnt # dd if=/dev/urandom of=/mnt/test8k.bin bs=8k count=250000 conv=sync status=progress 2036400128 bytes (2036 MB, 1942 MiB) transferred 18.013s, 113 MB/s 250000+0 records in 250000+0 records out 2048000000 bytes transferred in 18.084497 secs (113246167 bytes/sec) why the difference? My non-expert observation is that when swap gets used, all writes start backing up and it starts thrashing. it's a thing I'd expect to see with slow media like microSD but not on faster hardware. I have the following sysctls set to minimise swap use vm.pageout_oom_seq=120 vm.pfault_oom_attempts=-1 vm.pageout_update_period=0 vfs.zfs.arc.min=536870912 vfs.zfs.arc_max=1073741824 this is a rpib4 with 8GB how can I fix? -- From nobody Tue Dec 19 19:19:44 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SvmjF0ncjz54mCG for ; Tue, 19 Dec 2023 19:19:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SvmjD0z9Qz3G0v for ; Tue, 19 Dec 2023 19:19:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703013584; a=rsa-sha256; cv=none; b=J9LCjqmoQOuuQGxNH+PaTI6I1re/PCLjNJjc/POErdh3GrayRFN5EE9J0qzeNfFBRyMKwc J1w876l5BCfunrYjPFoyj15/I9oCUGP47xQB4QFpL/FfYHmqb7fP0L7RihUnwh/qK2YlYG Krmtq5CpuCEw6SH25uRiuKYBwaWUNOV1MxIyvL/DRkaOllRPd7lADqCIFFYYQDriYDAmAU TiGG2ZDKABCO4DTLSAPDf61jE/CgU5c/GeO2hiS9X9QcxcZi7NL+eEYKWYSkQrWHbpx6SX c/FfjA8w9SV3AjGsTf1UMg8rFXYlk90Klg40BDr8+k1yka7GeMn5ANerR9JfgA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703013584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5HcP1wInmYSzSaLBavwprLTXqQrkLtUfqYpKq9stUow=; b=T9eGKq4sHX4Qknli3QQMgCn1Nh/7gJTFwnVub5Oox9w11Q+xTV24EI0bTEFGMhZqnk+YnT 7KSdw5vS1BMN0391/ogIntp+aTnkr+RpwO+tAwP02qzE0CYMxcR0/MDSQV8z8t1rMGmY2T OuhxWkAeQZGJ3HqRPj2noDjJIj1enftMWon3QzIUs0Jkgwz+PJFA2VI4t0DhnKQKQZ/dH5 fFszfnPlh8ZuFA6KR2fvX6MeoXmWfLJxKQXt0ho/ZNDtdkyqvMx0y/WOmxVwHjxExvUMYT jEtkMs5S2v5QFeuZvnmoZxqzqx3gaPiPCkXuU2LjLBVjkZxKaN+HM+TyBt8MdQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SvmjD0486z2xX for ; Tue, 19 Dec 2023 19:19:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BJJJhXW059675 for ; Tue, 19 Dec 2023 19:19:43 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BJJJhId059674 for fs@FreeBSD.org; Tue, 19 Dec 2023 19:19:43 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 268005] rsync to FAT32 flash drive gets "Freeing unused sector" errors Date: Tue, 19 Dec 2023 19:19:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D268005 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org Keywords| |regression --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Dec 20 13:31:15 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwDwm1nssz54y0j for ; Wed, 20 Dec 2023 13:31:20 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwDwm1Mq7z4VmM for ; Wed, 20 Dec 2023 13:31:20 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703079080; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NT7xH20n4GD88bh/MHOyKyYdW/Ow49PtBogvmZLBRaM=; b=mKW3y6AsVZr29XLF7gudpa0MvHGQukKFO2MUVe+5+XgqSMFYxP28h52trQFaf3BoT+f73A TzdaCUYpMoU+Ek4DnquEddu4vyY1Wuuz9nNLuSY9LaDxxY8jgfQxD4wpEy1Ko/uD8eLd44 l7qTCa00v2Gnm6hAvsi+1jekJGvZSny544E6Xcseob1UeO2mCMK339DNn15Vlstjl4fogs wpynFyeOBXyZQUH14OQ1bmcauoucad6jnrtv3ZZW1vLqhJgZWwu+HHogLdNyTS3MdTJJUc yVLcG2+i1g2DaTSx9A1x6jM/edbqZNdt/C7PwkiAVoY5cRYkc8j6UU2sz53XtA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703079080; a=rsa-sha256; cv=none; b=Vi+szV02sVqPMcc65svpV2yvN7TT+0Mp0czEbU7ck9bKEWAAYXwz0aQZb7xVQ7GfFe7ze5 TSu2lwGDUBXODmd3x3M0wJeAxa2uzDIMYfTcx6PVorcXuZtYAkz3CWrVsZ7Y9aNs6VYzwr WJPuO0wsiEF1nNXV7g4uDDdTpO6dawgg6hNKiKFWx2CRrkpGDdrESDToTa3ZWxbXhUL3T4 GL64bSg3hLhLzVxhi76BjtcSn/ISjVHYfMG+M+5X1FQyFJvWYHoZRPezMdJqG9ge3xA9G2 0AQV0gmOVx8VlMgJxyUEDHQqW+CZafAfVrQoKY2JvfBTcjOSHN4unjQPbkKQSg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703079080; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NT7xH20n4GD88bh/MHOyKyYdW/Ow49PtBogvmZLBRaM=; b=uLsXN+aHYR5g1x/8eNHYecoft+VHBGX+XSir+FQWAJhujFVrOT7pJ44K7vP6mTiUtppRna +A8yzNr7UK5TOBIkBPO0OExtRICjGOjiUNFQm+gioAU8rI0PhPaknHYKeQBIRe/4S3rq6X 0Nos1gAsNZRT03mJiS0SATYCrL/VD2Tqzuw9PK7BHbeisQZSpwJi7kyST7brePt+Sch7FF Bnh11YPz79wY9kZhDmBycR3RjxqefJiqWJUkEHGJ+ZUYYRcRVuNWVJkLO3P2QHXfsJ3V8e NTgY5XFFD8GB0qui5NV4h6neFomGSYgHdhDGEO5/v2fut/7Nv6HQWBPhe4MvaA== Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SwDwl6pllz988 for ; Wed, 20 Dec 2023 13:31:19 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:ddbf:db2d:460d:b681] (unknown [IPv6:2001:470:923f:1:ddbf:db2d:460d:b681]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 80C0E160C1 for ; Wed, 20 Dec 2023 16:31:16 +0300 (MSK) Message-ID: Date: Wed, 20 Dec 2023 14:31:15 +0100 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: "freebsd-fs@freebsd.org" Content-Language: en-US From: Lev Serebryakov Reply-To: lev@FreeBSD.org Subject: ZFS pool hangs (live-locks?) after adding L2ARC Organization: FreeBSD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello! System in question is FreeBSD 13.2-STABLE stable/13-n256849-05c55eed44e5. I have 3 ZFS pools: one "simple" (nda1p2) (it is system pool with BE, root, etc), and two radiz1 pools: "zstor", consisting of 5 HDDs (daX) and "ztorr", consisting of 3 HDDs (adaX). Also, I have NVMe disk nvme0 (nda0, it is brand new AData Legend 960 2TB) with 1 GPT partition of type "freebsd-swap" (it is NOT configured or enabled as swap in the system!). Size of this partition is 1.6T. When I try to add nda0p1 (AData partition) as "cache" to "zstor" pool it is added without problem, but later pool hangs. I've experienced 2 hangs: (1) Right after adding cache and reboot import of pool hangs. When I tried to import pool by hands in single-user mode, I've seen that one kernel thread with name like "z_int_2_2" consume 100% of one core. I've waited for one hour without any result. After that I've removed NMVe physically, booted successfully and removed it with "zpool remove". (2) After that I've re-added "cache" device and everything worked for some time (10+ days). But suddenly one filesystem on the pool (only one!) starts to livelock: if you do "ls" on this filesystem it hangs forever, "ls" consume one core (100%) in system and again thread with name like "z_int_X_Y" consumes 100% of other core. "ls" could not be killed, only reboot (which hangs too after "all bufs synced"!) helps. But after reboot it reproduced again, with exactly same symptoms. This time I was able to remove chache device with "zpool remove", without detaching it physically. Status of pool: > zpool status zstor pool: zstor state: ONLINE status: Some supported and requested features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: resilvered 1.85T in 04:02:19 with 0 errors on Sat Dec 9 16:21:33 2023 config: NAME STATE READ WRITE CKSUM zstor ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da4 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 da1 ONLINE 0 0 0 da0 ONLINE 0 0 0 errors: No known data errors I have two non-default settings for zfs: vfs.zfs.min_auto_ashift=12 vfs.zfs.abd_scatter_enabled=0 I can not find any discussion about such problem on Internet. Also, "live" system doesn't have these "z_int_X_Y" threads at all. I want my L2ARC, I've payed for this NVMe! -- // Lev Serebryakov From nobody Wed Dec 20 18:55:17 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwN6d4FB7z55DnW for ; Wed, 20 Dec 2023 18:55:21 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwN6d3PXkz3GBp for ; Wed, 20 Dec 2023 18:55:21 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703098521; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ILgodZGsAr1kBrIIAlQDQtOaHg3Et1nySaKoxri4V6M=; b=aNSE10mBkXyrXq45GMIBslLP15ojM0zL8UhWXGWfc6SsTKEFAaDd56CzFqTd3MlQ+37tMB rsdS1LTwlX3fd45O2/eLMPBItwlr6WEgzIYsjOfYrtsZ87z/A4v6b8eYGcgy+Tt+8/24QL 4/3f9+tmq20cOBWcbAq5RJIkqfdvD0Hjk/jbETnhJQmiuedTrp2pPKmP5bZxaQFPJEVVb+ 78gc2rcxIAIqD1oZgzLoGMdq7dYXdnVYklCGc40vv0IGDKvdRLmv0Umo3YdlRem/+9grvc //ov2yIbXrWnlwktGrzzTa82cTMO1lAmNZA5wd0qs+Cj0HNT9XSHXdiIH32NZg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703098521; a=rsa-sha256; cv=none; b=qZtViWQqbdotm1fCwlhQJkV1U+gQPjlU6w/djXibjtxIkbZ45NDTCkmUvBrH3wMFtv0TzU WR6OO3wI065QZOKMGnHDaEZtQnn+12p2nKmGy/s/F6SbtTrgs2t5C+GvuMfYE3sex2HHlO SzN/TorFzwTyA42YCDGJOC6jar6DAok5uchJDVMAfGAFvzt4uzlZiRLUMR2Uux5tKSgfcg K1eOMt0iNjYEh4D6WZm7fM4iT2SFnZDoOF6sX03YvrzvieD7gwLJilBEL/AwKN8GDFt8so PrtFgbkq9WgmIJsavu6H71ZFCLVfw5DrjUl8xA0TTXWSfjaHIdwBIRON6dVzkA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703098521; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ILgodZGsAr1kBrIIAlQDQtOaHg3Et1nySaKoxri4V6M=; b=caQ0+GoxXrqoGM94mNCYXcrucci5tx0PqT+gZkZmWTPgmMbModSPG2dJXmX0IfLPBx56ib aJVFRhKz7o9uFHrywAXvq2SzwLAVtQszL+Bt6xMYSID2PVkP0ofejR6frXLO3ZNGVLIGB6 H74ARrk5PBDN+4x2WtdB2ukvGMkcDlgD481UXI9/QxVubMNoPsmfv/5Ihf21+cSg6BFVIu aRh6SiwGlAGqKKurijYYCenN+AEUISpl1tLeJzDe4YTWnAocxIewRw//1fUS9IYYQNLWNu wrwg54/ozgNXz75Bh7bC7/8sNKUaf8740iMg/ihq+lnkpM2ZqHo/4FwjI7WmWA== Received: from onlyone.not-for.work (onlyone.not-for.work [148.251.9.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SwN6d1m3Gzm5p for ; Wed, 20 Dec 2023 18:55:21 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:bdf7:4bd8:b09:3db7] (unknown [IPv6:2001:470:923f:1:bdf7:4bd8:b09:3db7]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 0349D16157 for ; Wed, 20 Dec 2023 21:55:18 +0300 (MSK) Message-ID: <61f814b2-06b5-4a4e-8e4b-0ff04c9f988c@FreeBSD.org> Date: Wed, 20 Dec 2023 19:55:17 +0100 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: ZFS pool hangs (live-locks?) after adding L2ARC Content-Language: en-US From: Lev Serebryakov To: "freebsd-fs@freebsd.org" Reply-To: lev@FreeBSD.org References: Organization: FreeBSD In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 20.12.2023 14:31, Lev Serebryakov wrote: > I've seen that one kernel thread with name like "z_int_2_2" consume 100% of one core. It was "z_rd_int_%d_%d" -- // Lev Serebryakov From nobody Thu Dec 21 03:48:14 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Swbxn3BbKz54F0W for ; Thu, 21 Dec 2023 03:48:29 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Swbxm0sGtz4VXb for ; Thu, 21 Dec 2023 03:48:27 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=AYhtuz1z; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703130505; bh=9zRXOlmT3HZ4xG8MtVsR63yyIuvo3n/j9fFuHioNKEg=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=AYhtuz1ztRM+BQ/FwoOFKpPbedHJwyQlC9WIrC4+F3SzkwRA9c2g+IgXSg2bVWnnXt2MPBB3ZiGL/7FSZfe2AzbFe9clpTHiDN3ow5BAFG6FyBEY2wUip0r1SpnKGekmWkpnfnH0v3e4hQvfH3g0Y3cklsmHF7qY2bWVdErpSvu6A7yJk4+QEN5JdRLc1mHvMLUMGh6p/UQH1+MupMXK0AfHxTEvwcWbbneJPkxpN0vamUD4UYCacRM9+rfuQdUDXRaYGWUQ70X5E7F4OH1YUFUky379sW60LNzwT9InmHHsgSfUcpg1LQh73qdlhJacGfXDp3IWuMr4tidVeyag4w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703130505; bh=XT7pMtcCwxwD2KoyzQ0O2Mw4cvIPMd0lwGNV0r6bkmj=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=J9Hj72DdXx8Xg/RrO1xCasEo9gXYjj3ztDjJnNVWT24uUekmjhwlMfcNpD+VRkNkNqtRPY4etKLQyoLFeKa8vIV2Qu6NTmopJ5u13MLMy+ROIFdk9bFrk2UMfwl8JjRjE7QBPX02GF+Iu8nA61suLqVQfqvuxMrRRj03iiNFyCI1gCP6QnrDXxTFyO3auQwFTZNxOR0eXayIkmL9En5MN6KUdR8stFlRNI8NtTqZl0XGqWaI4xoI8cVDUOKDBvVp0nfgRSKQ1c1IEDTP2wF+CIA+zTBVKU8XjJx1ZUucmw7Dvq1h/x7iks8aG+GzosEwtaeTgXC0A3Omw+Hx1UftSg== X-YMail-OSG: RgceatoVM1mqR_APqVPc.__6_0y5hafOHMY8vgLs83_EBtVwV2llnV_oLEx1x__ G04IvqlaYFvrT7Ej6QgMDVByVb9AcBACsckf.5q9YO9S3AfKaudNVGEYi14eOzBh3Wrut91f_9PD 41UOa2kzZIUnNb1mporidreW_I5G.DfSZ0GIdd7S7ykDrg74FdPkc1NYVSYUuUtBeV1yiua9rA5e G.twj9Th76CmvPM._6GKD1QjCkED7k1wZn9lowH0PXCv0XEg6khEhDfvEi8AsO5QQfmoa7ulD8XD NYUOYxMYWc6..btTOuCtk9nKE68kLsllxDpTz.BVMQE7XLrzfhuofyoAN.Uh0MrK_yd9exPY6KNJ WQ24mAwzaZC5i5I_tAcdYwyuCwVgdfxjXzV9ZIG838Vd_fdgwLTUt61IcPp9oIc6XmRjv7uq9uIL JJHl8ouZWKAhZ6cZ9XREQHzwXocukfBIZVtdkcIsnKN.VxvbY4qOwGDIcJiRJ7X7dWsFhwJ0jUZ3 iSMuvfsh1bh3dK9CfM0XyMiPMVCfEsmx73zCwNgaxOAKc6sGHXnvwTdhCfnTRK__X40F1LJcfioP xjxOGnXuYvqYsVSU8xu_wiPIaNmngQPsq8DRTKS8gA2P5IcQEZi4JT_5tied63s1pUbhtiKffs2g nVA0O95Ek6sKQRPfRoIxLui.2jHbiWaw69Fa.zePCU8rvwEuBHXVsTx3bXbcLucWMnvR99Se9wx1 DSiorTlFzCWrJZnu8Q1wikaahurVyP_VNb25SKFgCXRbIqUAxslc3FLRw2O9iegd8dJl3GuTL6Tr MIh3mTUgLDM1aoVaplDwnukND81d3UC1ZWW7zT684f0iwl531vsgGBxDI6sk5cB_L8QE9mlPH_6g VXIwcbSqjwCU_EiCPpZrpS7ws_ccCCZOBa.dosO7TVb3CBeFUOdFiaAoziCY5qLL7JQSQyCdZcEa 2mhwNMKFUp7zYMhX0fSJuKzsJQ6ToQ6wM2JOog_OqiyethAC4uNXoMa2WPKsHdvnGcRE4SwuGS5M 1JuLaxMdLoSmqLev.BRFbpNu93DtPnH46Km9OlyDh_qw3UWnprGnn4N.gtMf_kAdRK2b.8a2zc.G l9WoIFxl0jo_Z.J0GL6dq7MU3Qhd3uflrONvx0jzksryoXngYQL0VUMPJY_HlrOSC3pjo_IoUjbd CdXTSqnsV.YYoIdENHi26GvzJb8uaSV2mGJcALwuCto6qnxjFV8aKYA8vZ024IY7eHhfqjP_MLsb oXWZYD30OWoHeEmzgHKuHUWsVgEHFKhlYu9.9qSRj1V7B_4P1nkJaM4ON8K2iUvEaeuzbWqpfsSd UMcBITy_3mHMH5eBt0tux0lZNXzQrCw9klrSIBG9cBwQo9VPin402iGQnZ7X8PlmtesgBAFa_dlz geDUVe8nFu78d.eXMGPwm3KN1gsWqvqLK7tMrKhXQsa_sCbLc2rDjTosu.fHMu8IaZCEb0kRIb__ kZNzEiJ9kFjK1QlYE0Ohk18zzoC8SDhL4mVNDIPeHlK711p3chM23sJl3qA2M8uSDzcBrEuOB4Av x.dCeJhY_6gKuN05E.jhOmU.hT7ATSRjBOuRbkYlc6Ey2io3NgFz_ifxyYNCFfD6FOOjtanj5dc_ 05qRT5JnliIG7.CeCafRWK63Zc0KKY5nEQNZqV3Tc2MMMCXL.mIYnUE3fI966_HVMhpSvKclhZyq PeGxqv_cLxzfTv9vYVfQNMWFCAy6GGqdYmw_poiRoADY1g8wsje1XUAKYk9ijQ0omiizLzXyANIT 1QflsDLH5CRcaF6P8KI7F7aHDbQSUw.aEgWPVoHbC10j7d6geGiJhdMK6nhCwucLmgx0d.v3Di4J afe4NGr2OgRJ3d4XZf8GdJ3muWe8EiPLZGjRPuHug9Y7qBThHKfs5n9KxSFxVmpB1F5uK7EWFkFE u0awCUppfFiDKvBqz4LvmvuY4k4SGw6GvgXF4KdfpdhKg1h4Xlcvfd9DM8w90niVFp954u3ymZLS S79TLwodHpMp7FkFDyTZqSS_48wgp9f33wS_jq2oVMWxNLIfNe4e_0PfrVuTO5efTI75aAbBKoyW sBbKbUmCRmwU2jexObn2Vb542I7s_RP1oDc1ie88COG7woQOHGD9vC.mJYA6y1G..6N6nnnw51yT zbhDASE4T3GV84RSTMEb25KHMcHfklx489AE6uqmrnPK9oafk.ExrQupxtj1M2l9PQLakEF5ZPP8 JGAvlSKg8AMKIJOebFfMnYa5xppGwU71OHzypXkAXJ6yb_DDjlseB7z05ZnV9KsdElC7BxHw3hg- - X-Sonic-MF: X-Sonic-ID: b99971e9-85a6-4864-a53e-9c02d2c5fa35 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Thu, 21 Dec 2023 03:48:25 +0000 Received: by hermes--production-gq1-6949d6d8f9-k52jv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7817bfaaf62c7f3ac612ae56f00a9097; Thu, 21 Dec 2023 03:48:25 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Wed, 20 Dec 2023 19:48:14 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.206:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.206:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Swbxm0sGtz4VXb X-Spamd-Bar: --- Just an example for comparison/contrast: 8 GiBYte RPi4B "C0T" SOC USB3 SSD boot media (no other media) # uname -apKU FreeBSD generic 14.0-STABLE FreeBSD 14.0-STABLE #0 = stable/14-n266002-2ef9079ece5a: Sat Dec 16 08:49:23 UTC 2023 = root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC = arm64 aarch64 1400501 1400501 (So, = FreeBSD-14.0-STABLE-arm64-aarch64-RPI-20231216-2ef9079ece5a-266002.img = based.) # more /boot/efi/config.txt=20 [all] arm_64bit=3D1 dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don dtoverlay=3Dmmc dtoverlay=3Ddisable-bt device_tree_address=3D0x4000 kernel=3Du-boot.bin [pi4] hdmi_safe=3D1 armstub=3Darmstub8-gic.bin # My additions: over_voltage=3D6 sdram_freq_min=3D3200 arm_freq_min=3D2000 force_turbo=3D1 # gpart show -p =3D> 63 468862065 da0 MBR (224G) 63 1985 - free - (993K) 2048 102400 da0s1 fat32lba [active] (50M) 104448 468757680 da0s2 freebsd (224G) =3D> 0 468757680 da0s2 BSD (224G) 0 128 - free - (64K) 128 451979136 da0s2a freebsd-ufs (216G) 451979264 16777216 da0s2b freebsd-swap (8.0G) 468756480 1200 - free - (600K) # more /boot/loader.conf # Configure USB OTG; see usb_template(4). hw.usb.template=3D3 umodem_load=3D"YES" # Multiple console (serial+efi gop) enabled. boot_multicons=3D"YES" boot_serial=3D"YES" # Disable the beastie menu and color beastie_disable=3D"YES" loader_color=3D"NO" # swapoff /dev/label/growfs_swap # dd if=3D/dev/urandom of=3D/dev/da0s2b bs=3D8k count=3D250000 conv=3Dsync= status=3Dprogress ^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s 60557+0 records in 60556+0 records out 496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) Adding: vm.pageout_oom_seq=3D120 vm.pfault_oom_attempts=3D-1 to /boot/loader.config gets the same sort of result: # more /boot/loader.conf # Configure USB OTG; see usb_template(4). hw.usb.template=3D3 umodem_load=3D"YES" # Multiple console (serial+efi gop) enabled. boot_multicons=3D"YES" boot_serial=3D"YES" # Disable the beastie menu and color beastie_disable=3D"YES" loader_color=3D"NO" # vm.pageout_oom_seq=3D120 vm.pfault_oom_attempts=3D-1 # swapoff /dev/label/growfs_swap # dd if=3D/dev/urandom of=3D/dev/da0s2b bs=3D8k count=3D250000 conv=3Dsync= status=3Dprogress ^C1085161472 bytes (1085 MB, 1035 MiB) transferred 50.000s, 22 MB/s 134642+0 records in 134641+0 records out 1102979072 bytes transferred in 50.816711 secs (21705046 bytes/sec) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Dec 21 15:50:52 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwvzQ0xVBz54wgB for ; Thu, 21 Dec 2023 15:50:58 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwvzN6ZFFz3HPf for ; Thu, 21 Dec 2023 15:50:56 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=ammUE4cI; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=Gt0UfVPj; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.21 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 660283200AB3 for ; Thu, 21 Dec 2023 10:50:55 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Thu, 21 Dec 2023 10:50:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703173854; x=1703260254; bh=iAlxTLe2Sg E2By+JiNo6Ycp2hx5C9PXeh0H8GUuCRz0=; b=ammUE4cItk1e0uehVp+dwhwTLM /cnxlAbbFHgRPPzxDvxNoqLGgf2bKiaw3nG9jjgu9htIwEFrdzKYLoemeUedHSdp HOYBLU1FRJRQ68DUnUXT+QHVxZBERJFGCsisCO1O6vlDLZzyPmn934eojuraXUyO LQv0ozK24vsrjNKeTAFn+Ob2mmCL9IiOITsdT0SlnVfQ2J9nEquje/2kwqMW2M9O BA4Tqj76q8814krF1ok8KsYUDip2I2Xua3q8FatgxWyplOiGseTZgjIgQczpZiNL AWQ0pSFewUDb+6j54EO5ab9jd5/UT3TFe+MwN8XCRTYXSYwI1OPydbKY3otg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703173854; x=1703260254; bh=iAlxTLe2SgE2By+JiNo6Ycp2hx5C 9PXeh0H8GUuCRz0=; b=Gt0UfVPjKcxohBWwuMh/6TQNImTgkR5GYV3XAK2TlFoy ybxpWqHI7VBp3sNIA/r01MjmTxThPGvKwxt6a3E1sFW7jbFVAIwzTOdhpxwxWXwW mS/VHjbfXXdP5udpK6CJBvH44UQpjkOJ64AqCsetVm5XQ/CJ6M15igrrKGi14Qel YPioEdizt218kWpVRukk2ovYlbn9vw3qVkejCnoa/2vwKwJEZXA5mdiAeI0WuJ6h V6uzNvYWVXAvAcLUuIo7WHS8T52GIqgskgdnjg74UOW6C/THB4Q2JdFIKErA5RQE t229yYen8+vLIVQnUZpvymFeIjte8XH1R6Be2NfNnA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduhedgheeiucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepfedtgfethfefkeejhefhheetteevjeeifeefudegheegfeelfeeggeevve dugedvnecuffhomhgrihhnpehgihhthhhusgdrtghomhenucevlhhushhtvghrufhiiigv pedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Thu, 21 Dec 2023 10:50:54 -0500 (EST) Date: Thu, 21 Dec 2023 15:50:52 +0000 From: void To: freebsd-fs@freebsd.org Subject: Re: measuring swap partition speed Message-ID: Mail-Followup-To: freebsd-fs@freebsd.org References: List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [-3.96 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.96)[-0.960]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.21:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.21]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.21:from]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SwvzN6ZFFz3HPf X-Spamd-Bar: --- On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: ># swapoff /dev/label/growfs_swap ># dd if=/dev/urandom of=/dev/da0s2b bs=8k count=250000 conv=sync status=progress >^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s >60557+0 records in >60556+0 records out >496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) 22MB/s is usable, I think. In my context, I'd be satisfied with that. My context differs from yours slightly in that yours is SSD and mine is spinning rust. This is unusable: # dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s because it's way too slow. Swap never gets fully reclaimed, thrashing happens, loads of other followon effects happen. The same partition formatted as ufs reports 113 MB/s. Multiple swap partitions have been tested, then converted to ufs. Results are the same. There are no reported errors in smartctl. Long smartctl tests run monthly. 5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0 9 Power_On_Hours -O--CK 001 001 000 - 48992 196 Reallocated_Event_Count -O--CK 100 100 000 - 0 197 Current_Pending_Sector -O--CK 100 100 000 - 0 198 Offline_Uncorrectable ----CK 100 100 000 - 0 I can't find any hardware problem here. Possible workarounds, bearing in mind I'm not versant in C so it's not like I can fix this myself in code: 1. swap as swapfile and not partition [a] 2. swap as nfs [b] 3. swapoff & swapon script running every minute [c] 4. just turn all swap off and reboot after crashing (undesirable) 5. use another OS that doesn't have this problem [a] not tried yet, and i hope it works. Legacy info suggests swap as partition is usually faster than filesystem-based swap. But the reverse might be the case here. [b] also not tried. This, I imagine, would be filesystem only (I'm unsure a zfs volume can be exported to look like a mountable partition to the client) [c] https://github.com/Freaky/swapflush.git - usually works but maybe i need to run it every minute instead of every five mins. For testing, this script was disabled. Any additional suggestions on how to overcome this problem gratefully received. -- From nobody Thu Dec 21 17:32:37 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwyF21bYmz552n8 for ; Thu, 21 Dec 2023 17:32:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-24.consmr.mail.gq1.yahoo.com (sonic304-24.consmr.mail.gq1.yahoo.com [98.137.68.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwyF059M2z3T7c for ; Thu, 21 Dec 2023 17:32:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=lekTUfp+; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703179969; bh=HV1I+8cKDmisaEE263BU4OlTEQ2YRVqT/s7oZk8pDIY=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=lekTUfp+48r/z31A/tnD9R6LD/eSp09wDH5um2S56yqQw2BpJMcJF8KtDI1ysEb3YOQSmnxRxpOxlZi08wVf2DvFiaLdqqUw12xk5jMkzTZI6oTZ0fSo2zBtp289XXU5M/0UVyRgdEP6Aw/tnJRVvOEAgwUmk1+BgjEWGQYVg2nVj/Ed74AZwCdTz9aOgc1uIPAtKvuGS6OO40enHnmnp/n2tQnJEWCadG72enn3xQgunyE5R7jmrRvUF5jkWPxWFqd1RClO3x1DMNIkdAdf+zd+numy2BK2wtq94rMmgaWfLSr5EDCWvMr5ZG/IGZc47yOUpKV+pMiKYYtDWWUMww== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703179969; bh=v8fpi8jXgWOL5yGZNXHQY/5IlOQ+nZG/GYc/7n65Crw=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Oo9df8eVyIgMFYNuW+F3vepJLgNqpRJO/twJ2VJh4ugT/gP293Am/yRmf/RCgpEc3RbC8Y6pPYeSGkZIaOvpZpt1cJ841dQWA4/8Vl/VzPK6vIxdXBIvN0VK5TpBV6G6/Ll0Qcn+JamI2U10QKeXr73Wnwi1xo03rrrRZwNJHz/3L4XmxCZNMCZeTw5yArAA7drbyQv+qpdSHiyXuwXDMf02mzQwY1+WE/vfxLDob8nkyEHLfcTp6NTv/xjjQHKYrp1Ier1LdyWmRjJkvqY5dlMSK5z1xlAug8O2PUlom9w5obFakqM46xNRS2rKIB7xHZ9yruUpmUOmU72To13q0g== X-YMail-OSG: pJYySQwVM1kYylhfV0R22OqkTbC2e3FEquE25kpUsrSp6sH5TLoLGsRllhNKWll sSZpsEEJP_WTRyHjY70EbxhSwuHuptooI3Oi83nzADOO8_sIA4RF8LqHXezHYqi_yRqJSVpVlgzm jI9QxPNE7TcR0CwFNYLdY.KqU39BB_k89FBzFKlKi4lFrZHGwaDNOXt9BjeqHJBN6Me5XRdXxJPT OxXoWjcqBw48fI44qLWWvh.s3HrDdcD44oTTxCpnPxSvBbDRUWrI5Ga3Ec53DI8OMui_vejpkIlP K8ykShGqkDsfhCucGx46OSQ80e4sGXwseFjmcLKO1Sdh4UcRE06wmK8rv5CzYLUPFfi3Lxhdsppz pzJ8BHQKNlyEOjrvS9.YHZdBS.oKBn9HvBO6qfDDzueMP8XFrJK5Kb_stvglLBtuEaK0dSafd1.t fnEz9D033pHYZl3zRlAmzI5KrA1qLHb8fkV4amn6YEejp8e02pcbJtzKEJc7wwuIyjpd8IuI2suk BUMaiRABtX4b1_y.122Hbjo8OlEEo.drk7EQGX4SclE63JI5veWF5zSjEl9PjneyU40O_Bh51bOf WB8lvXLV48irJCShYQ0T3Yoqw4mmN557rcGqiLOMwFkuTt9VAisMLtlrReYLF36y_f10JG5uxWBm 1ORsdVCgqOWl.HvNDfrk2YG1Fm5mPHbO7cTNgERrnRFTJwks5kKq8iXph_ocUGv6re.7e1NcedWH 9WmPz6yOOVSJBLKCGGBycnNphIf7vfYcE0icvkzX9fSSO9pNY2Zco1KiGGQrbWXkDHawNMCvV9Gf GHAbLLq5NlMzQmH8eKVdZ87wLwB3PHWq47xZqV_mWT4ae7BM7ziXwubDv5CMQeMjxdqeltB14X.k iLGj94vgWxXH17u4tHI4R6ViMZ3JHLPpJ6RJPVnS12s8jyhVKdhqsMXClIfdpKjUSIRFl5rUY_CI um6PcRtJy5h.84SRaNz79ENO__GntgAqA_GBC6vMJwMZrtxbfhIkh9RnnW8fsOKwMcY6o4lSPHr5 TwjPk42EXT9euG9EAWLF9QEoGgXFOh6PEUNgscKgMVb5tw._RqDN9yhlynao5nJaS2Y62.Gsyh_y 65LGOKeYkd9L0se1QiRpeW0UNKcDvI8XVpYINMGHael34PmlNcuZUbRWy9GX3uBnDmc.LxyA_26R bBu.AsOIPjwgU9dSpnDBjozyD.HK6.v446CIa8YY5KwTEpkaJVcX1y07uJX4Xzjl1.6wJadnGw6t ZovIHrnwBW7GkpZJ1CYTQsVNoic8Voxr5ik0W9Dp.G91c3n07lQiVwAaeo7_O6BD60uuGOM2TFGH T.5ALy6n5M8k_ZMTZ6x2JUBBP8EftA2fd56zk_5.b_Li7MLu1cP9e9zOAVTE5kvNUSkS4ESijnlU 7DEC9oM2.sLvvtDjR9KFajk5D3HOw5VAcT3.ZnRYPiPzQADBm4PqRbhCdrx3_NhHvEfPKZBSrnXo ZpzVJ7upDVs54jFVc_zKNj3RAB2cPO1xFmvNQ_S_xFADjoaxmYnZ9UBq0CynmJH0novqUbQdB6e5 xSu.huDu08JC3mrybc0QU9jlBtGTBhJDfWhfRsnJogRMMoVzc7Jhqjg6TOh6Ilam69BEL0QNVNbL fraoD_672N4MXLQ2jkOD2cYVcb.3DTijWAQnH7MLb6r3m6U3AtUC3jNHshSDdQQvNUVNoDq96Yun je624.WxaB.p0QCYMQAWulras0HHvOvh3SLxchmg.m8fcH0q2XU.eft0WdAy1E5SEEHDEfEfSEwO 3Na90LopEaqJP2IzCBIWlaZ5CPzCYrFggoa6uyRbZr3dficOFlmEEIU3vCyuxJ0E3sbdbglMaBrE pyyRHs3m1VqPGaVh0N0oAY73v77UnBzjDyW2BvPnhkVSSSe5H4cRjL8fLVLR3Vl4SoN6zQdkYmCm jNMzbiezks5SXH04HyImsPQMz6WA0ROGS9eOULxAyrPH2TeMMKWGggKuQPxsmiFUHtGRATvhvKUQ BVR8EIrc.JGJsf6U9OAaQfwG4iTBvA63R7bttpD2oWKci.ORGbYfqN4q1ImLsxkgsPBuerw0gh.H Gp3iwlmdJJ7VDvMfnZ8duJ6_yvWLhmomq1z6wp3zthfVTthmKmeOBDEtsiPyt9tnhIdoM38rjRqy oleN0af1D2vKaCI15g7s3z3QdNRjjuOFmo1lr_Mi_AKIo6FVGK9xLza9P.P3ykOjI0s6a469wVhS mqyBHbZNDySKjjgruDoOfdDBczWRNK4wCr1ewPUqxsv4cBwIwrK2aDlMCi1frp57uBkyndPRmGUk - X-Sonic-MF: X-Sonic-ID: 47a4ed77-b76b-4dd0-ba41-73c882a5d7de Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Thu, 21 Dec 2023 17:32:49 +0000 Received: by hermes--production-gq1-6949d6d8f9-k52jv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ea2ea5096d116082818022b62ccf8aec; Thu, 21 Dec 2023 17:32:48 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Thu, 21 Dec 2023 09:32:37 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.205:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.205:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SwyF059M2z3T7c X-Spamd-Bar: --- [For this message I'm replying to just the one point because I expect that it is rather important to your context.] void wrwote on Date: Thu, 21 Dec 2023 15:50:52 UTC : > On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: >=20 > . . . > Possible workarounds, bearing in mind=20 > I'm not versant in C so it's not like I can fix this myself in code: >=20 > 1. swap as swapfile and not partition [a] >=20 I never recommend such because of the "trivial and unavoidable deadlocks" issue. I just quote Konstantin Belousov on this (copied from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D206048#c7= ): On 2017-Feb-13, at 7:20 PM, Konstantin Belousov = wrote on the freebsd-arm list: . . . swapfile write requires the write request to come through the filesystem write path, which might require the filesystem to allocate more memory and read some data. E.g. it is known that any ZFS write request allocates memory, and that write request on large UFS file might require allocating and reading an indirect block buffer to find the block number of the written block, if the indirect block was not yet read. As result, swapfile swapping is more prone to the trivial and = unavoidable deadlocks where the pagedaemon thread, which produces free memory, needs more free memory to make a progress. Swap write on the raw partition = over simple partitioning scheme directly over HBA are usually safe, while = e.g. zfs over geli over umass is the worst construction. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Dec 21 18:15:42 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwzBQ4DHLz5550H for ; Thu, 21 Dec 2023 18:15:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwzBQ1P0Pz3XWf for ; Thu, 21 Dec 2023 18:15:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703182542; a=rsa-sha256; cv=none; b=vV3w5uvyWzPVXMouWcoKRliFmRYpLOBzd1GWsWsk0f5SYzFmj8hP2l2WH4edLEG+sV5A67 ZwR00poKLeCteVOis1t20thWmuLHhqKSt0hVe/WNsjAO8ZUvy1AIPdWLWgj/rMKSocOph9 HAypY93HoKMdqIWwnmr7f1kju9ZduQg+5+JEwsMR4Pf5MfDciU9RWzUhtqaEVZvxHSq/w7 3bgDdlV7lhxsc+YQsKtsZYx9yk3VShQi8ab7rmsArky9HMCeuaxTrZ1Hdak0NH67/zy1qM TG/CiTtSe3iH0aAi2NMopYl0DPrhjuTEXFCNEU0fCADX7OGt6M5rSnmApK37EA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703182542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=z6ThHo6ivGILoMzkl7ZBUgBhH9alOXyEyfRcL7e0GDs=; b=ZBB/l2JhKpFl8KZKJ58UOiJ++3Emwbd/zZ7q4/x5vCz0krIYT+hUkBDixn7FewPctJzOMx Yb0gcoWt59j0TQysVCtRw5wpgguq2F7FRDZS5EbzAoNq9zFBX36htg3QLBiFsT9dDb1TT7 sE7pQnwimAKBSaYY4ro+AKTfnN880RC/Lp1YqpVpoY5vdd8VCJtlf7BZ6+f9QaRTgEmrBT fH5VDLKO4Ki+u1ZWJwSZSk/Dum5dog+5rIzuvSX4+1Dw2bkSUG3w2OWCqNaH1Fs5jTDrFs DVQZCLUlA9z55bCgnDR2SWa07/JcDA/yVjk/XePFARk4mp26GIqp7gZRad7U4Q== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SwzBQ0LMkzjSC for ; Thu, 21 Dec 2023 18:15:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BLIFfgt079866 for ; Thu, 21 Dec 2023 18:15:41 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BLIFfKl079865 for fs@FreeBSD.org; Thu, 21 Dec 2023 18:15:41 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275870] [unionfs] [zfs] kernel panic on umount Date: Thu, 21 Dec 2023 18:15:42 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275870 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |crash Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Thu Dec 21 18:36:10 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SwzfQ47xrz555qv for ; Thu, 21 Dec 2023 18:36:30 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SwzfP0K49z3b6C for ; Thu, 21 Dec 2023 18:36:28 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=fJ8WU1T2; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703183786; bh=6JpI267D+zflN+Twokyek4iAWDWPeoykjSMKaQkFAQg=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=fJ8WU1T29omlD6KiMjerZnp/ujiraIrQzXV1htOsZo/RckFa2W92YxJ1KlZl+pFYn8YdBE0uiN7iUCPcRjWaFJEnQeD7Yn9Y12QuBuOIkyoCOYpcEo9A/PsWHz9ytIaQVVBzaTzwuVgRAJRCxm712sxa+2UMW/et6VSVs66mrR8B7EK4f8XXgMMnwX2X1Q4uYkCQ0u98f0sYwzcN2ZvjAme4ZS2YG7k3x2IjXBrRDP+vksiUrfqx7AzKXFXh4tBf6/PfYyH5Evez2FH76QHGGZxfctBLJJgn56hMPhB4JK2Leb5dmj2ltJc0PnZ0kjcM53BbG0oRmS6oLqOCK3wpZA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703183786; bh=o2Usr4ffIz7+LoccAA/Y3Mu2vWETWQxG21cbRw34vbF=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=bmi/oIkqhoqcFWroGtQGOT0kI0UDCZxxiaUU6iiCs/6Nn2tJaf5Ar57KBbiyiIIaC7GaItnKUp6lcnCHGAf+HHV+FNAP0jQcgRIYkg3V7D3ki2HojnjDd/GxcUMdJSKrL5bZkCQ9bcCpWvM4PBguIuTb/OHY7YUwAOqCDQMlzB5hVbvfrSKrWuRmbbcAwGaqXKLjrPo6hqXPZrD1AVUXRNDHziXnkTnt9xVmXcgM8uh0BujyDRadEz+JRJCpi9GVgja6/Ks22LY2jzyh+QyuO3SfhY7eEfykucKe2scDLhMN6r0x2zSZDfxSzEC3fChyIdZXLUE3h+M1+mdHAaJXZw== X-YMail-OSG: fldAhmcVM1kAABYM4YObFXon8T0gtPIz8D_xE9QDmHMUEwD9mOhxYFu6iMRYAMK dH6ttnxiTRPprpZaUIyljQl7hdassvFQh99JKn0dqQBgyPXHQVVxfOQwFqQetkli70fG_FBexzZG 28.YqD5ZsCVtIXEM60Tkm4A2yOakb1RbUdfBK6FKtkJ5glQnh6zqNOz3jEQ6C8Yx_EuyXruPFn0M lXLzNHB.wS_T4yuVTCdJL3CERo3.uveTqH.QInqiXS7rjhOPXbnf9_3fevxJXif2HaxePmfpmQ7a hQe5gv7lrzJteBrcJSrz9OxSBpNNxkvY6jjgQUOQ2rmO4VaLV.TDzh5dVQr849GNLj2ceRZO3A7I mX3E_NfGw8ZupXGmBiS6XsR3GQflcbWL8ZVkESU6y9uqSLTyVpqzrCKBv6Hd.nEKISkgzJv9Fjez 68QNRIVqHFBio1wc7FhnZqcuWPHFItFyCWkw_Nfc0hjCGOCSZ19.0QhCMP_hPiwyeej_XI2dhI.y IMSZt79j0BOjDyAby6OnME01E2Gqu7yl5hq2fmeOp7zaIOEjLDZhBX0lKkF_aMyvzK63Fv62RSNm AWmtHG.GsCxCIJAekeqlnHijFSc45qh8B0Z0rJ_Op1wNdLZHt7TuYcAba.K2R91Xr6iw5DyD6M8F 2KjkYyENMcMRyUakJrFb41nESV_xLY1jd69SgE5uKuDmbUkXesUQ3QCk0g.0ZTPrE1Gu6XNV5bVP 30mz7xSbKCoTbXojGFjbftBwuvPQ9D94T2HBSZ.wr4xLxjipNhI8Zmi4_BYdl7vBT_j4QZ30DUt_ Q9zPdRutWL4qwP865EZVMiW6p_kfJEOSZIMbqMkUHLCh7TPzuECIIdW7KHfP4EkcBiaXGcj79q4W wlNkKAfawwdo1RPsW7zneJydpcqbj394yRv7oSw1mJ1O8CEg5wb516nKqDuXyhMOVZoXySWCkTwY ICbgma10u2BfuMov53X6pQFBut27tOnFZQzcaa5itpaaTH1e.rLLY..ngzrD655KaNxyYsvA6OH1 3ivKFp287U4.l1IPN2kRiy_NWbmzrEp9bk14HH9hOs1Sl70Nfuha4Vpot.aj6le6Cwds0hhGdsnE jf7Wh6txmvU1E1_r4h0Ay3Lu4MX9MhPOCU96Wq4B9ycFEIOgBKDmFq2t5q71oH.5knQ1SkVRdaH3 OaD9c_ZuOQKZYEKF3b_8IL8ZzQYxRoBPFQOCwQ1hajsAwaXr5xL.IURW4dTNiI5BzEpcTAwUV.Dx EsyCORZdPg39FBt4Sjk9bQ9dhWqM_Jwed.iucgXTWoMS1buFZoBavWrIRSaVzHD3ZpakSXFe57ri UrmyIX0nkVAq4Jn3STNSHrH4vCro4vXJjg9wNYVIVWdj3ShQ_qo7Vw3GX4vrzy2LwCDBk9E9k._P h_MlFxNBLhHYyLS6l1zlaXf7a2rZzUkerXZkVJYD02zgMkHeuCES3NMWddrK6w1jWpEhQqCbyPI9 qvmFT5DqNleAH5hekfL_Dl8E7pu5kLKeGDOmYRrRAyQFYJWUao4ITsb4.luRVVLa33.ZlY8I33Qw LCYm4sGBcve7EzJR9hjsLKhQxiHLRsuV7o7Ucy.LOLV9i4vf87Y_602iAO7xW0uQ545gm2Jo_gak zNlYvRMj0mijU5vDwz_0EMut_AvtDYOerPKFmjnh6fH1cp16f_fhvY4OcODvLXnNLUW4yKPc3Iui sPBc4s5bYwy8EuCxsc535B9Z4wfbkjVKx3ieRhEX8IIKId2M4_lype0lo.eKy.11Q2pMMo8WwDIO gJWBnrrKWxlhHgcQC88N6.BurHprybXt1R19C4mS5HGGBeXTe5rXaACQNFnNM9tohG0dhrAJIerz UgXT4Tf4M0BgPXxfqmwuy3hziASUgt5cglbDkf5z4OQe1Y0helNZl603OaiOsQ2KQwivjz9ZTPPC hmKbPHE2uoK8CegjvVIkZTH0tdZuKedxZmSpYPQW34XPWqfcpnnKSxeClEZ8dJPYYmqy86g2HM7O mLZ96c15nErAgeVQz_27sI5T9XSbyol6OugiTyNcJUsg40wB59UIp8ifOiQso8ECoYtb.bIKLPwa tGmK5jowcU.DYpW158RHOMWfhT5zK.PXLdPShNO5bMUd5ZTWKXDOxNKnFX4czLVAtejJggDHuAST dYnUfS.OuQn_ijTHlc2ocfklMvR2grTNhEkWcd8TEH7sfAHoPAtOwo4gHddJ1.1bBiBenLcDIGdv ZbqJmv_gfIV6OBUlPSGd4XKwbBEyc6VlLlKX.SBj2WOOhYabGVzc9lcH9.JI.f4AcN0e0vLaqiHj V_g-- X-Sonic-MF: X-Sonic-ID: dee42a02-7368-41c5-8458-948758c7150d Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Thu, 21 Dec 2023 18:36:26 +0000 Received: by hermes--production-gq1-6949d6d8f9-7dnvp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f721cea4849e239702ff1ab86c68f112; Thu, 21 Dec 2023 18:36:21 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Thu, 21 Dec 2023 10:36:10 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SwzfP0K49z3b6C X-Spamd-Bar: --- void wrote on Date: Thu, 21 Dec 2023 15:50:52 UTC : > On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: >=20 > ># swapoff /dev/label/growfs_swap > ># dd if=3D/dev/urandom of=3D/dev/da0s2b bs=3D8k count=3D250000 = conv=3Dsync status=3Dprogress > >^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s > >60557+0 records in > >60556+0 records out > >496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) >=20 > 22MB/s is usable, I think. In my context, I'd be satisfied with that. > My context differs from yours slightly in that yours is SSD and mine > is spinning rust. I do not have access to spinning rust to test for comparison. Others likely do. > This is unusable: > # dd if=3D/dev/urandom of=3D/dev/da0p4 bs=3D8k count=3D250000 = conv=3Dsync status=3Dprogress > ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s My point is that the performance seems to be strongly tied to the media type contributions to the performance. There is no general problem with partition swap based performance. (But the type of test was set up to match yours, not to be realistic for paging activity.) The paging access pattern likely ends up doing lots of seek activity, making for lots of accumulation of latencies. It also is likely is a mix of read and write activity. Mixes of small reads/writes to fairly random palces tends to worsen performance compared to sequential. Paging is not a good match to large sequential writes as the only activity. Perhaps someone with fio background can fully specify how to run a noticeably more realistic benchmark for making swap perforamnce judgments, perhaps monitored via gstat during its operation. > because it's way too slow. Swap never gets fully reclaimed, > thrashing happens, loads of other followon effects happen.=20 >=20 > The same partition formatted as ufs reports 113 MB/s. Multiple swap = partitions > have been tested, then converted to ufs. Results are the same. This uses the file system caching and larger than 8K writes --and only writes, not a realistic mixing of reads and writes or the more random distribution of where the reads and writes would be form/to. conv=3Dsync does not prevent the caching effect for sequential activity as I understand. I suggest using: # gstat -spod to get an idea when the actual I/Os are like in each of whatever relevant contexts are of interest (actual operation with the swap performance issue and benchmarking). So far as I can tell only you can provide such information, as the issue is not readily repeatable by others. =46rom a broader view, actual-operation examples from "gstat -spod" output might be of more general interest for your type of context. > There are no reported errors in smartctl. Long smartctl tests run = monthly. >=20 > 5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0 > 9 Power_On_Hours -O--CK 001 001 000 - 48992 > 196 Reallocated_Event_Count -O--CK 100 100 000 - 0 > 197 Current_Pending_Sector -O--CK 100 100 000 - 0 > 198 Offline_Uncorrectable ----CK 100 100 000 - 0 >=20 > I can't find any hardware problem here. Possible workarounds, bearing = in mind=20 > I'm not versant in C so it's not like I can fix this myself in code: >=20 > 1. swap as swapfile and not partition [a] (1) is subject to "trivial and unavoidable deadlocks". After suffering such, I avoid always avoid this form. > 2. swap as nfs [b] I've never used nfs for this but it likely has the same issue as (1). > 3. swapoff & swapon script running every minute [c] If this ways works for bringing everything into RAM, it seems to be an approximation of not having swap in the first place and would be subbject to (4). > 4. just turn all swap off and reboot after crashing (undesirable) (I tend to have active SWAP partion(s) that has about 3.8*RAM space because of doing a form of high load average "poudriere bulk" runs.) [I have multiple SWAP partitions because of using the same media in various machines that have widely different amounts of RAM. I form a total active swap space that is appropriate to the RAM present for the boot. Other than that I'd use just one partition.] > 5. use another OS that doesn't have this problem You omit the alternative of using media for the swap/paging space that avoids the problem. There are such around. Is there a blocking issue for going the direction of also having a separate swap media that has helpful characteristics? I will note that the RPi4B shares its USB3 bandwidth across the 2 USBC ports: they are not independent channels. Having sustained I/O that competes for the bandwidth can be a bottleneck issue of itself. A similar point can happen at the media level when the swap space I/O and other I/O are to the same media. (For spinning rust, that includes more time spent seeking: additional latency.) > [a] not tried yet, and i hope it works. Legacy info suggests swap as = partition is usually > faster than filesystem-based swap. But the reverse might be the case = here. >=20 > [b] also not tried. This, I imagine, would be filesystem only (I'm = unsure a zfs volume can > be exported to look like a mountable partition to the client) >=20 > [c] https://github.com/Freaky/swapflush.git - usually works but maybe = i need to run it every=20 > minute instead of every five mins. For testing, this script was = disabled. >=20 > Any additional suggestions on how to overcome this problem gratefully = received. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Thu Dec 21 18:56:21 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx05V0LVJz547sk for ; Thu, 21 Dec 2023 18:56:30 +0000 (UTC) (envelope-from SRS0=yuxr=IA=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx05T4tyYz3g0f for ; Thu, 21 Dec 2023 18:56:29 +0000 (UTC) (envelope-from SRS0=yuxr=IA=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Thu, 21 Dec 2023 19:56:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1703184982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PbsUI1AQLsB8wt9I3atnKWK/iaGpiuO6MxizQ7F6tZE=; b=LCvaDAI8rbTbVaJaao87f4q3KsucHyj+YZdDdJFguL/gAPLKgqVfZcMzZoiNTh7OvkKsYX QeIddO6sdyn4lq+jbKIGuUbE+fI2obY8gML1fvOYUQpf970xVJ9xbXHI5E9/C0+7WAjO+B 8UpZftgmi6eigUaK8BHkJwilMOrFUVq8BXTTzz2CTvd24lGiAflhSibKCwyx4IMuCIOsEB BOZrZosvBT6BITkChh6Kty+inK0FnS7Y4Gulkx+AUwyEt/tG2SenG7UhVjWNAOQajV0IkH 5yhVezG8OWO6TXEx4zdOYtQGknnxBjksKsYz1AuyRlR5sbzYd1jwJuEJazmBAA== From: Ronald Klop To: void Cc: freebsd-fs@freebsd.org Message-ID: <1439231787.9333.1703184981914@localhost> In-Reply-To: References: Subject: Re: measuring swap partition speed List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9332_1497520887.1703184981868" X-Mailer: Realworks (683.53) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Sx05T4tyYz3g0f ------=_Part_9332_1497520887.1703184981868 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit A bit weird that a simple dd is so slow. Just a quick thought. Are your partitions aligned properly? Is other IO going on on the same time? What does gstat say about %util, queue length and all the other stats while running the dd? Or "iostat -x -d 1". Could you try if another disk has the same issues? How is your disk connected? USB-to-SATA-adapter? Any output of dmesg? Regards, Ronald. Van: void Datum: donderdag, 21 december 2023 16:50 Aan: freebsd-fs@freebsd.org Onderwerp: Re: measuring swap partition speed > > On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote: > > ># swapoff /dev/label/growfs_swap > ># dd if=/dev/urandom of=/dev/da0s2b bs=8k count=250000 conv=sync status=progress > >^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s > >60557+0 records in > >60556+0 records out > >496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec) > > 22MB/s is usable, I think. In my context, I'd be satisfied with that. > My context differs from yours slightly in that yours is SSD and mine > is spinning rust. > > This is unusable: > # dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress > ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s > > because it's way too slow. Swap never gets fully reclaimed, > thrashing happens, loads of other followon effects happen. > The same partition formatted as ufs reports 113 MB/s. Multiple swap partitions > have been tested, then converted to ufs. Results are the same. > > There are no reported errors in smartctl. Long smartctl tests run monthly. > > 5 Reallocated_Sector_Ct PO--CK 100 100 050 - 0 > 9 Power_On_Hours -O--CK 001 001 000 - 48992 > 196 Reallocated_Event_Count -O--CK 100 100 000 - 0 > 197 Current_Pending_Sector -O--CK 100 100 000 - 0 > 198 Offline_Uncorrectable ----CK 100 100 000 - 0 > > I can't find any hardware problem here. Possible workarounds, bearing in mind I'm not versant in C so it's not like I can fix this myself in code: > > 1. swap as swapfile and not partition [a] > 2. swap as nfs [b] > 3. swapoff & swapon script running every minute [c] > 4. just turn all swap off and reboot after crashing (undesirable) > 5. use another OS that doesn't have this problem > > [a] not tried yet, and i hope it works. Legacy info suggests swap as partition is usually > faster than filesystem-based swap. But the reverse might be the case here. > > [b] also not tried. This, I imagine, would be filesystem only (I'm unsure a zfs volume can > be exported to look like a mountable partition to the client) > > [c] https://github.com/Freaky/swapflush.git - usually works but maybe i need to run it every minute instead of every five mins. For testing, this script was disabled. > > Any additional suggestions on how to overcome this problem gratefully received. > > -- > > > > ------=_Part_9332_1497520887.1703184981868 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit
A bit weird that a simple dd is so slow.

Just a quick thought.
Are your partitions aligned properly?
Is other IO going on on the same time?
What does gstat say about %util, queue length and all the other stats while running the dd? Or "iostat -x -d 1".

Could you try if another disk has the same issues?

How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?

Regards,
Ronald.

 

Van: void <void@f-m.fm>
Datum: donderdag, 21 december 2023 16:50
Aan: freebsd-fs@freebsd.org
Onderwerp: Re: measuring swap partition speed

On Wed, Dec 20, 2023 at 07:48:14PM -0800, Mark Millard wrote:

># swapoff /dev/label/growfs_swap
># dd if=/dev/urandom of=/dev/da0s2b bs=8k count=250000 conv=sync status=progress
>^C478830592 bytes (479 MB, 457 MiB) transferred 22.001s, 22 MB/s
>60557+0 records in
>60556+0 records out
>496074752 bytes transferred in 22.790754 secs (21766491 bytes/sec)

22MB/s is usable, I think. In my context, I'd be satisfied with that.
My context differs from yours slightly in that yours is SSD and mine
is spinning rust.

This is unusable:
# dd if=/dev/urandom of=/dev/da0p4 bs=8k count=250000 conv=sync status=progress
^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s   

because it's way too slow. Swap never gets fully reclaimed,
thrashing happens, loads of other followon effects happen.
The same partition formatted as ufs reports 113 MB/s. Multiple swap partitions
have been tested, then converted to ufs. Results are the same.

There are no reported errors in smartctl. Long smartctl tests run monthly.

5 Reallocated_Sector_Ct   PO--CK   100   100   050    -    0
9 Power_On_Hours          -O--CK   001   001   000    -    48992
196 Reallocated_Event_Count -O--CK   100   100   000    -    0
197 Current_Pending_Sector  -O--CK   100   100   000    -    0
198 Offline_Uncorrectable   ----CK   100   100   000    -    0

I can't find any hardware problem here. Possible workarounds, bearing in mind I'm not versant in C so it's not like I can fix this myself in code:

1. swap as swapfile and not partition [a]
2. swap as nfs [b]
3. swapoff & swapon script running every minute [c]
4. just turn all swap off and reboot after crashing (undesirable)
5. use another OS that doesn't have this problem

[a] not tried yet, and i hope it works. Legacy info suggests swap as partition is usually
     faster than filesystem-based swap. But the reverse might be the case here.

[b] also not tried. This, I imagine, would be filesystem only (I'm unsure a zfs volume can
     be exported to look like a mountable partition to the client)

[c] https://github.com/Freaky/swapflush.git - usually works but maybe i need to run it every      minute instead of every five mins. For testing, this script was disabled.

Any additional suggestions on how to overcome this problem gratefully received.

-- 
 

  ------=_Part_9332_1497520887.1703184981868-- From nobody Thu Dec 21 20:24:37 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx23S1wWwz54DDR for ; Thu, 21 Dec 2023 20:24:52 +0000 (UTC) (envelope-from bakul@iitbombay.org) Received: from mail-pl1-x62a.google.com (mail-pl1-x62a.google.com [IPv6:2607:f8b0:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx23S07Xnz4KK9 for ; Thu, 21 Dec 2023 20:24:51 +0000 (UTC) (envelope-from bakul@iitbombay.org) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1d3f2985425so7708955ad.3 for ; Thu, 21 Dec 2023 12:24:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=iitbombay-org.20230601.gappssmtp.com; s=20230601; t=1703190290; x=1703795090; darn=freebsd.org; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=mBph6f3m0/UhhP0lxeZ+loiBeFx9c4Yid7L+tO9MshM=; b=UBAqX8/nJJRWL/xXcYNlJmAl1ZlTdajA9VZz4GeBbSoJ/kKfgPs/zRvqfegc9Wz/ru fVwJua5jLNNvdukNOxxk5C/zp7l5qH8tjLoFbG4aL77uOA4fd+dV00DtgTFb/RmosGV1 WbhVQ+ouyP99w6y4RkQuXtbLWYkR/1cKjtTTy9Q0Z0hiS/QBaZZf3/Wser3P4LLCnmPU 6SQVRLGi9FlJvCkcnfR2Jk5PRxgfB6gze1k4KtXCJ5RFE2dOyOEe+sCk+z20V7p2uD8X rc4FRqlPBHM7NbtuvTxNK64cWY/CRsV9MkatSPNzVtyNVnJ6LaAAUR9kKvWWWk0dcQ4R xpbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703190290; x=1703795090; h=to:references:message-id:content-transfer-encoding:cc:date :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mBph6f3m0/UhhP0lxeZ+loiBeFx9c4Yid7L+tO9MshM=; b=slx5RDFR4fR39hSrvT1BcNV/RT+Uo0S8PVUAB3MhwydNm01gR3V472DnSL5yMsmrQ5 eKMvOokigvBhMrGwAPYq57NTxmU7jMKNYJCSCfbSwu4V62oTKlJmrze4nnV3QJ+P/Nlv 2/S4mbthHMIY/3/BU0p88P1c+zyIFb9njzk/SA81bTiUGiHbev8QKEjAfGzTGlXJt2s0 DnceIwiqFQSCE3hR259GZuUOegwXyQwukJV+t8BmdtBdsQJL7goEk74jzCvTDct3S1fU S5FWOhOacNkP6lWFArf3GqSz+0upRchMK4xW5NRqnUVcBMkpZ5/w5FyapyRdj7PxDnvB Hs/A== X-Gm-Message-State: AOJu0YyWxqs+cF/VnsGmKkXHFnBllvzmvV/a0u5Wqm1gSKeMW5C4q2/u furB3XVMlpq0Ab709yaRtn4/encxhiML X-Google-Smtp-Source: AGHT+IEfODcR2UTrcXoWlgST+Vdd9sZjofJ8gN4uIAnN9e22qEGhPjIwDDxnd6EL9gohJMBhuSxuEQ== X-Received: by 2002:a17:902:d507:b0:1d0:6ffd:836a with SMTP id b7-20020a170902d50700b001d06ffd836amr187362plg.117.1703190289660; Thu, 21 Dec 2023 12:24:49 -0800 (PST) Received: from smtpclient.apple (107-215-223-229.lightspeed.sntcca.sbcglobal.net. [107.215.223.229]) by smtp.gmail.com with ESMTPSA id a10-20020a1709027d8a00b001d38ca8cbc2sm2042539plm.156.2023.12.21.12.24.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Dec 2023 12:24:48 -0800 (PST) Content-Type: text/plain; charset=utf-8 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed From: Bakul Shah In-Reply-To: Date: Thu, 21 Dec 2023 12:24:37 -0800 Cc: freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <6B1D921F-0640-49F0-9DFB-07B65C056F6A@iitbombay.org> References: To: void X-Mailer: Apple Mail (2.3774.300.61.1.2) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4Sx23S07Xnz4KK9 On Dec 21, 2023, at 7:50=E2=80=AFAM, void wrote: >=20 > # dd if=3D/dev/urandom of=3D/dev/da0p4 bs=3D8k count=3D250000 = conv=3Dsync status=3Dprogress > ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s Is this on a pi4? Could it be that you have connected your USB disk to a USB2 port? Use one of the middle USB3 ports. Another possibility is your disk's USB port is a USB2. From nobody Thu Dec 21 21:03:41 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx2wc54MQz54Gp4 for ; Thu, 21 Dec 2023 21:04:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx2wZ6Fb3z4MPL for ; Thu, 21 Dec 2023 21:03:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=G6cpx04r; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703192635; bh=3R0zBayWi3betHKi4g8lAtcnxgIFAkre4oAfFUF7gM4=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=G6cpx04rZvRm8CdtgED/ihzTzBe0q5XE6xV1JtEokpDsgS3AyS6EIY28kg7TqcbrF8qIhDaP6dYPKsRn2Jo7Q+2TlaJKrEYnKkFYuKbeFKwMIs58q4Sxn9Q6AKhCyF3+w5G+litLP8FbtUhUOP62OiQCk2750Ba9oJTZ3osNBN9ho1Fmmdv9eW8Ra3B8VUzRmNN0rubkzZpMsOPmxwnWRP0iOOJfhrKv9tGyyFdJ22PS5U1BOwYK6b8RTGzYDRy9OJDUWXVrctYYTQFOVIxItfg5dQWCCEzN4Zrm1WLoOL2cX6sK5Z0NPX99ipjkR8O0jKK4pTMAzdRCSgm9NJW/kQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703192635; bh=LG56rLqrRjffnqELTb5b2Pz0O0CakDw92zERMdp5b+w=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=nubIsXRE73oxykbPccAsRSvivSnSJTmLsywAxPFR5jHOavB1ef/80WC9aEJJrci9XwdjrHCAUysA0jjUkDbvqw0n7m2SGzsft+JDcUHNsN0EuOTol1EYPCqqMbsiF7AVCH2WnRGMRpp0QaWbyBgXXsvvl79QzIyZIDa2wfLTjzy94cwaFvB994CtWapmy7hvCTBoDo9eUi3kxsFuNTCuL9oI8PtGQJ+scxkIVZU5ET0WGVqWiIF6X3HV+OfO0ESuu6iKkwdb8UTeuvqMXHc9AJP8H+b18bulIpyI+cuzwqWGjT4BWnHoKtY3hSC+j91Q+o6aYHGhx2QWwP6nY8HDnw== X-YMail-OSG: gkY4gjQVM1lwm4DsufZQgTHAT3ri2g4vGVtWzF_deESZOFH1vb0FobZXPkuYihE qR0yX5tfuhppvdNm0zejaeaoOxRmWAH1SI85s31cIa24RnPuoMWkSOr_1X2wSXwGBaEgpFGgUFgp Fs17zL6qS1NFVVV9zpk8IcVyyIx3aZkKBXguQFSVBJGaZoYCApnqZMbj_d9t4J9CUzlPr6OuIhmq DzxfJHayDeh5n6oDiAk7EgUyUo12J1DGdE5dcQBZXVu1lK__ANwNATRsNkXOmTJOEa3rCskj2pl. VYPEMEgAd7oZ5Toz3nJw2_5.x0DHNGrcaArTQrU1disPkoI6QhKzz2yldHdDvs3.1LMaHNrPEuB1 Bp0gywKKrsvrD8hxIvjtfYGJH5NWPxmrFtNE1kuHAxvf_74gjF4T_bnvFPcKvi2ZdmH51E9kCQnF iTIzfA7AmiSXpwuhYfIowc6ffEn2WkdMGJSNb46_EWrHgHqw8getm619aazIPAOvp3NlMvacEtcJ 8rfm1P7Kip07pXZMt7qKwxiD48lZO1ZV77ASqrCupk1KNCN_HnOK7MP8JusxzCPyYAMWjcruOQez isRfDittkHC8u5TB_3njc2jlOF.mWrD7Dd0X5TbpJzK6Qahmj05IgHitf9SQaU.46eW0fGHhsvz0 J0viNqgb1KKU3pF5hZ_DcqK1PVm_YBA0aqSEPBfZoGoFXcUchXEs9Ij4YugjoRftggGCtvsGc4Zz 8XiIdXL6clZGEz5e3Y5DH0Tg9Jak1DgO6t6UCH1K3cAAoOpggBZovgrACm_JVbfKXKCL5w08jxP9 ybHfu1ZDmRxr1oGZLuYMb3G57Xe2oYwMOekUUOMKVbxMorNcVajj0TgMvXLGNFyMnBKH3e3c5iez H6CUsZV1nJhpc.rOWNIZARvNnW8kNjJjx84iskDCfZARZ0GyAxmowVqTSe5Z.4gHyjXifJm_uhJO 8ga1ZgNGdTDA6kmGid.UDIoymeMhrMBxa_m3vdP_FfE2oidrPL9TSXz5WZwOMX.Li_OFHMRYw6QJ 3YZFpTlKnpektWyD.GBoPF.wzbblVIScrtnOQb8n8ao0xGe_hMLw1KIdIBanPgNHzwHu2l_BHYIx 8aLgYPJJ.Em93Gn.a5M5mMPqukkztNHwmTH5S2stiMYf18q_HrH3CteQ4QVIVmt5TT.4bZs10veL q_taIEmaijPUjCqV.pN_GnU9zpw2tHBYzoAzxETus6diMUivpQsVvJikz2imktPWfxEFbCYsptva bXQ7IT7CHO9kP5ybmL8X6hTl_.tRHwTJQqKengH6zRfowCb5bbE2M7oZhP0XenwHiGXGyOTAMPOw .rh1We8cMx8Dp4bAe19kfXcP0nKEoFViaguJgv15QWOed4L_1.wA1hd2AKidUGeiOnuWKqDLMJqA 75snrFPAWKBDc3ochigjisW.rWOHgJpu1EjfPNqOHP7ayWGJvOjYIqV0.IN6878CnndPV2zJ4kPf Bj4HhLm5eyYe752e6MHHC8Z99TQOzVH3JRp6ewGhNlLi0eKW3KUA2Ow8m_vq1JGSzfXS3U5v9wBi PPFh.OymqH5hRWM7VTK9kuWui1t_L0KdGgY34fo3G3DOCxbnfSe7cZ4AocGy9WR9nnScIqTsARyU ScCTeyFSRV18zVVMJ8O3IhaFO5HCUdbmL9g9xJrVS098JZla3jxbNUkkSvXk9w7JMXH7YHOWHYCg Ef7uvsw__ocoBc6kbRgcAJk3PbvFKEvOf38yDR0gibk_8W_zROtYXohu4Y4S5JyuwlOw233W5OJK vydMZRJzLHUzfabsUKxxeJ3rBBvZUMygoB7TtqFvTJihQdJpC860v319paPr9Ir2Xsivd4YsYXmE x9T8T9HxZsbS1ZejaZlBU9OV6x3lBp7PAWharbnDY5C.QrgnVJRt8Wg9vGRkf1xaLR0uE.PMJKQl TsakoiXa4A.NrwQU.Ap47GOQvKBPY4NT5.CENRhlhpJBeq4bk.GqkCSBqFLAFeezc90uX0i399Aq _h0MrANBu2sSUu2L6LlNdtOHmtk3Mzwpo2XPvqhLB4NQ8oJLSCrTX6WxyaDETIRoFAu3Lyc6vNHg _nTTiytwEdYM0z.ZDhhmTAWQmIay0sdQNj7ABctzbz0Ob1t9KTjXaLo7mJHxk1sMMRkeHj3b9p3C jWH1gz2VzAXhBSKWutjexFfQp1BCpn8d6hOxH5b1RJ2uIEkQXd9pRFtZJqaR1uCkyyUbtaYrho8V fFxKM2qzDSryUI6S7DAlt6Bcg77TfF5uWm.a_DB6ZO_iTPAPACaFsPHL3Y.ea1VbccPUmpRP4H4k - X-Sonic-MF: X-Sonic-ID: 2faf0906-f81f-4396-bfb9-ec4eb366e4e4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Thu, 21 Dec 2023 21:03:55 +0000 Received: by hermes--production-gq1-6949d6d8f9-ghhkt (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 0596b7c5c0c42bb8f6c37c390cb4b0b1; Thu, 21 Dec 2023 21:03:51 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Thu, 21 Dec 2023 13:03:41 -0800 To: Warner Losh , void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.45 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.953]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_TO(0.00)[bsdimp.com,f-m.fm,freebsd.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Sx2wZ6Fb3z4MPL X-Spamd-Bar: --- Warner Losh wrote on Date: Fri, 15 Dec 2023 15:41:10 UTC : I'm trying to figure out what to do with the fio, what adjustments to the example are appropriate, both for the orginal message's context and for what I have access to. > . . . > Now, having said all that, your best bet is to run a FIO test. fio is my > go-to choice for doing > benchmarking of storage. Do a random workload with a 8k write size (since > that's the page > size of aarch64) I'm guessing the figure in question is: # sysctl vm.stats.vm.v_page_size vm.stats.vm.v_page_size: 4096 Is that correct? > on one of the swap partitions when it's not in active use. > I suspect you have > a SSD, and that it will kinda suck, but be in line with the swap > performance you are seeing. > > I use the following template for my testing (128k should be reduced to 8k > for this test, Again: # sysctl vm.stats.vm.v_page_size vm.stats.vm.v_page_size: 4096 > though > I've not looked at how much we cluster writes in our swap code, so maybe > that's too pessimistic). Is that based on: # sysctl vm.phys_pager_cluster vm.phys_pager_cluster: 1024 I'm unclear if vm.phys_pager_cluster is relevant or, if it is, how vm.phys_pager_cluster should be put to use here. Where is the 8k figure from? > You might also try reducing the number of I/O jobs, since I'm measuring, or > trying to, what the > best possible sustained throughput numbers are (latency in this test tends > to run kinda high). The original E-mail reported: rpi4 and a usb3-connected disk So: 4 cores and the USB3 is one channel that the 2 USB3 ports share. (There are 2 separate USB2 ports.) Later "spinning rust" was mentioned for the media. (I've only USB3 SSD or USB3 NVMe access.) > ; SSD testing: 128k I/O 64 jobs 32 deep queue Spinning rust, not an SSD for the original message. (But I've no access to spinning rust to test: USB3 SSD or USB3 NVMe access.) Again: 128k's replacement? 64 jobs and 32 deep: might not be a good match to 4 cores and single channel USB3 spinning rust? Or my context: 4 cores and USB3 SSD or USB3 NVMe? > [global] > direct=1 > rw=randread Only reads? I'd have guessed a mix of reads and writes, not necessarily in fixed proportions. So I'm not sure if rw=randwrite was the intended replacement or if something else that would end up exploring mixes would be the intended replacement. > refill_buffers > norandommap > randrepeat=0 > bs=128k 4k? 8k? something else? > ioengine=posixaio > iodepth=32 ??? > numjobs=64 ??? > runtime=60 > group_reporting > thread > > [ssd128k] === Mark Millard marklmi at yahoo.com From nobody Thu Dec 21 22:34:48 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx4xN42THz54MWf for ; Thu, 21 Dec 2023 22:34:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx4xN0jCgz4T8N for ; Thu, 21 Dec 2023 22:34:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703198088; a=rsa-sha256; cv=none; b=Qj0/w9DXX+yl2PJKnhWXFiUnFFDDV7TTMzjM/DcReBcdPxGIdRZh4O9lxYUBBgQ+IebHIx 0T7Q0dFsdal0WU91EVrRipYbBkEsZxO2oC8I7XX6MDr/jWJDk8Ij6LWsoy4FhWPdfU8zKn ttJTjRwKio5+zuVNGsHPTtsP+5VSKXAHBscjnEzS7+tAN/YJEAZCEoKdqD9EVKi+QBZ3CU I6g8rTZSwPglRSEbtzWUSSDEjkbOAhfQgT8R1Is7qnxKgTztavyxEXwv9RLfpVdqBe7FRe sJMLQK56mnXxFiug0FfFlo9Hc/S0XB1vRu7/UYLX4AU4fuPyaH3bPT6sicxf9A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703198088; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=d1fZInjJ+xWz6yTpIwAubaD6QdsGnI/uZZNoejXSWDs=; b=egwk3+0g3SGXk5VGKIq5tbc14ThMSXSUkM3JZAwE3aRHOF6SOFYwMSFzlI1g9w320kZfK2 RIVzaQIkUxDzTSEhw5uuFN/gaRkv48JAmgPLQDFsFyEXOEVHq3V0H0RgTX5NFpDrl3OkaA XFtCYjP2h8v4exuKueWjndUx05/4fL8e3Ssgrr3wRlqzZocqhZjGoCAX//Os7tcTmHcUWf uS5rGp1TIa/UTCKLKoS/l6RF5Ks/y13ntbUL0TR68NTOv1buFyAeq6pAg+vuN8FWwYJ2RJ c2nBLip5caNjXn/YoeCfa7xIj47LbRLMrRKcK5lrQcXn48MWNtIJyadbZ3PMjA== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Sx4xM6v60zrKY for ; Thu, 21 Dec 2023 22:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BLMYlfU079759 for ; Thu, 21 Dec 2023 22:34:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BLMYlbh079758 for fs@FreeBSD.org; Thu, 21 Dec 2023 22:34:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275871] [unionfs] [nullfs] [zfs] corrupt filesystem Date: Thu, 21 Dec 2023 22:34:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275871 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Thu Dec 21 23:50:15 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx6ct0rDGz54QKN for ; Thu, 21 Dec 2023 23:50:38 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx6cr6hz0z4cgZ for ; Thu, 21 Dec 2023 23:50:36 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=ebUl1E4l; dkim=pass header.d=messagingengine.com header.s=fm2 header.b="A Ewjbsp"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 2763B5C00FE for ; Thu, 21 Dec 2023 18:50:36 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Thu, 21 Dec 2023 18:50:36 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:subject:subject:to:to; s=fm2; t=1703202636; x=1703289036; bh=AdU1NefESgxpOzSfLECGtWj3cB7RiPDUAL2b0DGPnzo=; b= ebUl1E4lFevGjDKiXsWkZWFAuL4QxMgXqZgYDPbUBpgiH2dbauSOJbZJIUTtIuzR lT/neKj6tJ72KwoFY4swQTU1hJqwMAVMWb2hEDl454U4j/3dTwGJ5u8oY68ANiII 3/KeLLZhwK7DZYK1Gd1azuDqzKUVw9PmGY+dBQCMMzP7mTmPwM//Ia0BQuWTCRdd 6V8Gn+Wpiev8j4Xv+uSfvAUGZ6eW3knrkV8ZfzCfJ2mzw/NDcqfh5YS0MoAynNJ/ d4oSaIux2hBNpUfXNafWn/nhcANybiYjbFZMki1aF4pU3jAdypq8RI03ua6Xf2r4 E/OudfCcxPRaqI0w0nJ1oA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm2; t=1703202636; x= 1703289036; bh=AdU1NefESgxpOzSfLECGtWj3cB7RiPDUAL2b0DGPnzo=; b=A EwjbspiXKPC0g7IfJ8G+zvFMu/Y/M6VIjiSF9xjwPVCTAQwgRTTOUHOt3gObwU+u HC3DBAkTamn361Ss+doJYfimeDI+GcIsfUyIUA0HiJ/SFkjH1SJLjzl1aWDXBVLR N6+AZPrsYEFO2NkZfmv3W/+rl75zNjxbSosURG1EzaxuzUl7HH0cit2rn9urhQma km8CZXAa25A8l3gtNaemaw7D7bO9s/j1xRp23Zta0tRQkGhY6WrlYGeVtKeCxxyG gmjRb2l+OD1PJUNcn9rmRvZdetXfZCi+eS1iYb3LhBZ3ftLIf5giIgQzuPq0L/xx /1sPzguYvo6EskyifS9yA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduiedgudehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggft rfgrthhtvghrnhepgeeuieduueeluedugeeiuedvtdettedvgfffueektdefjeeiueehje fhhedvledtnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhepvhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id DAB472A2008B; Thu, 21 Dec 2023 18:50:35 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: In-Reply-To: <6B1D921F-0640-49F0-9DFB-07B65C056F6A@iitbombay.org> References: <6B1D921F-0640-49F0-9DFB-07B65C056F6A@iitbombay.org> Date: Thu, 21 Dec 2023 23:50:15 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.19 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sx6cr6hz0z4cgZ X-Spamd-Bar: ---- On Thu, 21 Dec 2023, at 20:24, Bakul Shah wrote: > On Dec 21, 2023, at 7:50=E2=80=AFAM, void wrote: >>=20 >> # dd if=3D/dev/urandom of=3D/dev/da0p4 bs=3D8k count=3D250000 conv=3D= sync status=3Dprogress >> ^C11862016 bytes (12 MB, 11 MiB) transferred 40.063s, 296 kB/s > > Is this on a pi4? Could it be that you have connected your USB > disk to a USB2 port? Use one of the middle USB3 ports. Another > possibility is your disk's USB port is a USB2. Yes it's a pi4, disk is on usb3 and it's what I guess would be called a = default=20 usb3 interface (in other words, the connector is usb3) --=20 From nobody Fri Dec 22 00:05:01 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx6y133vnz54SC8 for ; Fri, 22 Dec 2023 00:05:29 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx6y05WsTz4f7W for ; Fri, 22 Dec 2023 00:05:28 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=fKkaOE6S; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=GHedZTaq; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 5A9EC5C01F4 for ; Thu, 21 Dec 2023 19:05:28 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Thu, 21 Dec 2023 19:05:28 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703203528; x=1703289928; bh=oNsKr+INrr aLYR5U34gih8xsoM1DtCUlQU8xC3Zz7zg=; b=fKkaOE6ShYjTDlmfpnCIs7FI+B 0i+cQZSpVhKUCDbn8LpLIdZEqSJtWFDbfINkqZGdS2QAmdQMcdCrkSrwW++woWc5 uQe1BC3aSN01SNVet3MNPDpmtLgr9CsejzaZgdCqWVeJ6+jz1j754wYmp3PyUXb2 x3JCGXNCRlZclU/2SBbLLoFckmTvirmQQXKSkVBH+ycvIC/cJXqkTTds63UIL6ih WGq5kf8kjGcfyQX+qBoeTMixipbnTKb73+XsWcskoe5sSWDr8lrUp/i0exlLHUkc xSdkjNxEZfb5xqWVzN4/jReKUAq4K5eaMRhyjcaIpaH+mXNDvux/18y5PVyQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703203528; x=1703289928; bh=oNsKr+INrraLYR5U34gih8xsoM1D tCUlQU8xC3Zz7zg=; b=GHedZTaqiaGCbtkMr7nLsp0W85q+J0M865ahtHbx83lb CVj3gFM/ZueaJq85B8fTaqZ9rBidWK3GSF52CBNeCakgItLKTrZIupj0f7E3+4Gh 8egQdtOA4RPXqvhUWgZY3PZX4/bLIkRgsYpa3CwP8i9Kz8PVKy5y7INpX0F62C8Y FIMTQzzao6Ra3jaqFKssTvMM1HxEbP0/G+4mvdXa51oY/zVEjxjcx+G1e1treeuy wg52ytxvdrIPHjfNjTn4Dlkw5JyKdkCUqzCzFOVDVJ+P88jH+314y72I3rk9rVlb K6KK/WVTk1OpGYNj5pYw65/hi3DeUAjVuAbdX+rRsg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduiedgudejucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 18CE72A2008B; Thu, 21 Dec 2023 19:05:28 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> In-Reply-To: <1439231787.9333.1703184981914@localhost> References: <1439231787.9333.1703184981914@localhost> Date: Fri, 22 Dec 2023 00:05:01 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.19 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28:c]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sx6y05WsTz4f7W X-Spamd-Bar: ---- On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote: > A bit weird that a simple dd is so slow. > > Just a quick thought. > Are your partitions aligned properly? no idea. How would I check? The system when installed, the auto-zfs option was selected, block size set to 4k. The swap was changed from the default 2gb to 12gb. geli encryption is active for the zfs filesystem but not for swap. It's only the swap that shows performance issues. > Is other IO going on on the same time? not really, the tests were done with cron jobs deactivated, almost completely idle, swap was not being used at all. > What does gstat say about %util, queue length and all the other stats > while running the dd? Or "iostat -x -d 1". I'll try that next, thanks > Could you try if another disk has the same issues? no spare or equivalent disk for this arch. > How is your disk connected? USB-to-SATA-adapter? Any output of dmesg? Regular usb3 connection. da0: 400.000MB/s transfers da0: 953869MB (1953525168 512 byte sectors) da0: quirks=0x2 da0: Delete methods: -- From nobody Fri Dec 22 00:19:39 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sx7Gn1pDKz54STH for ; Fri, 22 Dec 2023 00:20:01 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sx7Gm5PNbz4g5B for ; Fri, 22 Dec 2023 00:20:00 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=s1H8uayd; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=f8+ao67A; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 650625C0243 for ; Thu, 21 Dec 2023 19:20:00 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Thu, 21 Dec 2023 19:20:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703204400; x=1703290800; bh=fpQKu0Cdxf ux9Wp8vVDMk1zyx/yES8HOyXNRc8ARF74=; b=s1H8uaydlN7JADmR3TvypVXwSu zGndMZ5tbA98YDVWYGY0MajzzLtBdFN6TtF/Qh5o6fJ0pvT+Ph4dvt9Ktg8tViZM hORGHjJYjrFqTtTtZtS8v5CRsMZrjVcXuYrBZh6O2GTVhtPSPLqDEXVGTCqrvyse pWeKNx4iegj7IHnxVl5kPTaJh1UvsmhtsInqWmGT7h5MZ/NqNcNhuVdumGav0NQM iC4oYBjqS57JMeGg0y/GiZv9Vgpuwxa+jr4jZrVrptc9TGoRpKTGI3ULwEU3ulAl poSFnP95LpufXh/UMN2CUCSDeTiL9SBFxFwu9e6+iCA38osDHg5OZ8Hql8/w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703204400; x=1703290800; bh=fpQKu0Cdxfux9Wp8vVDMk1zyx/yE S8HOyXNRc8ARF74=; b=f8+ao67Aw0AGtcNyo1k9VlI6j9eAQ1ObDYZ3M6ybt8rc a9g8ebEblz7BGVHmqn05uEg3ADRMptmiWoEY7/SW1Vp3sQ2FzsXaTAhbIZb5xrid Eh2YKedc0058uZ9Ylp2fvc2NlS6sk6ScS676//3ct35DvqY+TIHo1KiT18ReSBN4 h5uYG3gkz2eX4+T4q6OgKcIDH9qVgM7ZTTsNNolp/l99h7yBQdFbqj0VPUIQF4w0 ZQXIxXEtGP/4tMJWzshBqMOT4xrow385KyKSW0mG1xpubF16bAk1v9iFn2Bey+Ga 2amnRnYtSCidjD2EumVG/+nuEKMG9mvAaI4cDvPX8g== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduiedgvdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 2BFD52A2008B; Thu, 21 Dec 2023 19:20:00 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: <2dcc21bb-f446-41fa-871b-2c37cdf9afe0@app.fastmail.com> In-Reply-To: References: Date: Fri, 22 Dec 2023 00:19:39 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.19 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28:c]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sx7Gm5PNbz4g5B X-Spamd-Bar: ---- Hi Mark & Warner, I tried using fio with Warner's suggestions as a template, and pasted the results in the latter part of this message. On Thu, 21 Dec 2023, at 21:03, Mark Millard wrote: > # sysctl vm.stats.vm.v_page_size > vm.stats.vm.v_page_size: 4096 Here, this setting is: # sysctl vm.stats.vm.v_page_size vm.stats.vm.v_page_size: 4096 > # sysctl vm.phys_pager_cluster > vm.phys_pager_cluster: 1024 It is 1024 here, too. Only the one usb3 port is occupied. fio output follows. No idea yet if the settings used are suitable for this context and would welcome suggestions. I think --filename can be a device? Output suggests it used hw.ncpu instead of --numjobs=8 ~~~~~~~ # fio --name=randread --ioengine=posixaio --rw=randread --direct=1 --bs=8k --refill_buffers --norandommap --randrepeat=0 --iodepth=32 --numjobs=8 --runtime=60 --group_reporting --thread --size=2048M randread: (g=0): rw=randread, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=posixaio, iodepth=32 ... fio-3.36 Starting 8 threads randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) Jobs: 4 (f=4): [_(1),r(2),_(2),r(1),_(1),r(1)][21.2%][r=288KiB/s][r=36 IOPS][eta 04m:05s] randread: (groupid=0, jobs=8): err= 0: pid=135125: Thu Dec 21 16:43:00 2023 read: IOPS=43, BW=351KiB/s (359kB/s)(22.6MiB/65986msec) slat (nsec): min=889, max=1948.6k, avg=4172.45, stdev=36488.07 clat (msec): min=1108, max=11660, avg=5644.23, stdev=1282.12 lat (msec): min=1108, max=11660, avg=5644.24, stdev=1282.12 clat percentiles (msec): | 1.00th=[ 1183], 5.00th=[ 3171], 10.00th=[ 4933], 20.00th=[ 5269], | 30.00th=[ 5470], 40.00th=[ 5604], 50.00th=[ 5738], 60.00th=[ 5873], | 70.00th=[ 5940], 80.00th=[ 6074], 90.00th=[ 6342], 95.00th=[ 6812], | 99.00th=[10671], 99.50th=[10939], 99.90th=[11610], 99.95th=[11610], | 99.99th=[11610] bw ( KiB/s): min= 208, max= 3760, per=100.00%, avg=1535.05, stdev=128.23, samples=245 iops : min= 26, max= 470, avg=191.88, stdev=16.03, samples=245 lat (msec) : 2000=3.25%, >=2000=96.75% cpu : usr=0.00%, sys=0.12%, ctx=22712, majf=0, minf=0 IO depths : 1=0.3%, 2=0.6%, 4=1.1%, 8=4.9%, 16=69.6%, 32=23.6%, >=64=0.0% submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0% complete : 0=0.0%, 4=94.7%, 8=2.9%, 16=1.5%, 32=0.9%, 64=0.0%, >=64=0.0% issued rwts: total=2895,0,0,0 short=0,0,0,0 dropped=0,0,0,0 latency : target=0, window=0, percentile=100.00%, depth=32 Run status group 0 (all jobs): READ: bw=351KiB/s (359kB/s), 351KiB/s-351KiB/s (359kB/s-359kB/s), io=22.6MiB (23.7MB), run=65986-65986msec # -- From nobody Fri Dec 22 03:30:21 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxCVQ4Xmkz54fPB for ; Fri, 22 Dec 2023 03:30:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxCVQ3D2Zz3QdF for ; Fri, 22 Dec 2023 03:30:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703215822; a=rsa-sha256; cv=none; b=OLcVCEEoLCk5Nh39Vc6XswNMt/KgpbVeV07saHTiOYt8R6829jnQIA52mVsDzANcajQcPt Y6yd76ACwnhlPHhA7IxkBYEXTZAn6eq5pBes+spdcb1JaDXuzYn1VL3pMRucCWPwFNEAmh 2GTet6+VljRCW4033TZmI6l9GOU5oU1ZyuccR1RD8piPll6SRl9Mvvftq6n9NK89Rei2po 22tjfmE4fL1/j+nQMbwIHWryDYH3M7MhuFL77Api7pLM3qKReIGKZ4oJ5jyNZwnCVA2+cI Gzp9j21lvjXm6Sy1pC0YsWwAS9Os5tZHdRfo/7MXCOyqLdibGXvMuGDeR/xdxQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703215822; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YBZQVxd8FZWatscBMS+afugzhWELP1TPBH4voYsf9lg=; b=nt0PI6n7qlxEUwvYnxc+RG2IQvKe7fI84WDxgSiRmJ9EzfoLoAW1ktDRMHlJR8710yy2eR 8aMKCsva1hWhRhjqyWiwYvOHjAlw8lC0rBpI2DpE3QsQSprFnVSQQSbh54mX/K4nghosCu 9Zsmu0jvC7tOwczPr193c/G1Rav+uEtyJsaweh7MLg4EHbENN3zLpfd/Va+zYmWLhq5lvp CNr2Loyhm8+/qR4Z5T1hm0msQC/5sX2Y+vCKqduTgEgLwwLGoPfBeHXUM4n1RqYX4FP8JW fZEPmh3f4FOwu/+WKL2znC9OvlDDQhP3QEAAdsOUOGSFnPEJ2bspTjPB/pb7ag== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SxCVQ2BQmz106S for ; Fri, 22 Dec 2023 03:30:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BM3UM7W031061 for ; Fri, 22 Dec 2023 03:30:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BM3UMrK031060 for fs@FreeBSD.org; Fri, 22 Dec 2023 03:30:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275870] [unionfs] [zfs] kernel panic on umount Date: Fri, 22 Dec 2023 03:30:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jah@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275870 Jason A. Harmening changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jah@FreeBSD.org --- Comment #1 from Jason A. Harmening --- Created attachment 247192 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D247192&action= =3Dedit Suggested patch, this fixed the panic in local testing. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Fri Dec 22 03:33:37 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxCZ90YdFz54fGT for ; Fri, 22 Dec 2023 03:33:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxCZ86Nngz3RH4 for ; Fri, 22 Dec 2023 03:33:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703216016; a=rsa-sha256; cv=none; b=aRxaQfSCRDJRD7Ge8SifS10KAJ1X2h5twbk/dF/Lf4NMECWDaBu4g1x0vruEbhiq1dlO1L /p2TJYgbT1dW8vAeTpx6dnuNF0ZNzsDA1kIwJ9uyuHGbi2FEzNKd8FM2c3eZHam5q5tHJA AkAPRcChkrJf8JlUdtvuV4vx+JFjr9ma+hAslKItNaWtzc8TBgsf135geuBQ/X/Zs4QDj6 cLaPPAosTTusuXOT+416C+jRQSW2SiFstw286vum7FNhXdG4TgDh7DlzGFqxuWlsDiQNGo LPEqNyXJ3AOcYZXvepz7hF6hkpMQmkJxoVKQBrNvt+E0H+xsCj+Rv46FG69oxw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703216016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74B9rh670hXiIG1KHoaukkJFvSOME1QzEPbt0+gGrqg=; b=iFmXrZqcNUNUbOVx7MCUj8NNdbeQm2hip8UCiSa5i9ZcNoTxUlx5N35p0h2/FHhUKddirb ZbAm6UH/hIQ9BeC1ytRX5759fpqDqY24UNvpyzaZ/gD/KsfQk3IXc2V1EtzgbPo/o90TS3 A54zH+XOmVxvxulOy+qdxGCzSw2XWWt02lBNjpPulqmc8L5QcqrmycUar8MRAvex4ijpbW k0T8WYg+597I2CnH6KmjuxgtfUAP1JZ5cbqo0CiLYViHaw2ooV+WKYIVrWsK96D5PPfiOQ FVLOLEObbyFVeOsISJeKEc/bNZRv2qLwDr5vDMrVtArqlcIhhXGsnE/SkCQGWw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SxCZ85TM3z10hS for ; Fri, 22 Dec 2023 03:33:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BM3XaYM041291 for ; Fri, 22 Dec 2023 03:33:36 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BM3XaB5041290 for fs@FreeBSD.org; Fri, 22 Dec 2023 03:33:36 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275870] [unionfs] [zfs] kernel panic on umount Date: Fri, 22 Dec 2023 03:33:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jah@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275870 --- Comment #2 from Jason A. Harmening --- It looks like this was caused by the vfs_unregister_upper() calls I added in 14. I'm a bit surprised that zfs rollback obliterates the v_mount field of= the upper/lower vnodes, but then I'm not familiar with that code. In any case, unionfs should probably cache the ref'ed upper/lower mount objects returned= by vfs_register_upper_from_vp (as nullfs already does), because there are seve= ral cases outside of unmount() in which unionfs directly accesses ump->um_[lower|upper]vp->v_mount which aren't likely to be safe in the pres= ence of a concurrent (recursive) forced unmount. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Fri Dec 22 04:19:45 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxDbl1Dgdz54hwx for ; Fri, 22 Dec 2023 04:20:03 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-21.consmr.mail.gq1.yahoo.com (sonic314-21.consmr.mail.gq1.yahoo.com [98.137.69.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxDbj5zdHz3TdS for ; Fri, 22 Dec 2023 04:20:01 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="m/gXu85Y"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703218799; bh=3i2gS91VcrMRRwnSEn+CMKOeq7k6AhPZXFByDfjCWnw=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=m/gXu85Ygi0768nOwRXrJZjU+e8k8IKsQFKXuwRfeLpMqAkMIyZb+DsC2+UtSoQTVU4V8YB4b7g/7eaYkEk5RGulUVpCD4T8ci6la15cO3eJEJVJMbbSi0JyOyp1y0r0dx3wQ24ckW1gUCk5zQqv3Q61n/00RR4QHAYSlMIa3Im4K4soeMfPGDOHdSMyG96nflEr2bvsEKuTfan9qu/oCFh6s8XWxmDNR+SX30kkT07QbTAnVh1prED7PWrcd9kMwpJNhscaLwzqVMp5ZosRRlTybzO4XuRquAJYh3Su1p+7Sp8AcXQIxWlJ7024Z15oboyki6RWq/jy4e9tbC9Qag== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703218799; bh=DmUl4qoi6ZpvXbHf+ah1s3bWQOCUE86wfZWTfZSBj6G=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=QUp/7yE38e25tv4HFSgaWrxxtKnIrAND85KQJB3M4zuviSef/64q7G2HfqDPzgvhDcSeM4GMs9ShjojNZ2SDmBr1oDoo+jC72zwNBn9ACWaAbpHn7ow1hRrCReU5E22AXebwCZl5P1/V8wzC3oktkBWSWLFbDqI+NBPOWYMbutAFogpGxGTxGUSHDBFXewTfwx/fs6NS17ZPk6RaGcVNYYJNMALfYAPWcqHKcJhDx7dgDd2ccpUiaGE42kzGOLwm0aTdhT27/3kHd+kyz5Csb/GJnPx1XrDSDRspWiysxckXM2ids5ldIU+pFMsoU3IlUI9FZQ7XST2R9yufAhHE5Q== X-YMail-OSG: GH1WkWcVM1lRD99ra.._XgRD_L7.f51QoHAK0wMfugm6j1nz62BTHta36PZ4iCI ZjTRFf.p4sEavLrZU.sWk.nFa_A6HZe628tB2SGX1svQY4Qa5mifFHoFv_KWie4AHLqz6yEJPBfj Wq1fkebFub_0BdjFYxFCpmSK7M1cmKt46NECt2MxnwAWpsseWDai1j48Cp8yJXsFvsidqPYFZx.H NI_tH5laslL3o8zruYywjtOwP9mI.4ixhtkuqrwNkKMcfbTAhx3S_xFA07n3il.4o4ui0FRs6qI4 DLPu_RiYvbwA6aay.TjMm1UskFgDWGmLBRkxPtIvC3OCFqZ6TzJGPezYct0ycNNSwdB3vEqr3asA TcC6gIyF2816Igrv1LnO8teBh9jU4vM0p0YVuwebVWy1lhDOX8E19WD6OsgrmOqWMIk9V5.AHfy0 RyiCfdMjgejlXSUqg7jtc2BCJCQwk86nzBjwdZvKaQGX7k6F75j9FuR9Z5nd7_ULlr8adziRX3n5 C8TtgEbRuDVDubzTGfk8BsZa6cZsDSgE0kNBcf3h5.cd8VLVI1YUyBA5Xdk3Mvijn27EFD0N0qcZ Fjd_1oi2leTRJZFynld.h2T0JhsQjKSMjAwThNJwyDZF3YGnzuWaUroleYIhysCTBUI.kf44FxMZ 2eTB1IvffIBWNym8yp20HkfAnuDV9OFjIKY2M43iBOB3Q4T8OkI0p24R0Pj5UodmXGj2wyvK39Xn gQfeZOzKuT94VD9nBID40dNM6k6gpLTtnW2UW9kGvUwse5m5BjDnN5QyMo9g1Ewjg42HFAL5_I4q LZpOOPQV2iGrR5aiqZku77p3QsShHxkNv796zwEt1E6Vs4fbmFlK_3eUsMQoyxvzxcoWNNy7Fvoy FDZLGW9dvxqPJKJb_0qqZWr3TIfPi8bSfH43nXC9ZRp9Ws.2Ic76LRz6vE13PlXboXXbh_WAFVve IhVOSKcXsE2PshirsuTWVItu4ucdcOHOHRXqTNH4pjV6fYR6UPwIwbaC9P6ejA04uESXNJ39DPLu Z1IJW1WKScTQP_eCwuQ_r2r0MqMqKoXdH4a40dEAX3QJ9lmI2gSY9zl7Cb9R8RmHiZl0LS2ytCJa iHP72TyXzMPHyBwCRFufAmUpgi8lisuTT5vP74KzlHeB2UVCsjXb_te7m65fABlQZetTEa7uqpM0 X3hIJK8cWZpl6JQ3dOjXnM4dHgUFTsDYEgN_h3zcj4xs4vLpzCvi56rWAqGjQQ.9NbhFMzmxeowI Sgd.JV6KksZfAtwT5f7DTHBiYu.NSWjQXy6o9TmXF6yrIzpSlR9XEFK0dHNI0IaEyTpnENL0f9VU RQreckUuDET.wp8v4DT87kkg2oeFnQt.84hQzx5UjRErTaPpeFmxznOyArkS6Id6DvkeU9weyvtX f5qbn0holorJ1YVckVwd.KNy6Vzqp55hNEQMFUs4__jTspfxYMx3zNi3n_NZ2iSudfcqedFxSqAu K7Laqexm02udcQjH.X9YubP_HOfqMv8fV6wOsISglBjtZrr0DukZPmFzP2VD0ao6dxbzUu3Kx9jL rfkmSTEUrHPanaSjYYxEGu.IRTsAWTlhoe10ikg_FfAQV.EncgLZ2nwMbMmM8pUPiFOujoZ3fAKX 35QmDj948_VuDg20iI6sYbrxlWs.OS46MmgEWG043WVoJssaV58Wc2C52jRsHySmUch_oAjz9RRS RtQIwv0F799wwQu0zCNg5e_8csyK2smK23PgXJoW2_H46HGxuQD6c_AuiFMqtnl0ztH094SwN7lX D3Ea2cV5Amvn99QikfNpA4s2nGFQ8qX3ftyR1g9_RyRDE6EJPlisO_6dNY72Pp23eVCFy5oQn8pe HUVVB7nzukXfSqSi0.PKWjn2C6KH5cAz97ZuRd0qeDbQvofk0JzbbbaycQiihBWvwgVI1Kfi3QAl 3UsD1tIFebgKImUE.MIYiCyooxRvJR8.FQs1ItkG6R9OxkRR56OLnOrBzpoeRwx6C2BWxVa4bkBN wbew1HMYnpLrDxaZjbDhxJHR4Q1uFt8l3vewbds8OTZhM8cuXJwN93SAPuVvR87s8FqgtWc9c7iL hizc4esbJeCewfPm9n_UpVs0XZhDxfltO3MzaHMAnK9cje3NVI3.XxiQaqDU95wEmNhwiNUfnNjh Xy9LP6i3WOEOeuhjsk2Myod7o9bp2IjQjWjHcZzzYDBIqiL3MzzEj.ze.GOqnWGvHIhCMejZCZS2 qgA1wRCZkQ4wA1IYvkMp6OW1V2GTQYQYQGHp3QF0g.suWWUHXeVZ75iw4ediI7r0MH1TivffXg5B AJmSADUbJ X-Sonic-MF: X-Sonic-ID: 73e56ac1-fbac-4d5b-aee6-a3f23b6756c4 Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Fri, 22 Dec 2023 04:19:59 +0000 Received: by hermes--production-gq1-6949d6d8f9-qkzts (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 8f12744bf7d5972e2fabfb8040e97407; Fri, 22 Dec 2023 04:19:56 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Thu, 21 Dec 2023 20:19:45 -0800 Cc: Warner Losh To: void@f-m.fm, freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.84:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.84:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SxDbj5zdHz3TdS X-Spamd-Bar: --- void wrote on Date: Fri, 22 Dec 2023 00:19:39 UTC : > Hi Mark & Warner, >=20 > I tried using fio with Warner's suggestions as a template, and > pasted the results in the latter part of this message. >=20 > On Thu, 21 Dec 2023, at 21:03, Mark Millard wrote: >=20 > . . . >=20 > fio output follows. No idea yet if the settings used are suitable for = this context > and would welcome suggestions. I later supply the output from running the same command, be it suitable or not. Again: the context is my old USB3 SSD stick, not spinning rust. Also my particular usual arm clocking configuration for the RPi4B. (I forgot to mention earlier that it has a case with a fan and heatsinks.) Also, this was not a swap space test command. So it is relevant that the file system involved was UFS, not ZFS. # swapinfo Device 1K-blocks Used Avail Capacity /dev/label/growfs_swap 8388604 0 8388604 0% I'll note that monitoring with top showed that the --numjobs=3D8 lead to 8 threads (twice the number of hardware threads). I'll note that the run left behind: # ls -C1 randread.*.0randread.0.0 randread.1.0 randread.2.0 randread.3.0 randread.4.0 randread.5.0 randread.6.0 randread.7.0 that I later deleted. > I think --filename can be a device? Output suggests > it used hw.ncpu instead of --numjobs=3D8 >=20 > ~~~~~~~ > # fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread = --direct=3D1 --bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 = --iodepth=3D32 --numjobs=3D8 --runtime=3D60 --group_reporting --thread = --size=3D2048M > randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) = 8192B-8192B, (T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32 > ... > fio-3.36 > Starting 8 threads > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > Jobs: 4 (f=3D4): = [_(1),r(2),_(2),r(1),_(1),r(1)][21.2%][r=3D288KiB/s][r=3D36 IOPS][eta = 04m:05s] > randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D135125: Thu Dec 21 = 16:43:00 2023 > read: IOPS=3D43, BW=3D351KiB/s (359kB/s)(22.6MiB/65986msec) > slat (nsec): min=3D889, max=3D1948.6k, avg=3D4172.45, stdev=3D36488.07 > clat (msec): min=3D1108, max=3D11660, avg=3D5644.23, stdev=3D1282.12 > lat (msec): min=3D1108, max=3D11660, avg=3D5644.24, stdev=3D1282.12 > clat percentiles (msec): > | 1.00th=3D[ 1183], 5.00th=3D[ 3171], 10.00th=3D[ 4933], 20.00th=3D[ = 5269], > | 30.00th=3D[ 5470], 40.00th=3D[ 5604], 50.00th=3D[ 5738], 60.00th=3D[ = 5873], > | 70.00th=3D[ 5940], 80.00th=3D[ 6074], 90.00th=3D[ 6342], 95.00th=3D[ = 6812], > | 99.00th=3D[10671], 99.50th=3D[10939], 99.90th=3D[11610], = 99.95th=3D[11610], > | 99.99th=3D[11610] > bw ( KiB/s): min=3D 208, max=3D 3760, per=3D100.00%, avg=3D1535.05, = stdev=3D128.23, samples=3D245 > iops : min=3D 26, max=3D 470, avg=3D191.88, stdev=3D16.03, samples=3D245= > lat (msec) : 2000=3D3.25%, >=3D2000=3D96.75% > cpu : usr=3D0.00%, sys=3D0.12%, ctx=3D22712, majf=3D0, minf=3D0 > IO depths : 1=3D0.3%, 2=3D0.6%, 4=3D1.1%, 8=3D4.9%, 16=3D69.6%, = 32=3D23.6%, >=3D64=3D0.0% > submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% > complete : 0=3D0.0%, 4=3D94.7%, 8=3D2.9%, 16=3D1.5%, 32=3D0.9%, = 64=3D0.0%, >=3D64=3D0.0% > issued rwts: total=3D2895,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0 > latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32 >=20 > Run status group 0 (all jobs): > READ: bw=3D351KiB/s (359kB/s), 351KiB/s-351KiB/s (359kB/s-359kB/s), = io=3D22.6MiB (23.7MB), run=3D65986-65986msec # fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D1= --bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 = --numjobs=3D8 --runtime=3D60 --group_reporting --thread --size=3D2048M randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, = (T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32 ... fio-3.36 Starting 8 threads randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) Jobs: 8 (f=3D8): [r(8)][100.0%][r=3D17.4MiB/s][r=3D2222 IOPS][eta = 00m:00s] randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D100241: Sun Dec 17 = 12:42:53 2023 read: IOPS=3D2486, BW=3D19.4MiB/s (20.4MB/s)(1168MiB/60125msec) slat (nsec): min=3D888, max=3D1494.1k, avg=3D3446.83, stdev=3D8015.07 clat (msec): min=3D58, max=3D334, avg=3D102.70, stdev=3D22.68 lat (msec): min=3D58, max=3D334, avg=3D102.70, stdev=3D22.68 clat percentiles (msec): | 1.00th=3D[ 70], 5.00th=3D[ 75], 10.00th=3D[ 79], = 20.00th=3D[ 84], | 30.00th=3D[ 89], 40.00th=3D[ 94], 50.00th=3D[ 99], = 60.00th=3D[ 105], | 70.00th=3D[ 111], 80.00th=3D[ 118], 90.00th=3D[ 136], = 95.00th=3D[ 150], | 99.00th=3D[ 171], 99.50th=3D[ 180], 99.90th=3D[ 211], = 99.95th=3D[ 228], | 99.99th=3D[ 268] bw ( KiB/s): min=3D 8246, max=3D24640, per=3D100.00%, avg=3D19903.38, = stdev=3D281.80, samples=3D960 iops : min=3D 1028, max=3D 3080, avg=3D2487.90, stdev=3D35.24, = samples=3D960 lat (msec) : 100=3D52.92%, 250=3D47.07%, 500=3D0.01% cpu : usr=3D0.17%, sys=3D5.01%, ctx=3D1016480, majf=3D0, = minf=3D0 IO depths : 1=3D0.1%, 2=3D0.1%, 4=3D0.1%, 8=3D0.7%, 16=3D75.3%, = 32=3D24.0%, >=3D64=3D0.0% submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% complete : 0=3D0.0%, 4=3D93.2%, 8=3D4.7%, 16=3D2.0%, 32=3D0.1%, = 64=3D0.0%, >=3D64=3D0.0% issued rwts: total=3D149497,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0 latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32 Run status group 0 (all jobs): READ: bw=3D19.4MiB/s (20.4MB/s), 19.4MiB/s-19.4MiB/s = (20.4MB/s-20.4MB/s), io=3D1168MiB (1225MB), run=3D60125-60125msec After swapoff instead: # fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D1= --bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 = --numjobs=3D8 --runtime=3D60 --group_reporting --thread --size=3D2048M randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, = (T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32 ... fio-3.36 Starting 8 threads randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) randread: Laying out IO file (1 file / 2048MiB) Jobs: 8 (f=3D8): [r(8)][100.0%][r=3D17.6MiB/s][r=3D2259 IOPS][eta = 00m:00s] randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D100211: Sun Dec 17 = 12:07:16 2023 read: IOPS=3D2471, BW=3D19.3MiB/s (20.2MB/s)(1161MiB/60119msec) slat (nsec): min=3D889, max=3D1912.6k, avg=3D3415.21, stdev=3D8521.17 clat (msec): min=3D54, max=3D762, avg=3D103.32, stdev=3D28.95 lat (msec): min=3D54, max=3D762, avg=3D103.32, stdev=3D28.95 clat percentiles (msec): | 1.00th=3D[ 69], 5.00th=3D[ 75], 10.00th=3D[ 79], = 20.00th=3D[ 84], | 30.00th=3D[ 89], 40.00th=3D[ 93], 50.00th=3D[ 99], = 60.00th=3D[ 105], | 70.00th=3D[ 110], 80.00th=3D[ 117], 90.00th=3D[ 136], = 95.00th=3D[ 150], | 99.00th=3D[ 174], 99.50th=3D[ 188], 99.90th=3D[ 518], = 99.95th=3D[ 558], | 99.99th=3D[ 617] bw ( KiB/s): min=3D12336, max=3D24912, per=3D100.00%, avg=3D19913.12, = stdev=3D267.11, samples=3D954 iops : min=3D 1542, max=3D 3114, avg=3D2489.14, stdev=3D33.39, = samples=3D954 lat (msec) : 100=3D52.77%, 250=3D47.01%, 500=3D0.11%, 750=3D0.11%, = 1000=3D0.01% cpu : usr=3D0.18%, sys=3D4.93%, ctx=3D1005861, majf=3D0, = minf=3D0 IO depths : 1=3D0.1%, 2=3D0.1%, 4=3D0.3%, 8=3D1.3%, 16=3D74.8%, = 32=3D23.6%, >=3D64=3D0.0% submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% complete : 0=3D0.0%, 4=3D93.4%, 8=3D4.5%, 16=3D1.9%, 32=3D0.2%, = 64=3D0.0%, >=3D64=3D0.0% issued rwts: total=3D148600,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0 latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32 Run status group 0 (all jobs): READ: bw=3D19.3MiB/s (20.2MB/s), 19.3MiB/s-19.3MiB/s = (20.2MB/s-20.2MB/s), io=3D1161MiB (1217MB), run=3D60119-60119msec The difference: Jobs: 4 (f=3D4) (yours) vs. Jobs: 8 (f=3D8) (mine) is not obvious, given the same commands. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Dec 22 09:37:42 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxMfR10Zgz551hD for ; Fri, 22 Dec 2023 09:37:51 +0000 (UTC) (envelope-from SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxMfQ5ttMz4Mky for ; Fri, 22 Dec 2023 09:37:50 +0000 (UTC) (envelope-from SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Received: from rwvirtual98.colo.realworks.nl (rwvirtual98.colo.realworks.nl [10.0.10.102]) by mailrelayint1.colo2.realworks.nl (Postfix) with ESMTP id 4SxMfH11GBz1Vq; Fri, 22 Dec 2023 10:37:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1703237863; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6a8XbPENsCiYh5VJZR4BGLkVktiNKR1gZHyNMOzETUI=; b=i9pH9N9N7MEVa7o41qfM7aBB8gkBYZk1cE4Q0NLxInibcCAoELcq+u9E7fpplhv0lZBWhg nUeIxrg6xGRa26a69VYwEIHGalw5FnOf9j8YkjUsPGBFNlo87vAcnhiqaJTKdSs46fWTEg +4gAEAhTAvwb0u+kreGBhnFbYND+zsnH7JcNXl7DeM9uiFdzeCMvMmoOeqVzlREBOQs9WE neZnIC5KhJTkU1rsMSuOqfEaoSra6SYA+jAsGT2bt0XbEZ/KH3faVxDuNkEDVppJIvHPQr vLsU6UrUI2Apb40v7sHLaSO0Jna+sneUFfkRyWsnyIbbGbQ605PdfTGpS0f2yA== Received: from rwvirtual98.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual98.colo.realworks.nl (Postfix) with ESMTP id DF7EAA02BB; Fri, 22 Dec 2023 10:37:42 +0100 (CET) Date: Fri, 22 Dec 2023 10:37:42 +0100 (CET) From: Ronald Klop To: void Cc: freebsd-fs Message-ID: <1137397689.3080.1703237862787@localhost> In-Reply-To: <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> References: <1439231787.9333.1703184981914@localhost> <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> Subject: Re: measuring swap partition speed List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_3079_229342031.1703237861046" X-Mailer: Realworks (683.54) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual98 [10.0.10.102] with HTTP; Fri, 22 Dec 2023 10:37:42 +0100 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:121.0) Gecko/20100101 Firefox/121.0 X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL] X-Spamd-Bar: ---- X-Rspamd-Queue-Id: 4SxMfQ5ttMz4Mky ------=_Part_3079_229342031.1703237861046 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: void Datum: vrijdag, 22 december 2023 01:05 Aan: freebsd-fs Onderwerp: Re: measuring swap partition speed > > On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote: > > A bit weird that a simple dd is so slow. > > > > Just a quick thought. > > Are your partitions aligned properly? > > no idea. How would I check? The system when installed, > the auto-zfs option was selected, block size set to 4k. The swap > was changed from the default 2gb to 12gb. geli encryption is > active for the zfs filesystem but not for swap. It's only the swap > that shows performance issues. > > > Is other IO going on on the same time? > > not really, the tests were done with cron jobs deactivated, almost completely idle, > swap was not being used at all. > > > What does gstat say about %util, queue length and all the other stats > > while running the dd? Or "iostat -x -d 1". > > I'll try that next, thanks > > > Could you try if another disk has the same issues? > > no spare or equivalent disk for this arch. > > > How is your disk connected? USB-to-SATA-adapter? Any output of dmesg? > > Regular usb3 connection. > > da0: 400.000MB/s transfers > da0: 953869MB (1953525168 512 byte sectors) > da0: quirks=0x2 > da0: Delete methods: > -- > > > > Can you provide more concrete information? Like: # usbconfig list ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA) ugen0.3: at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) ugen0.4: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) ugen0.5: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3. And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you. About the alignment. You wrote in your first mail: "# gpart show => 40 1953525088 da0 GPT (932G) 40 532480 1 efi (260M) 532520 2008 - free - (1.0M) 534528 4194304 2 freebsd-swap (2.0G) 4728832 4194304 4 freebsd-swap (2.0G) 8923136 4194304 5 freebsd-swap (2.0G) 13117440 4194304 6 freebsd-swap (2.0G) 17311744 4194304 7 freebsd-swap (2.0G) 21506048 4194304 8 freebsd-swap (2.0G) 25700352 1927823360 3 freebsd-zfs (920G) 1953523712 1416 - free - (708K) " Your dd tests were on da0p4 which starts on block 4728832. Assuming "gpart list" shows "Sectorsize: 512". (4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good. AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out. Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin. Regards, Ronald. ------=_Part_3079_229342031.1703237861046 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: void <void@f-m.fm>
Datum: vrijdag, 22 december 2023 01:05
Aan: freebsd-fs <freebsd-fs@freebsd.org>
Onderwerp: Re: measuring swap partition speed

On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote:
> A bit weird that a simple dd is so slow.
>
> Just a quick thought.
> Are your partitions aligned properly?

no idea. How would I check? The system when installed,
the auto-zfs option was selected, block size set to 4k. The swap
was changed from the default 2gb to 12gb. geli encryption is
active for the zfs filesystem but not for swap. It's only the swap
that shows performance issues.

> Is other IO going on on the same time?

not really, the tests were done with cron jobs deactivated, almost completely idle,
swap was not being used at all.

> What does gstat say about %util, queue length and all the other stats
> while running the dd? Or "iostat -x -d 1".

I'll try that next, thanks

> Could you try if another disk has the same issues?

no spare or equivalent disk for this arch.

> How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?

Regular usb3 connection.

da0: 400.000MB/s transfers
da0: 953869MB (1953525168 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
da0: Delete methods: <NONE(*),ZERO>
-- 
 


Can you provide more concrete information? Like:

# usbconfig list
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)

And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3.
And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you.

About the alignment. You wrote in your first mail:
"# gpart show

=>        40  1953525088  da0  GPT  (932G)
           40      532480    1  efi  (260M)
       532520        2008       - free -  (1.0M)
       534528     4194304    2  freebsd-swap  (2.0G)
      4728832     4194304    4  freebsd-swap  (2.0G)
      8923136     4194304    5  freebsd-swap  (2.0G)
     13117440     4194304    6  freebsd-swap  (2.0G)
     17311744     4194304    7  freebsd-swap  (2.0G)
     21506048     4194304    8  freebsd-swap  (2.0G)
     25700352  1927823360    3  freebsd-zfs  (920G)
   1953523712        1416       - free -  (708K)
"

Your dd tests were on da0p4 which starts on block 4728832.
Assuming "gpart list" shows "Sectorsize: 512".
(4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good.
AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out.

Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin.

Regards,
Ronald.
  ------=_Part_3079_229342031.1703237861046-- From nobody Fri Dec 22 09:59:26 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxN7Q36mZz552SV for ; Fri, 22 Dec 2023 09:59:30 +0000 (UTC) (envelope-from SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxN7P1dsxz4QPN for ; Fri, 22 Dec 2023 09:59:29 +0000 (UTC) (envelope-from SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=SD1t+Q6G; spf=pass (mx1.freebsd.org: domain of "SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl" designates 194.109.157.24 as permitted sender) smtp.mailfrom="SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Date: Fri, 22 Dec 2023 10:59:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1703239166; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1+61QWft+NhOOAqGeRidYRz8+Zo04FPInMd7U/tlm8Q=; b=SD1t+Q6G4m2KSRP5zFtFtW12Tuq4a4o7JhQNiAaSC813Bi1yImZYemsIHcGSv1OjN8p384 tEgChl6eUUu1yI1DeSJhvWUIlXfu3yydrhNfpZ/lCxUgm+PmnUcCbm+zZNVIRJP47PVFAz VMv7EMUjYra7Kc3iD47h/sMIcYmlNsjvU26uI47cApMzRjC+XGG2a6/kzM5E0AmgLjd58m 2st+0sRHmPj3em+9d8cUjy++eukgzpg647OAd6QsBlO1TrPsc0anfKyh8Kw1uCk0lRlr93 RDJXf2HXFGYTSvw/FXXIF2dogaL6+iQytYeYMonwPyiXdfJ52iT4QNbMoFTZrA== From: Ronald Klop To: freebsd-fs , void Message-ID: <478987228.2760.1703239166835@localhost> In-Reply-To: <1137397689.3080.1703237862787@localhost> References: <1439231787.9333.1703184981914@localhost> <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> <1137397689.3080.1703237862787@localhost> Subject: Re: measuring swap partition speed List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_2759_327864946.1703239166820" X-Mailer: Realworks (683.54) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Result: default: False [-3.20 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; FREEMAIL_TO(0.00)[freebsd.org,f-m.fm]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ZERO(0.00)[0]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; HAS_X_PRIO_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; TO_DN_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=ck8o=IB=klop.ws=ronald-lists@realworks.nl] X-Rspamd-Queue-Id: 4SxN7P1dsxz4QPN X-Spamd-Bar: --- ------=_Part_2759_327864946.1703239166820 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Ronald Klop Datum: vrijdag, 22 december 2023 10:37 Aan: void CC: freebsd-fs Onderwerp: Re: measuring swap partition speed > > > Van: void > Datum: vrijdag, 22 december 2023 01:05 > Aan: freebsd-fs > Onderwerp: Re: measuring swap partition speed >> >> On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote: >> > A bit weird that a simple dd is so slow. >> > >> > Just a quick thought. >> > Are your partitions aligned properly? >> >> no idea. How would I check? The system when installed, >> the auto-zfs option was selected, block size set to 4k. The swap >> was changed from the default 2gb to 12gb. geli encryption is >> active for the zfs filesystem but not for swap. It's only the swap >> that shows performance issues. >> >> > Is other IO going on on the same time? >> >> not really, the tests were done with cron jobs deactivated, almost completely idle, >> swap was not being used at all. >> >> > What does gstat say about %util, queue length and all the other stats >> > while running the dd? Or "iostat -x -d 1". >> >> I'll try that next, thanks >> >> > Could you try if another disk has the same issues? >> >> no spare or equivalent disk for this arch. >> >> > How is your disk connected? USB-to-SATA-adapter? Any output of dmesg? >> >> Regular usb3 connection. >> >> da0: 400.000MB/s transfers >> da0: 953869MB (1953525168 512 byte sectors) >> da0: quirks=0x2 >> da0: Delete methods: >> -- >> >> >> >> > > > Can you provide more concrete information? Like: > > # usbconfig list > ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) > ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA) > ugen0.3: at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) > ugen0.4: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) > ugen0.5: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) > > And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3. > And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you. > > About the alignment. You wrote in your first mail: > "# gpart show > > => 40 1953525088 da0 GPT (932G) > 40 532480 1 efi (260M) > 532520 2008 - free - (1.0M) > 534528 4194304 2 freebsd-swap (2.0G) > 4728832 4194304 4 freebsd-swap (2.0G) > 8923136 4194304 5 freebsd-swap (2.0G) > 13117440 4194304 6 freebsd-swap (2.0G) > 17311744 4194304 7 freebsd-swap (2.0G) > 21506048 4194304 8 freebsd-swap (2.0G) > 25700352 1927823360 3 freebsd-zfs (920G) > 1953523712 1416 - free - (708K) > " > > Your dd tests were on da0p4 which starts on block 4728832. > Assuming "gpart list" shows "Sectorsize: 512". > (4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good. > AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out. > > Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin. > > Regards, > Ronald. > Just did the same test on my RPI4 running FreeBSD 15.0-CURRENT of Dec 14. Used an unused partition on spinning disk. My used swap is on an ssd nowadays. [root@rpi4 ~]# dd if=/dev/urandom of=/dev/da1p2 bs=8k count=25000 conv=sync status=progress 193200128 bytes (193 MB, 184 MiB) transferred 10.024s, 19 MB/s 25000+0 records in 25000+0 records out 204800000 bytes transferred in 10.592984 secs (19333551 bytes/sec) NB: using bs=128k makes it go to 69MB/s. # gpart show => 40 1953458096 da1 GPT (931G) 40 102400 1 efi (50M) 102440 8388608 2 freebsd-swap (4.0G) 8491048 1944967088 3 freebsd-zfs (927G) ugen0.5: at usbus0 umass1 on uhub0 umass1: on usbus0 da1 at umass-sim1 bus 1 scbus1 target 0 lun 0 da1: Fixed Direct Access SPC-4 SCSI device da1: Serial Number 575843314538354834414536 da1: 400.000MB/s transfers da1: 953837MB (1953458176 512 byte sectors) da1: quirks=0x2 # usbconfig list ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA) ugen0.3: at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA) ugen0.4: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) ugen0.5: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA) Found some more things to check: Do these settings have something non-default? # sysctl kern.geom.disk # sysctl kern.cam And your disk prints this in dmesg. How does that influence things? I haven't seen this before anywhere. da0: Delete methods: Because of this I'm particularly interested in: sysctl kern.cam.da.0.delete_method and sysctl kern.cam.da.0.trim_count I might be totally of in my thinking here, but you never know with such unpredictable problems. Regards, Ronald. ------=_Part_2759_327864946.1703239166820 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: vrijdag, 22 december 2023 10:37
Aan: void <void@f-m.fm>
CC: freebsd-fs <freebsd-fs@freebsd.org>
Onderwerp: Re: measuring swap partition speed

 

Van: void <void@f-m.fm>
Datum: vrijdag, 22 december 2023 01:05
Aan: freebsd-fs <freebsd-fs@freebsd.org>
Onderwerp: Re: measuring swap partition speed

On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote:
> A bit weird that a simple dd is so slow.
>
> Just a quick thought.
> Are your partitions aligned properly?

no idea. How would I check? The system when installed,
the auto-zfs option was selected, block size set to 4k. The swap
was changed from the default 2gb to 12gb. geli encryption is
active for the zfs filesystem but not for swap. It's only the swap
that shows performance issues.

> Is other IO going on on the same time?

not really, the tests were done with cron jobs deactivated, almost completely idle,
swap was not being used at all.

> What does gstat say about %util, queue length and all the other stats
> while running the dd? Or "iostat -x -d 1".

I'll try that next, thanks

> Could you try if another disk has the same issues?

no spare or equivalent disk for this arch.

> How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?

Regular usb3 connection.

da0: 400.000MB/s transfers
da0: 953869MB (1953525168 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
da0: Delete methods: <NONE(*),ZERO>
-- 
 


Can you provide more concrete information? Like:

# usbconfig list
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)

And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3.
And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you.

About the alignment. You wrote in your first mail:
"# gpart show

=>        40  1953525088  da0  GPT  (932G)
           40      532480    1  efi  (260M)
       532520        2008       - free -  (1.0M)
       534528     4194304    2  freebsd-swap  (2.0G)
      4728832     4194304    4  freebsd-swap  (2.0G)
      8923136     4194304    5  freebsd-swap  (2.0G)
     13117440     4194304    6  freebsd-swap  (2.0G)
     17311744     4194304    7  freebsd-swap  (2.0G)
     21506048     4194304    8  freebsd-swap  (2.0G)
     25700352  1927823360    3  freebsd-zfs  (920G)
   1953523712        1416       - free -  (708K)
"

Your dd tests were on da0p4 which starts on block 4728832.
Assuming "gpart list" shows "Sectorsize: 512".
(4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good.
AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out.

Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin.

Regards,
Ronald.
 

Just did the same test on my RPI4 running FreeBSD 15.0-CURRENT of Dec 14.
Used an unused partition on spinning disk. My used swap is on an ssd nowadays.

[root@rpi4 ~]# dd if=/dev/urandom of=/dev/da1p2 bs=8k count=25000 conv=sync status=progress
  193200128 bytes (193 MB, 184 MiB) transferred 10.024s, 19 MB/s
25000+0 records in
25000+0 records out
204800000 bytes transferred in 10.592984 secs (19333551 bytes/sec)

NB: using bs=128k makes it go to 69MB/s.

# gpart show
=>        40  1953458096  da1  GPT  (931G)
          40      102400    1  efi  (50M)
      102440     8388608    2  freebsd-swap  (4.0G)
     8491048  1944967088    3  freebsd-zfs  (927G)

ugen0.5: <Western Digital Elements 25A2> at usbus0
umass1 on uhub0
umass1: <Western Digital Elements 25A2, class 0/0, rev 3.00/10.04, addr 4> on usbus0
da1 at umass-sim1 bus 1 scbus1 target 0 lun 0
da1: <WD Elements 25A2 1004> Fixed Direct Access SPC-4 SCSI device
da1: Serial Number 575843314538354834414536
da1: 400.000MB/s transfers
da1: 953837MB (1953458176 512 byte sectors)
da1: quirks=0x2<NO_6_BYTE>

# usbconfig list
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)


Found some more things to check:

Do these settings have something non-default?
# sysctl kern.geom.disk
# sysctl kern.cam

And your disk prints this in dmesg. How does that influence things? I haven't seen this before anywhere.
da0: Delete methods: <NONE(*),ZERO>

Because of this I'm particularly interested in:
sysctl kern.cam.da.0.delete_method
and
sysctl kern.cam.da.0.trim_count

I might be totally of in my thinking here, but you never know with such unpredictable problems.

Regards,
Ronald.
  ------=_Part_2759_327864946.1703239166820-- From nobody Fri Dec 22 16:59:10 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxYSc0XYDz54C86 for ; Fri, 22 Dec 2023 17:00:00 +0000 (UTC) (envelope-from void@f-m.fm) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxYSZ39QGz3XP0 for ; Fri, 22 Dec 2023 16:59:58 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=u6q3fvWv; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=hAFKMFja; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 64.147.123.19 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id 56A253200B21 for ; Fri, 22 Dec 2023 11:59:56 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Fri, 22 Dec 2023 11:59:56 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703264395; x=1703350795; bh=urScdkpprG n6AJS4c1aFynnWoHAE15SlUeND96Wq7hY=; b=u6q3fvWvNlB9uv2bFc06+1XsxE vUBGeOoh+xvBdJOrJ991tig14rsSYKbCxXsRQuol8ytL+w/eiOrgpxg23NatnY+h htQCQmQJzI5I7zpTx6Fma94TGz6OwnhD9l243SpeNeOPxqLTHQxJugkzEHzsSUj5 FCcndJvccCEYg6PgPW1lguZ1CvGu7kFH25KpmujiCkryw3YKyZ2hPv+jA27x++u7 YKtagOFsbW441cSZPVgkp8hQNZw8He6n8EUMC+Ba4hok2TmPEbahzQ2UmMS+a2BN PeaiT3+KzjkmhjFmtIHwpjOIp8GZvlhJz9G9LWJQQ1ro9pTEmDqTd7XmiWFw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703264395; x=1703350795; bh=urScdkpprGn6AJS4c1aFynnWoHAE 15SlUeND96Wq7hY=; b=hAFKMFjaiZrCAbGPZBh5qQun5jnQ7pNtv1dU108b16Zd pfrMrf/qGoCfz2lTmncdTJmqO0q7K5zOR15TSeT3Geh//zaxuBMkKeJb+MpmDJMz KnR/1A0F5WJhPD1w8XRNFtaJPgQdVEoKTubFGgCz/wnidEC4P4grOu1EPVYL4zUh LdNgBmSWmPGWdkFbwnwtS6yaOvbDCCVKcAL6oob+J81ugusmKZLm7OAutTJ5+Ete CY5U8hdtwFJnxTTXMKaMDnAiKxbHDqEsV4Doi9Bvyu2QAqfIc+9541nIPjTioYzp Wnj66Lko9QbIVZIexWP1Yas0WYohBwamaDm33QlG/A== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvddujedgledvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 8092C2A2008B; Fri, 22 Dec 2023 11:59:55 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: In-Reply-To: References: Date: Fri, 22 Dec 2023 16:59:10 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.46 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.970]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[64.147.123.19:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4SxYSZ39QGz3XP0 X-Spamd-Bar: ---- My assessment of the "system being inactive while testing" may have been inaccurate. By "being inactive" I was looking at avg load being 1% or less, and for swap use being 0. Maybe the initial "inactive" test should have been in single user mode [1] because the tests in this mode show no (I mean much less) of a speed issue. Apologies for not considering single user mode till now. The results in single user mode seem to me to infer that there is no problem with the hardware. I have been able to reliably create the problem by rebooting the computer, then running something that does not need to be one huge chunk of data, that doesn't load the system that much, so ran 'make installworld' and in another terminal ran the write-to-swap-partition test [2]. It shows in this context that writing to the filesystem effectively blocks writing to swap. 507 kB/s compared to 16 MB/s. I don't know if this is unique to arm64 or if it's also the case on other arches, but it seems suboptimal to me. [1] single user mode test across various blocksizes: Enter root password, or ^D to go multi-user Password: Enter full pathname of shell or RETURN for /bin/sh: root@:/ # dd if=/dev/urandom of=/dev/da0p7 bs=4k conv=sync status=progress dd: /dev/da0p7: end of device2040 MiB) transferred 223.000s, 9591 kB/s 524289+0 records in 524288+0 records out 2147483648 bytes transferred in 223.905183 secs (9591040 bytes/sec) root@:/ # dd if=/dev/urandom of=/dev/da0p7 bs=8k conv=sync status=progress dd: /dev/da0p7: end of device2045 MiB) transferred 131.001s, 16 MB/s 262145+0 records in 262144+0 records out 2147483648 bytes transferred in 131.214263 secs (16366236 bytes/sec) root@:/ # dd if=/dev/urandom of=/dev/da0p7 bs=16k conv=sync status=progress dd: /dev/da0p7: end of device2038 MiB) transferred 82.003s, 26 MB/s 131073+0 records in 131072+0 records out 2147483648 bytes transferred in 82.420743 secs (26055136 bytes/sec) root@:/ # dd if=/dev/urandom of=/dev/da0p7 bs=32k conv=sync status=progress dd: /dev/da0p7: end of device2028 MiB) transferred 61.030s, 35 MB/s 65537+0 records in 65536+0 records out 2147483648 bytes transferred in 61.621686 secs (34849479 bytes/sec) root@:/ # dd if=/dev/urandom of=/dev/da0p7 bs=128k conv=sync status=progress dd: /dev/da0p7: end of device2032 MiB) transferred 49.002s, 43 MB/s 16385+0 records in 16384+0 records out 2147483648 bytes transferred in 49.394171 secs (43476459 bytes/sec) root@:/ # [2] test swap partition writing while multiuser and 'make installworld' # dd if=/dev/urandom of=/dev/da0p7 bs=8k conv=sync status=progress ^C57442304 bytes (57 MB, 55 MiB) transferred 113.382s, 507 kB/s 7013+0 records in 7012+0 records out 57442304 bytes transferred in 114.630376 secs (501109 bytes/sec) test while multiuser and 4k write to zfs: # dd if=/dev/urandom of=test4k.bin bs=4k count=2500000 conv=sync status=progress ^C9595363328 bytes (9595 MB, 9151 MiB) transferred 115.001s, 83 MB/s 2360113+0 records in 2360113+0 records out 9667022848 bytes transferred in 115.957392 secs (83367025 bytes/sec) # dd if=/dev/urandom of=/dev/da0p7 bs=8k conv=sync status=progress ^C7127040 bytes (7127 kB, 6960 KiB) transferred 112.235s, 64 kB/s 872+0 records in 871+0 records out 7135232 bytes transferred in 112.309547 secs (63532 bytes/sec) From nobody Fri Dec 22 18:17:34 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxbBW0NWlz54Hdx for ; Fri, 22 Dec 2023 18:17:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxbBT5Ffrz3dvc for ; Fri, 22 Dec 2023 18:17:53 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=YnAD1gSa; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.206 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703269071; bh=fIVCScX9ZKR/E9bra2d2rJBd+uINhCKb4dk/GabKBj8=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=YnAD1gSaBb/EkrZVadQLwRWID8GzMDEMo8suS1WGULWgWHCU9/i1C5TVHQPZxVuwg9Vk9m8TbgDnlL8iMh7cuN3B5uTYR8Fl8Nu/krBUW4ruR2WUPQl44XSs0vs0fVFM/0VNB5SjoEdmVbWZUUnI1HLcGViAv9/7ICLk9TYnkvrjrbWAsONrv+3ThGkX0GooBd0JBmkX6H+D0M8abgnk8204MDA4lGOqDhUkaLvgTdaMPyG2FhaAWIFr6y9uoLau9aiOUTH63a/XfWNoBgKrl1PjmNgs/5LjkOtyexKx+p2N6QQqA8SHdvFWRwNi5e/LJtgPI8w2/LecDk+NO4w46w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703269071; bh=bUCOlLC9b79Oyb8eIthuZ5USS7hvF783AoPyKK23MCh=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=YciF4fghNcUcsigWsmEsw+XcySyRbOO2hJPUVlMy0ulUBewi0CnEvLDUA1u4uVEDLuwZOSetJViMHOCphqCBjkz8FuMq4mCYivm9m4d79lXHDC3WIo4stHUCo3h0RWgR4n8T4UIBmxmzuWgKNpgCmyx0U7rXE40U5wR3JcwKbgosaNep95272nEHSwUeA3tg7/rqYDnXIrMnlHWUA7ea+iThHgWaQRyAuCRpjUOTGFFqIS6wEt5HCFkqcvn6mM98j7nH9FDoaVi7mdNEuFU976czQ4VrC1vbglAJ0/gqGmjrj38xhJsRS+gXI5f9xBhnnUwzUVwTPFfcZ++K7CuHiQ== X-YMail-OSG: kYqa.UUVM1nmY6opETZfLi8X6ocm9zNR1w8jd9kffkXaJPLiYXJ3jFYozOl2.M6 MNQPnYsGBUupYLA9aqkUM_2uVQmJ2zM9l7OgwMFX.yT8W__yax4QNJD9E6tpp2g2fUdwDbE8XeHk 6yotAD.9aQC8dCA0yKDHHmRSi6EnkJY6HBjbOetSW4AILEAEUkn0_zZxg92hK1Sx8tsCl0SGHXec 01jntrR7EOGLKPz_eeZwaUao5BaoQuBBNulZ4UXfK3I3Sj5_8MvXEf2n0ScNC6qynBYwCjwQC4wQ oUJIwgVAb0hIzrw1hGGYq.aErISXOTb8KHJICDkYvUHd0BAl.unhSdTSyPdGk7EY223BzKC5fSX9 kRzImLvcEAcQCA_pxcT12Ayozn6TATCPo5qLYSdXKcKi7t_fQHVe5vcNrJObLIaFDWI8r1PhWn.5 s6AWW1M2_mahNSz2flTuvYEBF.ekln_F0dH0yOcwHUe_BENe43TAE0D.opU_fKMQT8QQV5bP1T7a 3JV7b2KW5gytHLIes3qKkI2FOUuYSi1LjRm9sqPUew6WW1Kc0wMi3ACEwRSv5ybtrH_L4ZAZtUZo XSnQiOgbYRSTBeLQmok_FGn35TlPkxd87ZCqsL2jw2XQei5IXGuz6TRwf8vxvlpb_GhY7mViJ3Vl aJWZ343atJxR10LYRJGP1bYsA8UbJicx_2LV2jz4rkB2CLjpbva3FlP73pAxdYOQrw58e6M2zWc_ rzlHn.mlxymGHwPvInpHlLQN3MKcEROyexCWeW6zfqnwW1y67C6IzxEBzCwtT6LUNXwOblTUE7Hg A8wv93gqmM91hWZspmusUN.LiAJUEiG8MW0.EWqTVAVjYUFuIjFnSh192kIm6lImlgtQ3987BSIo LHARoSTP4vTf3MtUej3aCpEzaRUU4wfGW_7_bpEGMqjWJMA5I1x6rv3cnhVD_EzHwP.NCzqCugDF V8mX_cn4LBrlD9oGdqwxXYFqyamd1J9tKZdZU7wdUhwkIHBCs.G93i2rl2_AilMVFaEu7Y1ioX13 jCnpJ7y9msmk.1gy8FQhH2jR322vIKVoVAb.SQB5e6prLz6kGUflMygcvj1dZdPWk7ZyVyYEUNnv YQG7B9p3pj7DokQejp74.lsg1VqLAqG42L8zeh7Mvl3oVV8DXNuOIlnKv49G6H63_2Wczy4O9PPq JWpZpOYiAjeRNKpejV2PxgttfdH1GavxjFPGn_m5mBNN12t_gh3Jaz.OR1.R6dy650wGnzzVZ9ZZ G23OGZT90mYkgTNrogt17eQ7vn.6ABg9cXmiL1h1R3abgzl.cmE3C1R0i1pngAJJpgRPyUjMN0WT zKRgZKQ6jUWTLUfRAhMyqlZ6lSE2Dlw5p9Q6PJpyDBFMV1uALnWCBlM7fg7kJJ4NAA76gK.km9nO KVP8C2Xh_LcUhAiF7j_uV_YzjBRFMyXKJHm7dHsosWtZrVPjuVru5MB4wKUAbEbCBh.LxxCDMykl rOV_3wTr8ND_r3oMncNJy.DVq_Jx58FW479tvCqNgbQYB8WKvVpPc0S5wpUTOV_D3Ke2DJ4a4Uts GPG9uRJ7hPZ6gjLpcVjINia4PDtHtvFJFF1xIYUCvj0GLG9MPRIx1S1bGEpPoYX413PyPBuC.wm_ mYVdniEBH5KmrqeW0MlatexZhlLqE5l4yggtO4ZSuVBzkH7ZXml5IzmNiG1sA7NKFDjLnvwApl9v BRy1l8g0hlUoPeF.qkYcxLBJ9UAsVIBZKi2PVm9E9SkvVdJxbfmGtUdcAnudzR0HnGya5xk0pHz5 oBC7GRWha6HWEPOXzk0q97KFtao9UjP2K6l3W8vr3FrZmtQel8QC93kSuE.JoU2AT3RZX609HRz5 sO7VB_o6kexkVZHsM9qukD_MzvX_.LyAf95tde07TUVenfwtT3ydxlj9fvYs5NunAbZXRRJ5QjM1 xP_jQLTpwv3J.t4z76o_ktg79G9lVmqFwaEA08G49_MP4apPTFOn1s2t68KCWgIOZnLq1KHKsQqt gHr9gpOpdOvzMceYMO4qvIv1c6GZ7nJkVSpt154ct3unzD03xSqB51tSYUU5YWLOFX07ZtGAEFMa CzdxUUHbRpB4uyhvgbWTE6X5SIGb8H2vgT7KNFMmQkZfGIQnFTkGfb5x6hK5LiwavjhYcTU83A_W FYFW.pZCqnhxCYjAyCblWXQJX7o.ui60sZiR24ep7X9uBOX8iD8UFgemzH87gRX7Ar6es9gLhlrQ x2K_I4u0AkXLuLRnV5eK42VH2USVyHhIKO6CZ2ToM9JMc4nV.uiQMNDypVwirzB5Ag4nB9bUnOpF E X-Sonic-MF: X-Sonic-ID: 6db3f37b-d4bc-4606-9110-2c63e9c4f8d7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Fri, 22 Dec 2023 18:17:51 +0000 Received: by hermes--production-gq1-6949d6d8f9-c9pk7 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID cfdd67e59e1210986cdb849dd55cb704; Fri, 22 Dec 2023 18:17:46 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: <84A99C91-9DB2-4611-A040-2261C1C46CDE@yahoo.com> Date: Fri, 22 Dec 2023 10:17:34 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: <84A99C91-9DB2-4611-A040-2261C1C46CDE.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.206:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.206:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SxbBT5Ffrz3dvc X-Spamd-Bar: --- void wrote on Date: Fri, 22 Dec 2023 16:59:10 UTC : > My assessment of the "system being inactive while testing" may have > been inaccurate. By "being inactive" I was looking at avg load being > 1% or less, and for swap use being 0. Maybe the initial "inactive" = test=20 > should have been in single user mode [1] because the tests in this = mode > show no (I mean much less) of a speed issue. Apologies for not = considering=20 > single user mode till now. The results in single user mode seem to me = to=20 > infer that there is no problem with the hardware. Suggestion: Compare/contrast what you see via "gstat -spod" for single user vs. not when you are not deliberately running the swap I/O test but other things are similar to when you do. > I have been able to reliably create the problem by rebooting the > computer, then running something that does not need to be one huge = chunk of data, > that doesn't load the system that much, so ran 'make installworld' and = in another=20 > terminal ran the write-to-swap-partition test [2]. I suggest monitoring what "gstat -spod" shows during a make installworld run (no competing writes to swap). Then with both going in overlapping time frames: what is noticably different? I wonder if UFS vs. ZFS contributes for the RAM-related bandwidth = limited RPi4B. (One core can saturate the RAM-related subsystem depending on how effective the RAM caching happens to be for the access patterns involved.) > It shows in this context=20 > that writing to the filesystem effectively blocks writing to swap.=20 > 507 kB/s compared to 16 MB/s. I don't know if this is unique to arm64 = or=20 > if it's also the case on other arches, but it seems suboptimal to me. I do not have a built world to install on my stable/14 snapshot media. I'll have to switch to a main [so: 15] media (that is not up to date) if I'm going to provide some sort of matching type of activity comparison/contrast. This would be using a newer USB3 NVMe media. Probably UFS instead of ZFS instead: I may not have both of the main [so: 15] media available to me for now. If I do this, it will likely not be quickly. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Dec 22 20:08:22 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxdfK3536z54PfJ for ; Fri, 22 Dec 2023 20:08:41 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-55.consmr.mail.gq1.yahoo.com (sonic307-55.consmr.mail.gq1.yahoo.com [98.137.64.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxdfH3msyz4L6v for ; Fri, 22 Dec 2023 20:08:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=UFyXLAaa; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703275717; bh=E0DZVRPyWK+mKovpB+oteHHIrMQlEewyD/ibsXPbEO0=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=UFyXLAaaNcq0qxN4d/6JY1h+a8zbZdfObXosY6Sn0nCrVr1dvpp2At6vTpLDqDanYAqceyrEzkhG3jOPMx3ncscXcTFRQnlfk4fbJzs1fASN+bNp8dDgPx9vcY/oFV93nYwHZm7/qVm//lKcPSwgw/I1dvc2KVlc4bPGHuvYE78OA0f6AkmhOvW3xAhCAi5yAHVAQhXOOx5G+niQ3g6kjcMKwjyqGBT6Nh9mtM/UmEOCf/yW6UJyM64hykMXKAEu+QrjLKoiBkEBw+qoKC3Ws+y9C14wyQQZfYRRfkRf4cFmTEGaS9XbebdvrLDMG1ftmKxn3ffCbaf4pTyICpZMTg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703275717; bh=JE0gH2JMy4yk2ZPUuv0wTiaRQpTVLP1DzlSxjnEePOR=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=Cdc3bbgZ3t6hWz6yfo0ngsJkggCYpSCbWnLQRvyi9xfpprKeJgU2GJwtBTAIwtarbHinJesVEQzaYsYFEiIX9JgvCuqgo4R4WaYcYnkbwwp2Knf9ShlPQDpkBJeAUvUUDm9dYmQxe2UecJrC6DHire77+vfcdJl1Eho/eq90aIiq5oo9bXVujIrHl9EhruZ5ftA6H+a/dJc1B0ukPRvQ3XLZOszbbeeG+XOhEiMWRr+vVdpBzoWaOLSHiM0H+1jY8iLzIdJL49kqifAAm8zCkckJ/uJRwBCpAQgQ9JLDU2TSg6rCQPizetGHrNHe+mN3d4zmv5bnjAp6vpG6HmL6jg== X-YMail-OSG: rySN3MwVM1mJxSLjTgd159y9EA0B78eo_1m5mwJ75Vq0k0NOhN67CtVcTMR910Q sCYJPZD_Em8q_VjlQmrdZFZtfckMfSR7_FFQkJKUd0.sbz2zXAQhvP665CqbBuZxSON.YGV5zqKi 62GE.z1V5DO8AtsJ0JnZxnWa8JXCRjDGg_2g4x1MhTxSkRPqHInnE5PwLdrmB..p.YSzrIwApQHb LPlacQgo7O3QF33xJbbs7CFviRODKeFGwrWj2YnzFNr8v_9PHQufRZ52V.dpOoLxb3PqO1OnS5jR Lm5wPvqOfJs4w3Kv6KF2qXBCtCqA6zdBWOzO30qjSpKyQm3ND8ife6hkU.nWjGQyAIipXa1UFfka oEzbhLWpFWEHU.9ZcYunoRwBP57dKxh.EWdxq4j6tI.BWGx6LWx5jDwy8kNl_fNeVF1SVLX9F7Iy elIKByhElQXsghFev0IYR58D2u4frmEseQ1xJK9ibP2TfBweMVZb2ouvYy7upxWMtFizSZfwV8qb PBFaTFDVZnh3GtGTYEYnL5SjQetHI5CgLKeBKfa75oFLjX6F5Zkr.6huX46dpaySRJuGa_f1GgWs B5zFFfRrRyB8gbPZyrITluQyu31khb6OVt8fRE4KN58lcZZ02Z4rNjW3aNfBkdeWG2i7LdV6o.8q ipAVPc9sJ_9mkvIA6h365aDRM8LUPnssSLOSKSimfDJeL_nJ0qOZYH0ZyL0qgHJbsXribEJomHjF zZjQKzDJhFlvv2AR4NI0TO_LYlbXoe9IoZt2ExqeryEaxglz0f.GofExh.IfdlNCsKTUabuOAAKu cjG0EyPC7wfdc7C_JNfGuRNBkRRFS5dcLDd4KdeYkkEn8Ny00c_jizcFakvtIBTLvQCXUTLUjvDG mt2ToXRqnGqz.kCplAW1qmNmeK31gPPk1MN8ko36uzMkZiHHCcDy_mq.2cOE45mGV5k0npTuo5W3 bnCEYq6bSUPdU6oz.enQq9G1f17V4J.LkZ.QkN_kwCi7Fg.Arv7.hnc7mspjmzQeCkEobJFVWdx2 v5E8oVDqjMu1T1xVjJHSuYPXlQBg8kChBGZGuoIlfDg5ND6bX3mkTI9j_Vh4Ca8rows.V8iyH4NA CFFK4XIWRF60TKMHd89cqXhZ3B_XDtX7H9gC_udnTtQWI0FDOVGs3strtPo.zWyiW__BFgIjnH2D 0WJjqFIBuUXd74V.y8UKJwHM3mGSZeJWA.c7gHb2XjVgYE0PyPhuUltV0T39bHjJ9DXx7z2mzAle t6zGXNjr4rTUL8u5_H9.CMUQNSxp3UKotvWKzxZGOXMEmMZfqd_7l9CgnY00fnWQmTe_uffWovEY Wl0VOcb59YRjAQHk67ZcxLWD.Ebpz1zdcWQs44aJDrJsYHVTR2CjhloBr0_T6d0Zp4g0gOJsebIh c33.bENor61VsOWU3kszXtwrsKsWhghQAokJHMZACw5ouZSX_FT7G8.OhfsnWTqfES3ljZLKtDJ9 S1PwXrLghT9cQNL8TmWa6zGJfKW9LLNL1hR5tT9yTrlHEYA2aKlMPOPDDoWYlIRsnQPC_ftBRoNM 7yo6tmdh3xae.uVi6zocjbu9ozsmZQgdrBqoyPbLp5yA1lOtJlfvTVZlqIGMqYFMFo2IT1H0.Jc8 6bqI9opJmhPLMNBTY_gyRI8jXJSg2cXCaqllNAL5Txv6kYHKRux_AHL1xW_RCKkdL4wcmC8w6ZtI Y98KKR3G4V6FIgOyj5avS4HVfZcI0qbuciWEYE2BzNJ5rbzIFjOREMoCgO5z0GC_nnH6TvPXOeRm 5f3ULTihaIIw3HCFP8SDBq.1O7lyJ38jydihEUc4fOythKt2l.uBqf3Vg_qPVFpZThBn4rfMZbp1 8BWvBd_AW.ZQGNXe0KsStp3nSkQ_nV.WYpHhM3_QjCfK_aek2CK7SMzoI4BvedLvD0eAAtrna5Zt PifhIpJAHKbIvG2LBCB2I9N_pxKH6vKsJGSHUW8z2Tup6ztGlpbE7hTwo2HrM3847yvmMQm8_xCJ qojHKlCefgHyTu4CRT0ZSHty.DS77FtlhWB_3OKAgWiONo4zR.910.iipPiVk05LhJ65XQMenLDu ZpfcKe5nWKo8N38tfuTt3HX7uRWRCH_aAmmOFx_oYi2QxieSXgwcPZG6pnuXTBhnummFaJAR5c8c iQEmsVDhvdSLtPws5wsDYb23U9gljlq1DdM.plFqM5McpOsv_JEayQy0wltn7n79unPu6EPsVkw3 h7gmfpTb8LcnnZifS.mP3mxJyV8lW5p7OoHYDePEsdSR2BMMg1Uza_LgPzkhJYWuHVQfe2YUXp6Z 3 X-Sonic-MF: X-Sonic-ID: f990c685-d2fb-4930-9c73-2fd378848b65 Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Fri, 22 Dec 2023 20:08:37 +0000 Received: by hermes--production-gq1-6949d6d8f9-hnk4w (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID fb5db664a4faa8f6f2d34b13e2bdf9c8; Fri, 22 Dec 2023 20:08:33 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Date: Fri, 22 Dec 2023 12:08:22 -0800 References: <84A99C91-9DB2-4611-A040-2261C1C46CDE@yahoo.com> To: void , freebsd-fs@freebsd.org In-Reply-To: <84A99C91-9DB2-4611-A040-2261C1C46CDE@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3774.300.61.1.2) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.31:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.31:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SxdfH3msyz4L6v X-Spamd-Bar: --- On Dec 22, 2023, at 10:17, Mark Millard wrote: > void wrote on > Date: Fri, 22 Dec 2023 16:59:10 UTC : >=20 >> My assessment of the "system being inactive while testing" may have >> been inaccurate. By "being inactive" I was looking at avg load being >> 1% or less, and for swap use being 0. Maybe the initial "inactive" = test=20 >> should have been in single user mode [1] because the tests in this = mode >> show no (I mean much less) of a speed issue. Apologies for not = considering=20 >> single user mode till now. The results in single user mode seem to me = to=20 >> infer that there is no problem with the hardware. >=20 > Suggestion: Compare/contrast what you see via "gstat -spod" for single > user vs. not when you are not deliberately running the swap I/O > test but other things are similar to when you do. >=20 >> I have been able to reliably create the problem by rebooting the >> computer, then running something that does not need to be one huge = chunk of data, >> that doesn't load the system that much, so ran 'make installworld' = and in another=20 >> terminal ran the write-to-swap-partition test [2]. >=20 > I suggest monitoring what "gstat -spod" shows during a make = installworld > run (no competing writes to swap). Then with both going in overlapping > time frames: what is noticably different? >=20 > I wonder if UFS vs. ZFS contributes for the RAM-related bandwidth = limited > RPi4B. (One core can saturate the RAM-related subsystem depending on > how effective the RAM caching happens to be for the access patterns > involved.) >=20 >> It shows in this context=20 >> that writing to the filesystem effectively blocks writing to swap.=20 >> 507 kB/s compared to 16 MB/s. I don't know if this is unique to arm64 = or=20 >> if it's also the case on other arches, but it seems suboptimal to me. >=20 > I do not have a built world to install on my stable/14 snapshot > media. I'll have to switch to a main [so: 15] media (that is not > up to date) if I'm going to provide some sort of matching type > of activity comparison/contrast. This would be using a newer > USB3 NVMe media. Probably UFS instead of ZFS instead: I may not > have both of the main [so: 15] media available to me for now. >=20 > If I do this, it will likely not be quickly. I have ZFS media available. so I'm using that here. The main [so: 15] context for the ZFS test goes back to 2023-Sep-21 or so for main: # uname -apKU FreeBSD CA72-4c8G-ZFS 15.0-CURRENT FreeBSD 15.0-CURRENT #118 = main-n265447-e5236d25f2c0-dirty: Thu Sep 21 09:13:36 PDT 2023 = root@CA72-16Gp-ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1500001 1500001 I normally use -j$(sysctl -n hw.ncpu) for installworld but, if I gather right, you effectively used -j1 (implicit), so I'll do both styles, -j4 being appropriate for RPi4B's. I'll note that I use ssh sessions instead of the serial console (no video connected). The I/O for the serial console would greatly slow the make installworld, waiting for output. Of source I do not know your make installworld context, so I'm just doing what I normally relative to such issues. I've done one serial console example as well, for reference. Booted multi-user but no make installworld in progress (ssh session): # dd if=3D/dev/urandom of=3D/dev/da0p7 bs=3D8k conv=3Dsync = status=3Dprogress ^C562470912 bytes (562 MB, 536 MiB) transferred 26.043s, 22 MB/s 69267+0 records in 69266+0 records out 567427072 bytes transferred in 26.274117 secs (21596428 bytes/sec) Booted multi-user but with make installworld in progress (no -jN) (ssh = session): # dd if=3D/dev/urandom of=3D/dev/da0p7 bs=3D8k conv=3Dsync = status=3Dprogress ^C591372288 bytes (591 MB, 564 MiB) transferred 32.001s, 18 MB/s 73785+0 records in 73784+0 records out 604438528 bytes transferred in 32.872265 secs (18387493 bytes/sec) Booted multi-user but with make -j4 installworld in progress (ssh = session): # dd if=3D/dev/urandom of=3D/dev/da0p7 bs=3D8k conv=3Dsync = status=3Dprogress ^C551559168 bytes (552 MB, 526 MiB) transferred 35.001s, 16 MB/s 69285+0 records in 69284+0 records out 567574528 bytes transferred in 35.917269 secs (15802274 bytes/sec) Booted multi-user but with make -j4 installworld in progress (serial = console): # dd if=3D/dev/urandom of=3D/dev/da0p7 bs=3D8k conv=3Dsync = status=3Dprogress ^C642711552 bytes (643 MB, 613 MiB) transferred 31.000s, 21 MB/s 78521+0 records in 78520+0 records out 643235840 bytes transferred in 31.024779 secs (20732971 bytes/sec) Compared/contrasted to yours, it again suggests seek-time as a potential notable contribution to the results in your context: the spinnig rust spends far more overall time between command, limiting the command processing rate. Plugging in and using a separate USB3 SSD suitable for use as swap might be a help, despite the shared bandwidth for the 2 USB3 ports. Even using the USB2 port could be useful, and might have a more independent bandwidth. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Dec 23 05:05:19 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxsYW4WmTz54yc1 for ; Sat, 23 Dec 2023 05:05:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxsYW3GKHz3WN1 for ; Sat, 23 Dec 2023 05:05:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703307919; a=rsa-sha256; cv=none; b=X8DI3WfigZvyuUkhcxhDzUjcakX8+XbLywhx8LYYhV8lvVmV7wvTQmXe0eL85D4yDe6U1B vxODtNvKLvFQXutaLlH3giCMAfor+eTgIGjTGNitBBMrBlTWw0NPdTcgFEPmYdEa4cJq8w ZCRx38zZyn06DLOnNYPTGFTDsWwZiEnz7IsNbxyrN6NMjUSBPPEdPJRTlX1elL/H3jAeTm zjT21UlFUdwL/5CgdKaSy3kKTaC2k1tbxFs23pjUVXEhd/jvvYFyBZEFRGu/prXRvLvveT Tab48GO3f+6b1mh4VYsMeHFXEOvE4WMtg76PW1FKVoZOdlYDbwi/EN2t1outOA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703307919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KtzY9GQieDNicfjNjIbu0JKgME+jwHFU+JauMyORGUM=; b=bYKuP7lxe27VmkMQfJEs1UzVLD7+++lsJ4AHkQ2iaOWCBe5ALX2EP3HK9/VkU/t9Z/h/9c wykb6K2Qmd+TtIQ+wsQrFE54wkhQsu9hUW9FD7JjY6slSye4obJivlK3ixdO+p4BzveZdq xngFj/q/W6uvynFU5IXg3owHw+xbjoFD1BTIea3qdfdJ6LPTI3rvDtSvblW20Jmle0LoEV vM/vvaw0vMUs5xcMMBFyYlT1awe8bBcuAL7iIjPcJmRKT3ccdTQwjPLC4OdlWLishLfLkk BnscGqwRbvCbxl/eA82cgAe6u/ZKfQdMtDRA3guC8dDxLZJF0kf6jEOBBBv/Ww== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SxsYW2D3Qzm9n for ; Sat, 23 Dec 2023 05:05:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BN55JRa032575 for ; Sat, 23 Dec 2023 05:05:19 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BN55JOV032573 for fs@FreeBSD.org; Sat, 23 Dec 2023 05:05:19 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275871] [unionfs] [nullfs] [zfs] corrupt filesystem Date: Sat, 23 Dec 2023 05:05:19 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jah@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: jah@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275871 Jason A. Harmening changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jah@FreeBSD.org Assignee|fs@FreeBSD.org |jah@FreeBSD.org --- Comment #1 from Jason A. Harmening --- The fundamental issue here is that ZFS doesn't correctly respect cnp->cn_namelen in its mkdir operation, which breaks unionfs shadow directo= ry creation. I've filed https://github.com/openzfs/zfs/issues/15705 to address this upstream, but in the meantime it can be worked around in unionfs using the attached patch. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 06:05:55 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SxtvS1smwz552w1 for ; Sat, 23 Dec 2023 06:05:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SxtvS0r7cz3Zqx for ; Sat, 23 Dec 2023 06:05:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703311556; a=rsa-sha256; cv=none; b=aydinKvCInJ3IvoQtjHs6yY/+vEa6MP/dJJVdzYczNVQjxBkzOCWEAelfzjAmOrgjjIcoa c9YufM7pIVSXmrOYNE8qT1cpDTLE7ed4jIhHJ1FtPI9OJIJM66F/IcpLX0lJRsSSt8mINL Ul8W+F1FW1sw7Ae7iSv9LvUhW3T+L6yN+wikZyx2ey2zlziiUjS4h0zXaKhiUctbmyg1OO 9s+KDQqBMpEnqQV5fD1xUzeMfFnGQlgQtSEZLrxeb7NGPZOOEkkbjhoF2ZEJkdApULSRHz qma7umM7vDi+s7dEjGIxCUSdCSKc8boh6D0JKlSZv/d82iGKKbTCDJbjB+WOOg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703311556; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BRuiDjI41JzCaZ2q8muSFKoPLnYinzmvjxQroyLXsUs=; b=uN4cDzdpI8Nyc1ey2n6pdP8oTtnKZy1kYptYbFO2HH9yGAODIUVviXvPz40NuBG/7ZR/YD 1zkwMJxZTK35LB0/K6hFyWoVpo0F4eDdQZlk10hgMNwM8xzhTyYKYKg4S9m6N4RlV2cq+x X6UyM2YSQOfhaJj8LyU3ZwuG2b2estw0s2WNBRlSjN0Tt8X3V0rNkNUiCcF1eEDR1AxDru 3ECoZMDXkMqocoBHJHX+EgdUStHXBuqxn8TFO05apOjzoaEAex1rC6nOO17eHHVy2zJ8te zsRDwvXcI0U/UZPrIystv0elE/WE7RV5Hd9lzhzvErdG0WCUqz+6SUyyb9fw3A== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SxtvR6vdhzp9T for ; Sat, 23 Dec 2023 06:05:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BN65t2h023445 for ; Sat, 23 Dec 2023 06:05:55 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BN65tHL023444 for fs@FreeBSD.org; Sat, 23 Dec 2023 06:05:55 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275870] [unionfs] [zfs] kernel panic on umount Date: Sat, 23 Dec 2023 06:05:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-RELEASE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jah@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: jah@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275870 Jason A. Harmening changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|fs@FreeBSD.org |jah@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 12:36:50 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sy3Zx2QN8z54SLy for ; Sat, 23 Dec 2023 12:37:13 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sy3Zw0bx4z4ZBP for ; Sat, 23 Dec 2023 12:37:12 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b="KjTnzs/v"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=hQAF7k6D; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.26 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id E786C5C00EC for ; Sat, 23 Dec 2023 07:37:10 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Sat, 23 Dec 2023 07:37:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703335030; x=1703421430; bh=CVqhpzUCnW /D1MgF2GRcut0q3D7CMZkkFJx8Z07Ea7s=; b=KjTnzs/v1UHq+h5x6BHPVYm/dy 2puMQKR6TbBP5MktQC3VNtYNkRgK2fHpa70gvGvU/KTNrclYPrTPe7Ke/EFkZgiG iYpajiEV6U9fJb6Gql5lkcoa+lRj+dviQMR3ifoUHV0WSodyY3P39yBgyszL4fvB snls4008uZiRChQX44cNfrEswstXYquH7c+mTgmU6hXlPbrrpMakORAH43WTKmVM /4ACG687+Pz8hZBLrImltcfelm4/vTfa7Sbo0WQ68MeiUJ8M+YJQi9lZnnhflmwT 9uPznoNhtMpnDbfKt1jqNOuslVxpHgYgGuHWNZ8lmoorp/nhb/jXHV6EbFTA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703335030; x=1703421430; bh=CVqhpzUCnW/D1MgF2GRcut0q3D7C MZkkFJx8Z07Ea7s=; b=hQAF7k6DUbZAqMT1ZvXrT+8X0A0d7MKrH1VsGrkXdmcj In7pySmkQJfSq7NITma3Q9RwFLRkCBCG7846XnE+3wvDuPUFP+b8/ntL/5TU8mOW AH1RlzmBiJSWv/T2Mr82O0rGH8SbNeYre/F6b5LutRhCdV0R7cyCjrS1+iNhWgW9 oP3NhRURv6KqD60yrahFCIYYwL0QvwgaihF/0Hsj6/mZZT/znRQ2bzYgMFR97RBo ZIP8MbrxrwwRkO+Rf/71GwJoqAX+Hsx3vXQudsLqtYjy3gAVW0f51n26Uw8P1fZX Bg8QD1wKOTA92TslbQRNqLsHVfLGTgyH5cGN2M8YpA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduledggedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id ACB8B2A2008B; Sat, 23 Dec 2023 07:37:10 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: <271589cd-102b-4035-bc0b-1bceba240b66@app.fastmail.com> In-Reply-To: References: <84A99C91-9DB2-4611-A040-2261C1C46CDE@yahoo.com> Date: Sat, 23 Dec 2023 12:36:50 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.17 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.982]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sy3Zw0bx4z4ZBP X-Spamd-Bar: ---- On Fri, 22 Dec 2023, at 20:08, Mark Millard wrote: > I normally use -j$(sysctl -n hw.ncpu) for installworld but, > if I gather right, you effectively used -j1 (implicit), so > I'll do both styles, -j4 being appropriate for RPi4B's. Interesting. Advice long ago was to not use -j for the installworld part so I've never used it at that stage. Other ways of testing, in place of 'make installworld': git -C /usr/ports pull --ff-only periodic daily find / -name "*.core" -mtime +2 -ls (snip results) > Compared/contrasted to yours, it again suggests seek-time as a > potential notable contribution to the results in your context: > the spinnig rust spends far more overall time between command, > limiting the command processing rate. Plugging in and using a > separate USB3 SSD suitable for use as swap might be a help, > despite the shared bandwidth for the 2 USB3 ports. Even using > the USB2 port could be useful, and might have a more independent > bandwidth. Historically there's been an issue (I didn't delve into it deeply at the time) where if both ports on the usb2 bus were occupied (like keyboard+mouse), one knocks the other off the bus. Doesn't matter what make/model the keyboard or bus are. This was last identified when the system was at 13-stable. At present, there is a wifi module plugged into the usb2 bus (ralink, shows up as run0). It'll page down and up when the system gets busy, to the extent i've had to take it out of monitoring. It's the only thing plugged into the usb2 bus. I've avoided having to plug in more than one usb2 device into this bus since encountering the problem. Having two things plugged into it makes one or the other malfunction. Having one thing plugged into it flaps when the system gets busy. It's unknown to me at this stage whether anything similar affects the usb3 bus. Right now, there's the usb3 disk plugged into one port and a usb2 serial cable (for a UPS) into the other. Something might be affecting usb in general and I've till now considered it to be usb2-only because that's where it was most obvious. So I'm thinking what I'm seeing might be some effect with usb2 and/or 3 being the root cause. Things I'll be trying: 1. creating a swap partition on a usb3 stick plugged into the usb3 bus and running the tests 2. take the disk, plug it into an amd64 usb3 bus, try the tests on the plugged-in disk thanks for your help, it's appreciated. From nobody Sat Dec 23 14:37:02 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sy6FB6KRPz54Zq7 for ; Sat, 23 Dec 2023 14:37:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sy6FB52Ccz3DHG for ; Sat, 23 Dec 2023 14:37:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703342222; a=rsa-sha256; cv=none; b=gNvbHBGOsnhbDab6u8zE49NU/v5n3wzqZqpfCQVJk25ybzdXR8RviSt3gsC4efufzjYEjT 466cqoTmXh5X+af5V9sDL83ekbXG9oyTzfYU2NXefUs1JLcYmTkjwkAq5sw+v9Ml2EnHi7 O/jy/91LV+GBnrvRXGkv+S9U/FMkw8/WYK+PIpAWucIbjWwoVfy2+xpeowd+KPtCOYWVLO biRfHLDlAglQHCIgwd8ugEfU5u+1ttAF4Z/yPZmWlzB23kD4jMyNsejjhRm6632RNs/M/r wasRD3ViDr7wSa70zpdj6IfpSZgsSKYLIQ1gilyeOsylOg7pVgQMZ4ok7/oAKg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703342222; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kV2TZoy637C26YNN3NZOBl98GZqixFmUO6ryNsu+fgc=; b=cCKpfF7TUsBSZVa8oL4PkKYbVFbPGdDxVgYnjes6Vj+H/a/tNUs/geveKeR5QckFqwR7xY ZeztDi3N2r582OJVp0x5RjmxKu6GreKdXLJZChcgWuu0y5L57eqw3xNUUHXBc2msekO0F1 apOK1OT5RH2JY1XY0mqer9mzEPmVIxtEvC25PZ6GIMFOXmT47WBU5KHPc6QzpFZWHUjfZU 3V0s5a1jmn93GZGuHAb+XEhDOJYSPkxxpTDSIurIrumBi4yl7XOohxRLASw4GVLDOBbK4l yhzDG73pvUlCQbGyh2+UNS4SKpiEKjCTvrq+coK8Ky2ruHrEaFfNSl9RntLCiQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Sy6FB401cz13LT for ; Sat, 23 Dec 2023 14:37:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BNEb2lK011591 for ; Sat, 23 Dec 2023 14:37:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BNEb2cA011590 for fs@FreeBSD.org; Sat, 23 Dec 2023 14:37:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275597] Samba: smbd sometimes aborts by PANIC when 'vfs objects = cap' Date: Sat, 23 Dec 2023 14:37:02 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: uratan@miomio.jp X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275597 --- Comment #3 from uratan --- I will close this topic with this. /* * how to make samba4 from ports (memorandum for me in the future) */ (and for those who got here) [1] Extract files from freebsd-dist/ports.txz +------------------------------------------------------ |# cd / |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/net/samba416 +------------------------------------------------------ Also these files are needed to make samba416 +------------------------------------------------------ |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/Keywords |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/Mk |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/Templates |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/lang |# tar xzf ...somewhere/freebsd-dist/ports.txz usr/ports/ports-mgmt +------------------------------------------------------ [2] Other preparations These packages were required additionally to make in my environment | p5-Parse-Yapp-1.21.pkg | libtextstyle-0.22.pkg | cmocka-1.1.5.pkg | bison-3.8.2,1.pkg | p5-JSON-4.10.pkg | m4-1.4.19,1.pkg | pkgconf-2.0.3,1.pkg (I wonder... 'make' would have added them on during 'make'ing ?) The distfile: samba-4.16.11.tar.gz will be downloaded automatically but if you already have it, place it to /usr/ports/distfiles/ [3] Do (first) make +------------------------------------------------------ |# cd /usr/ports/net/samba416 |# make +------------------------------------------------------ Then some configuration menu is appeared, simply select to choose/keep default settings +------------------------------------------------------ |lqqqqqqqqqqqqqqqqqqqqqqqqqqq samba416-4.16.11 qqqqqqqqqqqqqqqqqqqqqqqqqq= qqqk |x lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq= qk x |x x+[x] ADS Active Directory client(implies LDAP) = x x |x x+[x] AD_DC Active Directory Domain Controller(implies PYTHO= Nx x |x x+[ ] CLUSTER Clustering support = x x |x x+[ ] CUPS CUPS printing system support = x x |x x+[x] DOCS Build and/or install documentation = x x |x x+[x] FAM File Alteration Monitor = x x |x x+[ ] GPGME GpgME support = x x |x x+[x] LDAP LDAP client = x x |x x+[ ] MANDOC Build manpages from DOCBOOK templates = x x |x x+[x] PROFILE Profiling data = x x |x x+[x] PYTHON3 Python 3.x bindings or support = x x |x x+[x] QUOTAS Disk quota support = x x |x x+[ ] SPOTLIGHT Spotlight server-side search support = x x |x x+[x] SYSLOG Syslog logging support = x x |x x+[x] UTMP UTMP accounting = x x |x xqqqqqqqqqqqqqqqqqqqqqqqqqqqqq VFS modules qqqqqqqqqqqqqqqqqqqqqqqqqqq= qx x |x mqqqqqv(+)qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq64%qqqq= qj x |tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq= qqqu |x < OK > = x |mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq= qqqj +------------------------------------------------------ It takes about 30min to finish in my environment, I've got same binary to samba416-4.16.11.pkg +------------------------------------------------------ |# pwd |/usr/ports/net/samba416 | |# ls -F |Makefile pkg-descr pkg-plist.cluster |distinfo pkg-plist pkg-plist.python |files/ pkg-plist.ad_dc work/ | |# ls -F work/ |.PLIST.mktmp .patch_done.samba416._usr_local |.build_done.samba416._usr_local .stage_done.samba416._usr_local |.configure_done.samba416._usr_local README.FreeBSD |.extract_done.samba416._usr_local pkg-message |.license-catalog.mk samba-4.16.11/ |.license-report samba_server |.license_done.samba416._usr_local stage/ | |# md5 -r work/stage/usr/local/sbin/smbd \ | /usr/local/sbin/smbd |e281293697932aff5d3c32c211da3a4b work/stage/usr/local/sbin/smbd |e281293697932aff5d3c32c211da3a4b /usr/local/sbin/smbd +------------------------------------------------------ [4] How to make again after changing source codes +------------------------------------------------------ |# pwd |/usr/ports/net/samba416 | |# cd work/samba-4.16.11/ |# vim source3/modules/vfs_cap.c |# cd ../../ | |# make | ... <-- do for the configuration menu |=3D=3D=3D> Options unchanged |# ... ???? NOTHING is DONE !!!! | |# ls -lrt work/ |total 1 |-rw-r--r-- 1 root wheel 0 Dec 23 13:45 .extract_done.samba416._usr_local |-rw-r--r-- 1 root wheel 215 Dec 23 13:45 .license-catalog.mk |-rw-r--r-- 1 root wheel 93 Dec 23 13:45 .license-report |-rw-r--r-- 1 root wheel 0 Dec 23 13:45 .license_done.samba416._usr_local |-rw-r--r-- 1 root wheel 0 Dec 23 13:45 .patch_done.samba416._usr_l= ocal |-rw-r--r-- 1 root wheel 958 Dec 23 13:45 pkg-message |-rw-r--r-- 1 root wheel 3094 Dec 23 13:45 README.FreeBSD |-rw-r--r-- 1 root wheel 8412 Dec 23 13:45 samba_server |drwxr-xr-x 36 root wheel 2048 Dec 23 13:49 samba-4.16.11 |-rw-r--r-- 1 root wheel 0 Dec 23 13:49 .configure_done.samba416._usr_local |-rw-r--r-- 1 root wheel 0 Dec 23 14:13 .build_done.samba416._usr_l= ocal |drwxr-xr-x 4 root wheel 512 Dec 23 14:13 stage |-rw-r--r-- 1 root wheel 52433 Dec 23 14:17 .PLIST.mktmp |-rw-r--r-- 1 root wheel 0 Dec 23 14:17 .stage_done.samba416._usr_l= ocal | |# rm work/.stage_done.samba416._usr_local \ | work/.build_done.samba416._usr_local | |# make | ... <-- do for the configuration menu |=3D=3D=3D> Options unchanged | ... re-make started (takes 5min to finish) +------------------------------------------------------ [5] How to install samba4 of ports +------------------------------------------------------ |# pkg info | fgrep samba4 |samba416-4.16.11 Free SMB/CIFS and AD/DC server and client for Un= ix |# pkg delete samba416-4.16.11 | |# pwd |/usr/ports/net/samba416 |# make install | ... <-- do for the configuration menu +------------------------------------------------------ [6] How to rollback to samba4 of packages +------------------------------------------------------ |# pwd |/usr/ports/net/samba416 |# make deinstall | |# pkg intall samba416-4.16.11 +------------------------------------------------------ you can confirm installation status from /var/log/messages | "pkg-static" means install/deinstall from ports | "pkg" means install/delete by pkg +------------------------------------------------------ |# bzcat /var/log/messages.0.bz2 | fgrep pkg : : |Dec 16 17:34:38 oxygen pkg-static[11504]: samba416-4.16.11 installed |Dec 16 18:27:13 oxygen pkg-static[39133]: samba416-4.16.11 deinstalled |Dec 16 18:27:27 oxygen pkg-static[39645]: samba416-4.16.11 installed |Dec 16 22:23:23 oxygen pkg-static[65811]: samba416-4.16.11 deinstalled |Dec 16 22:26:15 oxygen pkg[65987]: pkg upgraded: 1.20.8 -> 1.20.9 |Dec 16 22:27:52 oxygen pkg[66699]: samba416-4.16.11 installed |Dec 16 23:00:40 oxygen pkg[81824]: samba416-4.16.11 deinstalled |Dec 16 23:01:17 oxygen pkg-static[82502]: samba416-4.16.11 installed |Dec 16 23:24:09 oxygen pkg-static[96335]: samba416-4.16.11 deinstalled |Dec 16 23:24:40 oxygen pkg-static[96976]: samba416-4.16.11 installed |Dec 17 00:01:49 oxygen pkg-static[13207]: samba416-4.16.11 deinstalled |Dec 17 00:02:39 oxygen pkg[13431]: samba416-4.16.11 installed |Dec 17 00:32:43 oxygen pkg[27009]: samba416-4.16.11 deinstalled |Dec 17 00:33:14 oxygen pkg-static[27678]: samba416-4.16.11 installed +------------------------------------------------------ ... My smbd has been running great ever since! (Please let me know if there are any problems with the workflow) --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 15:52:55 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sy7yv4nyPz54gMc for ; Sat, 23 Dec 2023 15:54:47 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sy7yt518Nz3Mln for ; Sat, 23 Dec 2023 15:54:46 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b=E2dAiHX2; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=c8YgN1O6; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.29 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 2A7555C0130 for ; Sat, 23 Dec 2023 10:54:46 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Sat, 23 Dec 2023 10:54:46 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703346886; x=1703433286; bh=UVP58xnOxp U01iIwZRQsMXAmoJROdFzDm/R0mwTh+o0=; b=E2dAiHX2oW9lBXWiCN27w0jIM5 aVMQ98MCalklOAVpoAyagGLvxbxlegkQfZtehPOXA/InZXLqYhdcTlMJxrKGLZ5I +giEUtRNQvrV0k9XTQ67cxCtsAHIC7wyMdfJb/kqO3lACezAWQ5a0MyPvGBBvgBC RmHjy4pxZWSfwtAZjwNgVInIPkkRvSpmffw3362XAr8HlYdDpQwtWs18nCphcmW6 kuIGcrIy2mCydmN9So2m48+SW4o4RRRMGWt5ik4cLoku+ryJJiY1l2nS8Rv6iWQd LUmjIJkKopFot7pWlgJ4pc9huhDYxFyZG05oG4A4KWOwhniP9EIk6JPZz2rA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703346886; x=1703433286; bh=UVP58xnOxpU01iIwZRQsMXAmoJRO dFzDm/R0mwTh+o0=; b=c8YgN1O6Udp6hLr4CdGJXT+X3whzJ2jpkNXT6QWI4ZYd nwcP3qfknVGnC5+VJVwPGHAqabKQYQrqNNkEu71OCzdRoDZmXKU53V85fckzU+Sc 5QReQL71jJZxRGifXC5B1qQzgB2fGq6pZ3KC6WeEKqtUZegL1RTNd+XEspvUWtUW 59UoCSfvJiukQkBKp9/wage9mZD3eyzCR8CFYcyVQ/LTLoD3CzakRF8+DsrxwcB2 YL+AeWspP7RT43XSK7ehI4UelYCPLcRQ3qn76iGhmKfzAGABysLCDMMNRPTpe9Y3 jgk+zSmwDrC71xq/rbovyiKLrbFk6UFrhVH75EBVFA== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduledgkedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucgoufhushhpvggtthffohhmrghinhculdegledmne cujfgurhepofgfggfkjghffffhvffutgesthdtredtreertdenucfhrhhomhepvhhoihgu uceovhhoihgusehfqdhmrdhfmheqnecuggftrfgrthhtvghrnhepfeejheefieeludejud elheejteeffeetgfetiedvgeelfeehtedtjedulefhveehnecuffhomhgrihhnpegsshgu rdhtohenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpe hvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id D9DEE2A20094; Sat, 23 Dec 2023 10:54:45 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: <6781d21e-fdff-447e-a1ef-1e514d411e08@app.fastmail.com> In-Reply-To: <1137397689.3080.1703237862787@localhost> References: <1439231787.9333.1703184981914@localhost> <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> <1137397689.3080.1703237862787@localhost> Date: Sat, 23 Dec 2023 15:52:55 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[66.111.4.29:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sy7yt518Nz3Mln X-Spamd-Bar: ---- Hello, On Fri, 22 Dec 2023, at 09:37, Ronald Klop wrote: > Can you provide more concrete information? Like: > > # usbconfig list # usbconfig list ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA) ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA) ugen0.3: at usbus0, cfg=0 md=HOST spd=LOW (1.5Mbps) pwr=ON (24mA) ugen0.4: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=ON (450mA) ugen0.5: at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (36mA) The power supply is the approved 5.1V 3A one. > And the output of "devinfo -v". That will give people much more > information to work with instead of "anecdotal evidence" about your > hardware being usb3. devinfo -v : https://bsd.to/ADPj detailed dmesg is here: https://bsd.to/Q5tx thanks for you help From nobody Sat Dec 23 15:59:21 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Sy84b0yRmz54gLW for ; Sat, 23 Dec 2023 15:59:43 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Sy84Z5NBwz3N61 for ; Sat, 23 Dec 2023 15:59:42 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm2 header.b="g/+J7acn"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b="Hq/5hpRl"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.29 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 78D785C014E for ; Sat, 23 Dec 2023 10:59:42 -0500 (EST) Received: from imap46 ([10.202.2.96]) by compute6.internal (MEProxy); Sat, 23 Dec 2023 10:59:42 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:subject :subject:to:to; s=fm2; t=1703347182; x=1703433582; bh=Z0I0dxjqZo jgjj9XO9bRdbf0cG1+AYC7mEvFeQt0OEI=; b=g/+J7acnuAJnp6mMdaX9dNvYuJ BXZnbeiGbW3o5UNgy27zC+gkBBKHH9Nf2C795KGrmRXcuucVVPQ7FK8BZeo7srL6 ecOfz6sCRkFOlBkC3AFB7lnzFbaaYPqTxpNPcva3pWxAJO6yGXQtz5kUXh79JYcX JoAlaxUgDGbMofPZ/n9nAof9HiXoTuCxpThR0Q9dIPDD8btxmC4tCL7jUkxjFJa8 gXWSHlcI/w19sgiH59c9uHnG9a+UZTrwD/2TprgdOAVr0Io9mYyfJMJWJseh1zaa 81GmI0QcmOuUB2tCro55vQIYraHkiPWc7Pg6am4ZfxqtKyQj2Jn5Pupzvvng== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:subject:subject:to :to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm2; t=1703347182; x=1703433582; bh=Z0I0dxjqZojgjj9XO9bRdbf0cG1+ AYC7mEvFeQt0OEI=; b=Hq/5hpRl7+JLJu4rU0bt8UW/Du5Jy5LJ9FsHy6RYqbO2 Bv5dcCvSgGVPZeXvuJt2l91Ynw6JAsoWmVYKXx7DTbReeXxGBZ+axn8JLqNQY7fY 1vpUPLmDbZLO+TsUdPiPmsdU9BdTNjx7mqNE19afqth2DD9/RdUs2E+dSLlVRpLG VqS7v8VnfmAGohAwqElt9yh2EN2C9XEDOr60Woyd35X+GdkiaNQZ7JDAfRM/2o4h HIVFSJVxkkRBFHbJ3VwgPW4hkhFQIg55Fy4CWGZcPnN6jgF8vACozoDC66XrPDRI /P9QFjYLncNwO8QfW04U1KoQJTo7l3vKM/3ExrHIDw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrvdduledgkeegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderredtnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffr rghtthgvrhhnpeeitedvueehtdehtddvhfeuhfevhedvieelvdeiffehveelheegfedule ejudekvdenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhm pehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 417B52A2008B; Sat, 23 Dec 2023 10:59:42 -0500 (EST) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.9.0-alpha0-1364-ga51d5fd3b7-fm-20231219.001-ga51d5fd3 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Message-Id: In-Reply-To: References: Date: Sat, 23 Dec 2023 15:59:21 +0000 From: void To: freebsd-fs Subject: Re: measuring swap partition speed Content-Type: text/plain X-Spamd-Result: default: False [-4.49 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[66.111.4.29:from]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm2,messagingengine.com:s=fm2]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[f-m.fm]; DWL_DNSWL_NONE(0.00)[messagingengine.com:dkim]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org] X-Rspamd-Queue-Id: 4Sy84Z5NBwz3N61 X-Spamd-Bar: ---- On Thu, 21 Dec 2023, at 17:32, Mark Millard wrote: > [For this message I'm replying to just the one point because I > expect that it is rather important to your context.] > swapfile write requires the write request to come through the filesystem > write path, which might require the filesystem to allocate more memory > and read some data. E.g. it is known that any ZFS write request > allocates memory, and that write request on large UFS file might require > allocating and reading an indirect block buffer to find the block number > of the written block, if the indirect block was not yet read. Are nfs writes the same as a "write request through the filesystem write path" ? If not the same, then the same deleterious effect in the context? -- From nobody Sat Dec 23 19:41:23 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyF0P0LJfz54vn4 for ; Sat, 23 Dec 2023 19:41:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyF0N6DKcz4GRs for ; Sat, 23 Dec 2023 19:41:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703360484; a=rsa-sha256; cv=none; b=OPdoJqbH585lU9hygOAjDTsU4EpDNy3Wk/xTpp7dnF9Db5Pcu+Z/xRE8bT2LQFfXLEXztv sSYSiF1FpxEow3QX5FGNnf+K1PeVJKDqE57OQIPGUVPGsmPx/qXPNXR0soSl5kzDy2SvaK 9pVp71PkZjTdchEJCIFxqAnFTFqo8JdMbG7z0QrO7hmATcByw50YnQe4MobAtIxrR+7yOH 4QQN27DHaCxWHvcqhpfzqLtmUQINlJhrBbpF0sSyiaeQqNcp+hzJCk+YWn4WmZ4SbAxlyn BZyLdHqSKiDLBDRS0AOL8yivbmBRpBBRNRpcAaQHHzzx3mMA/6BcZV1FJLFSIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703360484; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zWHVNcpUWtB0P/x/W6qJ7BGNB7uCf1BolXHg0H03l9U=; b=c7EYGXshI6IGZrfaAq9AZUYSsI8nxDLK2/GCQSFmot7v1U2ezjmhnNFE2QjNBdOJiyNCg0 rDkt6c66G9bxc85aI06YsbrDnOD5j4TZ6BGXqPeeXM8urhqTc3WoLlkHwgA0ZXp4rHb2nQ nN9ynwfgJaI97WWjugU/gilOMWCYSiLmjkJcL5s0oTaYBJid4YhA4h4y9ulWK2lYtSGq45 rWfdUxpjqZ+4UKK1mFyIicTbjXkZKYZNzm/7oi8/ElzDEHgoby+y6B9Le6mpTxi4SJxya0 NkAASmOHA29b6ZPCOIztRouKDIh1HgYW8d+4lUU8yYKTT++z9zP7+bqhonMLZQ== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SyF0N5JlZz1C9S for ; Sat, 23 Dec 2023 19:41:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BNJfOYs062193 for ; Sat, 23 Dec 2023 19:41:24 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BNJfOi8062192 for fs@FreeBSD.org; Sat, 23 Dec 2023 19:41:24 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275597] Samba: smbd sometimes aborts by PANIC when 'vfs objects = cap' Date: Sat, 23 Dec 2023 19:41:23 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275597 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mckusick@FreeBSD.org --- Comment #4 from Kirk McKusick --- (In reply to uratan from comment #3) Thanks for your detailed report and final comment #3 that details how to ma= ke it all work. For completeness, I suggest that you close this bug report as you have figu= red out how to make it work. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 19:55:43 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyFKC6byWz54wR8 for ; Sat, 23 Dec 2023 19:55:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyFKB5w6sz4HTY for ; Sat, 23 Dec 2023 19:55:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=L8BgYS7O; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703361356; bh=s9hDYdZ16H4ZlCeSE5Dgs1fOIlAtcp4nvYxQm1lKIZs=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=L8BgYS7OsG341Slq/B0pcaXVjv8gwaIMaX9Ct3hyY80YRxSaTRbhKMSGSVOt3+gMTlL/OZreaSQhq4d8pR0PV1+DstKtYl4+XU3XhBWuEcMBZX9OkmBfNJy1/XHK3y5dy/38uCaZMZNR44QaulTuHIj/a/2cb78ScGRmrzVRxvK6sR1uqUEPBtVhjZryu0o1JIoLimNq8boJealt8q5EoIM6j9ixmEU34ZuPEQZvGHgAyAwHJ2aVQ66jtNuSx23VQjCWRkPg71NUgfd+veMljS56zh+ceO8NgTcn2LdyVzByd5zlvmrCyWwUcny0sA3RGrth0DQ/Hfdgrbevy6M17w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703361356; bh=kpCrh2sBF5WxjfNl87/AWmDoiWnkf+8Rk8YlzI2NB/S=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=I+ThOl9BhOXKbz3giGEQAPIz7kvEpbYcBfo/yu+EL0F+ED+AvkwLz9wrxMzbliPOTdiF2pHtzv3412JitBDDltV9ILM17VfyXZUsVr3FDHt+Qklwj2L46ANbtX4s6oVBHerw9I0dPdNtgkXLki7zEPG3xYA1WcEOujBHcvh2ge+r2AP4X3ve2X3z/IXmJqc2XCayj9ZnsEDerVpbuqsPsqy0DeYQ9v97NalKw4TWhVjmQH+RnLs+4sz2xfXQ/88arf3NfjNg6IukRY/hyZD1m2T7m2/p6ZnU1IZbI8kvqtBxRgrfRvii/hRq44V0B5L9eU4HqodFncrwGHYbC09tzg== X-YMail-OSG: 2B5..v0VM1mcuGhHEkKv2LbeOTYOjdJU_YcPGUlQiC8hwKZf49RMrnHM2TxwI42 tuAib5VNCYhjZviWOLIClT7mBz2tA0z7tKal8diiwWq923_PkVxLQr8uMovFzlJkcI.rqFOhCE_4 jLQ0bg2zJYa6V6TbndiA29LoXObiw9mTXp8oNZGrhD0ugR0vWLZoKZKYfrI7WsUajeET5wsenQIs 3oEVzBYluw0ZtRz8N5jI0lxfsa4FXAZTnGDbTwQ0Qbr.P85kY6kq5UaV8gJtNd9Wufinb1nQxQEZ pxK1Eq5JCjcZWly6XL1fRhw69KyIdEL9iybJaDWiPw245p7MBLmfdTYH_UDrl9XBjkvTw2dxdijK uk2ZXkV1Oq10F1jM9ki6KZ65K_VrEpQ5zHA5gRVnrMqq6V1dZdVBGqEF1Vq35bg_Czim4qGsDYIW qqOetWgjR.w9lP63HiX6uo.16JXMYfD6MlE3I1WD9egIpeD9qrTpA8TC.WvC904CBbIRe3T1EOov oFSk4Z1Ufg2_jLqXYdRhXOhmx38lFKDrnvDuGTE9cTLiMg0L2U71HWFuw.mKO154TMUdtZzvqO3c mZcLeo9mkGLAddXAvhLmmSRruPc3fHb0iN0QWr43UmpWC_piRMXHzdi.l02ztNzUHmsf02xoC1D8 l5cdSc.eDEChgzfpfxlxNJQP.qY7DGREuORn5U1gglxugcUx4TCHAQi.oeDeB38S73Faow4cFu4I 10sx0n0QQClbj8F5nzWsMLcK.ollaLCwEC6rt6SvINlyWCFozQPyiiOjFI9hRU8Fip8YRUllAG3a NT2NfQfuxk4ON0y6Y1YO_7WooxocojFh0A7wZC73z0nUQ1HVlr9vnaQz10XN4A1VEgcuj3KtKaqN XivIzhj0gGEF0mZ0OsKhK3Lej5dCZ1u4vbcwLRs5hGKt5hwo4.yQS4HQVjmvVuNnHb1i1foHYlym mwwiYZpZyD6iafVzYEiMZkf90m9lW5Y4tGKHx3I_uwqnmxL5BPnY.evd_P6YCnR5zzypSc2eIa8j p3O1ddaiI3qHMzfNmw.ZRxcyPNJf6s4NhzYyFQoFI0w46stTwOonQeXGY_oSwKn3jXnB9XG29AT7 TQbg4gUgkDwT1WYVA42o7WpA4pq2IUIqL8jIoeUIu0ETFoRoxE92jgZsm4CCMbvs8UHwid9xB0B. Teip4Q_g7pHPPQCj7UYRlZG9KycSG58yY2OjjTYn4PrzcO7JU9cMrWTfLDN.u9pvf1lyEDvnPZd0 5SXJyHlEx.ZmfR_CXpXSmsIBIwuG5Sg3.uOIeGUZWX05vUhpEdVUfadWBvWkkf7JFKluuq03x.7D Hb9DbvS2nVyZysvFoKnFnoFVGCmtxBKjEOIhXc83g4BZDi0ybyo7beG6klgwghcEl6Vp1pvFQufy UtI2A9oYER1NcRyP.I72tX6QwNa9jFXqsS3klNr7ogEDqD9BMkR_i3368u1c4r_AD20c7XwPXo1k 6fGzoowYK7xegDqwTcViuVOxrnYxRES2kOU1MdVZ5uOg.5CSvyQyU2471bpLecYlWKzKtPUo3USk n2iSDdKtPBl1870E2TfQeKAX.sBuXnO3gdx5wDacf22FT3CJk4QrOLX0kBeg0mz8hfrpkfZv.3rX K3cdf8cG0PshmVtL5_Y_OVuHRXTHbWAfPwACnGiyJ22d7a8pfInek_edTRq.cfZ20L8wCz7B_ZR3 T5ocAs.RKoPO6SkXuq1OWL8e8sxzBY3fTwQYsg6W4iAhmc1sKrr4RoFEwNRROVJcSD5WwDqiywmR 4V6y6rslRp5pBX76yq09KwXEhFxs3j5x4L4MZ6Jz.gMy1Y7TjO5yHN.zOmzfVU3MAHxeoYUwOM7i PN8KTVnrC0SAQ_XaCBL4JX6t6yXePE_NZ94NsUKAxCyzQbhkb0cism1JIPIVJgxrHuox9WE1Y1.D DBk622xU8xEEUePmCFG.8emSV1euWPlzopOWWy7vcViYn7V6ZrbspNnjqp1yFMlrPzxlG5sHr3CJ xkvvfFtRVXnpOBF5iwrpRp.dhpXUG7CGZWB2wAydTjv9TuharSx9wTxVeyUZQ6CKqKOCcGct9N_g Kxl_Ir1wZofu1_F8vS5X6KqNE5IQgVEsYGhghqRZqkvPstTVFu2cYOK8uKMWHmanYre6hpMNI1Rt HaekKZvc5dRztxHWIPbC5VS1qghGf6XC.HACN4YzWtvM1sq_n3zAfSoPxEk0t0S8iqdGqjBrzKAn q8bwxD7GuLesG3_TxO67wrxHQ2LkPvgbOBAVuD9OVwKbhSavHUYihrHfoXNDvpbtu.HuU4Hkp4sc m X-Sonic-MF: X-Sonic-ID: 5f096bad-d59a-455e-a00c-f4d26a533b35 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sat, 23 Dec 2023 19:55:56 +0000 Received: by hermes--production-gq1-6949d6d8f9-dpfkp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c555999766c6a14270b03fbe6c7207d7; Sat, 23 Dec 2023 19:55:54 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: Date: Sat, 23 Dec 2023 11:55:43 -0800 To: void , freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyFKB5w6sz4HTY X-Spamd-Bar: --- void wrote on Date: Sat, 23 Dec 2023 15:52:55 UTC : > On Fri, 22 Dec 2023, at 09:37, Ronald Klop wrote: > > Can you provide more concrete information? Like: > > > > # usbconfig list >=20 > # usbconfig list > ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=3D0 md=3DHOST = spd=3DSUPER (5.0Gbps) pwr=3DSAVE (0mA) > ugen0.2: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DSAVE (100mA) > ugen0.3: at usbus0, cfg=3D0 md=3DHOST spd=3DLOW (1.5Mbps) pwr=3DON (24mA) > ugen0.4: at usbus0, cfg=3D0 md=3DHOST spd=3DHIGH = (480Mbps) pwr=3DON (450mA) > ugen0.5: at usbus0, cfg=3D0 md=3DHOST spd=3DSUPER = (5.0Gbps) pwr=3DON (36mA) >=20 > The power supply is the approved 5.1V 3A one. I use CanaKit power supplies that have more margin for the power: 5.1V 3.5A > > And the output of "devinfo -v". That will give people much more=20 > > information to work with instead of "anecdotal evidence" about your=20= > > hardware being usb3. >=20 > devinfo -v : https://bsd.to/ADPj >=20 > detailed dmesg is here: https://bsd.to/Q5tx That last indicates another potentially significant difference from my example benchmarks done for comparison to yours: GEOM_ELI: Device da0p3.eli created. GEOM_ELI: Encryption: AES-XTS 256 GEOM_ELI: Crypto: software That likely contributes to slowing down I/O for da0p3 and possibly to leaving less time for I/O to other partitions when da0p3 is in use. If you had the resources to test avoiding the encrypted partition for your type of media, that might prove interesting. I've never had occasion to want to use any encrypted partitions for my FreeBSD activity. I'm unsure of the GEOM_ELI processing-time contributions vs. "gstat -spod" reporting. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Dec 23 20:11:40 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyFgd4FBqz54xMD for ; Sat, 23 Dec 2023 20:11:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyFgc36lzz4JP0 for ; Sat, 23 Dec 2023 20:11:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=VzdbFlTb; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703362314; bh=NcncRRGjCfcQxL0MqAQPJrmhMlLg6UZUxYBguwU4UqE=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=VzdbFlTb9SsYTgSgerWQAE2gHvU5Hd9ljneLNBlisvB+xlZbQQ/4wwMUHYqUtxd671oKcR3FCFmMZkuEQTAnjjom7SahPJt9zGJ3ceHpDgiPFRUob8M/foZme0rclp/aPV+KeTaqIBUp1GY7b3vjPQV+E6Bar8uGoVaOp47TEcSX8MvWz21PVcnfHm8OUlLsx2fEoehF6i+TE53xe7q9frWtLvVedAnwVqFGcLl4qgtsg3nv7umzEnBrOWrBJeEbNX27kPyQ2B9mmJKe+/YfUGdFpahL+wc0tJQK2b/mw56PfC4P7G42D087M+OnJIEQxPXK+ETXKgoXN71WKIkg8Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703362314; bh=PB+80AQtznWre/hxty36W2IX+ExUckTgf8pjcKTtwpq=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=op7lwwkgg2/7KIpCiGlpCcmHQK6Nsa6/SDm7sIvifETm3AMa2iSns/G34ua5uq0udskytCnlQZrd7optNYjegj72aJxFySYeusyguKUGyi6C9Dh5RCiuHul33Ql278PgVjgYt2uLg5jR6wqRm3UTWP7q8RMud+g9LtaFds/5DWFzId+5GkB6jPSpoaZ2zFdv9SbnT3BbPYXYBQnlGGDIUqq4dVSOeN4Le3aKGTkS+nV54IO90fXBvvaXaSSvXA+V/1edq2hqedteGw8JO3ejmlg11xAD1463Dg3N+x34JJwFI1pX+zk3g5uz9JnGN2q1Z1r+076NduF23hgnRJbCtQ== X-YMail-OSG: 92RbzCUVM1lHsjRABp6olI3cHQOsi6.dbazH57fVsQEyNibAm86MpQUT8TXQEnu nocEqLFZsZMfEL.n0wTKCwhff6H6.fiEdpVJUePHZ.OnpSmtln5Ka3l85Ir4vgCFMITMSZeJ4DGf YHi1d7HKUbDpgzV4Dp6JWPoRS41gp7ju9DE4PFLnV6uEfMGSQTg0WsPqrFa6A5DgWW_PX3tNebw7 TjXEG24mTwTO5eq8rvqbZbWCKYWTcXj2aFZg__sy3LUWVIX7evm4.vbdZTagL_6vJ47sqsLcMCsP 7mm8xuc9dAaf8qzYa_3QJ5WHXQiRCkFHCrTSZA90MhZOZdhPKGflMl_j5MYsC3wcgw_l_4U4a65r g1hRI0ltkhKjQr4eK0yBv4d61yigGGrjOhgX297h5pKvocmww8EcAYRIW3NsQCoH2OJfy3hSQ.kP iyLcOSNG.3_Yzp_KpjjJ1Wxf_9MzTsGUTjulhzXUvCzHrZMu.ZFNPEleLAiwqfKAEj3aBozcWgju KsfvauMFTbMIgORH6ZeYi9iq5aehu851Qa0tiXXon1PT8ruTFsIaE3nt90wgAQ07fvqV.WMor3Th wDJ2XTX7kIgO4TV32nC8rDKMyFp1nxVVzoTFshacH3uMFOFOCJMGI7tLAFZwyAlO.fhTq9x90FHG oLLLC5iErZ7iL2ha71YH9izGMzxvrV1aKJL22azFV9L6eO2rmp5mDCAtNZ7FEWoA84wvae2tJPp_ SO72mZtpKnaaX8gO81le732JHpgFN8TjW4ZRwXekkCzq.yAhEGg6rlJYzdaAqabbKUieABmg2V0l LsKWU1EZ5eFflIWpUSnCMibXvN4nn0C6hLbODM2xH.66v_70unttvvtiBMMcQZVWa12whUozQ6JV 3Dp1ETrj6fCBkDEoU3wDTUhnrJb37_xdFNEycUNk3OCFbGFw7xUgOdBC8ZVw0UupqWiQQVgrW8Ns wt56Wy9PVn.turKsudq6ZZNMQXvQU6Fs7tWt_4QuCYJju9dO8E.R_kQG8TU1aHdhQo.2QCGUKMZb oSVsE1UTLpw8FJztAPqNnN3S7RoWI84w0CwOYJyRSsVmLBcynKM5DX6NlN36EzOxfSym1EF.5Tju SHqiQhzHkaIsOGx.gumlo_qR2JriuTvt9m2exTdO1Z6TcQNlSp3OHwfTrKl87Ow3MF6w_EQ.mcnt UZIsSexATDSGk26kYeEPnNd6rz0WKmLzAGXmtLBICq90HLw9M_.cziL_0J0AaWyoL0p2ubmU6ViD vifP2h1vlmePxqdgb1EEhVx7hFe0VPJVDQc_noRPwyFd5KeHYGNonJcOuMlqH.S6wYy5otGOBvO. HMthSkwcxdc00Z5UnNSfYZ7RahoRnzIKiocTiIus.qbBRzdEqK5fJhDOC73a84xbmA7WBXAKyKtC yHAzeJTKbdhqHalrNdqY_lqPRss2fEis7l9WJQi8DCf9M7i5fISex9YuIumqV.CRPFtyci_kcuW5 1NAcb8mmv2riSkUKVvFBu_jBDjk7pRmNA4fnTi2YXBZaFjm_RwFz9e3bS2LQIQc7ZiT.XVC4UMqW DxPtfa2tJ72a2aspv7rF3QxCnR8B1Zj7pL8CsuVFK4KKJuRljUXoz59Oisb28kLOBOzXPgS.DlqW Sbg3pBvRzir4z72NPGFGpfn4TzwTi9XA5EX6dDyvGKpmV0DUO4yqlm.DDF9I8S7xxB3gHPKjZ0CD 6XhdOh1E0kAnNViD8Xgmyr.IugKjZijz9QzNq_EKYFoP9A9NyN1e1FLaGqVF8Ee2n4tjxjOO9c.O lTmiAZFNxUlpKfTBQ2mCUmndPzrxTanzofI0FVvlnzRNHLDJUg9cgO_DX2nTYoWXWXkBeT_EdbD5 KFw0g.DXowT5XwkTe5ugHS9iaqIuBTDWPnE.b9ntnepHlnKnrttcLD5PmZXAmNtZfoeuaTDD2UXq XlRvSVOYsvA78crx0644dyIiJp1C2BUwmfEJmiIyCA1Ak4FtgSZL9771X1QxzMo40edRrO9cqWPb UyhsYuXGvopcO7XV8sJYgUIKg_ZdXrUaULtkHBsoexAZBfRc_OKZPpQqFNPMdwneJQJ_9Yk.BmH0 zgZJQEEAXKcrXhYB42ZlMDHPnUBjo3hsUmZxFWx0EdUQ_k.ZsvjnHw2c8ZFdKO1e7H5LntopJUs1 dv8IUKwkc7QtortCTO6tp6u4weKJDo_3zvY0u9ewlkoclDj624VE.CBsES9xb.LUZ2NWSKoyZu9. BUo8l9feM5OFa_KYS.Q8gnSRUCqoYR6RBT.oXhW_PKaCeU2mP1ZcBDcEPIT45aYF6Z5PdpIsKVGD X X-Sonic-MF: X-Sonic-ID: 3ad4c935-b135-4546-8c89-48f921b31f47 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sat, 23 Dec 2023 20:11:54 +0000 Received: by hermes--production-gq1-6949d6d8f9-qkzts (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c612515a163a443a2ac6ae3ca0ab9b1d; Sat, 23 Dec 2023 20:11:51 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Message-Id: <8DD13AF2-C4E3-470E-B748-92828FC20D17@yahoo.com> Date: Sat, 23 Dec 2023 12:11:40 -0800 To: void@f-m.fm, freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3774.300.61.1.2) References: <8DD13AF2-C4E3-470E-B748-92828FC20D17.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.147:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyFgc36lzz4JP0 X-Spamd-Bar: --- void wrote on Date: Sat, 23 Dec 2023 15:59:21 UTC : > On Thu, 21 Dec 2023, at 17:32, Mark Millard wrote: > > [For this message I'm replying to just the one point because I > > expect that it is rather important to your context.] >=20 > > swapfile write requires the write request to come through the = filesystem > > write path, which might require the filesystem to allocate more = memory > > and read some data. E.g. it is known that any ZFS write request > > allocates memory, and that write request on large UFS file might = require > > allocating and reading an indirect block buffer to find the block = number > > of the written block, if the indirect block was not yet read. >=20 > Are nfs writes the same as a "write request through the filesystem = write path" ? > If not the same, then the same deleterious effect in the context? I expect that NFS would have the issue: SOMETHING(s) might require NFS to allocate more memory during the = attempt to do a paging I/O of interest But I'm no NFS expert. Basically, any additional layers that create additional memory allocations in managing the paging I/O contribute to such issues. The more allocations in trying to do the paging I/O, the worse the issue. Even just the fact that umass is involved for the USB I/O contributes some risk, if I understand right. (Seems to be rather low risk of itself.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Dec 23 21:14:22 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyH3g6HBxz550sq for ; Sat, 23 Dec 2023 21:14:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyH3g4bTBz4PYn for ; Sat, 23 Dec 2023 21:14:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703366063; a=rsa-sha256; cv=none; b=VdTGg1qUTxL6ThqXrqFuTjIjHarY0OGd4yafGCf5u66yIEa2DqihC524RNuT9eDs+9mH9a +EwfCZSa2niMraNLV5+EQcrKbq1GaeDDunQMoeOrwunJ6ajLF/TUxEiO9Hn+oE+MnioLjy wR3W9ODQsWQlFA9ra5akoeh60k4lmbCgjHY5JSPy9WA4ABizIaR41/27zlyRE11Tpv0z3b fxlrkFNd0FSGZz8GuWYsmREybD6bSKEPP9RSYYx8VtMfrFVRuAs1HVLTHmsYwN40lVSQUG JL2ClMdyP9R86M3Em8nprS+33mI/9a6Y1SYzpmMa1pHj91MvHVqlHiSQcMUEOw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703366063; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dJO+L6DkV0Vd/5E8wh+rBv4P923AH1GTiCs3UKc4Fas=; b=CrpCPG2IH+/USVYeCwfoF/3mYCcwBdrIqdvcX9YTO8gT4k0EPM79PlvlMSLLorszqTkAFb FHw2rKV46tzkzmD9vviJRIKpSUQMs8AwdK98pfWfHKmT0EUqhkrjysnqzVvusUCP8pqo03 poS6RwpqPVWSZSBCdUQkKFzkrX+2J2egoxwj6bJ4kw0o+TezTFHXhVP5Qal9Hz6jy2cN/g pr4f55k5BfjMlPqnZQUlMbSzxSHzyxUEhUUoYPHkuVo0mcm6LW8XUS2+WsuyLgrnnZBCQD whbDkAk5tNtSNGsSMJ2aExx6N+0ZRQe1kGUjRmMsrsYT18bnsuLjABGl01KM9g== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SyH3g3fl2z1FWS for ; Sat, 23 Dec 2023 21:14:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BNLENG0010260 for ; Sat, 23 Dec 2023 21:14:23 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BNLENuc010259 for fs@FreeBSD.org; Sat, 23 Dec 2023 21:14:23 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275597] Samba: smbd sometimes aborts by PANIC when 'vfs objects = cap' Date: Sat, 23 Dec 2023 21:14:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: marklmi26-fbsd@yahoo.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275597 Mark Millard changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |marklmi26-fbsd@yahoo.com --- Comment #5 from Mark Millard --- (In reply to Kirk McKusick from comment #4) I'm confused. Comment #2 includes a patch that has not been made to the port: QUOTE |#define Q_HACK_U 1 |#if Q_HACK_U | newdirent->d_pad1 =3D result->d_pad1 ; | newdirent->d_namlen =3D result->d_namlen ; | newdirent->d_pad0 =3D result->d_pad0 ; | newdirent->d_type =3D result->d_type ; | newdirent->d_reclen =3D result->d_reclen ; | newdirent->d_off =3D result->d_off ; | newdirent->d_fileno =3D result->d_fileno ; |#else /* Q_HACK_U */ **115 | memcpy(newdirent, result, sizeof(struct dirent)); |#endif /* Q_HACK_U */ END QUOTE While a more directly useful variant of the patch would be appropriate as an attachment, this submittal is reporting a bug and an example source code fix/workaround. It seems early to close this bug report. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 21:19:28 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyH9p27Qkz551XD for ; Sat, 23 Dec 2023 21:19:42 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyH9n42pyz4Q8j for ; Sat, 23 Dec 2023 21:19:41 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20230601.gappssmtp.com header.s=20230601 header.b=fXBBumwa; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::62d) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ej1-x62d.google.com with SMTP id a640c23a62f3a-a234dc0984fso305453766b.0 for ; Sat, 23 Dec 2023 13:19:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1703366380; x=1703971180; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=YVvYFyS/TB5JqAVd5J29h47ymdD/CyXCkj3tXa/ec9Q=; b=fXBBumwabSHgmV5baRAMPkl835iPNjQ3T8okPx/tXLYuClKpnEoQUxjnOTSt+dWHOK CvS0rwCtz9LikwYC8YnDyWeZ9W3d0SeCICoeGLOgignOQ5HiPpPM17coOU4kzVzEZvty dwg/hymvKdQx3EE0VVwE6lQfztKEzbpywQ4jyPglmgNzFJ5zu/aupxS3lEfq1RGu0Zqo I0JWg+HFJ4v6MeC0y8FdDZx4Cd6ExBEvdL7WrOrgZsIE1vXwncVpZu3Bk9jdQiqTpR2u dzakpPOQ6HMnJon0okhFRPcxNM2RNhE7VdXWMZvOmgg7L0KHLYtPMMZ+COPIlVOGbFGZ 2oJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703366380; x=1703971180; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=YVvYFyS/TB5JqAVd5J29h47ymdD/CyXCkj3tXa/ec9Q=; b=M1BUl3vgALBSVZSDJUvjYUt5B9SPvnR40MnRUcxnveg3Jnu31tcbLQ2Q6IjhpxDxdo rmt1cJdcGIe+xhnJXRFdBE4EnV3PgVf5JHdSI3cSx52jW9LLN/RKgH+Lr3iVvVK143E7 sf5i1DYW9uH1Jx8dW33dt19winISpXlfAIKWr0Lh0uIcweHhbkVn/cQY+u5Pdh5mZzEd brdT53OE7cL1UVbMpMdeo5+ePI4g84Ai2kLZZLGxPMbyTJVsfs90hzRESUgeUFiKdpLZ fxKn9qUq+9waO6D/IvHgUaXp08Nsi15I5qisiqHoQyNFTyP0tGPexEv3cDd0KbBjhES/ izoQ== X-Gm-Message-State: AOJu0Yxpcb6qm7tjig0muQLBoWSIriSLuiL4rpradE+jyOsIm6AgqNor C306knAflcqN700vFc8DpDnvIGnTvK5cIrX/5nJYtBucTfTCZjTtJc26Av/ag7U= X-Google-Smtp-Source: AGHT+IEscmjia9edp+BW3RM5Nl699dYmTBLsX0RBdP3Uk//muMbVdx1Bl8sD8toQtrSIgCBnmAYI0cHn6aFvdDH8aIA= X-Received: by 2002:a17:906:6954:b0:a26:97dc:67d9 with SMTP id c20-20020a170906695400b00a2697dc67d9mr1732723ejs.68.1703366379679; Sat, 23 Dec 2023 13:19:39 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <2dcc21bb-f446-41fa-871b-2c37cdf9afe0@app.fastmail.com> In-Reply-To: <2dcc21bb-f446-41fa-871b-2c37cdf9afe0@app.fastmail.com> From: Warner Losh Date: Sat, 23 Dec 2023 14:19:28 -0700 Message-ID: Subject: Re: measuring swap partition speed To: void Cc: freebsd-fs Content-Type: multipart/alternative; boundary="000000000000fbc1c2060d33e351" X-Spamd-Result: default: False [-3.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20230601.gappssmtp.com:s=20230601]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; FREEMAIL_TO(0.00)[f-m.fm]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; R_SPF_NA(0.00)[no SPF record]; RCVD_TLS_LAST(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62d:from]; DKIM_TRACE(0.00)[bsdimp-com.20230601.gappssmtp.com:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; DMARC_NA(0.00)[bsdimp.com]; TO_DN_ALL(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com] X-Rspamd-Queue-Id: 4SyH9n42pyz4Q8j X-Spamd-Bar: -- --000000000000fbc1c2060d33e351 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Dec 21, 2023 at 5:20=E2=80=AFPM void wrote: > Hi Mark & Warner, > > I tried using fio with Warner's suggestions as a template, and > pasted the results in the latter part of this message. > > On Thu, 21 Dec 2023, at 21:03, Mark Millard wrote: > > > # sysctl vm.stats.vm.v_page_size > > vm.stats.vm.v_page_size: 4096 > > Here, this setting is: > # sysctl vm.stats.vm.v_page_size > vm.stats.vm.v_page_size: 4096 > > > # sysctl vm.phys_pager_cluster > > vm.phys_pager_cluster: 1024 > > It is 1024 here, too. > > Only the one usb3 port is occupied. > > fio output follows. No idea yet if the settings used are suitable for thi= s > context > and would welcome suggestions. I think --filename can be a device? Output > suggests > it used hw.ncpu instead of --numjobs=3D8 > > ~~~~~~~ > # fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D= 1 --bs=3D8k > --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 --numjobs= =3D8 > --runtime=3D60 --group_reporting --thread --size=3D2048M > randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, = (T) > 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32 > .. > fio-3.36 > Starting 8 threads > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > randread: Laying out IO file (1 file / 2048MiB) > Jobs: 4 (f=3D4): [_(1),r(2),_(2),r(1),_(1),r(1)][21.2%][r=3D288KiB/s][r= =3D36 > IOPS][eta 04m:05s] > randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D135125: Thu Dec 21 16:= 43:00 2023 > read: IOPS=3D43, BW=3D351KiB/s (359kB/s)(22.6MiB/65986msec) > slat (nsec): min=3D889, max=3D1948.6k, avg=3D4172.45, stdev=3D36488.0= 7 > clat (msec): min=3D1108, max=3D11660, avg=3D5644.23, stdev=3D1282.12 > lat (msec): min=3D1108, max=3D11660, avg=3D5644.24, stdev=3D1282.12 > clat percentiles (msec): > | 1.00th=3D[ 1183], 5.00th=3D[ 3171], 10.00th=3D[ 4933], 20.00th= =3D[ 5269], > | 30.00th=3D[ 5470], 40.00th=3D[ 5604], 50.00th=3D[ 5738], 60.00th= =3D[ 5873], > | 70.00th=3D[ 5940], 80.00th=3D[ 6074], 90.00th=3D[ 6342], 95.00th= =3D[ 6812], > | 99.00th=3D[10671], 99.50th=3D[10939], 99.90th=3D[11610], 99.95th= =3D[11610], > | 99.99th=3D[11610] > bw ( KiB/s): min=3D 208, max=3D 3760, per=3D100.00%, avg=3D1535.05, > stdev=3D128.23, samples=3D245 > iops : min=3D 26, max=3D 470, avg=3D191.88, stdev=3D16.03, s= amples=3D245 > lat (msec) : 2000=3D3.25%, >=3D2000=3D96.75% > cpu : usr=3D0.00%, sys=3D0.12%, ctx=3D22712, majf=3D0, minf=3D= 0 > IO depths : 1=3D0.3%, 2=3D0.6%, 4=3D1.1%, 8=3D4.9%, 16=3D69.6%, 32= =3D23.6%, > >=3D64=3D0.0% > submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, 64= =3D0.0%, > >=3D64=3D0.0% > complete : 0=3D0.0%, 4=3D94.7%, 8=3D2.9%, 16=3D1.5%, 32=3D0.9%, 64= =3D0.0%, > >=3D64=3D0.0% > issued rwts: total=3D2895,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0 > latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32 > > Run status group 0 (all jobs): > READ: bw=3D351KiB/s (359kB/s), 351KiB/s-351KiB/s (359kB/s-359kB/s), > io=3D22.6MiB (23.7MB), run=3D65986-65986msec > # > 5s+ of latency on the average, max latency of 12s! Woof. No wonder you hate life. Warner --000000000000fbc1c2060d33e351 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Thu, Dec 21, 2023 at 5:20=E2=80=AF= PM void <void@f-m.fm> wrote:
Hi Mark & Warner,=

I tried using fio with Warner's suggestions as a template, and
pasted the results in the latter part of this message.

On Thu, 21 Dec 2023, at 21:03, Mark Millard wrote:

> # sysctl vm.stats.vm.v_page_size
> vm.stats.vm.v_page_size: 4096

Here, this setting is:
# sysctl vm.stats.vm.v_page_size
vm.stats.vm.v_page_size: 4096

> # sysctl vm.phys_pager_cluster
> vm.phys_pager_cluster: 1024

It is 1024 here, too.

Only the one usb3 port is occupied.

fio output follows. No idea yet if the settings used are suitable for this = context
and would welcome suggestions. I think --filename can be a device? Output s= uggests
it used hw.ncpu instead of --numjobs=3D8

~~~~~~~
# fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D1 = --bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 --= numjobs=3D8 --runtime=3D60 --group_reporting --thread --size=3D2048M
randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, (T= ) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32
..
fio-3.36
Starting 8 threads
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
Jobs: 4 (f=3D4): [_(1),r(2),_(2),r(1),_(1),r(1)][21.2%][r=3D288KiB/s][r=3D3= 6 IOPS][eta 04m:05s]
randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D135125: Thu Dec 21 16:43= :00 2023
=C2=A0 read: IOPS=3D43, BW=3D351KiB/s (359kB/s)(22.6MiB/65986msec)
=C2=A0 =C2=A0 slat (nsec): min=3D889, max=3D1948.6k, avg=3D4172.45, stdev= =3D36488.07
=C2=A0 =C2=A0 clat (msec): min=3D1108, max=3D11660, avg=3D5644.23, stdev=3D= 1282.12
=C2=A0 =C2=A0 =C2=A0lat (msec): min=3D1108, max=3D11660, avg=3D5644.24, std= ev=3D1282.12
=C2=A0 =C2=A0 clat percentiles (msec):
=C2=A0 =C2=A0 =C2=A0|=C2=A0 1.00th=3D[ 1183],=C2=A0 5.00th=3D[ 3171], 10.00= th=3D[ 4933], 20.00th=3D[ 5269],
=C2=A0 =C2=A0 =C2=A0| 30.00th=3D[ 5470], 40.00th=3D[ 5604], 50.00th=3D[ 573= 8], 60.00th=3D[ 5873],
=C2=A0 =C2=A0 =C2=A0| 70.00th=3D[ 5940], 80.00th=3D[ 6074], 90.00th=3D[ 634= 2], 95.00th=3D[ 6812],
=C2=A0 =C2=A0 =C2=A0| 99.00th=3D[10671], 99.50th=3D[10939], 99.90th=3D[1161= 0], 99.95th=3D[11610],
=C2=A0 =C2=A0 =C2=A0| 99.99th=3D[11610]
=C2=A0 =C2=A0bw (=C2=A0 KiB/s): min=3D=C2=A0 208, max=3D 3760, per=3D100.00= %, avg=3D1535.05, stdev=3D128.23, samples=3D245
=C2=A0 =C2=A0iops=C2=A0 =C2=A0 =C2=A0 =C2=A0 : min=3D=C2=A0 =C2=A026, max= =3D=C2=A0 470, avg=3D191.88, stdev=3D16.03, samples=3D245
=C2=A0 lat (msec)=C2=A0 =C2=A0: 2000=3D3.25%, >=3D2000=3D96.75%
=C2=A0 cpu=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 : usr=3D0.00%, sys=3D0.12%, ct= x=3D22712, majf=3D0, minf=3D0
=C2=A0 IO depths=C2=A0 =C2=A0 : 1=3D0.3%, 2=3D0.6%, 4=3D1.1%, 8=3D4.9%, 16= =3D69.6%, 32=3D23.6%, >=3D64=3D0.0%
=C2=A0 =C2=A0 =C2=A0submit=C2=A0 =C2=A0 : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 1= 6=3D0.0%, 32=3D0.0%, 64=3D0.0%, >=3D64=3D0.0%
=C2=A0 =C2=A0 =C2=A0complete=C2=A0 : 0=3D0.0%, 4=3D94.7%, 8=3D2.9%, 16=3D1.= 5%, 32=3D0.9%, 64=3D0.0%, >=3D64=3D0.0%
=C2=A0 =C2=A0 =C2=A0issued rwts: total=3D2895,0,0,0 short=3D0,0,0,0 dropped= =3D0,0,0,0
=C2=A0 =C2=A0 =C2=A0latency=C2=A0 =C2=A0: target=3D0, window=3D0, percentil= e=3D100.00%, depth=3D32

Run status group 0 (all jobs):
=C2=A0 =C2=A0READ: bw=3D351KiB/s (359kB/s), 351KiB/s-351KiB/s (359kB/s-359k= B/s), io=3D22.6MiB (23.7MB), run=3D65986-65986msec
#

5s+ of latency on the average, max la= tency of 12s!=C2=A0 Woof. No wonder you hate life.

Warner
--000000000000fbc1c2060d33e351-- From nobody Sat Dec 23 22:01:54 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyJ6q648Pz554PK for ; Sat, 23 Dec 2023 22:02:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyJ6p6kCCz4TwS for ; Sat, 23 Dec 2023 22:02:10 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=clBMJpts; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703368928; bh=5co/4Id38Vi5iy7gBb1pybqa6Xwbxaigq3958gYmAvc=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=clBMJpts0R/3OozL5RkjXuZrN+Zhz55glXZQtwgeuZ06aoSTuIb9jjO/AYaTQ1+ZuikS51wAGZxefQoWhbhG0F5jxgoIm6I5XS/4K7IlFEb4jJrlr20xrEnAiWKSHjasQqdoZj1aSgYuRG1SgTmUoxT23XZVIxMB4CZDnVRJdk/OxBEuoRZ2ZYZQWhxTFieB+Vp8HC2V4R6oUKrUU6MslnmqFjBw3RgtZZ2flVKNeP1aN3ouiBcE8g8AulBCUS8JqhFLn6ONv57rUX7f80Uzmnc4Qua5aBXsxjzdI4PbJ62vf4ob0VxSwAxD8JL3VXrAg59dOyDb6SNYLi7k6vhJdw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703368928; bh=p2+N4UJ9IKM1NhnqsAFoOOjQevkzFzXoBjy2i7cHUdz=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=De/Cyi7k0QHlI8elMrcHB7/3COs9s/hGN8Xp8o7+inh7lvEG8Pihi5/E2hCS2C/i2d181KRS+PHiYZkTT9/SP0wIWBRQV9hB6viFEUHZNA5Lkp+TeK+fsavXJF9En5iU94wCm9E40J+qCbTVGOl38VoPk0ZGFk7ybFFyMU+4l5FRDLLYkHjpjOSVaEERJI6V7hr2vxVu2SW/IHSsdEdoW8pDQY+1YmimLEi2aC1JmoTNad/mKgFNfC21YG/UO6m8/ccDm89894edWinzO/PnBAN01OV+Ue7dngLWKlTCgR7TFulXMTV0qc9vmTU9endXdXLwnjszfWb5tdp0tsUJqA== X-YMail-OSG: m8Ank54VM1kdUlXWqivmjGamPckfEx.y74n0q44yNgw4Dz61GqwqdOvlloAyLx1 8si9Wvjx588nEQgygeWNDEDswfgbGJ9IQG.onV5Ik2vU4JuwRqUF52PyYkNj6A.3uCbvUmja4IRV 0QQfGJL_MxajHLogFmoELcKgcGx09RKvKBYSTHIcxYFXYG7Yq2RRE_2UZ0Acxv0LNzIgMw7tIx37 dXJAUsgcqvc6ozFfHNYIgKkmVHNwcQZn_tWSwVIDkJIPT09BbVZCIK9wcXV1L1TMka8SQauBczOF HIrp7Co_JjzTa3gdEe55S31juX8KD9exA3_hgl4Wguuo62zaXju63GINZIum9mv71k8SKC1Dhswa qJvlAuM4t_P61s3_gqk0NTvV5U5.6CqkukQetH1s2gqmkBzFeloRhIKB5PMjqjob__HOjhnv0Dd4 ogFN7p5UtqUEgNBWG3vxslQ7x3LSfz1pYxfNYaoFHy9AP_lnprO61vJg6XJYx3Y45cWwPXJ6dZHz nCufU68nety.jzVnKpYpxWadrGN5mvV6XRqnFxyICuCmqpusm9hXS1eF0.Hq7.UOIHzZtAtJje6_ 1QsI3cdXWP28U6U_xb8YDPUsl8iXRzGrAFe7_j87O.nY7Da5IE4vaTqK.qPHph024imLGvDL7Trx 91t0JPOutdtzOSRS2XJ3kIQBF_TWSN3nGk1mt6SKeL4_x.pjREpsHDsdh5lLU8iHQ06ETPevohC_ b10ewn_3naWNFbjg819g1s3ygUM8ERXqfGc2soxhuxVRp7.gnbchkT9UYQmf2TT.Vdfcy7jnjjCe WNacsSJVOdhA2bWIM9HDqZZYrW.oPhe6AKi6Me75tggdha0ZuuXsOqOToeVMceYKYBda23IBPHZm FsHHtsLTa8ersbY53dE_cKGoXnonydyMdRwjmezXuzWWYMmqwnvXOliAGU9c.Pdnr9V87KX.aNJE 985K2_S.Z.a1BMUNW6lAZTAqW3j6dab0EFDwrNWx7pXsV5hrujRcUGpecjaeIoNI8ye6dWS0FvHR XewT7Uq8A12No4owJKc5YKJKPKv5.OYINNjTbsCxM_SDu6jfuBHtaAR1ZwNMe_Q5HF9x8xRUxjYf PA9uHwzKq30DhEexn8oP8_FquGoHbnRMnYWfdimVPQXZlJRtihlhL3SEuKysxnPj4snGY4GGT7XY gVZswF08WjP8ZDpVtQQgCRZhz0LxzDVBIFIw4_8OFnVvaFKiTSMa3Lw7SskCSg8ZAiU4mXTSxxgt OkTBljAvEx3fie0I2wdsrvcbF7jG4KrGs3RR1u8B1c8Hx3ptuZgqfYb4u_w7dqCL_vXtf5OcEZ0t f6iaLS24eUul.ZbxMrZQkYLA0z1LU3ziXFhm_68.9jO_tgM9Nan9D4ZFAaWFG8.10Ynmp4vcLV28 vC3lk6WW.d9H0FvjUfu_TNRqxEeubj5ZVKP0EnOfoEL0g.pkRCyUOLPGHspqDFOInpjlAHQNO5LN xSTSJUDf6NXFzkupc55b6dPHz4c7usGlgwMCHDMkSPxef29x99pjU.0HRI3tjCqLlLPzbQ7NUq2X pkyOhY43ErhARqbsObhw42UyK.J1HSAkUdLxaaTmq8FNraBYdQZVZTC10VT3AmQa9kpWAFw.mSpf 8g3CkaizFqgrHkpeHc7mKpiVdUiIVBMorg15XspZMJCIG5Kf0.ctQOj5wzWKBZeJEkCAgaSJAdDI zy1VcZV_ldQGltylMPlG3z3Jk09qwzWqhj1HxW94VKMPbmqfhR0Fov5rXX1..HbSNAlggsLPuh4m NTBVN_z37i_yFSCqDra_VmxJNr7mBDFQU5DtgQJC8KiLTWWdTJf.5Dl02NC6sm6xl7L17oUb8tBp MWEE7q1usHbykt0cbQlxxwa.sDZo8VF2VWzLg8ZwXR4cGUoV.pXnSiqnaPweHPu692ajAb28dB8s 2TzKHE6gEjnGHRtx_J5CfyqrNDNtubpSkX7bPiwLyyrT8vR1okGN_dbR4Q75gsfbOFmL.SXa6Hqr QJpK_TH_5n9HlMyrzaxd6wBQQ58rY5eJb4mYbvdqmy8MHOsi3TdJzRodPxNEOGkWSb0dbOTGPR2h K7AlkS1TQPgRzdCXk0Geh9mZEHoai9g4o.LbTETpdLH8jryT2nOuWzjixG48sqIo8EzdIBiLTdef 6l4yldr9Lcd9tk7jOKAsAd48qOddVciwCWnXnIcUxJwen0y3t698MoGkoaoduVGhYUOyY63N7T.F HMA433J.spQ1FUwIq.XXh8fMF3fvXOW18tCr6z_WFWWRWGVLIRxmJdESImFvsEJ36N3SKMA5YVox UAw-- X-Sonic-MF: X-Sonic-ID: 08219059-c4ef-4997-898e-c8d5ffb8a359 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sat, 23 Dec 2023 22:02:08 +0000 Received: by hermes--production-gq1-6949d6d8f9-pmzmd (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 35e5fd75b3d7c541da96d31b572c1809; Sat, 23 Dec 2023 22:02:05 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Date: Sat, 23 Dec 2023 14:01:54 -0800 References: To: void , freebsd-fs@freebsd.org In-Reply-To: Message-Id: <81D464BA-FABE-4F25-B2BE-3DB9D85E3E64@yahoo.com> X-Mailer: Apple Mail (2.3774.300.61.1.2) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyJ6p6kCCz4TwS X-Spamd-Bar: --- On Dec 23, 2023, at 11:55, Mark Millard wrote: > void wrote on > Date: Sat, 23 Dec 2023 15:52:55 UTC : >=20 >> On Fri, 22 Dec 2023, at 09:37, Ronald Klop wrote: >>> Can you provide more concrete information? Like: >>>=20 >>> # usbconfig list >>=20 >> # usbconfig list >> ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=3D0 md=3DHOST = spd=3DSUPER (5.0Gbps) pwr=3DSAVE (0mA) >> ugen0.2: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DSAVE (100mA) >> ugen0.3: at usbus0, cfg=3D0 md=3DHOST spd=3DLOW (1.5Mbps) pwr=3DON = (24mA) >> ugen0.4: at usbus0, cfg=3D0 md=3DHOST spd=3DHIGH= (480Mbps) pwr=3DON (450mA) >> ugen0.5: at usbus0, cfg=3D0 md=3DHOST spd=3DSUPER = (5.0Gbps) pwr=3DON (36mA) >>=20 >> The power supply is the approved 5.1V 3A one. >=20 > I use CanaKit power supplies that have more margin for the power: >=20 > 5.1V 3.5A >=20 >>> And the output of "devinfo -v". That will give people much more=20 >>> information to work with instead of "anecdotal evidence" about your=20= >>> hardware being usb3. >>=20 >> devinfo -v : https://bsd.to/ADPj >>=20 >> detailed dmesg is here: https://bsd.to/Q5tx >=20 > That last indicates another potentially significant difference > from my example benchmarks done for comparison to yours: >=20 > GEOM_ELI: Device da0p3.eli created. > GEOM_ELI: Encryption: AES-XTS 256 > GEOM_ELI: Crypto: software >=20 > That likely contributes to slowing down I/O for da0p3 and > possibly to leaving less time for I/O to other partitions > when da0p3 is in use. >=20 > If you had the resources to test avoiding the encrypted > partition for your type of media, that might prove > interesting. >=20 > I've never had occasion to want to use any encrypted > partitions for my FreeBSD activity. >=20 > I'm unsure of the GEOM_ELI processing-time contributions > vs. "gstat -spod" reporting. FYI: the 2023-Dec-16 stable/14 snapshot that I'm using has: # strings /boot/efi/start4.elf | grep VC_BUILD_ID_ VC_BUILD_ID_USER: dom VC_BUILD_ID_TIME: 10:50:39 VC_BUILD_ID_VARIANT: start VC_BUILD_ID_TIME: Mar 17 2023 VC_BUILD_ID_BRANCH: bcm2711_2 VC_BUILD_ID_HOSTNAME: buildbot VC_BUILD_ID_PLATFORM: raspberrypi_linux VC_BUILD_ID_VERSION: 82f3750a65fadae9a38077e3c2e217ad158c8d54 (clean) So fairly modern RPi* firmware compared to what FreeBSD used to use. What vintage are you using? IT may have different defaults than what my FreeBSD snaphot context reports. Another issue may be: bcm2835_cpufreq0: current ARM 600MHz, Core 250MHz, SDRAM 400MHz, Turbo = OFF bcm2835_cpufreq0: max/min ARM 2147/600MHz, Core 750/250MHz, SDRAM = 400/400MHz What does: # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq report for your context? The default for the FreeBSD supplied config.txt in the snapshot I'm using is: # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 1500/-1 600/-1 dev.cpu.0.freq_levels: 1500/-1 600/-1 dev.cpu.0.freq: 600 Adding arm_boost=3D1 to config.txt in order to have 1800 instead of = 1500: # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 1800/-1 600/-1 dev.cpu.0.freq_levels: 1800/-1 600/-1 dev.cpu.0.freq: 600 Then there is how I normally use all the RPi4B's that I (sometimes) have access to: # more /boot/efi/config.txt=20 [all] arm_64bit=3D1 dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don dtoverlay=3Dmmc dtoverlay=3Ddisable-bt device_tree_address=3D0x4000 kernel=3Du-boot.bin [pi4] hdmi_safe=3D1 armstub=3Darmstub8-gic.bin # over_voltage=3D6 sdram_freq_min=3D3200 arm_freq_min=3D2000 force_turbo=3D1 # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq dev.bcm2835_cpufreq.0.freq_settings: 2000/-1 dev.cpu.0.freq_levels: 2000/-1 dev.cpu.0.freq: 2000 An unfortunate issue is the signed output of: # sysctl -x hw.cpufreq.sdram_freq hw.cpufreq.sdram_freq: 0xbebc2000 root@generic:~ # sysctl hw.cpufreq.sdram_freq hw.cpufreq.sdram_freq: -1094967296 0xbebc2000 =3D=3D 3200000000 (3200 [MHz] in terms of config.txt content. More modern RPi* firmware always uses that figure by default for sdram_freq_min as well: so fixed RAM frequency across alternate arm frequencies.) Also, FreeBSD does not have a hw.cpufreq.sdram_freq_min but the RPi4B does have sdram_freq_min . Based on config.txt as FreeBSD supplies it: # sysctl hw.cpufreq.sdram_freq hw.cpufreq.sdram_freq: 400000000 (That last is 400 [MHz] in config.txt notation.) This is based on the RPi* firmware default for sdram_freq_min for the vintage of RPi* firmware FreeBSD currently uses for its snapshot builds. If you are using the combination: dev.cpu.0.freq: 600 hw.cpufreq.sdram_freq: 400000000 that may be causing extra time to be taken compared to alternatives. Of course you may be using powerd or some such. I use fixed frequency configurations normally. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Dec 23 22:41:15 2023 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyJzv6XsHz556ZG for ; Sat, 23 Dec 2023 22:41:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyJzv22rbz4ZwX for ; Sat, 23 Dec 2023 22:41:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1703371275; a=rsa-sha256; cv=none; b=sUZ1HFqwhsQSj1BaJ7u++Iv/oMRiEaLAs+EgkFhVvgQYx5DDh9xUgJ87dE0yNQccNpR83J /oYaxsH2PPebkCn9UPmiGPgEkx2U665xaAgqVKOdWJgnvD33Hunyx81mBn3L0raeNWdz3+ RWTd1q+1K3Hx9gJyOiRFvoyopYkJQRvX0J2bz2E1EOQ5v8jRnMRpqdWfV48WRjT7b/W8oU JWRxAk56z4GEXALibtpOKBgvLqR/7twRd39qX17rERi4+7ozZatyDhwcggPKnWeJYN3TaB Fb43s0QIA6sw7oVR4BJkViIAQ8XiqMevVKHOGkL7agUGIOE7TGHG+uRvYdydOg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1703371275; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KDFgxS7l+Rjd1NEZnGA/n4R6wA4LZQMRSJDYPYHrsJE=; b=BHRor/1fpx0CJyC2799pHg0nMgER0G5RS56TSp+XrGXbLp4Y+u+74Ok5YoLBjZLqOxcsyr zkXG1AbGLgutJo//SkwFrceJkSpDm2BbC9iB6o397Ad2L9CIHiSo/0hblouuZaZMoyac/3 3Sv7NlD0lK8X3SuXFlpR6obw5CJzkRdJPwALml4KKGW5j5sYpBtyIpzYQwQqwZjV8Ba3Dc +VXgkVc2rIvByEKEprOcQhCXxprSJJgMsV5ybs0DCznF69IO8dbIh5UacbRHNgjkQptls5 3rKLHgtKCbHw80pIufvZJapZhzYdgoSkUIxywfqsen5MqNQEtCtNWxvVh2kMYw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4SyJzv17mmz1HD1 for ; Sat, 23 Dec 2023 22:41:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 3BNMfFoD037916 for ; Sat, 23 Dec 2023 22:41:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 3BNMfFUT037915 for fs@FreeBSD.org; Sat, 23 Dec 2023 22:41:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 275597] Samba: smbd sometimes aborts by PANIC when 'vfs objects = cap' Date: Sat, 23 Dec 2023 22:41:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D275597 --- Comment #6 from Kirk McKusick --- (In reply to Mark Millard from comment #5) I obviously did not read it closely enough as I thought the patch had been made. I have poked some freebsd folks to try and find out who needs to take actio= n to move this forward. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sat Dec 23 23:24:34 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyKyD6lKhz558dt for ; Sat, 23 Dec 2023 23:24:52 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-21.consmr.mail.gq1.yahoo.com (sonic317-21.consmr.mail.gq1.yahoo.com [98.137.66.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyKyC3WK6z3C8w for ; Sat, 23 Dec 2023 23:24:51 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=PatBt85o; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703373888; bh=AGiWZ0GtHqd9soebrNcSMEMYlI8S01f+/KIFA3wgS3U=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=PatBt85o3f+ZrKr5sNHY++rp1K3hAiFErOs4bITQ348b9/IRYKMjhZ0MhsJ/c4CQC3DTbKCWRIyltFczh8Ux14ZCgUsbYitd5yMdztP71Xd1q2tfzsn4NJMDyZq2RFdrF344PF9KFpf+1jV97t85gJ5a/q4rSPtDsZgH/mU7HDhbRllHCm35rpU5RyK+j6S34Hz8CY8s1JSzQX/H/J5r8NAd8jiCCNZQWhmCDf892KD2D1UwbE/aBgdlngYPLHBrZNZA0r6/JZix5VvjNbRHqHjLQhLjekmkzWr10GbK6rg97MoSaMQPZ/aCHyuRj85jKgp91/WA9rN6BkoETtPPwQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703373888; bh=NZR5KHbSjpNw402V787ya0pmWjF1lM7WGDGXTYDkX9z=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=hrQy1kHxuioyGqRQjvA65XtJF3mK0z/FOrG0QL6Xye87NozQwHQ2Y69fPZufRGwiCLwcf4yVVD3DR69FgrW/A4ZbD99YtLo4TheR2KEVcSP1UT7zD1eW4ZsF60k8FrP2O9hBGxRYX5n9ZEULvuIkro5HmxKjodw1LJ0lqOrlKrmoiA32DgzFbjkeKPchey04nAelApoOL4h+qkHcnSgB0mooblyXNtQYZ6XX0RFEKKtwVTjG8g5PpUEhLAD9+g0dlTLvwhnrZyyXz5CiRclGmWacT6qtXvDt7E5e444fb9g4c3legUSI+CIiSTke5AWkuN5GlCBUVYiYILH4f1EV7Q== X-YMail-OSG: rU8.fJAVM1lWp1uq186g1b_OGTCuMJEk4IBsYONZRrjwAslbYkl2nnynlFMcyEZ L7uuggEGWGdnZRtAJoRhzRDaaE3nXfmFZzs6jMpiRm1HVntw4I3dXdTPBtn6jbrVnLfAvDfmuX9m NNEehne7xcpFYCu15k_CMED.lweFe1KdVF.vVYCP2RXAg9qH2abKIlvDGfwIpC2zGE6ROzMSa43K WTMia3KZKxFcEDJzHuZS141a14nbaPEHSpfqOfHMArebDBMfCoun9dJNfBbitW4h.8PMl8MqJm0B 0gievnendZTtkkcVnfnqYeiR9_I81Op5h1_zZVaDtUb9LXEiJrl5a73vMOAOCXFqBSfgiDTVA2RI IhrKCklGRvHpmKZbIo8SbTRfAS5ohBxH.5miRlVm6wn4InNB0jyrxfsR6vCXfILXaIAjGsOLGOYu Fov_BhGeyb4yQCwKkwh9CYNpOmtv2IBwudrhsAVtZ0mnPaOiRVLRHwXgaAgTM4G.JlRQSX2Xt.gK xJ1RV5E6Vg49dCVmoUT5POaX6qPdYiXZqZ9HDPOyZmgJDzOolVoKS1Avw.nLTuiEDCnM59EvsTWm jDKmfBAV5D0ts999XsHec4M8i2yIyLfKNlLR29eBsWf8s0ixlS2QsCnqU9jkV0b2Jx3H2IjlBkxR WDi4Mai6cIX1zGaWWjhW2mbhThvIVy_Y9SukqF0IXxtRL7YvU4pxjgs61X66RVp4G0rIFbeILyp8 qMhmoZoRKyIy2xrBROzn.h1D42flk1s_ODychoC5bdngNMcvGjp8jbs6EEhOX4jGYKjeLGHq3N8_ U6nxDO4DWsrG3zHJlox705OL8tRbhE_21xQueK9BWt2MZsp7BpCyswjA9XA5yUfGZSOvhIr3Mtqs 1zJcTOzyX50Wr1k4JRlbTSDJxujw.wWKq8ewukI09xbDKnM3kQ6YQb45F.kPX9FEbu9_R95jeLB8 4PfUZ.jkKIW4JJ5e7kb1Q3UH.i5l..452twi6RqUa0q0bLmlKzSynrdrG6hw72eIkawTK039udg1 qTFKl__s7TYD.1tibIg.94T8QP6j5BmAfO5_JZRGqJFW6i.q9nMKeg.oNG0fk0Ky0J8kyrabPBML E7LzLO8CzMYjZBf7Z5idQEyw33acKX9q_64hOB6S_mcZ9la4evd3bu0.9EOsgMh6tkcnaApOtlrc pu7i53cTyY2ztgIVRDsnzONTJGDCJ.E0uAd7u6DNAgKjWEHdcSQaadAo4BawyJBSgQ4pdGebh9.G iA2adKsild3idC2JzA_DpGtvmgGH8lzttYyy_3Txhcls7AwY_0GqcLZSv..L0IrNnScRQjeBpNth U1WiTfacGajhvpH7dCwIwTdX74bY2LvQRfU2jjpTnqy5nJx9npehygpTVIwtmUjreCY9mh7JWNBP EgjTQJ6o7widT4v9Gxk1soWJdzaCxUAhmxsH1eGdzDA7KTjL9l9JQFJA4_7h9MphR4nSnyN3r4Yh u7CrO4nxz6iVQeZxdd1eYdygS3N7lu.4PFrw_Hne3VTGb8c5QyE9zZEMuHh05kNqKHds0EhG8BJF juN1DgGFbF_arrz8ceKBBE8iyYGQ1YaYHi0SJnuESbdNRGyRVOnEHadQQxcb8LyohUXtv7mg3CbX _OcGnvZ6zzB4B7RSykUFj7yQ20RY2R0K_kT_ZIaX7VNoZFQc9VxZXOKCNxGpTDY1YZolJL9d09ry Bz51DJaalVAnv9Xaz7ybYyUlfx2LC7SmUHsa9L85AXHmxKSk0b2i9Lm8kvT7N8.bCmWZzyhYl_co 5pkciRNOosIC6PbrgJzMlXnRbpJLT8ym0ly6XN8yliuwdBEkn2y8JX.5853b.250X.LU8qG2neHL 9bZRs2OYX4Jd2TfC91oKk2dM7UDAPwOEq_jq4Y1kdDXnnG_h8H1RA.PBea9uf_GfYaXw0iE39M6A ZuuQW998m24iUnJT_mCBb3H.WWKGCXOeXIkJ8znba3R.5oSAjxQFiefKauxrGW5DFKwCmZyM7IHr 8KEa.Cb7XlChWHDTLKYz.Ly7wbwr6dpqZcZhhXjNgxhrURSsdZgJzN8Tb3YcsLZnKXL8BlndBcex nocm4LaGyhhdIGvCXqEHs.8hMvzFKyZOXNwmw_p0zSKQ00e05pMiM8SQkXJQEI0HHmZADAbT0bSj yM8fhqhf6D7Oi26DbnDO7xBe.3ccRrQw0DaJOJHfnkZZ8IAujPdCnxqM8wosDAWW0ciOFEQ.50bn 4FQI1tiGuCQZvuOSo5zCFSIKoyvYYcMl6z29QlP9jGmfbSn4Sc4m2dCcRZZ1kiu819bpJtCyce1s q X-Sonic-MF: X-Sonic-ID: 47fa48de-65ac-45db-b189-204992988392 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Sat, 23 Dec 2023 23:24:48 +0000 Received: by hermes--production-gq1-6949d6d8f9-x28h5 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ae133fe4d8995c93ae69ee93a40199e9; Sat, 23 Dec 2023 23:24:45 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Date: Sat, 23 Dec 2023 15:24:34 -0800 References: <81D464BA-FABE-4F25-B2BE-3DB9D85E3E64@yahoo.com> To: void , freebsd-fs@freebsd.org In-Reply-To: <81D464BA-FABE-4F25-B2BE-3DB9D85E3E64@yahoo.com> Message-Id: <68CE5AE9-4800-49B6-BC00-14F5D3AB464A@yahoo.com> X-Mailer: Apple Mail (2.3774.300.61.1.2) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.147:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyKyC3WK6z3C8w X-Spamd-Bar: --- On Dec 23, 2023, at 14:01, Mark Millard wrote: > On Dec 23, 2023, at 11:55, Mark Millard wrote: >=20 >> void wrote on >> Date: Sat, 23 Dec 2023 15:52:55 UTC : >>=20 >>> On Fri, 22 Dec 2023, at 09:37, Ronald Klop wrote: >>>> Can you provide more concrete information? Like: >>>>=20 >>>> # usbconfig list >>>=20 >>> # usbconfig list >>> ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=3D0 md=3DHOST = spd=3DSUPER (5.0Gbps) pwr=3DSAVE (0mA) >>> ugen0.2: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DSAVE (100mA) >>> ugen0.3: at usbus0, cfg=3D0 md=3DHOST spd=3DLOW (1.5Mbps) pwr=3DON = (24mA) >>> ugen0.4: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DON (450mA) >>> ugen0.5: at usbus0, cfg=3D0 md=3DHOST spd=3DSUPER = (5.0Gbps) pwr=3DON (36mA) >>>=20 >>> The power supply is the approved 5.1V 3A one. >>=20 >> I use CanaKit power supplies that have more margin for the power: >>=20 >> 5.1V 3.5A >>=20 >>>> And the output of "devinfo -v". That will give people much more=20 >>>> information to work with instead of "anecdotal evidence" about your=20= >>>> hardware being usb3. >>>=20 >>> devinfo -v : https://bsd.to/ADPj >>>=20 >>> detailed dmesg is here: https://bsd.to/Q5tx >>=20 >> That last indicates another potentially significant difference >> from my example benchmarks done for comparison to yours: >>=20 >> GEOM_ELI: Device da0p3.eli created. >> GEOM_ELI: Encryption: AES-XTS 256 >> GEOM_ELI: Crypto: software >>=20 >> That likely contributes to slowing down I/O for da0p3 and >> possibly to leaving less time for I/O to other partitions >> when da0p3 is in use. >>=20 >> If you had the resources to test avoiding the encrypted >> partition for your type of media, that might prove >> interesting. >>=20 >> I've never had occasion to want to use any encrypted >> partitions for my FreeBSD activity. >>=20 >> I'm unsure of the GEOM_ELI processing-time contributions >> vs. "gstat -spod" reporting. >=20 > FYI: the 2023-Dec-16 stable/14 snapshot that I'm using > has: >=20 > # strings /boot/efi/start4.elf | grep VC_BUILD_ID_ > VC_BUILD_ID_USER: dom > VC_BUILD_ID_TIME: 10:50:39 > VC_BUILD_ID_VARIANT: start > VC_BUILD_ID_TIME: Mar 17 2023 > VC_BUILD_ID_BRANCH: bcm2711_2 > VC_BUILD_ID_HOSTNAME: buildbot > VC_BUILD_ID_PLATFORM: raspberrypi_linux > VC_BUILD_ID_VERSION: 82f3750a65fadae9a38077e3c2e217ad158c8d54 (clean) >=20 > So fairly modern RPi* firmware compared to what FreeBSD used > to use. >=20 > What vintage are you using? IT may have different defaults than > what my FreeBSD snaphot context reports. >=20 >=20 > Another issue may be: >=20 > bcm2835_cpufreq0: current ARM 600MHz, Core 250MHz, SDRAM 400MHz, Turbo = OFF > bcm2835_cpufreq0: max/min ARM 2147/600MHz, Core 750/250MHz, SDRAM = 400/400MHz >=20 > What does: >=20 > # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq >=20 > report for your context? The default for the FreeBSD supplied > config.txt in the snapshot I'm using is: >=20 > # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq > dev.bcm2835_cpufreq.0.freq_settings: 1500/-1 600/-1 > dev.cpu.0.freq_levels: 1500/-1 600/-1 > dev.cpu.0.freq: 600 >=20 > Adding arm_boost=3D1 to config.txt in order to have 1800 instead of = 1500: >=20 > # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq > dev.bcm2835_cpufreq.0.freq_settings: 1800/-1 600/-1 > dev.cpu.0.freq_levels: 1800/-1 600/-1 > dev.cpu.0.freq: 600 >=20 > Then there is how I normally use all the RPi4B's that I > (sometimes) have access to: >=20 > # more /boot/efi/config.txt=20 > [all] > arm_64bit=3D1 > dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don > dtoverlay=3Dmmc > dtoverlay=3Ddisable-bt > device_tree_address=3D0x4000 > kernel=3Du-boot.bin >=20 > [pi4] > hdmi_safe=3D1 > armstub=3Darmstub8-gic.bin > # > over_voltage=3D6 > sdram_freq_min=3D3200 > arm_freq_min=3D2000 > force_turbo=3D1 >=20 > # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq > dev.bcm2835_cpufreq.0.freq_settings: 2000/-1 > dev.cpu.0.freq_levels: 2000/-1 > dev.cpu.0.freq: 2000 >=20 >=20 > An unfortunate issue is the signed output of: >=20 > # sysctl -x hw.cpufreq.sdram_freq > hw.cpufreq.sdram_freq: 0xbebc2000 > root@generic:~ # sysctl hw.cpufreq.sdram_freq > hw.cpufreq.sdram_freq: -1094967296 >=20 > 0xbebc2000 =3D=3D 3200000000 >=20 > (3200 [MHz] in terms of config.txt content. More modern > RPi* firmware always uses that figure by default for > sdram_freq_min as well: so fixed RAM frequency across > alternate arm frequencies.) >=20 > Also, FreeBSD does not have a hw.cpufreq.sdram_freq_min > but the RPi4B does have sdram_freq_min . >=20 > Based on config.txt as FreeBSD supplies it: >=20 > # sysctl hw.cpufreq.sdram_freq > hw.cpufreq.sdram_freq: 400000000 >=20 > (That last is 400 [MHz] in config.txt notation.) >=20 > This is based on the RPi* firmware default for > sdram_freq_min for the vintage of RPi* firmware > FreeBSD currently uses for its snapshot builds. >=20 > If you are using the combination: >=20 > dev.cpu.0.freq: 600 > hw.cpufreq.sdram_freq: 400000000 >=20 > that may be causing extra time to be taken compared to > alternatives. >=20 > Of course you may be using powerd or some such. I use > fixed frequency configurations normally. >=20 Well, my assumption about the likes of the hw.cpufreq.sdram_freq being due to RPi* firmware looks to be wrong. The RPi* documentation changed from 400 MHz to 3200 MHz for RPi4B sdram_freq_min at: Before (400), Jun 8, 2021: = https://github.com/raspberrypi/documentation/blob/974995fabb184a2435a98e68= c1e728b346112f89/configuration/config-txt/overclocking.md After (3200), Jun 9, 2021: = https://github.com/raspberrypi/documentation/blob/920ff905995541f7ef1c6048= 2924a392143e9192/configuration/config-txt/overclocking.md The RPi* firmware should be setting things up to have 3200 MHz. Since that is not what FreeBSD ends up with in modern snapshots with the FreeBSD supplied config.txt , Likely FreeBSD has taken control of such. This might just be one example parameter that is overridden. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sat Dec 23 23:30:13 2023 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SyL4m5Fxwz5594W for ; Sat, 23 Dec 2023 23:30:32 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic316-55.consmr.mail.gq1.yahoo.com (sonic316-55.consmr.mail.gq1.yahoo.com [98.137.69.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SyL4l6JHCz3D8h for ; Sat, 23 Dec 2023 23:30:31 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=JXTndrS8; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703374228; bh=EqZjObwGOPq/oR+Ec/A6nHg2Ld7tHOMzV6YMakg7eVQ=; h=From:Subject:Date:References:To:In-Reply-To:From:Subject:Reply-To; b=JXTndrS8UdKyrz+mflOytCzuvZpd6XASo61Ecoc/tisT1dkfU96OjDCUCEBzIrTocVOt5ZWqJLe6mjCwOw3J5MHZ3AXmxKV2drD857cjwQJXFCaAoWSeGsQbjSdFhlXA2+FVE0C+lF6i9neNfcd/HVa2+WULXrFHitUDIOMlFpBeIrohlolnU9/jw8QLcqm8WkHfWekZxmAbi1L2VJLEMH+PLvyrpokK5dn/O4v1YtzIvjp9Fg5bw3f6JD6VVFbR53C2m3a261e8dCTemyXe/spXd4KyMaHsGbdR8f81/bCv6ZzPF3lNG0I2pFkLrG6Zf12qIHHRK8hM3bCz+h4jlQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1703374228; bh=KICqqoPwpLDWATWrbiVXxUeVF+/7AH9aDBcy2kAWmm3=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=DW/vr3Kz+8ZeP3eag70ZTMkOfBTWLc+bk6FZrePODmhaE0YdUagznEASDKxRO1JFcSC+6aEINzJ0K5iDCf2EJYgcAdgMiddOLNpz/lBmfqnl+8j2wNOdn5OCwDj3UCrIMY3FJhXT+Mdu6uDbcmZkd/cOoA3iigwa7C62vUMd0fga74cT9W7ijvTl0R4DKowdh6GUPqjJq2XAA4gwDmyDlUpG46WudnDQ2ieI2s58Pcxao5OCpygqKShRAyqJiGq2pHTczHXH4pnjGI5wlSstCAk29W9PHG4FUTArrpV9mjJ9cfi1vY+NmFgRsRJas/OqVVEw50OHlODM+TZjSnEVeA== X-YMail-OSG: mdBRFaEVM1mYjwh0JTo6_3FNZpL5VX2NR65YIj9xvae7LwiH7iNp3lR5PnCZBzH tmbhOG1KX1q5CpJ871y2D8fjG0q4uFT8n6FsOIYKImc7ZAA2EtZ5G.K9qpLPmxXbK1y.aki6zprO 2uyp29.4jp6FjzxDZOuimX.F8bSruIRQaWDWBghT1MjCN4cEEsZBVh84Sa8DSPZzTdNZUoNdsk6o f7dOO7qkQVIELnSw_kT1SA2S3N2NwIe8rxVah8zM7JrNw_FHTyEjlpt2NwGgbjOEvSAXZo7V.p6i AzXx5RjM6bOl4i7rsqKv77Q0I0417NtLtDSjnTgh8yfRf3dOaklIkWZLPblBTiM8r8sfiBYUV9nR 348VIGD1y0OX0myyp2DiDJaYQ9PRjqdTfGXZZuzk_Z6jRWgEcig5vB57tpqXMQv6U4IjCAzhX_5Y 3moJb_C4c_6WcXuj0y8sQXNKOFda6bosjQ4iP41WbP4YzhBgWpGsX2s3_cEeKZZfIwhF9dC7kVUK slE62CuBuGA0H8qgCP06UfyiQcQSFOdj3oe2N0wXG4T6C87Y6YcjWXe7Bz00GsjjtLH.7LySg4TY cffs0yZuDrK7W9XBvGMk3zZZJasPMUURcqEV_xN7iqCZoOhKg7VpP.JOAUwqmwaHqVop8.QRa2BZ w7DHvfo8WjsZOkyrMC9tJJzUZMLslUYnxlUQIq2IoCGXIiPcS16j4bK5z1MSs_ZPCNj6TajUtgsY rXAvkOv1jOvlrW_BonicqiDF46N2Apti6asHTxhb3LLJTW3BgZD0fv2VDJypWH2b6Pnr7Ff8LJzu SUVPnoHH0jMvWiCsxST4nUzq2QftIUmtFmyxZjp4rEHJ2Ak5iaw2esFqmr.eX..V6LkXrdkFdzda IPmHj0B6z4cq9VzRaMjaOnAFb2W2imHSKV38NZVqMekO_RUxW2R7lEuQhfSuOkN5kJF2ruHMhRx8 a63pGOCVgb.ffdlIUsh0POJO9.ws1pACTZctxJi0Rb4ZXkgHx595KyVOQNO4p6kRkeT3e.t5ULhZ dItXNjO_HoE195TRld7Fg204ezByMRn6j0al2Vi.0dmLJkRFs8_AjV2C.I2czvUlUhL0YvV.VwcU Un675blsOyGERYRd6OoLLfrNooc.4aLP8d3rSAv1Xem6.TalZD.fE7GMMBCGBcXQqPouz2GqY3Mn yFKpLi493HD3LcpUrG3p.L.ZqqQrsSvLKHOWponr9AJDhT2CZ_RPz8ONxeSTUf6c3lwWiDEuFTTg 2cZypkoIARTIfzkazm1w3XYlTf8hOEXJWRsmvDXj9ZLH.AOP7TO4OR6NKj0TXtYIhFqwfqht80gE 6lCRFwih9VG.PT54PlpRC4YUz8BdUlac9gAEERoVBLAS8xZf6GvbpfUr8CgduzirmInHCL10tm5p aISY3zgWu6f5LyLmb2mEpBn6aBeJphz6b4as_Ghj1eAGQMIteNTR1.Qj34lGL3vMTWEWd8cJur._ 3Vss_V9Hxeu4NBYtTzKpouMWsuJ_xcLRmQDlO_7Uiq027wDoMoEVPIXOPqvI__qfYKEuUdaelkgi TKudt00iw99Dc9GsTJaVswTDFHBfUg8LArJZgaCeTtdc1SX6xSyV744ey4af1xOVy_H_QGbeA5QH 2jcvkje1negxMgUeI_qeRR2xAVjR3D_3yhCztG7BaM0BjXMFW_CwO0W6eHmsuQ6nuxQu8oCFOzRi PQEa5cWM_Y9Gp08ypbo3HhQ8KnigRgp7UPa2h7Ov3sDYs0kLTfpShw_bpB2XEtn6C5W9YJac1cey rXxe2NBT4ujOGlTX2rg9zAOxpZIESHxEcWarm5SX5ZCc6GA14L135zIzZHMGnb9GneVAmpd5yXOS lr.oy66e4rjO.zdkwsz9zQ_INlQAyn4sTJSm.nuIevRxMOzlXRqFTNHK1UDROlq8DNdiiHwEMtBb OzZrIgD6KlF8w5paaP1wm1I.d3fJurLwXKITkThP_xfermN0mt6xurY.edGr_KETWYVdxzV1Kqcs n_NjMVZr6AE3GJPEGJftjQrft5Io1YpMMDTs1qGih7e2MDxCBzWYH6EPcEhiBkXGfc_A5xFiGHOZ nTL9rde4rwfIhMqDNArSjrKfcuf1GnG41WKtjWaBkOKgcAI.I3cGeNhxOEhX7CoIKmski_1z2J_k Jg47xn48iuE_pbYx5yx9imYBSKzDrJNlPbEz2FLKfsYL99WeLEpAA02sPauVmVrn9_zE3tRZoaUG CW.PtwmEdCstOF8OXUEqqELMIOguMhS1x1q8nfQ7XhC46XfLCy6JdAgRIi_4GRNmV_T74ZwPvLAk 8TEaO X-Sonic-MF: X-Sonic-ID: 71278ea2-f2b0-4295-abb1-5cfafa8d483e Received: from sonic.gate.mail.ne1.yahoo.com by sonic316.consmr.mail.gq1.yahoo.com with HTTP; Sat, 23 Dec 2023 23:30:28 +0000 Received: by hermes--production-gq1-6949d6d8f9-k52jv (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID ccb6de6cb11b41b4248e5f0bd7578096; Sat, 23 Dec 2023 23:30:24 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: Re: measuring swap partition speed Date: Sat, 23 Dec 2023 15:30:13 -0800 References: <81D464BA-FABE-4F25-B2BE-3DB9D85E3E64@yahoo.com> <68CE5AE9-4800-49B6-BC00-14F5D3AB464A@yahoo.com> To: void , freebsd-fs@freebsd.org In-Reply-To: <68CE5AE9-4800-49B6-BC00-14F5D3AB464A@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3774.300.61.1.2) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MLMMJ_DEST(0.00)[freebsd-fs@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.31:from]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.31:from]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[f-m.fm,freebsd.org]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SyL4l6JHCz3D8h X-Spamd-Bar: --- On Dec 23, 2023, at 15:24, Mark Millard wrote: > On Dec 23, 2023, at 14:01, Mark Millard wrote: >=20 >> On Dec 23, 2023, at 11:55, Mark Millard wrote: >>=20 >>> void wrote on >>> Date: Sat, 23 Dec 2023 15:52:55 UTC : >>>=20 >>>> On Fri, 22 Dec 2023, at 09:37, Ronald Klop wrote: >>>>> Can you provide more concrete information? Like: >>>>>=20 >>>>> # usbconfig list >>>>=20 >>>> # usbconfig list >>>> ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=3D0 md=3DHOST = spd=3DSUPER (5.0Gbps) pwr=3DSAVE (0mA) >>>> ugen0.2: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DSAVE (100mA) >>>> ugen0.3: at usbus0, cfg=3D0 md=3DHOST spd=3DLOW (1.5Mbps) pwr=3DON = (24mA) >>>> ugen0.4: at usbus0, cfg=3D0 md=3DHOST = spd=3DHIGH (480Mbps) pwr=3DON (450mA) >>>> ugen0.5: at usbus0, cfg=3D0 md=3DHOST spd=3DSUPER = (5.0Gbps) pwr=3DON (36mA) >>>>=20 >>>> The power supply is the approved 5.1V 3A one. >>>=20 >>> I use CanaKit power supplies that have more margin for the power: >>>=20 >>> 5.1V 3.5A >>>=20 >>>>> And the output of "devinfo -v". That will give people much more=20 >>>>> information to work with instead of "anecdotal evidence" about = your=20 >>>>> hardware being usb3. >>>>=20 >>>> devinfo -v : https://bsd.to/ADPj >>>>=20 >>>> detailed dmesg is here: https://bsd.to/Q5tx >>>=20 >>> That last indicates another potentially significant difference >>> from my example benchmarks done for comparison to yours: >>>=20 >>> GEOM_ELI: Device da0p3.eli created. >>> GEOM_ELI: Encryption: AES-XTS 256 >>> GEOM_ELI: Crypto: software >>>=20 >>> That likely contributes to slowing down I/O for da0p3 and >>> possibly to leaving less time for I/O to other partitions >>> when da0p3 is in use. >>>=20 >>> If you had the resources to test avoiding the encrypted >>> partition for your type of media, that might prove >>> interesting. >>>=20 >>> I've never had occasion to want to use any encrypted >>> partitions for my FreeBSD activity. >>>=20 >>> I'm unsure of the GEOM_ELI processing-time contributions >>> vs. "gstat -spod" reporting. >>=20 >> FYI: the 2023-Dec-16 stable/14 snapshot that I'm using >> has: >>=20 >> # strings /boot/efi/start4.elf | grep VC_BUILD_ID_ >> VC_BUILD_ID_USER: dom >> VC_BUILD_ID_TIME: 10:50:39 >> VC_BUILD_ID_VARIANT: start >> VC_BUILD_ID_TIME: Mar 17 2023 >> VC_BUILD_ID_BRANCH: bcm2711_2 >> VC_BUILD_ID_HOSTNAME: buildbot >> VC_BUILD_ID_PLATFORM: raspberrypi_linux >> VC_BUILD_ID_VERSION: 82f3750a65fadae9a38077e3c2e217ad158c8d54 (clean) >>=20 >> So fairly modern RPi* firmware compared to what FreeBSD used >> to use. >>=20 >> What vintage are you using? IT may have different defaults than >> what my FreeBSD snaphot context reports. >>=20 >>=20 >> Another issue may be: >>=20 >> bcm2835_cpufreq0: current ARM 600MHz, Core 250MHz, SDRAM 400MHz, = Turbo OFF >> bcm2835_cpufreq0: max/min ARM 2147/600MHz, Core 750/250MHz, SDRAM = 400/400MHz >>=20 >> What does: >>=20 >> # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq >>=20 >> report for your context? The default for the FreeBSD supplied >> config.txt in the snapshot I'm using is: >>=20 >> # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq >> dev.bcm2835_cpufreq.0.freq_settings: 1500/-1 600/-1 >> dev.cpu.0.freq_levels: 1500/-1 600/-1 >> dev.cpu.0.freq: 600 >>=20 >> Adding arm_boost=3D1 to config.txt in order to have 1800 instead of = 1500: >>=20 >> # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq >> dev.bcm2835_cpufreq.0.freq_settings: 1800/-1 600/-1 >> dev.cpu.0.freq_levels: 1800/-1 600/-1 >> dev.cpu.0.freq: 600 >>=20 >> Then there is how I normally use all the RPi4B's that I >> (sometimes) have access to: >>=20 >> # more /boot/efi/config.txt=20 >> [all] >> arm_64bit=3D1 >> dtparam=3Daudio=3Don,i2c_arm=3Don,spi=3Don >> dtoverlay=3Dmmc >> dtoverlay=3Ddisable-bt >> device_tree_address=3D0x4000 >> kernel=3Du-boot.bin >>=20 >> [pi4] >> hdmi_safe=3D1 >> armstub=3Darmstub8-gic.bin >> # >> over_voltage=3D6 >> sdram_freq_min=3D3200 >> arm_freq_min=3D2000 >> force_turbo=3D1 >>=20 >> # sysctl dev.bcm2835_cpufreq.0.freq_settings dev.cpu.0.freq_levels = dev.cpu.0.freq >> dev.bcm2835_cpufreq.0.freq_settings: 2000/-1 >> dev.cpu.0.freq_levels: 2000/-1 >> dev.cpu.0.freq: 2000 >>=20 >>=20 >> An unfortunate issue is the signed output of: >>=20 >> # sysctl -x hw.cpufreq.sdram_freq >> hw.cpufreq.sdram_freq: 0xbebc2000 >> root@generic:~ # sysctl hw.cpufreq.sdram_freq >> hw.cpufreq.sdram_freq: -1094967296 >>=20 >> 0xbebc2000 =3D=3D 3200000000 >>=20 >> (3200 [MHz] in terms of config.txt content. More modern >> RPi* firmware always uses that figure by default for >> sdram_freq_min as well: so fixed RAM frequency across >> alternate arm frequencies.) >>=20 >> Also, FreeBSD does not have a hw.cpufreq.sdram_freq_min >> but the RPi4B does have sdram_freq_min . >>=20 >> Based on config.txt as FreeBSD supplies it: >>=20 >> # sysctl hw.cpufreq.sdram_freq >> hw.cpufreq.sdram_freq: 400000000 >>=20 >> (That last is 400 [MHz] in config.txt notation.) >>=20 >> This is based on the RPi* firmware default for >> sdram_freq_min for the vintage of RPi* firmware >> FreeBSD currently uses for its snapshot builds. >>=20 >> If you are using the combination: >>=20 >> dev.cpu.0.freq: 600 >> hw.cpufreq.sdram_freq: 400000000 >>=20 >> that may be causing extra time to be taken compared to >> alternatives. >>=20 >> Of course you may be using powerd or some such. I use >> fixed frequency configurations normally. >>=20 >=20 > Well, my assumption about the likes of the hw.cpufreq.sdram_freq being Sorry: (RPi* config.txt notation) sdram_freq_min (since 400 is observed = to occur) > due to RPi* firmware looks to be wrong. The RPi* documentation changed > from 400 MHz to 3200 MHz for RPi4B sdram_freq_min at: >=20 > Before (400), Jun 8, 2021: > = https://github.com/raspberrypi/documentation/blob/974995fabb184a2435a98e68= c1e728b346112f89/configuration/config-txt/overclocking.md >=20 > After (3200), Jun 9, 2021: > = https://github.com/raspberrypi/documentation/blob/920ff905995541f7ef1c6048= 2924a392143e9192/configuration/config-txt/overclocking.md >=20 > The RPi* firmware should be setting things up to have 3200 MHz. Since > that is not what FreeBSD ends up with in modern snapshots with the > FreeBSD supplied config.txt , Likely FreeBSD has taken control of > such. >=20 > This might just be one example parameter that is overridden. =3D=3D=3D Mark Millard marklmi at yahoo.com