Date: Thu, 28 Dec 2023 12:24:14 +0000 From: void <void@f-m.fm> To: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: measuring swap partition speed Message-ID: <deac0f64-73cb-4708-b789-dc768a6353da@app.fastmail.com> In-Reply-To: <A539DD68-5FAD-4303-9FF1-350F17D90B26@yahoo.com> References: <0D286183-ACD6-4818-8E27-E03D7F16D288@yahoo.com> <A539DD68-5FAD-4303-9FF1-350F17D90B26@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Plugged the disk into usb3 interface on amd64 machine and ran the following [a] tests all at roughly the same time. The write test (2x git clone/pull) goes to a ufs partition. Writing 2x swap at the same time never got below 3mb/s
tl;dr :
1. in the arm64 context, the extreme decline in swap performance happens if swap
gets used, by anything (system)
2. in the [1] context, even when swap later becomes unused by the system,
the performance problem persists.
3. the situation is recoverable only via swapoff then on.
Then the swap partition can be written to in the tests at full speed.
Writing subsequent swap tests also run at full speed.
When the system swaps again, the problem returns.
4. the slowdown does not arise in an amd64 context beyond what one would expect
5. ufs or zfs makes no real difference
it wouldn't matter to me if the write speed was 3mb, 16mb or 22mb/s.
What causes problems is the sub-1mb/s speed (worst I've seen it is 300kb/s) and the
non-recovery of that swap space when load eases off. I'm probably doing myself no
favours using zfs in the arm64 context though.
thank you everyone who helped, I now have an idea of a workaround
########### amd64 tests of the same disk
# gpart show da4
=> 40 1953525088 da4 GPT (932G)
40 532480 1 efi (260M)
532520 2008 - free - (1.0M)
534528 4194304 2 freebsd-swap (2.0G)
4728832 4194304 4 freebsd-swap (2.0G)
8923136 4194304 5 freebsd-swap (2.0G)
13117440 4194304 6 freebsd-swap (2.0G)
17311744 2048 - free - (1.0M)
17313792 8384512 7 freebsd-ufs (4.0G)
25698304 2048 - free - (1.0M)
25700352 1927823360 3 freebsd-zfs (919G)
1953523712 1416 - free - (708K)
# newfs -U /dev/da4p7
/dev/da4p7: 4094.0MB (8384512 sectors) block size 32768, fragment size 4096
using 7 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
with soft updates
super-block backups (for fsck_ffs -b #) at:
192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880
# mount /dev/da4p7 /mnt
# cd /mnt
[a]
#### all the following tests were run at roughly the same time, in different
terminal windows. 2x git (ufs) & 2x dd.
# time -l git clone https://git.freebsd.org/src.git
Cloning into 'src'...
remote: Enumerating objects: 4437205, done.
remote: Counting objects: 100% (383246/383246), done.
remote: Compressing objects: 100% (28769/28769), done.
remote: Total 4437205 (delta 377514), reused 354477 (delta 354477), pack-reused 4053959
Receiving objects: 100% (4437205/4437205), 1.59 GiB | 6.10 MiB/s, done.
Resolving deltas: 100% (3526142/3526142), done.
Updating files: 58% (57132/97280)
/mnt: write failed, filesystem is full
error: unable to write file sys/contrib/dev/acpica/components/utilities/utxface.c
Updating files: 58% (57317/97280)
/mnt: write failed, filesystem is full
Updating files: 100% (97280/97280), done.
fatal: unable to checkout working tree
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'
639.76 real 566.78 user 41.47 sys
1054240 maximum resident set size
2566 average shared memory size
222 average unshared data size
127 average unshared stack size
475518 page reclaims
0 page faults
0 swaps
3 block input operations
15556 block output operations
422759 messages sent
1527197 messages received
606 signals received
3656856 voluntary context switches
851907 involuntary context switches
#
# time -l git clone https://git.freebsd.org/ports.git
Cloning into 'ports'...
remote: Enumerating objects: 6046014, done.
remote: Counting objects: 100% (936/936), done.
remote: Compressing objects: 100% (120/120), done.
remote: Total 6046014 (delta 923), reused 816 (delta 816), pack-reused 6045078
Receiving objects: 100% (6046014/6046014), 1.16 GiB | 5.28 MiB/s, done.
Resolving deltas: 100% (3650283/3650283), done.
/mnt: write failed, filesystem is full
fatal: sha1 file '/mnt/ports/.git/objects/pack/tmp_rev_cQdim5' write error. Out of diskspace
fatal: fetch-pack: invalid index-pack output
553.87 real 0.41 user 1.47 sys
6092 maximum resident set size
2400 average shared memory size
208 average unshared data size
118 average unshared stack size
451 page reclaims
0 page faults
0 swaps
4 block input operations
2 block output operations
152484 messages sent
414280 messages received
0 signals received
409098 voluntary context switches
571 involuntary context switches
#
# time -l dd if=/dev/urandom of=/dev/da4p4 bs=8k conv=sync status=progress
dd: /dev/da4p4: end of device2040 MiB) transferred 660.001s, 3242 kB/s
262145+0 records in
262144+0 records out
2147483648 bytes transferred in 660.734537 secs (3250146 bytes/sec)
660.87 real 0.44 user 10.61 sys
1992 maximum resident set size
20 average shared memory size
8 average unshared data size
132 average unshared stack size
133 page reclaims
0 page faults
0 swaps
0 block input operations
262145 block output operations
0 messages sent
0 messages received
660 signals received
262424 voluntary context switches
2182 involuntary context switches
#
# time -l dd if=/dev/urandom of=/dev/da4p5 bs=8k conv=sync status=progress
dd: /dev/da4p5: end of device2047 MiB) transferred 666.010s, 3223 kB/s
262145+0 records in
262144+0 records out
2147483648 bytes transferred in 666.162391 secs (3223664 bytes/sec)
666.16 real 0.57 user 10.25 sys
1992 maximum resident set size
21 average shared memory size
8 average unshared data size
138 average unshared stack size
133 page reclaims
0 page faults
0 swaps
0 block input operations
262145 block output operations
0 messages sent
0 messages received
666 signals received
262420 voluntary context switches
2867 involuntary context switches
#
# gstat -spod
dT: 1.007s w: 1.000s
L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w d/s kB kBps ms/d o/s ms/o %busy Name
1 1096 0 0 0 0.0 1096 19 20876 2.2 0 0 0 0.0 0 0.0 107.2| da4
[...]
2 1252 0 0 0 0.0 1252 8 10020 1.5 0 0 0 0.0 0 0.0 100.0| da4
When the simultaneous write test completes, swap speed in an amd64 context
reverts to normal, without any user intervention:
# time -l dd if=/dev/urandom of=/dev/da4p5 bs=8k conv=sync status=progress
dd: /dev/da4p5: end of device2046 MiB) transferred 200.020s, 11 MB/s
262145+0 records in
262144+0 records out
2147483648 bytes transferred in 200.166970 secs (10728462 bytes/sec)
200.16 real 0.45 user 9.69 sys
2016 maximum resident set size
16 average shared memory size
6 average unshared data size
106 average unshared stack size
130 page reclaims
0 page faults
0 swaps
0 block input operations
262145 block output operations
0 messages sent
0 messages received
200 signals received
262147 voluntary context switches
320 involuntary context switches
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?deac0f64-73cb-4708-b789-dc768a6353da>