Date: Thu, 28 Dec 2023 06:07:55 -0800 From: Mark Millard <marklmi@yahoo.com> To: void <void@f-m.fm>, freebsd-fs@freebsd.org Subject: Re: measuring swap partition speed Message-ID: <0D486016-8806-414F-B7F9-66519E8B0EBC@yahoo.com> References: <0D486016-8806-414F-B7F9-66519E8B0EBC.ref@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
void <void_at_f-m.fm> wrote on Date: Thu, 28 Dec 2023 12:24:14 UTC : > Plugged the disk into usb3 interface on amd64 machine and ran the = following [a] tests all at roughly the same time. The write test (2x git = clone/pull) goes to a ufs partition. Writing 2x swap at the same time = never got below 3mb/s >=20 > tl;dr : >=20 > 1. in the arm64 context, the extreme decline in swap performance = happens if swap=20 > gets used, by anything (system) >=20 > 2. in the [1] context, even when swap later becomes unused by the = system,=20 > the performance problem persists. >=20 > 3. the situation is recoverable only via swapoff then on. > Then the swap partition can be written to in the tests at full speed. > Writing subsequent swap tests also run at full speed. > When the system swaps again, the problem returns. >=20 > 4. the slowdown does not arise in an amd64 context beyond what one = would expect=20 >=20 > 5. ufs or zfs makes no real difference >=20 > it wouldn't matter to me if the write speed was 3mb, 16mb or 22mb/s. > What causes problems is the sub-1mb/s speed (worst I've seen it is = 300kb/s) and the > non-recovery of that swap space when load eases off. I'm probably = doing myself no=20 > favours using zfs in the arm64 context though. >=20 > thank you everyone who helped, I now have an idea of a workaround >=20 > ########### amd64 tests of the same disk Could you produce the same sort of report (gstat -spod and such included) for the arm64 context? The direct (well matched) comparison/contrast to amd64 could prove of interest to folks that work on arm64 support. > # gpart show da4 > =3D> 40 1953525088 da4 GPT (932G) > 40 532480 1 efi (260M) > 532520 2008 - free - (1.0M) > 534528 4194304 2 freebsd-swap (2.0G) > 4728832 4194304 4 freebsd-swap (2.0G) > 8923136 4194304 5 freebsd-swap (2.0G) > 13117440 4194304 6 freebsd-swap (2.0G) > 17311744 2048 - free - (1.0M) > 17313792 8384512 7 freebsd-ufs (4.0G) > 25698304 2048 - free - (1.0M) > 25700352 1927823360 3 freebsd-zfs (919G) > 1953523712 1416 - free - (708K) >=20 > # newfs -U /dev/da4p7 > /dev/da4p7: 4094.0MB (8384512 sectors) block size 32768, fragment size = 4096 > using 7 cylinder groups of 625.22MB, 20007 blks, 80128 inodes. > with soft updates > super-block backups (for fsck_ffs -b #) at: > 192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880 >=20 > # mount /dev/da4p7 /mnt > # cd /mnt >=20 > [a] >=20 > #### all the following tests were run at roughly the same time, in = different=20 > terminal windows. 2x git (ufs) & 2x dd. >=20 > # time -l git clone https://git.freebsd.org/src.git > Cloning into 'src'... > remote: Enumerating objects: 4437205, done. > remote: Counting objects: 100% (383246/383246), done. > remote: Compressing objects: 100% (28769/28769), done. > remote: Total 4437205 (delta 377514), reused 354477 (delta 354477), = pack-reused 4053959 > Receiving objects: 100% (4437205/4437205), 1.59 GiB | 6.10 MiB/s, = done. > Resolving deltas: 100% (3526142/3526142), done. > Updating files: 58% (57132/97280) > /mnt: write failed, filesystem is full > error: unable to write file = sys/contrib/dev/acpica/components/utilities/utxface.c > Updating files: 58% (57317/97280) > /mnt: write failed, filesystem is full > Updating files: 100% (97280/97280), done. > fatal: unable to checkout working tree > warning: Clone succeeded, but checkout failed. > You can inspect what was checked out with 'git status' > and retry with 'git restore --source=3DHEAD :/' >=20 > 639.76 real 566.78 user 41.47 sys > 1054240 maximum resident set size > 2566 average shared memory size > 222 average unshared data size > 127 average unshared stack size > 475518 page reclaims > 0 page faults > 0 swaps > 3 block input operations > 15556 block output operations > 422759 messages sent > 1527197 messages received > 606 signals received > 3656856 voluntary context switches > 851907 involuntary context switches > #=20 >=20 > # time -l git clone https://git.freebsd.org/ports.git > Cloning into 'ports'... > remote: Enumerating objects: 6046014, done. > remote: Counting objects: 100% (936/936), done. > remote: Compressing objects: 100% (120/120), done. > remote: Total 6046014 (delta 923), reused 816 (delta 816), pack-reused = 6045078 > Receiving objects: 100% (6046014/6046014), 1.16 GiB | 5.28 MiB/s, = done. > Resolving deltas: 100% (3650283/3650283), done. >=20 > /mnt: write failed, filesystem is full > fatal: sha1 file '/mnt/ports/.git/objects/pack/tmp_rev_cQdim5' write = error. Out of diskspace > fatal: fetch-pack: invalid index-pack output > 553.87 real 0.41 user 1.47 sys > 6092 maximum resident set size > 2400 average shared memory size > 208 average unshared data size > 118 average unshared stack size > 451 page reclaims > 0 page faults > 0 swaps > 4 block input operations > 2 block output operations > 152484 messages sent > 414280 messages received > 0 signals received > 409098 voluntary context switches > 571 involuntary context switches > #=20 >=20 > # time -l dd if=3D/dev/urandom of=3D/dev/da4p4 bs=3D8k conv=3Dsync = status=3Dprogress > dd: /dev/da4p4: end of device2040 MiB) transferred 660.001s, 3242 kB/s >=20 > 262145+0 records in > 262144+0 records out > 2147483648 bytes transferred in 660.734537 secs (3250146 bytes/sec) > 660.87 real 0.44 user 10.61 sys > 1992 maximum resident set size > 20 average shared memory size > 8 average unshared data size > 132 average unshared stack size > 133 page reclaims > 0 page faults > 0 swaps > 0 block input operations > 262145 block output operations > 0 messages sent > 0 messages received > 660 signals received > 262424 voluntary context switches > 2182 involuntary context switches > #=20 >=20 > # time -l dd if=3D/dev/urandom of=3D/dev/da4p5 bs=3D8k conv=3Dsync = status=3Dprogress > dd: /dev/da4p5: end of device2047 MiB) transferred 666.010s, 3223 kB/s >=20 > 262145+0 records in > 262144+0 records out > 2147483648 bytes transferred in 666.162391 secs (3223664 bytes/sec) > 666.16 real 0.57 user 10.25 sys > 1992 maximum resident set size > 21 average shared memory size > 8 average unshared data size > 138 average unshared stack size > 133 page reclaims > 0 page faults > 0 swaps > 0 block input operations > 262145 block output operations > 0 messages sent > 0 messages received > 666 signals received > 262420 voluntary context switches > 2867 involuntary context switches > #=20 > # gstat -spod >=20 > dT: 1.007s w: 1.000s > L(q) ops/s r/s kB kBps ms/r w/s kB kBps ms/w d/s kB kBps ms/d o/s ms/o = %busy Name >=20 > 1 1096 0 0 0 0.0 1096 19 20876 2.2 0 0 0 0.0 0 0.0 107.2| da4 > [...] > 2 1252 0 0 0 0.0 1252 8 10020 1.5 0 0 0 0.0 0 0.0 100.0| da4 >=20 > When the simultaneous write test completes, swap speed in an amd64 = context=20 > reverts to normal, without any user intervention: >=20 > # time -l dd if=3D/dev/urandom of=3D/dev/da4p5 bs=3D8k conv=3Dsync = status=3Dprogress > dd: /dev/da4p5: end of device2046 MiB) transferred 200.020s, 11 MB/s >=20 > 262145+0 records in > 262144+0 records out > 2147483648 bytes transferred in 200.166970 secs (10728462 bytes/sec) > 200.16 real 0.45 user 9.69 sys > 2016 maximum resident set size > 16 average shared memory size > 6 average unshared data size > 106 average unshared stack size > 130 page reclaims > 0 page faults > 0 swaps > 0 block input operations > 262145 block output operations > 0 messages sent > 0 messages received > 200 signals received > 262147 voluntary context switches > 320 involuntary context switches =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0D486016-8806-414F-B7F9-66519E8B0EBC>