Date: Thu, 21 Dec 2023 20:19:45 -0800 From: Mark Millard <marklmi@yahoo.com> To: void@f-m.fm, freebsd-fs@freebsd.org Cc: Warner Losh <imp@bsdimp.com> Subject: Re: measuring swap partition speed Message-ID: <B3D05A6F-329B-4EC4-85F6-3357131DFA61@yahoo.com> References: <B3D05A6F-329B-4EC4-85F6-3357131DFA61.ref@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
void <void@f-m.fm> wrote on
Date: Fri, 22 Dec 2023 00:19:39 UTC :
> Hi Mark & Warner,
>=20
> I tried using fio with Warner's suggestions as a template, and
> pasted the results in the latter part of this message.
>=20
> On Thu, 21 Dec 2023, at 21:03, Mark Millard wrote:
>=20
> . . .
>=20
> fio output follows. No idea yet if the settings used are suitable for =
this context
> and would welcome suggestions.
I later supply the output from running the same command,
be it suitable or not. Again: the context is my old USB3
SSD stick, not spinning rust. Also my particular usual
arm clocking configuration for the RPi4B. (I forgot
to mention earlier that it has a case with a fan and
heatsinks.)
Also, this was not a swap space test command. So it
is relevant that the file system involved was UFS,
not ZFS.
# swapinfo
Device 1K-blocks Used Avail Capacity
/dev/label/growfs_swap 8388604 0 8388604 0%
I'll note that monitoring with top showed that the
--numjobs=3D8 lead to 8 threads (twice the number of
hardware threads).
I'll note that the run left behind:
# ls -C1 randread.*.0randread.0.0
randread.1.0
randread.2.0
randread.3.0
randread.4.0
randread.5.0
randread.6.0
randread.7.0
that I later deleted.
> I think --filename can be a device? Output suggests
> it used hw.ncpu instead of --numjobs=3D8
>=20
> ~~~~~~~
> # fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread =
--direct=3D1 --bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 =
--iodepth=3D32 --numjobs=3D8 --runtime=3D60 --group_reporting --thread =
--size=3D2048M
> randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) =
8192B-8192B, (T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32
> ...
> fio-3.36
> Starting 8 threads
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> randread: Laying out IO file (1 file / 2048MiB)
> Jobs: 4 (f=3D4): =
[_(1),r(2),_(2),r(1),_(1),r(1)][21.2%][r=3D288KiB/s][r=3D36 IOPS][eta =
04m:05s]
> randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D135125: Thu Dec 21 =
16:43:00 2023
> read: IOPS=3D43, BW=3D351KiB/s (359kB/s)(22.6MiB/65986msec)
> slat (nsec): min=3D889, max=3D1948.6k, avg=3D4172.45, stdev=3D36488.07
> clat (msec): min=3D1108, max=3D11660, avg=3D5644.23, stdev=3D1282.12
> lat (msec): min=3D1108, max=3D11660, avg=3D5644.24, stdev=3D1282.12
> clat percentiles (msec):
> | 1.00th=3D[ 1183], 5.00th=3D[ 3171], 10.00th=3D[ 4933], 20.00th=3D[ =
5269],
> | 30.00th=3D[ 5470], 40.00th=3D[ 5604], 50.00th=3D[ 5738], 60.00th=3D[ =
5873],
> | 70.00th=3D[ 5940], 80.00th=3D[ 6074], 90.00th=3D[ 6342], 95.00th=3D[ =
6812],
> | 99.00th=3D[10671], 99.50th=3D[10939], 99.90th=3D[11610], =
99.95th=3D[11610],
> | 99.99th=3D[11610]
> bw ( KiB/s): min=3D 208, max=3D 3760, per=3D100.00%, avg=3D1535.05, =
stdev=3D128.23, samples=3D245
> iops : min=3D 26, max=3D 470, avg=3D191.88, stdev=3D16.03, samples=3D245=
> lat (msec) : 2000=3D3.25%, >=3D2000=3D96.75%
> cpu : usr=3D0.00%, sys=3D0.12%, ctx=3D22712, majf=3D0, minf=3D0
> IO depths : 1=3D0.3%, 2=3D0.6%, 4=3D1.1%, 8=3D4.9%, 16=3D69.6%, =
32=3D23.6%, >=3D64=3D0.0%
> submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, =
64=3D0.0%, >=3D64=3D0.0%
> complete : 0=3D0.0%, 4=3D94.7%, 8=3D2.9%, 16=3D1.5%, 32=3D0.9%, =
64=3D0.0%, >=3D64=3D0.0%
> issued rwts: total=3D2895,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0
> latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32
>=20
> Run status group 0 (all jobs):
> READ: bw=3D351KiB/s (359kB/s), 351KiB/s-351KiB/s (359kB/s-359kB/s), =
io=3D22.6MiB (23.7MB), run=3D65986-65986msec
# fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D1=
--bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 =
--numjobs=3D8 --runtime=3D60 --group_reporting --thread --size=3D2048M
randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, =
(T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32
...
fio-3.36
Starting 8 threads
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
Jobs: 8 (f=3D8): [r(8)][100.0%][r=3D17.4MiB/s][r=3D2222 IOPS][eta =
00m:00s]
randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D100241: Sun Dec 17 =
12:42:53 2023
read: IOPS=3D2486, BW=3D19.4MiB/s (20.4MB/s)(1168MiB/60125msec)
slat (nsec): min=3D888, max=3D1494.1k, avg=3D3446.83, stdev=3D8015.07
clat (msec): min=3D58, max=3D334, avg=3D102.70, stdev=3D22.68
lat (msec): min=3D58, max=3D334, avg=3D102.70, stdev=3D22.68
clat percentiles (msec):
| 1.00th=3D[ 70], 5.00th=3D[ 75], 10.00th=3D[ 79], =
20.00th=3D[ 84],
| 30.00th=3D[ 89], 40.00th=3D[ 94], 50.00th=3D[ 99], =
60.00th=3D[ 105],
| 70.00th=3D[ 111], 80.00th=3D[ 118], 90.00th=3D[ 136], =
95.00th=3D[ 150],
| 99.00th=3D[ 171], 99.50th=3D[ 180], 99.90th=3D[ 211], =
99.95th=3D[ 228],
| 99.99th=3D[ 268]
bw ( KiB/s): min=3D 8246, max=3D24640, per=3D100.00%, avg=3D19903.38, =
stdev=3D281.80, samples=3D960
iops : min=3D 1028, max=3D 3080, avg=3D2487.90, stdev=3D35.24, =
samples=3D960
lat (msec) : 100=3D52.92%, 250=3D47.07%, 500=3D0.01%
cpu : usr=3D0.17%, sys=3D5.01%, ctx=3D1016480, majf=3D0, =
minf=3D0
IO depths : 1=3D0.1%, 2=3D0.1%, 4=3D0.1%, 8=3D0.7%, 16=3D75.3%, =
32=3D24.0%, >=3D64=3D0.0%
submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, =
64=3D0.0%, >=3D64=3D0.0%
complete : 0=3D0.0%, 4=3D93.2%, 8=3D4.7%, 16=3D2.0%, 32=3D0.1%, =
64=3D0.0%, >=3D64=3D0.0%
issued rwts: total=3D149497,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0
latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32
Run status group 0 (all jobs):
READ: bw=3D19.4MiB/s (20.4MB/s), 19.4MiB/s-19.4MiB/s =
(20.4MB/s-20.4MB/s), io=3D1168MiB (1225MB), run=3D60125-60125msec
After swapoff instead:
# fio --name=3Drandread --ioengine=3Dposixaio --rw=3Drandread --direct=3D1=
--bs=3D8k --refill_buffers --norandommap --randrepeat=3D0 --iodepth=3D32 =
--numjobs=3D8 --runtime=3D60 --group_reporting --thread --size=3D2048M
randread: (g=3D0): rw=3Drandread, bs=3D(R) 8192B-8192B, (W) 8192B-8192B, =
(T) 8192B-8192B, ioengine=3Dposixaio, iodepth=3D32
...
fio-3.36
Starting 8 threads
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
randread: Laying out IO file (1 file / 2048MiB)
Jobs: 8 (f=3D8): [r(8)][100.0%][r=3D17.6MiB/s][r=3D2259 IOPS][eta =
00m:00s]
randread: (groupid=3D0, jobs=3D8): err=3D 0: pid=3D100211: Sun Dec 17 =
12:07:16 2023
read: IOPS=3D2471, BW=3D19.3MiB/s (20.2MB/s)(1161MiB/60119msec)
slat (nsec): min=3D889, max=3D1912.6k, avg=3D3415.21, stdev=3D8521.17
clat (msec): min=3D54, max=3D762, avg=3D103.32, stdev=3D28.95
lat (msec): min=3D54, max=3D762, avg=3D103.32, stdev=3D28.95
clat percentiles (msec):
| 1.00th=3D[ 69], 5.00th=3D[ 75], 10.00th=3D[ 79], =
20.00th=3D[ 84],
| 30.00th=3D[ 89], 40.00th=3D[ 93], 50.00th=3D[ 99], =
60.00th=3D[ 105],
| 70.00th=3D[ 110], 80.00th=3D[ 117], 90.00th=3D[ 136], =
95.00th=3D[ 150],
| 99.00th=3D[ 174], 99.50th=3D[ 188], 99.90th=3D[ 518], =
99.95th=3D[ 558],
| 99.99th=3D[ 617]
bw ( KiB/s): min=3D12336, max=3D24912, per=3D100.00%, avg=3D19913.12, =
stdev=3D267.11, samples=3D954
iops : min=3D 1542, max=3D 3114, avg=3D2489.14, stdev=3D33.39, =
samples=3D954
lat (msec) : 100=3D52.77%, 250=3D47.01%, 500=3D0.11%, 750=3D0.11%, =
1000=3D0.01%
cpu : usr=3D0.18%, sys=3D4.93%, ctx=3D1005861, majf=3D0, =
minf=3D0
IO depths : 1=3D0.1%, 2=3D0.1%, 4=3D0.3%, 8=3D1.3%, 16=3D74.8%, =
32=3D23.6%, >=3D64=3D0.0%
submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, =
64=3D0.0%, >=3D64=3D0.0%
complete : 0=3D0.0%, 4=3D93.4%, 8=3D4.5%, 16=3D1.9%, 32=3D0.2%, =
64=3D0.0%, >=3D64=3D0.0%
issued rwts: total=3D148600,0,0,0 short=3D0,0,0,0 dropped=3D0,0,0,0
latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D32
Run status group 0 (all jobs):
READ: bw=3D19.3MiB/s (20.2MB/s), 19.3MiB/s-19.3MiB/s =
(20.2MB/s-20.2MB/s), io=3D1161MiB (1217MB), run=3D60119-60119msec
The difference:
Jobs: 4 (f=3D4) (yours)
vs.
Jobs: 8 (f=3D8) (mine)
is not obvious, given the same commands.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B3D05A6F-329B-4EC4-85F6-3357131DFA61>