Date: Fri, 22 Dec 2023 10:37:42 +0100 (CET) From: Ronald Klop <ronald-lists@klop.ws> To: void <void@f-m.fm> Cc: freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: measuring swap partition speed Message-ID: <1137397689.3080.1703237862787@localhost> In-Reply-To: <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com> References: <D9C150E2-8453-463B-9A23-B0474D0AE390.ref@yahoo.com> <D9C150E2-8453-463B-9A23-B0474D0AE390@yahoo.com> <ZYRe3GFrf1fSIPjk@int21h> <1439231787.9333.1703184981914@localhost> <1dbf8804-f1ac-4578-a538-889744d7de9a@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_3079_229342031.1703237861046
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Van: void <void@f-m.fm>
Datum: vrijdag, 22 december 2023 01:05
Aan: freebsd-fs <freebsd-fs@freebsd.org>
Onderwerp: Re: measuring swap partition speed
>
> On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote:
> > A bit weird that a simple dd is so slow.
> >
> > Just a quick thought.
> > Are your partitions aligned properly?
>
> no idea. How would I check? The system when installed,
> the auto-zfs option was selected, block size set to 4k. The swap
> was changed from the default 2gb to 12gb. geli encryption is
> active for the zfs filesystem but not for swap. It's only the swap
> that shows performance issues.
>
> > Is other IO going on on the same time?
>
> not really, the tests were done with cron jobs deactivated, almost completely idle,
> swap was not being used at all.
>
> > What does gstat say about %util, queue length and all the other stats
> > while running the dd? Or "iostat -x -d 1".
>
> I'll try that next, thanks
>
> > Could you try if another disk has the same issues?
>
> no spare or equivalent disk for this arch.
>
> > How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?
>
> Regular usb3 connection.
>
> da0: 400.000MB/s transfers
> da0: 953869MB (1953525168 512 byte sectors)
> da0: quirks=0x2<NO_6_BYTE>
> da0: Delete methods: <NONE(*),ZERO>
> --
>
>
>
>
Can you provide more concrete information? Like:
# usbconfig list
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)
And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3.
And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you.
About the alignment. You wrote in your first mail:
"# gpart show
=> 40 1953525088 da0 GPT (932G)
40 532480 1 efi (260M)
532520 2008 - free - (1.0M)
534528 4194304 2 freebsd-swap (2.0G)
4728832 4194304 4 freebsd-swap (2.0G)
8923136 4194304 5 freebsd-swap (2.0G)
13117440 4194304 6 freebsd-swap (2.0G)
17311744 4194304 7 freebsd-swap (2.0G)
21506048 4194304 8 freebsd-swap (2.0G)
25700352 1927823360 3 freebsd-zfs (920G)
1953523712 1416 - free - (708K)
"
Your dd tests were on da0p4 which starts on block 4728832.
Assuming "gpart list" shows "Sectorsize: 512".
(4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good.
AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out.
Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin.
Regards,
Ronald.
------=_Part_3079_229342031.1703237861046
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit
<html><head></head><body><br>
<p><strong>Van:</strong> void <void@f-m.fm><br>
<strong>Datum:</strong> vrijdag, 22 december 2023 01:05<br>
<strong>Aan:</strong> freebsd-fs <freebsd-fs@freebsd.org><br>
<strong>Onderwerp:</strong> Re: measuring swap partition speed</p>
<blockquote style="padding-right: 0px; padding-left: 5px; margin-left: 5px; border-left: #000000 2px solid; margin-right: 0px">
<div class="MessageRFC822Viewer" id="P">
<div class="TextPlainViewer" id="P.P">On Thu, 21 Dec 2023, at 18:56, Ronald Klop wrote:<br>
> A bit weird that a simple dd is so slow.<br>
><br>
> Just a quick thought.<br>
> Are your partitions aligned properly?<br>
<br>
no idea. How would I check? The system when installed,<br>
the auto-zfs option was selected, block size set to 4k. The swap<br>
was changed from the default 2gb to 12gb. geli encryption is<br>
active for the zfs filesystem but not for swap. It's only the swap<br>
that shows performance issues.<br>
<br>
> Is other IO going on on the same time?<br>
<br>
not really, the tests were done with cron jobs deactivated, almost completely idle,<br>
swap was not being used at all.<br>
<br>
> What does gstat say about %util, queue length and all the other stats<br>
> while running the dd? Or "iostat -x -d 1".<br>
<br>
I'll try that next, thanks<br>
<br>
> Could you try if another disk has the same issues?<br>
<br>
no spare or equivalent disk for this arch.<br>
<br>
> How is your disk connected? USB-to-SATA-adapter? Any output of dmesg?<br>
<br>
Regular usb3 connection.<br>
<br>
da0: 400.000MB/s transfers<br>
da0: 953869MB (1953525168 512 byte sectors)<br>
da0: quirks=0x2<NO_6_BYTE><br>
da0: Delete methods: <NONE(*),ZERO><br>
-- <br>
</div>
<hr></div>
</blockquote>
<br>
<br>
Can you provide more concrete information? Like:<br>
<br>
# usbconfig list<br>
ugen0.1: <(0x1106) XHCI root HUB> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=SAVE (0mA)<br>
ugen0.2: <vendor 0x2109 USB2.0 Hub> at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (100mA)<br>
ugen0.3: <Silicon Labs CP2102 USB to UART Bridge Controller> at usbus0, cfg=0 md=HOST spd=FULL (12Mbps) pwr=ON (100mA)<br>
ugen0.4: <USB 3.0 Device USB 3.0 Device> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)<br>
ugen0.5: <Western Digital Elements 25A2> at usbus0, cfg=0 md=HOST spd=SUPER (5.0Gbps) pwr=ON (224mA)<br>
<br>
And the output of "devinfo -v". That will give people much more information to work with instead of "anecdotal evidence" about your hardware being usb3.<br>
And the complete output of dmesg instead of only the "da0:" lines. There is more information in there which can be valuable for people to help you.<br>
<br>
About the alignment. You wrote in your first mail:<br>
"# gpart show<br>
<br>
=> 40 1953525088 da0 GPT (932G)<br>
40 532480 1 efi (260M)<br>
532520 2008 - free - (1.0M)<br>
534528 4194304 2 freebsd-swap (2.0G)<br>
4728832 4194304 4 freebsd-swap (2.0G)<br>
8923136 4194304 5 freebsd-swap (2.0G)<br>
13117440 4194304 6 freebsd-swap (2.0G)<br>
17311744 4194304 7 freebsd-swap (2.0G)<br>
21506048 4194304 8 freebsd-swap (2.0G)<br>
25700352 1927823360 3 freebsd-zfs (920G)<br>
1953523712 1416 - free - (708K)<br>
"<br>
<br>
Your dd tests were on da0p4 which starts on block 4728832.<br>
Assuming "gpart list" shows "Sectorsize: 512".<br>
(4728832 * 512) mod 4096 = 0, so the partitiion is aligned on a 4KB page size. And the calculation is also 0 for 8KB (is this the page size of arm64?) which is good.<br>
AFAIK: this is properly aligned such that the hardware does not need to double write or read-before-write when paging out.<br>
<br>
Up until now I haven't seen anything which would explain the slow dd speed on of=/dev/da0p4 vs the quick dd on of=/mnt/test8k.bin.<br>
<br>
Regards,<br>
Ronald.<br>
</body></html>
------=_Part_3079_229342031.1703237861046--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1137397689.3080.1703237862787>