Date: Sat, 16 Dec 2023 11:16:31 -0700 From: Warner Losh <imp@bsdimp.com> To: freebsd-fs@freebsd.org Subject: Re: measuring swap partition speed Message-ID: <CANCZdfqi4ONxKMcuqZoYnCYL%2BJqoZaDCwcVEgtii_YZinhJmdQ@mail.gmail.com> In-Reply-To: <ZX2XDqnO2AKN8LTO@int21h> References: <ZXxis23iKT3iHDdt@int21h> <CANCZdfpgvE8UjX1XUgimbotpnrH-fXYWZU3kfAwD849bMrT2Vg@mail.gmail.com> <ZX2XDqnO2AKN8LTO@int21h>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000cdb043060ca4845f Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, Dec 16, 2023 at 5:24=E2=80=AFAM void <void@f-m.fm> wrote: > On Fri, Dec 15, 2023 at 08:41:10AM -0700, Warner Losh wrote: > > > What's the underlying hardware? > > my apologies, I should have supplied this detail initially. > It's a spinning rust TOSHIBA MQ01ABD100 > > > So the good news, kinda, is that if this is spinning rust, your swap > > partitions are > > on the fastest part of the disk. I'd expect 1 that's 12G would work > better > > than 6 that > > are 2G since you'd have less head thrash. Parllelism with multiple swap > > partitions > > works best when they are on separate spindles. > > ok that's easily fixable. It was originally one big 12G swapspace. > The reason it's split into 6 is for testing purposes. I've noticed when > swap gets used there's the marked slowdown even if the process using > it has paused. So I've found that using swapoff then swapon puts these > processes back into active memory and the system sluggishness goes, > for a short while until swap gets used again. With the swapspace split, > I can swapoff/on each partition individually. If there's 2GB free mem > and 3GB on swap, I can swapoff a couple of partitions, swapon them again > then swapoff/on the relatively full partition to get total swap use > back to 0. > I'm not sure that makes a lot of sense. Having one big partition with swap in use just means there's processes in the system that aren't paged into memory. That's totally fine most of the time. It makes for a larger RAM working set for everything else. At least that's the theory.. When you're exceeding the amount of RAM by a lot, weird things are going to happen... sounds like you are trying to cope as best you can with memory shortages... The best way around that is either to get more ram, or put less load on the system. > >The bad news is that your disk may be fine. I'd expect that as the ZFS > >partition fills up, > >the seek size will increase as greater distances have to be traversed to > >get back > >to the swap space. There's a sweet spot of a few tens of GB that drives > >usually can seek > >far faster than longer throws... > > (not an SSD) > > TYVM for the fio tip. Am building it now. > Great! Be careful with it: it writes to partitions. > I'm wondering really if I should reprovision this rpi4 for UFS rather > than zfs. Have had great positive experience with zfs as data (ie, bootin= g > ufs, zfs is a non-bootable single disk or array), less than ideal with > zfs-on-root. There's periodic backups and not much call for this device t= o > utilise zfs functionality that can't be provided on hardware with more > horsepower. > You can try that. I think you'll find that ZFS puts more memory pressure on your system. But the RPi4 likely has enough RAM to not be terrible (though given your swap issues, maybe not). UFS might be better, especially if you don't need the features of ZFS for this system. Warner > it must be said though that FreeBSD is *very stable* on this hardware. > > -- > > --000000000000cdb043060ca4845f Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Sat, Dec 16, 2023 at 5:24=E2=80=AF= AM void <<a href=3D"mailto:void@f-m.fm">void@f-m.fm</a>> wrote:<br></= div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bor= der-left:1px solid rgb(204,204,204);padding-left:1ex">On Fri, Dec 15, 2023 = at 08:41:10AM -0700, Warner Losh wrote:<br> <br> > What's the underlying hardware?<br> <br> my apologies, I should have supplied this detail initially.<br> It's a spinning rust TOSHIBA MQ01ABD100<br> <br> > So the good news, kinda, is that if this is spinning rust, your swap<b= r> > partitions are<br> > on the fastest part of the disk. I'd expect 1 that's 12G would= work better<br> > than 6 that<br> > are 2G since you'd have less head thrash. Parllelism with multiple= swap<br> > partitions<br> > works best when they are on separate spindles.<br> <br> ok that's easily fixable. It was originally one big 12G swapspace.<br> The reason it's split into 6 is for testing purposes. I've noticed = when<br> swap gets used there's the marked slowdown even if the process using<br= > it has paused. So I've found that using swapoff then swapon puts these<= br> processes back into active memory and the system sluggishness goes,<br> for a short while until swap gets used again. With the swapspace split,<br> I can swapoff/on each partition individually. If there's 2GB free mem<b= r> and 3GB on swap, I can swapoff a couple of partitions, swapon them again<br= > then swapoff/on the relatively full partition to get total swap use<br> back to 0.<br></blockquote><div><br></div><div>I'm not sure that makes = a lot of sense. Having one big partition with</div><div>swap in use just me= ans there's processes in the system that aren't</div><div>paged int= o memory. That's totally fine most of the time. It makes for</div><div>= a larger RAM working set for everything else. At least that's the theor= y..</div><div>When you're exceeding the amount of RAM by a lot, weird t= hings are</div><div>going to happen... sounds like you are trying to cope a= s best you can</div><div>with memory shortages...=C2=A0 The best way around= that is either to get more</div><div>ram, or put less load on the system.<= br></div><div>=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin= :0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"= > >The bad news is that your disk may be fine. I'd expect that as the = ZFS<br> >partition fills up,<br> >the seek size will increase as greater distances have to be traversed t= o<br> >get back<br> >to the swap space. There's a sweet spot of a few tens of GB that dr= ives<br> >usually can seek<br> >far faster than longer throws...<br> <br> (not an SSD)<br> <br> TYVM for the fio tip. Am building it now.<br></blockquote><div><br></div><d= iv>Great! Be careful with it: it writes to partitions.<br></div><div>=C2=A0= </div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;b= order-left:1px solid rgb(204,204,204);padding-left:1ex"> I'm wondering really if I should reprovision this rpi4 for UFS rather<b= r> than zfs. Have had great positive experience with zfs as data (ie, booting<= br> ufs, zfs is a non-bootable single disk or array), less than ideal with<br> zfs-on-root. There's periodic backups and not much call for this device= to<br> utilise zfs functionality that can't be provided on hardware with more<= br> horsepower.<br></blockquote><div><br></div><div>You can try that. I think y= ou'll find that ZFS puts more memory=C2=A0 pressure</div><div>on your s= ystem. But the RPi4 likely has enough RAM to not be terrible (though</div><= div>given your swap issues, maybe not). UFS might be better, especially if<= /div><div>you don't need the features of ZFS for this system. <br></div= ><div><br></div><div>Warner<br></div><div>=C2=A0</div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex"> it must be said though that FreeBSD is *very stable* on this hardware.<br> <br> -- <br> <br> </blockquote></div></div> --000000000000cdb043060ca4845f--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqi4ONxKMcuqZoYnCYL%2BJqoZaDCwcVEgtii_YZinhJmdQ>