Date: Wed, 18 Jul 2018 20:53:05 -0700 From: John Kennedy <warlock@phouka.net> To: bob prohaska <fbsd@www.zefox.net> Cc: freebsd-arm@freebsd.org Subject: Re: RPI3 swap experiments Message-ID: <20180719035305.GB92843@phouka1.phouka.net> In-Reply-To: <20180718060650.GA24566@www.zefox.net> References: <c6b8842a-fcc5-8e11-5a03-ba76eb3c5dea@sentry.org> <20180629233937.GC35717@www.zefox.net> <0f137e06-214a-3e8c-a216-f061ec04ac2c@sentry.org> <20180630005145.GA43801@www.zefox.net> <6f3406e2-71f3-d0c2-2b65-703e1a1d3c25@sentry.org> <8e92b2b7-da61-3efb-7231-9fac76b2c1d4@sentry.org> <ba33d8a7-a849-3893-8016-0765ebe1c51f@sentry.org> <2deaaec3-f78f-0b09-5ca7-27e14c6979f9@sentry.org> <20180704004554.GA61273@www.zefox.net> <20180718060650.GA24566@www.zefox.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Jul 17, 2018 at 11:06:50PM -0700, bob prohaska wrote: > It appears that some progress has been made in getting swap working reasonably > on the RPI3. A -j4 buildworld attempt running r336356 to compile 336431 failed > with "out of swap" but the worst read and write delays were less than 5 seconds, > a marked improvement over previous examples. > > In this case swap was split, 2 each 1GB partitions on USB flash plus 1 GB on the > microSD card. Previous attempts using 3 each 1 GB partitions on USB flash have been > repeatedly successful, while a single attempt using 3 each 1GB partitions on microSD > failed. So I've got a RPI3B+. YMMV. I've recompiled 12 ~7 times since June 27th and run into a handful of problems but haven't had issues lately. I think I started out with the stock images (*-RPI3-20180618-r335317 and *-RPI3-20180709-r336134). I had OOM problems, so added swap (typical swap-on-ufs). Could only get ~3.6G of swap without triggering the kern.maxswzone warning, and then had the pauses (forget the exact text, but basically FreeBSD telling me that something disk related had timed out after ~20 secs). That stopped being a problem as I took the stock images and created a real freebsd-swap partition (and maybe a SDCARD upgrade). Just a gut feeling, but I figured that the kernel recompile was causing enough buffering (and perhaps double-buffering with swap-on-ufs) that it was getting itself soft-wedged (so timeouts, and OOM even though it didn't look like I was getting anywhere near maxing out ~3.6G of swap). OOM, kill ntpd, then eventually kill part of the build and eventually the build would fail. I didn't know if there might have been some memory spikes that blew it up that I wasn't seeing with top. I could restart and it would eventually finish, but building everything takes too long as it is (very spoiled by beefy AMD64 box, but RPI3B+ destroys my old RPI1). So yes, I've (re)compiled everything on just an SDCARD. I'll note that said SDCARD is a SanDisk Extreme Plus 64G (UHS3 / V30, so ~30MB/s). I think I was using a SanDisk Ultra Plus (UHS1, no V-speed, so maybe ~10MB/s) initially because it was laying around. That was an educated guess on my part that something that could sink 30MB/s of video would make a better sink that something that averaged out to ~10MB/s data, and that spending more than that was a waste based on iostat stats (I think topping out at ~6+MB/s with -w5). I think I managed to get up to ~12+MB/s with some dd-bs=1M chunking. I theory, I think the RPI3 should basically have a ~60MB/s USB2 "backplane", so not sure what the bottleneck is. At the moment, I have /usr/src and /usr/obj NFS-mounted, mostly to waste a lot of time copying that work around while I try to figure out what is wrong with my attempted ZFS setup.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20180719035305.GB92843>