Date: Wed, 8 Aug 2018 15:51:26 -0600 From: Warner Losh <imp@bsdimp.com> To: Mark Johnston <markj@freebsd.org> Cc: bob prohaska <fbsd@www.zefox.net>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org> Subject: Re: RPI3 swap experiments ["was killed: out of swap space" with: "v_free_count: 5439, v_inactive_count: 1"] Message-ID: <CANCZdfrbT_fxU9kt4CuZaeMV9W6=E1G8nDAmOjTY_-46QobJ=w@mail.gmail.com> In-Reply-To: <20180808204841.GA19379@raichu> References: <20180731231912.GF94742@www.zefox.net> <2222ABBD-E689-4C3B-A7D3-50AECCC5E7B2@yahoo.com> <20180801034511.GA96616@www.zefox.net> <201808010405.w7145RS6086730@donotpassgo.dyslexicfish.net> <6BFE7B77-A0E2-4FAF-9C68-81951D2F6627@yahoo.com> <20180802002841.GB99523@www.zefox.net> <20180802015135.GC99523@www.zefox.net> <EC74A5A6-0DF4-48EB-88DA-543FD70FEA07@yahoo.com> <20180806155837.GA6277@raichu> <20180808153800.GF26133@www.zefox.net> <20180808204841.GA19379@raichu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Aug 8, 2018 at 2:48 PM, Mark Johnston <markj@freebsd.org> wrote: > On Wed, Aug 08, 2018 at 08:38:00AM -0700, bob prohaska wrote: > > The patched kernel ran longer than default but OOMA still halted > buildworld around > > 13 MB. That's considerably farther than a default build world have run > but less than > > observed when setting vm.pageout_oom_seq=120 alone. Log files are at > > http://www.zefox.net/~fbsd/rpi3/swaptests/r337226M/ > 1gbsdflash_1gbusbflash/batchqueue/ > > > > Both changes are now in place and -j4 buildworld has been restarted. > > Looking through the gstat output, I'm seeing some pretty abysmal average > write latencies for da0, the flash drive. I also realized that my > reference to r329882 lowering the pagedaemon sleep period was wrong - > things have been this way for much longer than that. Moreover, as you > pointed out, bumping oom_seq to a much larger value wasn't quite > sufficient. > > I'm curious as to what the worst case swap I/O latencies are in your > test, since the average latencies reported in your logs are high enough > to trigger OOM kills even with the increased oom_seq value. When the > current test finishes, could you try repeating it with this patch > applied on top? https://people.freebsd.org/~markj/patches/slow_swap.diff > That is, keep the non-default oom_seq setting and modification to > VM_BATCHQUEUE_SIZE, and apply this patch on top. It'll cause the kernel > to print messages to the console under certain conditions, so a log of > console output will be interesting. Experience tells me that worst case is about 10x-50x median (which kinda tracks the average that gstat reports). For SD cards, it may be much much worse. Like seconds to tens of seconds worst case. I don't think the swapper is well tuned to devices that have such a large dynamic range and suck so bad when driven to the brink. Warner
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfrbT_fxU9kt4CuZaeMV9W6=E1G8nDAmOjTY_-46QobJ=w>