From owner-freebsd-arm@freebsd.org Sun Aug 12 17:34:13 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB46A10775F7 for ; Sun, 12 Aug 2018 17:34:13 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (phouka1.phouka.net [107.170.196.116]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "phouka.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 383A472041 for ; Sun, 12 Aug 2018 17:34:12 +0000 (UTC) (envelope-from warlock@phouka1.phouka.net) Received: from phouka1.phouka.net (localhost [127.0.0.1]) by phouka1.phouka.net (8.15.2/8.15.2) with ESMTPS id w7CHWmIT005945 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 12 Aug 2018 10:32:48 -0700 (PDT) (envelope-from warlock@phouka1.phouka.net) Received: (from warlock@localhost) by phouka1.phouka.net (8.15.2/8.15.2/Submit) id w7CHWmxE005944; Sun, 12 Aug 2018 10:32:48 -0700 (PDT) (envelope-from warlock) Date: Sun, 12 Aug 2018 10:32:48 -0700 From: John Kennedy To: bob prohaska Cc: freebsd-arm Subject: Re: RPI3 swap experiments ["was killed: out of swap space" with: "v_free_count: 5439, v_inactive_count: 1"] Message-ID: <20180812173248.GA81324@phouka1.phouka.net> References: <6BFE7B77-A0E2-4FAF-9C68-81951D2F6627@yahoo.com> <20180802002841.GB99523@www.zefox.net> <20180802015135.GC99523@www.zefox.net> <20180806155837.GA6277@raichu> <20180808153800.GF26133@www.zefox.net> <20180808204841.GA19379@raichu> <2DC1A479-92A0-48E6-9245-3FF5CFD89DEF@yahoo.com> <20180809033735.GJ30738@phouka1.phouka.net> <20180809175802.GA32974@www.zefox.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180809175802.GA32974@www.zefox.net> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Aug 2018 17:34:14 -0000 On Thu, Aug 09, 2018 at 10:58:02AM -0700, bob prohaska wrote: > ... I've gathered up what I've been given and put it in > http://www.zefox.net/~fbsd/rpi3/swaptests/tools/ > in hopes it'll be useful. ... Bob, are you doing anything clever to track/graph resources over time? I've got my box patched up to 12.0-ALPHA1 (r337589+2a27f5c34b07) + patches, and haven't seen any logged output (assuming it'll end up in /var/log/messages) although I'm certainly exercising things. I typically beat things up with a buildworld-then-buildkernel. Patches I've applied: http://www.zefox.net/~fbsd/rpi3/swaptests/tools/pageout.patch http://www.zefox.net/~fbsd/rpi3/swaptests/tools/slow_swap.patch I haven't used the CAM_IOSCHED_DYNAMIC option or adjusted vm.pageout_oom_seq. Just using top isn't really going to help me go back in time and diagnose it after the fact and capturing it's output (and all the screen codes) over an extended period of time seems pretty ugly. I can see why my ntpd tends to get whacked. At ~18M, it's almost always in the top 5 of memory-using processes. Load seems lower than it normally does (~4 for a -j4) and it seems a bit more swappy than I remember it being (r337443). I added a vfs.zfs.arc_max="450M" to /boot/loader.conf, but it didn't seen to be anywhere near that originally and doesn't seem to have taken effect: [sysctl vfs.zfs.arc_max] vfs.zfs.arc_max: 195873280 AFAIK, that would take up more "wired" space, but wired is only ~197M and ARC claims to only be ~50M so ZFS doesn't see to be too piggy. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= [top -IatP -o res] last pid: 75860; load averages: 1.38, 1.72, 1.95 up 0+13:42:25 10:04:05 65 processes: 65 sleeping CPU 0: 1.6% user, 0.0% nice, 5.1% system, 13.7% interrupt, 79.7% idle CPU 1: 6.6% user, 0.0% nice, 9.0% system, 0.0% interrupt, 84.4% idle CPU 2: 2.7% user, 0.0% nice, 9.0% system, 2.7% interrupt, 85.5% idle CPU 3: 16.0% user, 0.0% nice, 7.0% system, 0.4% interrupt, 76.6% idle Mem: 596M Active, 31M Inact, 51M Laundry, 197M Wired, 32K Buf, 19M Free ARC: 50M Total, 14M MFU, 16M MRU, 2758K Anon, 558K Header, 17M Other 6757K Compressed, 26M Uncompressed, 3.99:1 Ratio Swap: 3584M Total, 820M Used, 2764M Free, 22% Inuse, 3736K In, 5568K Out PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 75841 root 1 35 0 373M 55M swread 2 2:27 14.35% /usr/bin/c++ -cc1 -triple aarch64-unknown- 75844 root 1 39 0 423M 46M swread 3 2:45 21.76% /usr/bin/c++ -cc1 -triple aarch64-unknown- 821 ntpd 1 20 0 18M 18M select 3 0:16 0.02% /usr/sbin/ntpd -p /var/db/ntp/ntpd.pid -c 75811 root 1 22 0 346M 14M swread 1 2:41 2.05% /usr/bin/c++ -cc1 -triple aarch64-unknown- 75815 root 1 21 0 389M 7252K swread 1 3:09 1.86% /usr/bin/c++ -cc1 -triple aarch64-unknown- 2308 warlock 1 20 0 14M 2748K select 2 7:30 0.78% top -IatP -o cpu 75749 warlock 1 20 0 14M 2744K select 2 0:18 0.79% top -IatP -o cpu 2515 root 1 20 0 11M 1828K kqread 0 0:27 0.06% tail -F /var/log/messages 930 warlock 1 20 0 20M 1460K select 2 16:36 0.07% sshd: warlock@pts/0 (sshd) 935 warlock 1 20 0 19M 1288K select 3 46:14 0.38% screen