From owner-freebsd-arm@freebsd.org Sat Feb 17 17:49:49 2018 Return-Path: Delivered-To: freebsd-arm@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0855F2395C for ; Sat, 17 Feb 2018 17:49:49 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from pmta2.delivery6.ore.mailhop.org (pmta2.delivery6.ore.mailhop.org [54.200.129.228]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2DBCB8B440 for ; Sat, 17 Feb 2018 17:49:48 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: de1b5e24-140a-11e8-b951-f99fef315fd9 X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 67.177.211.60 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [67.177.211.60]) by outbound2.ore.mailhop.org (Halon) with ESMTPSA id de1b5e24-140a-11e8-b951-f99fef315fd9; Sat, 17 Feb 2018 17:49:14 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id w1HHnfVx049827; Sat, 17 Feb 2018 10:49:41 -0700 (MST) (envelope-from ian@freebsd.org) Message-ID: <1518889781.91697.3.camel@freebsd.org> Subject: Re: Pi3 out of swap at < 50% From: Ian Lepore To: bob prohaska Cc: freebsd-arm@freebsd.org Date: Sat, 17 Feb 2018 10:49:41 -0700 In-Reply-To: <20180217173414.GB93736@www.zefox.net> References: <20180217162732.GA93736@www.zefox.net> <1518885801.91697.2.camel@freebsd.org> <20180217173414.GB93736@www.zefox.net> Content-Type: text/plain; charset="ISO-8859-1" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Porting FreeBSD to ARM processors." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 17:49:49 -0000 On Sat, 2018-02-17 at 09:34 -0800, bob prohaska wrote: > On Sat, Feb 17, 2018 at 09:43:21AM -0700, Ian Lepore wrote: > > > > On Sat, 2018-02-17 at 08:27 -0800, bob prohaska wrote: > > > > > > Running make -j4 buildworld on a Pi3 at r329360 tends to result > > > in??messages similar to: > > > > > > pid 33492 (llvm-tblgen), uid 0, was killed: out of swap space > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 11764, size: 28672 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 16080, size: 4096 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 15721, size: 20480 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28139, size: 65536 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 40544, size: 4096 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 58384, size: 65536 > > > pid 49735 (c++), uid 0, was killed: out of swap space > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28470, size: 4096 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 24889, size: 8192 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 28736, size: 4096 > > > swap_pager: indefinite wait buffer: bufobj: 0, blkno: 29736, size: 4096 > > > > > > even though swap usage appears to be less than one GB out of two available. > > > > > > So far it's been possible to use the -DNO_CLEAN option with make to pick > > > up where things left off and I've backed down to -j3 to see if that helps. > > > > > > There are two 1 GB swap partitions, one on USB flash and one on the microSD > > > card and both appear to be in use according to swapinfo. There are no warnings > > > in the boot messages and no explicit changes have been made to configuration > > > files apart from adding the swap entries to /etc/fstab. > > > > > > World and kernel are about two weeks out of sync, so top isn't perfectly > > > up-to-date with this kernel; could that account for the mismatch between > > > apparent swap usage and the "out of swap" messages? > > > > > > Thanks for reading, and any ideas! > > > > > > bob prohaska > > I suspect your swap devices are too slow to keep up with instantaneous > > demands. ?An sdcard can easily have read and write latencies as big as > > 30 seconds; they make pretty poor swap devices (have gstat running in > > another window during the build to see what I mean, look at the ms/r > > and ms/w columns). ?USB flash drives may not be much better. > > > I'm watching gstat now. With make -j3 running and three cc processes in > the top of top the usual value of ms/r and ms/w is less than 5 ms. An > eyeball average is probably less than 10 ms. Before cc got going (make > dominant in top) there were very brief excursions to 2000 ms, as you posit, > but they didn't last and didn't put any errors on the console.  > > The experiment ended when make stopped with: > > /usr/src/lib/libc/rpc/auth_time.c:115:10: error: unknown type name 'endpoint' > free_eps(endpoint eps[], int num) >          ^ > /usr/src/lib/libc/rpc/auth_time.c:143:8: error: unknown type name 'nis_server' > static nis_server * >        ^ > /usr/src/lib/libc/rpc/auth_time.c:144:49: error: unknown type name 'nis_server' > get_server(struct sockaddr_in *sin, char *host, nis_server *srv, >                                                 ^ > /usr/src/lib/libc/rpc/auth_time.c:145:5: error: unknown type name 'endpoint' >     endpoint eps[], int maxep)  > > among others. > > I'll update sources and restart buildworld with -j4, watching gstat more > carefully.  > > Thanks very much for the hint! > > bob prohaska Part of the problem is clang-tblgen, which requires HUGE amounts of ram; if you've gotten beyond that in the build, you may not see any more problems. -- Ian