From owner-freebsd-current@freebsd.org Fri Jul 29 19:40:52 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F58DBA8FE4 for ; Fri, 29 Jul 2016 19:40:52 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C642B1A58; Fri, 29 Jul 2016 19:40:51 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (envelope-from ) id <1bTDeL-0042WW-A6>; Fri, 29 Jul 2016 21:40:49 +0200 Received: from x4e34a829.dyn.telefonica.de ([78.52.168.41] helo=thor.walstatt.dynvpn.de) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (TLSv1.2:AES256-GCM-SHA384:256) (envelope-from ) id <1bTDeL-001TYy-0u>; Fri, 29 Jul 2016 21:40:49 +0200 Date: Fri, 29 Jul 2016 21:41:42 +0200 From: "O. Hartmann" To: Allan Jude Cc: freebsd-current@freebsd.org Subject: Re: CURRENT: memory leak? Message-ID: <20160729214142.77f91579.ohartman@zedat.fu-berlin.de> In-Reply-To: <18fe457f-c99e-8747-8692-e199f356f6d5@freebsd.org> References: <20160729200458.2bb2c6ca.ohartman@zedat.fu-berlin.de> <18fe457f-c99e-8747-8692-e199f356f6d5@freebsd.org> Organization: FU Berlin X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/AVSlqi9vHhAxbxLv/Lq_PtY"; protocol="application/pgp-signature" X-Originating-IP: 78.52.168.41 X-ZEDAT-Hint: A X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 19:40:52 -0000 --Sig_/AVSlqi9vHhAxbxLv/Lq_PtY Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Am Fri, 29 Jul 2016 15:03:04 -0400 Allan Jude schrieb: > On 2016-07-29 14:04, O. Hartmann wrote: > > > > I realise an exorbitant memory usage of FreeBSD CURRENT ( FreeBSD 12.0-= CURRENT #16 > > r303470: Fri Jul 29 05:58:42 CEST 2016 ). Swap space gets eaten up whil= e building > > world/kernel and/or ports very quickly. > > > > I see this phenomenon on different CURRENT systems with different RAM (= but all ZFS!). > > No box is less than 8 GB RAM: one 8GB, another 16, two 32 GB. An older = XEON Core2Duo > > server with postgresql 9.5/postgis acting on some OSM data etas up all = of its 32 GB > > and additional 48GB swap - never seen before with 11-CURRENT. > > > > I didn't investigate the problem so far since I realized this memory hu= nger of > > 12-CURRENT just today on several boxes compiling world, eating up all t= he memory, > > staring swapping and never relax even after hours from the swapped memo= ry. > > > > Is this a known phenomenon or am I seeing something mystique? > > > > Regards, > > > > Oliver > > =20 >=20 > Do you have the output of 'top', the first few lines >=20 > Specifically, is there very high 'Other' usage, on the ZFS ARC line? >=20 I had to reboot the worst box since it didn't answer on any input or ssh. I= also just restarted the other systems after buildworld/buildkernel of the most recent= src ... I try VBox - it complained last time not having enough memory after I simply buildworld/portupdated ... This is head of top just before I start buildworld/buildkernel on the 8 GB = (dual core/ 4 threads box): last pid: 1645; load averages: 0.13, 0.15, 0.28 up 0+01:08:06 21:24:01 61 processes: 1 running, 60 sleeping CPU: 0.2% us= er, 0.0% nice, 0.4% system, 0.0% interrupt, 99.4% idle Mem: 118M Active, 547M Inac= t, 4376M Wired, 667M Buf, 2788M Free ARC: 2411M Total, 836M MFU, 363M MRU, 32K Anon,= 27M Header, 1185M Other Swap: 64G Total, 64G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU CO= MMAND 1002 root 1 20 0 12434M 49192K select 2 0:39 1.34% Xo= rg 514 root 1 20 0 12748K 2252K select 0 0:22 0.27% mo= used 1027 ohartmann 1 20 0 125M 20488K select 0 0:02 0.14% wm= aker 1634 ohartmann 1 20 0 76020K 11988K select 2 0:00 0.09% xt= erm 1645 root 1 20 0 22260K 3588K CPU1 1 0:00 0.08% top 403 root 1 20 0 9588K 5060K select 2 0:00 0.01% de= vd 900 root 1 20 0 88148K 9768K select 2 0:00 0.00% se= ndmail 563 root 1 20 0 12608K 2476K select 0 0:00 0.00% sy= slogd 793 root 1 20 0 22764K 12632K select 0 0:00 0.00% nt= pd [...] This is the top after "make -j4 buildkernel buildworld" (filemon.ko enabled= ) and afterwards a "make update" in /usr/ports which resides on an ZFS 3TB disk a= nd is svn backed and after that 5 minutes wating/relaxing: last pid: 11935; load averages: 0.10, 0.55, 0.64 up 0+01:21:10 21:37:05 66 processes: 1 running, 65 sleeping CPU: 0.4% user, 0.0% nice, 0.6% system, 0.0% interrupt, 99.0% idle Mem: 308M Active, 2805M Inact, 4431M Wired, 775M Buf, 284M Free ARC: 2470M Total, 1007M MFU, 279M MRU, 32K Anon, 31M Header, 1153M Other Swap: 64G Total, 64G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU CO= MMAND 1002 root 1 21 0 12433M 44044K select 1 1:02 1.47% Xo= rg 514 root 1 20 0 12748K 2248K select 1 0:24 0.29% mo= used 1027 ohartmann 1 21 0 125M 16536K select 1 0:09 0.18% wm= aker 1645 root 1 20 0 22260K 4096K CPU2 2 0:01 0.08% top 1032 ohartmann 1 20 0 86260K 20344K select 3 0:06 0.06% xt= erm 1634 ohartmann 1 20 0 76020K 10304K select 1 0:01 0.03% xt= erm 21103 ohartmann 1 20 0 76020K 10296K select 1 0:00 0.02% xt= erm 21553 ohartmann 1 20 0 26776K 6136K select 2 0:00 0.00% ssh 793 root 1 20 0 22764K 12632K select 1 0:00 0.00% nt= pd 820 root 1 20 0 43744K 3184K select 0 0:00 0.00% sa= ned 1026 ohartmann 1 20 0 33592K 4028K select 0 0:00 0.00% gp= g-agent [...] I do not understand much of the memory partitioning, but starting with ~ 2 = GB free RAM and ending up with < 300MB free RAM - if correctly interpreted - seems weir= d. I'm out of the office, so I can not access the box that is literaly "explod= ing" right now. Hope you can see something in the glass sphere ... Kind regards, thank you very much in advance, Oliver --Sig_/AVSlqi9vHhAxbxLv/Lq_PtY Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBCAAGBQJXm7F2AAoJEOgBcD7A/5N8TKsH+wckErZE1JOe6yhN3nGFyA0r dxAzfKJpac5KLyV8QeZ5UakY6MdQNx4ZLchVeb750vrZfvpTrLcrKuJDXcDWtS9E p8zKh33O8NOW6frpGD6FOa9vLVtv7ecUGwz70Oi3xlkkiFQzGuU5rdfIdW2zvORy 8nUmiKdHsFUO8+iY0qk9a0jL3wvp+U9OeBqIKVj5jmSxN1CXaJ7rau+K6XABx7JS yjWWwomk9mBbIsxoA2m7Zu2p2A7yenfP8eFMzZsI9wwl9Suo+m9S8RwtLLPcBbS4 NPWrVJz+n55kE8gczwvzQx0LnO0UXy3VkaojaaBrhwjF+vZgI0fqIDKDLOWhIcA= =bdS/ -----END PGP SIGNATURE----- --Sig_/AVSlqi9vHhAxbxLv/Lq_PtY--