From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 17 18:10:43 2015 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 49402CD1 for ; Tue, 17 Mar 2015 18:10:43 +0000 (UTC) Received: from mx144.netapp.com (mx144.netapp.com [216.240.21.25]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "mx144.netapp.com", Issuer "VeriSign Class 3 International Server CA - G3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 189731E8 for ; Tue, 17 Mar 2015 18:10:42 +0000 (UTC) X-IronPort-AV: E=Sophos;i="5.11,417,1422950400"; d="scan'208,217";a="30251678" Received: from hioexcmbx03-prd.hq.netapp.com ([10.122.105.36]) by mx144-out.netapp.com with ESMTP; 17 Mar 2015 11:09:37 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com (10.122.105.36) by hioexcmbx03-prd.hq.netapp.com (10.122.105.36) with Microsoft SMTP Server (TLS) id 15.0.995.29; Tue, 17 Mar 2015 11:09:36 -0700 Received: from HIOEXCMBX03-PRD.hq.netapp.com ([::1]) by hioexcmbx03-prd.hq.netapp.com ([fe80::4c9:7f9e:4b9f:2c9c%21]) with mapi id 15.00.0995.031; Tue, 17 Mar 2015 11:09:36 -0700 From: "Gumpula, Suresh" To: "freebsd-hackers@freebsd.org" Subject: Re: BSD 8.1 and 9.1 memory increase Thread-Topic: BSD 8.1 and 9.1 memory increase Thread-Index: AQHQYN1kO9bXTk+QcE+8E2hd8pdV+Z0hK5sA Date: Tue, 17 Mar 2015 18:09:35 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.4.7.141117 x-originating-ip: [10.122.56.79] MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 Mar 2015 18:10:43 -0000 On similar notes , when I looked at procstat -v , it does not show us the = shared libraries resident usage . I have added a column to display the shared resident count. kern_proc.c: if (obj->shadow_count > 1) kve->kve_shared_resident =3D obj->resident_page_count; procstat_vm.c : if (kve->kve_private_resident || kve->kve_shared_resident) entry_total =3D(kve->kve_private_resident + kve->kv= e_shared_resident); else entry_total =3D kve->kve_resident; proc_total +=3D entry_total; shared_total +=3D kve->kve_shared_resident; private_total +=3D kve->kve_private_resident; mapped_total +=3D kve->kve_resident; printf("%8d ", kve->kve_shared_resident); printf("%8d ", entry_total); % sudo procstat -v `pgrep bash` PID START END PRT RES PRES SHRD = TOTAL REF SHD FL TP PATH 3485 0x400000 0x4b0000 r-x 128 151 0 = 151 2 1 CN-- vn /usr/bin/bash 3485 0x6b0000 0x6b9000 rw- 9 0 0 = 9 1 0 C--- vn /usr/bin/bash 3485 0x6b9000 0x800000 rw- 5 0 0 = 5 1 0 C--- df 3485 0x8006b0000 0x8006c9000 r-x 25 0 0 = 25 38 0 CN-- vn /libexec/ld-elf.so.1 3485 0x8006c9000 0x8006ef000 rw- 26 0 0 = 26 1 0 CN-- df 3485 0x8008c8000 0x8008ca000 rw- 2 0 0 = 2 1 0 CN-- df 3485 0x8008ca000 0x800914000 r-x 40 0 79 = 79 6 3 CN-- vn /lib/libncurses.so.8 3485 0x800914000 0x800b14000 --- 0 0 0 = 0 1 0 CN-- df 3485 0x800b14000 0x800b19000 rw- 5 0 0 = 5 1 0 CN-- vn /lib/libncurses.so.8 3485 0x800b19000 0x800c71000 r-x 318 0 331 = 331 75 37 CN-- vn /lib/libc.so.7 3485 0x800c71000 0x800e71000 --- 0 0 0 = 0 1 0 CN-- df 3485 0x800e71000 0x800e7c000 rw- 11 0 0 = 11 1 0 CN-- vn /lib/libc.so.7 3485 0x800e7c000 0x800ee2000 rw- 9 0 0 = 9 1 0 CN-- df 3485 0x801000000 0x801200000 rw- 35 0 0 = 35 1 0 C--- df 3485 0x7ffffffdf000 0x7ffffffff000 rw- 5 0 0 = 5 1 0 C--D df 3485 0x7ffffffff000 0x800000000000 r-x 1 0 0 = 1 40 0 CN-- ph Totals(pages) 619 151 410 = 694 Can somebody pick up this change if it looks good ? Thanks Suresh From: , suresh gumpula > Date: Tuesday, March 17, 2015 at 2:08 PM To: "freebsd-hackers@freebsd.org" > Subject: BSD 8.1 and 9.1 memory increase Hello VM experts, I am trying to figure out what has been changed from 8.1 to 9.1 which r= esults more memory footprint on all processes. looking at one of the big pr= ocesses we have on a idle machine, its about 35M resident size increase. Looking at map entries by procstat -= v shows me that two libraries , one is our internal library(libmgwd) and ot= her one is boost consume more now. There are no changes made with respect to process, just comparing after the= upgrade to 9.1. Are here any knows things with respect VM have been changed and could resul= t in more resident memory ? Can somebody please help on this to know what e= xactly causing this behaviour ? Details below. Thank you! 8.1 =97 % sudo procstat -v `pgrep mgwd` PID START END PRT RES PRES REF SHD FL TP P= ATH 2213 0x800a46000 0x807b41000 r-x 18209 22827 2 1 CN vn /= usr/lib/libmgwd.so.1 2213 0x807d40000 0x808a90000 rw- 3019 0 1 0 C- vn /= usr/lib/libmgwd.so.1 2213 0x833406000 0x833407000 rw- 1 0 1 0 CN vn /us= r/lib/libboost_atomic.so.1.56.0 2213 0x833600000 0x835600000 rw- 8134 0 1 0 C- df 2213 0x835600000 0x835800000 rw- 20 0 1 0 -- df 2213 0x7ffff6f99000 0x7ffff6fb9000 rw- 1 0 1 0 -- df 2213 0x7ffff719a000 0x7ffff71ba000 rw- 1 0 1 0 -- df 2213 0x7ffff739b000 0x7ffff73bb000 rw- 12 0 1 0 -- df 9.1 =97=97 % sudo procstat -v `pgrep mgwd` PID START END PRT RES PRES REF SHD FL TP P= ATH 2158 0x800a1c000 0x807c87000 r-x 23328 26760 2 1 CN-- vn= /usr/lib/libmgwd.so.1 2158 0x807e87000 0x808bd4000 rw- 3283 0 1 0 C--- vn= /usr/lib/libmgwd.so.1 2158 0x8336d7000 0x8336d8000 rw- 1 0 1 0 CN-- vn /= usr/lib/libboost_atomic.so.1.56.0 2158 0x8336d8000 0x8342cf000 rw- 2105 0 1 0 C-S- df 2158 0x8342cf000 0x8342ea000 rw- 27 0 1 0 C--- df 2158 0x8342ea000 0x8342f3000 rw- 7 0 1 0 ---- df 2158 0x834400000 0x834600000 rw- 511 0 1 0 C--- df 2158 0x834600000 0x836400000 rw- 7375 0 1 0 C--- df 2158 0x836400000 0x836600000 rw- 228 0 1 0 ---- df BSD 8.1 =3D=3D=3D=3D=3D=3D last pid: 5116; load averages: 5.22, 4.29, 2.34 up 0+00:10:15 17:3= 4:00 352 processes: 1 running, 350 sleeping, 1 zombie CPU: 0.2% user, 0.0% nice, 4.5% system, 1.7% interrupt, 93.5% idle Mem: 297M Active, 648M Inact, 139M Wired, 6948K Cache, 7520K Buf, 1862M Fre= e Swap: 1536M Total, 1536M Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMA= ND 2219 root 68 96 0 859M 226M ucond 1 0:24 0.00% mgwd BSD 9.1 =97=97=97 last pid: 5344; load averages: 5.17, 4.47, 2.79 up 0+00:26:12 17:2= 2:57 39 processes: 1 running, 37 sleeping, 1 zombie CPU: 0.2% user, 0.0% nice, 2.2% system, 0.6% interrupt, 97.0% idle Mem: 338M Active, 669M Inact, 147M Wired, 392K Cache, 7488K Buf, 1799M Free Swap: 1536M Total, 1536M Fre PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMA= ND 2158 root 68 40 0 874M 262M uwait 1 0:23 0.00% mgwd =3D=3D=3D=3D=3D=3D=3D=3D % ldd /usr/lib/libmgwd.so.1 /usr/lib/libmgwd.so.1: % ldd /usr/lib/libboost_atomic.so.1.56.0 /usr/lib/libboost_atomic.so.1.56.0: librt.so.1 =3D> /usr/lib/librt.so.1 (0x801002000) libthr.so.3 =3D> /lib/libthr.so.3 (0x801207000) ;;image size comparison on different builds shows TEXT size of a libray wen= t up by ~1.5M text data bss dec hex filename 118484184 139568805724944 13816600883c3ef8 devN_150110_0500/mgmtgateway/bed= rock/internal/x86_64/ulibso/libmgwd.so.1 119973051 139460085724944 139644003852cc63 devN_150110_1035/mgmtgateway/bed= rock/internal/x86_64/ulibso/libmgwd.so.1 BSD 9.1 =97=97 % size /sbin/mgwd text data bss dec hex filename 2106422 42388 371584 2520394 26754a /sbin/mgwd % size /usr/lib/libboost_atomic.so.1.56.0 text data bss dec hex filename 1309 264 2624 4197 1065 /usr/lib/libboost_atomic.so.1.56.0 BSd 8.1 =97=97=97 % size /sbin/mgwd text data bss dec hex filename 2091457 42364 371520 2505341 263a7d /sbin/mgwd % size /usr/lib/libboost_atomic.so.1.56.0 text data bss dec hex filename 1309 264 2624 4197 1065 /usr/lib/libboost_atomic.so.1.56.0 Thanks Suresh