From owner-freebsd-hackers@FreeBSD.ORG Sun Nov 4 16:36:17 2012 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11F6AD63 for ; Sun, 4 Nov 2012 16:36:17 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ie0-f182.google.com (mail-ie0-f182.google.com [209.85.223.182]) by mx1.freebsd.org (Postfix) with ESMTP id B1ED08FC16 for ; Sun, 4 Nov 2012 16:36:16 +0000 (UTC) Received: by mail-ie0-f182.google.com with SMTP id k10so9090433iea.13 for ; Sun, 04 Nov 2012 08:36:16 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=63yDsjz8PAzKhlYSRmP1Yuke8U3kKrQTEyn7nqTVvKY=; b=GiKnnCf8VU7NN0soy6m7E8/VisiG7ofGGaoIrFFg8Qd3iIVDPBWF9ms7qwByJwVIjO HLBZNVWLqi5sSqGPMbv5CGizh8UbWiPKJcOrbq/hbdlmSLe1xXauQq0R6t5R7nz9uwDU 253v9QsI2lX66FBznsX6QjRNitRP3HMGXRgXssM/2Zbm+WnjJNatzZvXE+PN+3/CoGIN YS7b2wJEG6xzBFBis5X4BIlaIPco9r8siCbnOc6+jhQxmO1e4a3LMpHnflnFdOjqcgOy RxXsIwrf+sdedKLcCir5BdSR9xb2Glk7oNihJVdnU+E53vqWn989hESB8E8iCbCIWtvy 5j1g== Received: by 10.50.88.168 with SMTP id bh8mr7197806igb.71.1352046976152; Sun, 04 Nov 2012 08:36:16 -0800 (PST) Received: from 53.imp.bsdimp.com (50-78-194-198-static.hfc.comcastbusiness.net. [50.78.194.198]) by mx.google.com with ESMTPS id l8sm4085807igo.13.2012.11.04.08.36.14 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 04 Nov 2012 08:36:15 -0800 (PST) Sender: Warner Losh Subject: Re: watchdogd, jemalloc, and mlockall Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1351968635.1120.110.camel@revolution.hippie.lan> Date: Sun, 4 Nov 2012 09:36:13 -0700 Content-Transfer-Encoding: 7bit Message-Id: <2D77347F-9913-4727-AA57-204AC266E15C@bsdimp.com> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> <1351968635.1120.110.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQn0m7lLtYXcP4VL5KoxO8e8URW5Ug4Q07oTUmBPfrNWS9UTrt60GjaBV1XbcXM+NPsy97mt Cc: Konstantin Belousov , freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Nov 2012 16:36:17 -0000 On Nov 3, 2012, at 12:50 PM, Ian Lepore wrote: > On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: >> On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: >>> In an attempt to un-hijack the thread about memory usage increase >>> between 6.4 and 9.x, I'm starting a new thread here related to my recent >>> discovery that watchdogd uses a lot more memory since it began using >>> mlockall(2). >>> >>> I tried statically linking watchdogd and it made a small difference in >>> RSS, presumably because it doesn't wire down all of libc and libm. >>> >>> VSZ RSS >>> 10236 10164 Dynamic >>> 8624 8636 Static >>> >>> Those numbers are from ps -u on an arm platform. I just updated the PR >>> (bin/173332) with some procstat -v output comparing with/without >>> mlockall(). >>> >>> It appears that the bulk of the new RSS bloat comes from jemalloc >>> allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect >>> that leads to wiring 8MB to satisfy what probably amounts to a few >>> hundred bytes of malloc'd memory. >>> >>> It would probably also be a good idea to remove the floating point from >>> watchdogd to avoid wiring all of libm. The floating point is used just >>> to turn the timeout-in-seconds into a power-of-two-nanoseconds value. >>> There's probably a reasonably efficient way to do that without calling >>> log(), considering that it only happens once at program startup. >> >> No, I propose to add a switch to turn on/off the mlockall() call. >> I have no opinion on the default value of the suggested switch. > > In a patch I submitted along with the PR, I added code to query the > vm.swap_enabled sysctl and only call mlockall() when swapping is > enabled. > > Nobody yet has said anything about what seems to me to be the real > problem here: jemalloc grabs 8MB at a time even if you only need to > malloc a few bytes, and there appears to be no way to control that > behavior. Or maybe there's a knob in there that didn't jump out at me > on a quick glance through the header files. Isn't that only for non-production builds? Warner