From owner-freebsd-embedded@FreeBSD.ORG Mon Oct 29 11:06:32 2012 Return-Path: Delivered-To: freebsd-embedded@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0F388B32 for ; Mon, 29 Oct 2012 11:06:32 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E81178FC0C for ; Mon, 29 Oct 2012 11:06:31 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q9TB6VZW028439 for ; Mon, 29 Oct 2012 11:06:31 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q9TB6VOB028437 for freebsd-embedded@FreeBSD.org; Mon, 29 Oct 2012 11:06:31 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 29 Oct 2012 11:06:31 GMT Message-Id: <201210291106.q9TB6VOB028437@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-embedded@FreeBSD.org Subject: Current problem reports assigned to freebsd-embedded@FreeBSD.org X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Oct 2012 11:06:32 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/172968 embedded [arswitch] probe/attach occasionally fails to find a P o misc/52256 embedded [picobsd] picobsd build script does not read in user/s o kern/42728 embedded [picobsd] many problems in src/usr.sbin/ppp/* after c 3 problems total. From owner-freebsd-embedded@FreeBSD.ORG Tue Oct 30 14:05:43 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 97C0A88C for ; Tue, 30 Oct 2012 14:05:43 +0000 (UTC) (envelope-from jack.mclauren@yahoo.com) Received: from nm15-vm4.bullet.mail.ne1.yahoo.com (nm15-vm4.bullet.mail.ne1.yahoo.com [98.138.91.175]) by mx1.freebsd.org (Postfix) with ESMTP id B58038FC08 for ; Tue, 30 Oct 2012 14:05:42 +0000 (UTC) Received: from [98.138.90.57] by nm15.bullet.mail.ne1.yahoo.com with NNFMP; 30 Oct 2012 14:05:42 -0000 Received: from [98.138.89.252] by tm10.bullet.mail.ne1.yahoo.com with NNFMP; 30 Oct 2012 14:05:41 -0000 Received: from [127.0.0.1] by omp1044.mail.ne1.yahoo.com with NNFMP; 30 Oct 2012 14:05:41 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 983823.64192.bm@omp1044.mail.ne1.yahoo.com Received: (qmail 87738 invoked by uid 60001); 30 Oct 2012 14:05:41 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s1024; t=1351605941; bh=Xd/T0fC7ITQgilwTxVKUCmnL2QPJtH+Ms53AHQnFlO4=; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=sPD+182cpR9y1kQ7HuD9+tv6pcKGYIhFg0ZGPXRtXYjyEuXpwEY8Tf0g5dIxYqI5tAx0U31Q7eNCja9wM9fRwiEOLeVfZ+xkc/u7vSFYwYK5bmzwuwDljp83sy+Lr0FbiSwd9FKxEOiEhxSEqKW/1dta0+d1zO18SMjmh6dSmVs= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:Message-ID:Date:From:Reply-To:Subject:To:MIME-Version:Content-Type; b=xiCYif807iN+BH20sE5pqcOOT6Rjfl5Nw8VbQUos7/31+6XvWJwU2F8INxrWiVNjGE7ErjIMwPDHWDx/8XJOiKylxxvE0rmEKdFf+Vds9trIIucqLcdN3JbltvuCN3QntRjVImVdheeiMHp8cJzobhwLqhWKOz+dvTUN48NhAkw=; X-YMail-OSG: SeIxRBYVM1mH4ekJPz3a90jnX5ReMcb05rSOL.Z2Vip7gmg xc3dnN5gLm.rrsII5knlIpxLXLJvH3HOzTPER0pPEvMCuMTT3JIZN_h1knsz Zvrk5_Y.ixZLjgwW5w2QPCQStDqEbADHJ_vpzPtFYjwWhm_fezJPuITAaxE7 yx8j_ao4CP5vHvw1v1nSz1Hk3aQxAYZymmWbabj8pzAqxEn5Kyq0xhlv9_Xt XWXxR21_orMJqSx1v84bksuQ9yCMojhaS4.ZPHQPp6nEFjtLlmS6Ewa8Gz5c 6WwclcxzImz.AvXSZi7QvI6hsZnTzz91X.5Vx0kmD3bcGL4mmxiyx_EpLfaS OToFgbuTX45u1CiFylb49O3adLe8d1mcWkxT5YR1sp68arQ8Bl_ArTh_Zb_D TK7_7fzQyiNPR2Sim.uDSLPy0wLeItTarahIO8xct6jpvDggs1v0- Received: from [89.165.120.140] by web126002.mail.ne1.yahoo.com via HTTP; Tue, 30 Oct 2012 07:05:41 PDT X-Rocket-MIMEInfo: 001.001, SGVsbG8KCkRvZXMgY3VzdG9taXppbmcgKGNvbW1lbnRpbmcgdW51c2VkIGRldmljZXMgaW4gR0VORVJJQykga2VybmVsIGluIC91c3Ivc3JjL2FtZDY0L2NvbmYgY2F1c2UgbmFub2JzZCBpbWFnZSBnZXQgbGlnaHRlciA_PwEwAQEBAQ-- X-Mailer: YahooMailWebService/0.8.123.460 Message-ID: <1351605941.79450.YahooMailNeo@web126002.mail.ne1.yahoo.com> Date: Tue, 30 Oct 2012 07:05:41 -0700 (PDT) From: Jack Mc Lauren Subject: GENERIC kernel configuraion To: "freebsd-embedded@freebsd.org" MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jack Mc Lauren List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 14:05:43 -0000 Hello Does customizing (commenting unused devices in GENERIC) kernel in /usr/src/amd64/conf cause nanobsd image get lighter ?? From owner-freebsd-embedded@FreeBSD.ORG Tue Oct 30 14:50:35 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 973AAB17 for ; Tue, 30 Oct 2012 14:50:35 +0000 (UTC) (envelope-from freebsd@psconsult.nl) Received: from mx1.psconsult.nl (unknown [IPv6:2001:7b8:30f:e0::5059:ee8a]) by mx1.freebsd.org (Postfix) with ESMTP id 45D878FC0C for ; Tue, 30 Oct 2012 14:50:32 +0000 (UTC) Received: from mx1.psconsult.nl (mx1.hvnu.psconsult.nl [46.44.189.154]) by mx1.psconsult.nl (8.14.5/8.14.4) with ESMTP id q9UEoOMX017534 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 30 Oct 2012 15:50:29 +0100 (CET) (envelope-from freebsd@psconsult.nl) Received: (from paul@localhost) by mx1.psconsult.nl (8.14.5/8.14.4/Submit) id q9UEoOFc017533 for freebsd-embedded@freebsd.org; Tue, 30 Oct 2012 15:50:24 +0100 (CET) (envelope-from freebsd@psconsult.nl) X-Authentication-Warning: mx1.psconsult.nl: paul set sender to freebsd@psconsult.nl using -f Date: Tue, 30 Oct 2012 15:50:24 +0100 From: Paul Schenkeveld To: freebsd-embedded@freebsd.org Subject: Re: GENERIC kernel configuraion Message-ID: <20121030145024.GA16785@psconsult.nl> References: <1351605941.79450.YahooMailNeo@web126002.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1351605941.79450.YahooMailNeo@web126002.mail.ne1.yahoo.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 14:50:35 -0000 On Tue, Oct 30, 2012 at 07:05:41AM -0700, Jack Mc Lauren wrote: > Hello > > Does customizing (commenting unused devices in GENERIC) kernel in /usr/src/amd64/conf cause nanobsd image get lighter ?? Here are two amd64 NanoBSD kernels I've archived: # ls -ls kernel1 kernel2 15 -r-xr-xr-x 1 root wheel 15572950 Apr 9 2012 kernel1 7 -r-xr-xr-x 1 root wheel 6921921 Jul 25 11:27 kernel1 # size kernel1 kernel2 text data bss dec hex filename 12011364 1331748 1137248 14480360 dcf3e8 kernel1 5155629 513459 637256 6306344 603a28 kernel2 The first one is FreeBSD 9.0, the second one FreeBSD 8.3 but 9.0 is not twice as big as 8.3. So yes, trimming /usr/src/amd64/conf can make a big difference! If memory is your constraint, trim the kernel as much as possible, don't forget to strip things like SCTP, NFS*, INET6, MAC etc. if you don't need them and leave "makeoptions DEBUG=-g" out too. If it's the size of your flash image you worry about, put in a line like makeoptions MODULES_OVERRIDE="" to not build/install kernel modules or include only those that you really need. /boot/kernel takes almost 50% of the space in a NanoBSD image with a generic kernel and all modules built. Another thing that can save a lot of space on your flash image is tuning CONF_WORLD, CONF_BUILD and CONF_INSTALL (see src/tools/build/options/ for an explanation of each option). This may take several iterations before you get it right though because sometimes it's not immediately visible which options are required by other ones (eg. if you build with INET6 enabled, you appear to need KVM and KVM_SUPPORT too). HTH Paul Schenkeveld From owner-freebsd-embedded@FreeBSD.ORG Tue Oct 30 16:05:07 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4AA8B8E8 for ; Tue, 30 Oct 2012 16:05:07 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from mail-ob0-f182.google.com (mail-ob0-f182.google.com [209.85.214.182]) by mx1.freebsd.org (Postfix) with ESMTP id 01E3B8FC0C for ; Tue, 30 Oct 2012 16:05:06 +0000 (UTC) Received: by mail-ob0-f182.google.com with SMTP id wc20so548543obb.13 for ; Tue, 30 Oct 2012 09:05:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer :x-gm-message-state; bh=sCuwzG6djZFv5s8N5kUUcrraF5Na6KqZCYtAGzHj7fY=; b=Zi879cb0NsGOXSvfrU0GWFusd6p/7yYbWa+R6ERfkZyX/wmIZSrrIontIjH7V7nABT oHnWD3IqWWJa8RPbOjOtYU5Y0ndF7H9aga5WPQ2lJJIeItT1R3u3LLVKdnKgbVA/DMAd 5WjtxotNg28stwFJE4VlN3zH00vp1XWWt9LMxS6R2Ggz1W6vPr9eb5TS9EPDn7AGY1L/ aIzKudP1/OnfA/IITe0Q2eYk+47vWdJ1Ze1kBlNrdyg2o6wdW/zo2LvEhex1Pv3weaaX c6xvWxa5aIsSmtylwC6Q3FptPxgHbmpas/pbTZumNcCZtpe8O+KjFWVUQ0eeCxyLVWtC YFWA== Received: by 10.60.10.133 with SMTP id i5mr28666599oeb.11.1351613105950; Tue, 30 Oct 2012 09:05:05 -0700 (PDT) Received: from [10.30.101.53] ([209.117.142.2]) by mx.google.com with ESMTPS id k8sm696549oee.11.2012.10.30.09.05.04 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 30 Oct 2012 09:05:04 -0700 (PDT) Sender: Warner Losh Subject: Re: GENERIC kernel configuraion Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii From: Warner Losh In-Reply-To: <1351605941.79450.YahooMailNeo@web126002.mail.ne1.yahoo.com> Date: Tue, 30 Oct 2012 10:05:02 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <94E7B5BC-89F5-4C99-973C-248B702A880A@bsdimp.com> References: <1351605941.79450.YahooMailNeo@web126002.mail.ne1.yahoo.com> To: Jack Mc Lauren X-Mailer: Apple Mail (2.1084) X-Gm-Message-State: ALoCoQn80UXu2pmWT3Ov7+NNA9AQbGzm8unsvECEsNMui2TObPonGVqVBv2N206akIZGy9990+/w Cc: "freebsd-embedded@freebsd.org" X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Oct 2012 16:05:07 -0000 On Oct 30, 2012, at 8:05 AM, Jack Mc Lauren wrote: > Does customizing (commenting unused devices in GENERIC) kernel in = /usr/src/amd64/conf cause nanobsd image get lighter ?? Yes. In fact, everybody does that. Warner From owner-freebsd-embedded@FreeBSD.ORG Sat Nov 3 11:43:08 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 54E0F5FF for ; Sat, 3 Nov 2012 11:43:08 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.friendlyhosting.spb.ru (onlyone.friendlyhosting.spb.ru [IPv6:2a01:4f8:131:60a2::2]) by mx1.freebsd.org (Postfix) with ESMTP id 15EB18FC0C for ; Sat, 3 Nov 2012 11:43:08 +0000 (UTC) Received: from lion.home.serebryakov.spb.ru (unknown [IPv6:2001:470:923f:1:dc:ffe:46f:81d2]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.friendlyhosting.spb.ru (Postfix) with ESMTPA id 8DFD44AC1C for ; Sat, 3 Nov 2012 15:43:06 +0400 (MSK) Date: Sat, 3 Nov 2012 15:43:00 +0400 From: Lev Serebryakov Organization: FreeBSD X-Priority: 3 (Normal) Message-ID: <1556142970.20121103154300@serebryakov.spb.ru> To: freebsd-embedded@freebsd.org Subject: pkg-ng and nanobsd? MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: lev@FreeBSD.org List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Nov 2012 11:43:08 -0000 Hello, Freebsd-embedded. Does somebody have solution for pkg-ng and nanobsd co-existence? I was able to hack "cust_pkg ()" function to install pkg-ng into chrooted environment (bootstrap pkg command), but after that "pkg add" fails, as it doesn't have `-F' flag and it reports double-installation as error :( -- // Black Lion AKA Lev Serebryakov From owner-freebsd-embedded@FreeBSD.ORG Sat Nov 3 18:38:49 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9F52D54D; Sat, 3 Nov 2012 18:38:49 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 6FB4B8FC0A; Sat, 3 Nov 2012 18:38:48 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA3IcfhX037325; Sat, 3 Nov 2012 12:38:48 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA3Icees010121; Sat, 3 Nov 2012 12:38:40 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Subject: watchdogd, jemalloc, and mlockall From: Ian Lepore To: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org Content-Type: text/plain; charset="us-ascii" Date: Sat, 03 Nov 2012 12:38:39 -0600 Message-ID: <1351967919.1120.102.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Nov 2012 18:38:49 -0000 In an attempt to un-hijack the thread about memory usage increase between 6.4 and 9.x, I'm starting a new thread here related to my recent discovery that watchdogd uses a lot more memory since it began using mlockall(2). I tried statically linking watchdogd and it made a small difference in RSS, presumably because it doesn't wire down all of libc and libm. VSZ RSS 10236 10164 Dynamic 8624 8636 Static Those numbers are from ps -u on an arm platform. I just updated the PR (bin/173332) with some procstat -v output comparing with/without mlockall(). It appears that the bulk of the new RSS bloat comes from jemalloc allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect that leads to wiring 8MB to satisfy what probably amounts to a few hundred bytes of malloc'd memory. It would probably also be a good idea to remove the floating point from watchdogd to avoid wiring all of libm. The floating point is used just to turn the timeout-in-seconds into a power-of-two-nanoseconds value. There's probably a reasonably efficient way to do that without calling log(), considering that it only happens once at program startup. -- Ian From owner-freebsd-embedded@FreeBSD.ORG Sat Nov 3 18:41:52 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 81827694; Sat, 3 Nov 2012 18:41:52 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kostikbel-1-pt.tunnel.tserv11.ams1.ipv6.he.net [IPv6:2001:470:1f14:13d6::2]) by mx1.freebsd.org (Postfix) with ESMTP id DBA558FC08; Sat, 3 Nov 2012 18:41:51 +0000 (UTC) Received: from tom.home (localhost [127.0.0.1]) by kib.kiev.ua (8.14.5/8.14.5) with ESMTP id qA3Ifh4X020600; Sat, 3 Nov 2012 20:41:43 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by tom.home (8.14.5/8.14.5/Submit) id qA3IfhMX020599; Sat, 3 Nov 2012 20:41:43 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 3 Nov 2012 20:41:43 +0200 From: Konstantin Belousov To: Ian Lepore Subject: Re: watchdogd, jemalloc, and mlockall Message-ID: <20121103184143.GC73505@kib.kiev.ua> References: <1351967919.1120.102.camel@revolution.hippie.lan> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="I0oWFe1KborvVxk0" Content-Disposition: inline In-Reply-To: <1351967919.1120.102.camel@revolution.hippie.lan> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=0.2 required=5.0 tests=ALL_TRUSTED, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Nov 2012 18:41:52 -0000 --I0oWFe1KborvVxk0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: > In an attempt to un-hijack the thread about memory usage increase > between 6.4 and 9.x, I'm starting a new thread here related to my recent > discovery that watchdogd uses a lot more memory since it began using > mlockall(2). >=20 > I tried statically linking watchdogd and it made a small difference in > RSS, presumably because it doesn't wire down all of libc and libm. >=20 > VSZ RSS > 10236 10164 Dynamic > 8624 8636 Static >=20 > Those numbers are from ps -u on an arm platform. I just updated the PR > (bin/173332) with some procstat -v output comparing with/without > mlockall(). >=20 > It appears that the bulk of the new RSS bloat comes from jemalloc > allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect > that leads to wiring 8MB to satisfy what probably amounts to a few > hundred bytes of malloc'd memory. >=20 > It would probably also be a good idea to remove the floating point from > watchdogd to avoid wiring all of libm. The floating point is used just > to turn the timeout-in-seconds into a power-of-two-nanoseconds value. > There's probably a reasonably efficient way to do that without calling > log(), considering that it only happens once at program startup. No, I propose to add a switch to turn on/off the mlockall() call. I have no opinion on the default value of the suggested switch. --I0oWFe1KborvVxk0 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCVZWcACgkQC3+MBN1Mb4hHhACguk/G8KdOYC2wQMMu6BH1WI8c BlkAnRwhcgc8SnQ62sV90VvzzrvX+cLf =s6o/ -----END PGP SIGNATURE----- --I0oWFe1KborvVxk0-- From owner-freebsd-embedded@FreeBSD.ORG Sat Nov 3 18:50:58 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8AB28BB9; Sat, 3 Nov 2012 18:50:58 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id 30B4D8FC12; Sat, 3 Nov 2012 18:50:58 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA3Iovk6037697; Sat, 3 Nov 2012 12:50:57 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA3IoZj4010150; Sat, 3 Nov 2012 12:50:35 -0600 (MDT) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: watchdogd, jemalloc, and mlockall From: Ian Lepore To: Konstantin Belousov In-Reply-To: <20121103184143.GC73505@kib.kiev.ua> References: <1351967919.1120.102.camel@revolution.hippie.lan> <20121103184143.GC73505@kib.kiev.ua> Content-Type: text/plain; charset="us-ascii" Date: Sat, 03 Nov 2012 12:50:35 -0600 Message-ID: <1351968635.1120.110.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 03 Nov 2012 18:50:58 -0000 On Sat, 2012-11-03 at 20:41 +0200, Konstantin Belousov wrote: > On Sat, Nov 03, 2012 at 12:38:39PM -0600, Ian Lepore wrote: > > In an attempt to un-hijack the thread about memory usage increase > > between 6.4 and 9.x, I'm starting a new thread here related to my recent > > discovery that watchdogd uses a lot more memory since it began using > > mlockall(2). > > > > I tried statically linking watchdogd and it made a small difference in > > RSS, presumably because it doesn't wire down all of libc and libm. > > > > VSZ RSS > > 10236 10164 Dynamic > > 8624 8636 Static > > > > Those numbers are from ps -u on an arm platform. I just updated the PR > > (bin/173332) with some procstat -v output comparing with/without > > mlockall(). > > > > It appears that the bulk of the new RSS bloat comes from jemalloc > > allocating vmspace in 8MB chunks. With mlockall(MCL_FUTURE) in effect > > that leads to wiring 8MB to satisfy what probably amounts to a few > > hundred bytes of malloc'd memory. > > > > It would probably also be a good idea to remove the floating point from > > watchdogd to avoid wiring all of libm. The floating point is used just > > to turn the timeout-in-seconds into a power-of-two-nanoseconds value. > > There's probably a reasonably efficient way to do that without calling > > log(), considering that it only happens once at program startup. > > No, I propose to add a switch to turn on/off the mlockall() call. > I have no opinion on the default value of the suggested switch. In a patch I submitted along with the PR, I added code to query the vm.swap_enabled sysctl and only call mlockall() when swapping is enabled. Nobody yet has said anything about what seems to me to be the real problem here: jemalloc grabs 8MB at a time even if you only need to malloc a few bytes, and there appears to be no way to control that behavior. Or maybe there's a knob in there that didn't jump out at me on a quick glance through the header files. -- Ian