From owner-freebsd-embedded@FreeBSD.ORG Mon Jul 30 08:35:03 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 42082106566B; Mon, 30 Jul 2012 08:35:03 +0000 (UTC) (envelope-from ray@freebsd.org) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id E94478FC0C; Mon, 30 Jul 2012 08:35:02 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) by smtp.dlink.ua (Postfix) with SMTP id AD23CC4934; Mon, 30 Jul 2012 11:34:55 +0300 (EEST) Date: Mon, 30 Jul 2012 11:34:18 +0300 From: Aleksandr Rybalko To: Adrian Chadd Message-Id: <20120730113418.97694fdf.ray@freebsd.org> In-Reply-To: References: <20120726143551.b2b0ce30.ray@freebsd.org> Organization: FreeBSD Project X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-embedded@freebsd.org, jasone@freebsd.org Subject: Re: jemalloc vs old allocator X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 08:35:03 -0000 On Fri, 27 Jul 2012 10:04:27 -0700 Adrian Chadd wrote: >> Just as a note, ray@ compared jemalloc-HEAD with an older jemalloc. >> >> I've just asked him to compare it to phkmalloc from -8. He'll >> hopefully post those results soon. >> >> Still, great work. :) >> >> >> >> Adrian >> >> >> On 26 July 2012 04:35, Aleksandr Rybalko wrote: >> > Hi, >> > >> > I believe what Jason Evans done many test before commit jemalloc, >> > but made by myself simple test for embedded environment. And get >> > following results: >> > >> > 1. old malloc: >> > Sum of RSS of userland programs: 28544K >> > >> > sysctl vm.vmtotal: >> > System wide totals computed every five seconds: (values in >> > kilobytes) =============================================== >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: >> > 19) Virtual Memory: (Total: 2372224K Active: 250316K) >> > Real Memory: (Total: 18096K Active: 11392K) >> > Shared Virtual Memory: (Total: 15824K Active: 5288K) >> > Shared Real Memory: (Total: 2644K Active: 2408K) >> > Free Memory: 2192K >> > >> > >> > 2. jemalloc with MALLOC_PRODUCTION: >> > Sum of RSS of userland programs: 21196K >> > >> > sysctl vm.vmtotal: >> > System wide totals computed every five seconds: (values in >> > kilobytes) =============================================== >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: >> > 19) Virtual Memory: (Total: 2380988K Active: 257820K) >> > Real Memory: (Total: 19048K Active: 12144K) >> > Shared Virtual Memory: (Total: 16180K Active: 5392K) >> > Shared Real Memory: (Total: 2664K Active: 2392K) >> > Free Memory: 2440K >> > >> > I check it on DIR-632(Atheros AR7242 with 32M RAM) between 10-20 >> > mins of uptime. Device behave identical in both cases. >> > >> > So if "something" new take more memory than before, that >> > "something" is not a jemalloc :) >> > >> > P.S. Many thanks to Jason for such great job! >> > >> > WBW >> > -- >> > Aleksandr Rybalko Hi all, so, unlike both jemallocs, Poul's version of malloc born with more free memory: Sum of RSS of userland programs: 25444K sysctl vm.vmtotal: System wide totals computed every five seconds: (values in kilobytes) =============================================== Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: 19) Virtual Memory: (Total: 2163524K Active: 59196K) Real Memory: (Total: 16656K Active: 10144K) Shared Virtual Memory: (Total: 5620K Active: 5200K) Shared Real Memory: (Total: 2552K Active: 2356K) Free Memory: 3140K But after half a day with some minor activity (logs on tmpfs, racoon connect attempts, etc.) free memory goes down to: Free Memory: 2316K and continue get more and more memory, after full day, it is around 1.5M free. -- Aleksandr Rybalko From owner-freebsd-embedded@FreeBSD.ORG Mon Jul 30 11:07:12 2012 Return-Path: Delivered-To: freebsd-embedded@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 92A90106566B for ; Mon, 30 Jul 2012 11:07:12 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 7B9A48FC1D for ; Mon, 30 Jul 2012 11:07:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q6UB7CAQ001744 for ; Mon, 30 Jul 2012 11:07:12 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q6UB7BPn001742 for freebsd-embedded@FreeBSD.org; Mon, 30 Jul 2012 11:07:11 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 30 Jul 2012 11:07:11 GMT Message-Id: <201207301107.q6UB7BPn001742@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-embedded@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-embedded@FreeBSD.org X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 11:07:12 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o misc/52256 embedded [picobsd] picobsd build script does not read in user/s o kern/42728 embedded [picobsd] many problems in src/usr.sbin/ppp/* after c 2 problems total. From owner-freebsd-embedded@FreeBSD.ORG Mon Jul 30 20:08:10 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 780DE1065673; Mon, 30 Jul 2012 20:08:10 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 331918FC17; Mon, 30 Jul 2012 20:08:10 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 7F4B1B962; Mon, 30 Jul 2012 16:08:09 -0400 (EDT) From: John Baldwin To: freebsd-embedded@freebsd.org Date: Mon, 30 Jul 2012 14:22:09 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p17; KDE/4.5.5; amd64; ; ) References: <20120726143551.b2b0ce30.ray@freebsd.org> <20120730113418.97694fdf.ray@freebsd.org> In-Reply-To: <20120730113418.97694fdf.ray@freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201207301422.10029.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 30 Jul 2012 16:08:09 -0400 (EDT) Cc: Aleksandr Rybalko , jasone@freebsd.org Subject: Re: jemalloc vs old allocator X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jul 2012 20:08:10 -0000 On Monday, July 30, 2012 4:34:18 am Aleksandr Rybalko wrote: > On Fri, 27 Jul 2012 10:04:27 -0700 > Adrian Chadd wrote: > > >> Just as a note, ray@ compared jemalloc-HEAD with an older jemalloc. > >> > >> I've just asked him to compare it to phkmalloc from -8. He'll > >> hopefully post those results soon. > >> > >> Still, great work. :) > >> > >> > >> > >> Adrian > >> > >> > >> On 26 July 2012 04:35, Aleksandr Rybalko wrote: > >> > Hi, > >> > > >> > I believe what Jason Evans done many test before commit jemalloc, > >> > but made by myself simple test for embedded environment. And get > >> > following results: > >> > > >> > 1. old malloc: > >> > Sum of RSS of userland programs: 28544K > >> > > >> > sysctl vm.vmtotal: > >> > System wide totals computed every five seconds: (values in > >> > kilobytes) =============================================== > >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: > >> > 19) Virtual Memory: (Total: 2372224K Active: 250316K) > >> > Real Memory: (Total: 18096K Active: 11392K) > >> > Shared Virtual Memory: (Total: 15824K Active: 5288K) > >> > Shared Real Memory: (Total: 2644K Active: 2408K) > >> > Free Memory: 2192K > >> > > >> > > >> > 2. jemalloc with MALLOC_PRODUCTION: > >> > Sum of RSS of userland programs: 21196K > >> > > >> > sysctl vm.vmtotal: > >> > System wide totals computed every five seconds: (values in > >> > kilobytes) =============================================== > >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: > >> > 19) Virtual Memory: (Total: 2380988K Active: 257820K) > >> > Real Memory: (Total: 19048K Active: 12144K) > >> > Shared Virtual Memory: (Total: 16180K Active: 5392K) > >> > Shared Real Memory: (Total: 2664K Active: 2392K) > >> > Free Memory: 2440K > >> > > >> > I check it on DIR-632(Atheros AR7242 with 32M RAM) between 10-20 > >> > mins of uptime. Device behave identical in both cases. > >> > > >> > So if "something" new take more memory than before, that > >> > "something" is not a jemalloc :) > >> > > >> > P.S. Many thanks to Jason for such great job! > >> > > >> > WBW > >> > -- > >> > Aleksandr Rybalko > > Hi all, > > so, unlike both jemallocs, Poul's version of malloc born with more free > memory: > > Sum of RSS of userland programs: 25444K > > sysctl vm.vmtotal: > System wide totals computed every five seconds: (values in kilobytes) > =============================================== > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: 19) > Virtual Memory: (Total: 2163524K Active: 59196K) > Real Memory: (Total: 16656K Active: 10144K) > Shared Virtual Memory: (Total: 5620K Active: 5200K) > Shared Real Memory: (Total: 2552K Active: 2356K) > Free Memory: 3140K > > But after half a day with some minor activity (logs on tmpfs, racoon > connect attempts, etc.) free memory goes down to: > > Free Memory: 2316K > > and continue get more and more memory, after full day, it is around > 1.5M free. It is ok to have "low" free memory. The kernel will just keep file data cached around forever if it can. When free memory hits a certain target, then the pagedaemon process will wake up and shuffle some inactive pages down to "cache" to free up more memory. However, until that happens cached file data will just stay in "inactive" and the kernel will keep using "free" pages. -- John Baldwin From owner-freebsd-embedded@FreeBSD.ORG Wed Aug 1 11:24:53 2012 Return-Path: Delivered-To: freebsd-embedded@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E5E2106566C; Wed, 1 Aug 2012 11:24:53 +0000 (UTC) (envelope-from ray@dlink.ua) Received: from smtp.dlink.ua (smtp.dlink.ua [193.138.187.146]) by mx1.freebsd.org (Postfix) with ESMTP id C7D148FC0A; Wed, 1 Aug 2012 11:24:52 +0000 (UTC) Received: from terran.dlink.ua (unknown [192.168.10.90]) (Authenticated sender: ray) by smtp.dlink.ua (Postfix) with ESMTPSA id 32150C4947; Wed, 1 Aug 2012 14:24:46 +0300 (EEST) Date: Wed, 1 Aug 2012 14:24:18 +0300 From: Aleksandr Rybalko To: John Baldwin Message-Id: <20120801142418.f56aa606.ray@dlink.ua> In-Reply-To: <201207301422.10029.jhb@freebsd.org> References: <20120726143551.b2b0ce30.ray@freebsd.org> <20120730113418.97694fdf.ray@freebsd.org> <201207301422.10029.jhb@freebsd.org> Organization: D-Link X-Mailer: Sylpheed 2.7.1 (GTK+ 2.20.1; i386-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Aleksandr Rybalko , freebsd-embedded@freebsd.org, jasone@freebsd.org Subject: Re: jemalloc vs old allocator X-BeenThere: freebsd-embedded@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Dedicated and Embedded Systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Aug 2012 11:24:53 -0000 On Mon, 30 Jul 2012 14:22:09 -0400 John Baldwin wrote: >> On Monday, July 30, 2012 4:34:18 am Aleksandr Rybalko wrote: >> > On Fri, 27 Jul 2012 10:04:27 -0700 >> > Adrian Chadd wrote: >> > >> > >> Just as a note, ray@ compared jemalloc-HEAD with an older >> > >> jemalloc. >> > >> >> > >> I've just asked him to compare it to phkmalloc from -8. He'll >> > >> hopefully post those results soon. >> > >> >> > >> Still, great work. :) >> > >> >> > >> >> > >> >> > >> Adrian >> > >> >> > >> >> > >> On 26 July 2012 04:35, Aleksandr Rybalko >> > >> wrote: >> > >> > Hi, >> > >> > >> > >> > I believe what Jason Evans done many test before commit >> > >> > jemalloc, but made by myself simple test for embedded >> > >> > environment. And get following results: >> > >> > >> > >> > 1. old malloc: >> > >> > Sum of RSS of userland programs: 28544K >> > >> > >> > >> > sysctl vm.vmtotal: >> > >> > System wide totals computed every five seconds: (values in >> > >> > kilobytes) =============================================== >> > >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 >> > >> > Sleep: >> > >> > 19) Virtual Memory: (Total: 2372224K Active: 250316K) >> > >> > Real Memory: (Total: 18096K Active: 11392K) >> > >> > Shared Virtual Memory: (Total: 15824K Active: 5288K) >> > >> > Shared Real Memory: (Total: 2644K Active: 2408K) >> > >> > Free Memory: 2192K >> > >> > >> > >> > >> > >> > 2. jemalloc with MALLOC_PRODUCTION: >> > >> > Sum of RSS of userland programs: 21196K >> > >> > >> > >> > sysctl vm.vmtotal: >> > >> > System wide totals computed every five seconds: (values in >> > >> > kilobytes) =============================================== >> > >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 >> > >> > Sleep: >> > >> > 19) Virtual Memory: (Total: 2380988K Active: 257820K) >> > >> > Real Memory: (Total: 19048K Active: 12144K) >> > >> > Shared Virtual Memory: (Total: 16180K Active: 5392K) >> > >> > Shared Real Memory: (Total: 2664K Active: 2392K) >> > >> > Free Memory: 2440K >> > >> > >> > >> > I check it on DIR-632(Atheros AR7242 with 32M RAM) between >> > >> > 10-20 mins of uptime. Device behave identical in both cases. >> > >> > >> > >> > So if "something" new take more memory than before, that >> > >> > "something" is not a jemalloc :) >> > >> > >> > >> > P.S. Many thanks to Jason for such great job! >> > >> > >> > >> > WBW >> > >> > -- >> > >> > Aleksandr Rybalko >> > >> > Hi all, >> > >> > so, unlike both jemallocs, Poul's version of malloc born with more >> > free memory: >> > >> > Sum of RSS of userland programs: 25444K >> > >> > sysctl vm.vmtotal: >> > System wide totals computed every five seconds: (values in >> > kilobytes) =============================================== >> > Processes: (RUNQ: 1 Disk Wait: 1 Page Wait: 0 Sleep: >> > 19) Virtual Memory: (Total: 2163524K Active: 59196K) >> > Real Memory: (Total: 16656K Active: 10144K) >> > Shared Virtual Memory: (Total: 5620K Active: 5200K) >> > Shared Real Memory: (Total: 2552K Active: 2356K) >> > Free Memory: 3140K >> > >> > But after half a day with some minor activity (logs on tmpfs, >> > racoon connect attempts, etc.) free memory goes down to: >> > >> > Free Memory: 2316K >> > >> > and continue get more and more memory, after full day, it is around >> > 1.5M free. >> >> It is ok to have "low" free memory. The kernel will just keep file >> data cached around forever if it can. When free memory hits a >> certain target, then the pagedaemon process will wake up and shuffle >> some inactive pages down to "cache" to free up more memory. >> However, until that happens cached file data will just stay in >> "inactive" and the kernel will keep using "free" pages. >> >> -- >> John Baldwin Hi John, I'm just do simple test in well known environment, since it is not a PC with 300 items in /etc/rc.d, but just small box with small count of processes and even w/o RTC. So every time at boot the box doing exact same job. Knowing that I can compare basically allocators. Of course it is not precise and I don't tweak malloc flags. Only simple test to know where we go :) Thank you John! WBW -- Alexandr Rybalko aka Alex RAY