From owner-freebsd-current@FreeBSD.ORG Wed Mar 22 14:11:32 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B9D4C16A401; Wed, 22 Mar 2006 14:11:32 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5056543D46; Wed, 22 Mar 2006 14:11:32 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 4EF285CF5; Wed, 22 Mar 2006 09:11:31 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 75803-05; Wed, 22 Mar 2006 09:11:30 -0500 (EST) Received: from [192.168.1.3] (pool-68-160-194-11.ny325.east.verizon.net [68.160.194.11]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 0418A5C50; Wed, 22 Mar 2006 09:11:29 -0500 (EST) Message-ID: <44215B1B.1080104@mac.com> Date: Wed, 22 Mar 2006 09:11:39 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Jason Evans References: <20060321184019.GX35129@funkthat.com> <47CC5AC0-1B44-4485-92A9-70751681A527@freebsd.org> <20060321222046.GY35129@funkthat.com> <1EB2EEE3-855C-4B76-81A6-1880526797CE@freebsd.org> In-Reply-To: <1EB2EEE3-855C-4B76-81A6-1880526797CE@freebsd.org> X-Enigmail-Version: 0.94.0.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: John-Mark Gurney , current@freebsd.org Subject: Re: core dumps are HUGE... X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Mar 2006 14:11:32 -0000 Hi, Jason-- First, thanks for your work on the new jemalloc. Jason Evans wrote: > On Mar 21, 2006, at 2:20 PM, John-Mark Gurney wrote: [ ... ] >> And yes, I'm pretty sure that I have a world from before your reduction >> in chunk size... Having a minimum of a 6meg core doesn't sound like a >> good idea... It's definately not going to play nice with small systems.. > > I don't think that a 6 MB core file is the big deal you are making it > out to be. A 50 MB core file is a pain, mainly because it takes a long > time to dump core. The smallest processes that people run often are likely going to be /bin/sh and the typical fodder of shell scripts (grep, awk, and so forth). Under 4.x, /bin/sh tended to have VSIZE of 640K and RSS of ~300K; under 5.x and later, /bin/sh has VSIZE of 1.6MB and RSS of ~1MB. Creating a 50MB corefile from a 2MB VSIZE process is unreasonable. Creating a 6MB corefile from a 2MB VSIZE process would be reasonable but probably could be improved by a factor of two. On the other hand, the staticly linked "t" program is: chuck 62964 0.0 0.1 232 128 p0 TX 8:09AM 0:00.03 /tmp/t 128K RSS, 232 VSIZE. A 6MB corefile is a factor of 25 larger, which strikes me as something that could be improved by more than a factor of ten. What else is being dumped that's not counted by VSIZE, aside from a header and maybe some per-thread state, (although aren't the per-thread stacks already part of VSIZE)...? Can jemalloc only create per-CPU arenas only for processes which are themselves multithreaded, when it's running on a multi-CPU system? Would that help reduce the amount of allocated but unreferenced memory that is involved for the common case of /bin/sh and friends? > Even so, consider that core dumps on OS X are in > excess of 90 MB, and that this hasn't caused the world to stop revolving > (though we should blame OS X for global warming). Dear Cthulhu, you're right. I was sure this was FUD, and I would have been wrong. :) The one-line test program from earlier in the thread creates a 147MB coredump under 10.4.5...I need to file a radar or two with Apple. -- -Chuck