Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 22 Mar 2006 10:19:39 -0500
From:      John Baldwin <jhb@freebsd.org>
To:        freebsd-current@freebsd.org
Cc:        John-Mark Gurney <gurney_j@resnet.uoregon.edu>, Jason Evans <jasone@freebsd.org>
Subject:   Re: core dumps are HUGE...
Message-ID:  <200603221019.43713.jhb@freebsd.org>
In-Reply-To: <44215B1B.1080104@mac.com>
References:  <20060321184019.GX35129@funkthat.com> <1EB2EEE3-855C-4B76-81A6-1880526797CE@freebsd.org> <44215B1B.1080104@mac.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday 22 March 2006 09:11, Chuck Swiger wrote:
> Hi, Jason--
> 
> First, thanks for your work on the new jemalloc.
> 
> Jason Evans wrote:
> > On Mar 21, 2006, at 2:20 PM, John-Mark Gurney wrote:
> [ ... ]
> >> And yes, I'm pretty sure that I have a world from before your reduction
> >> in chunk size...  Having a minimum of a 6meg core doesn't sound like a
> >> good idea...  It's definately not going to play nice with small systems..
> > 
> > I don't think that a 6 MB core file is the big deal you are making it
> > out to be.  A 50 MB core file is a pain, mainly because it takes a long
> > time to dump core.
> 
> The smallest processes that people run often are likely going to be /bin/sh and
> the typical fodder of shell scripts (grep, awk, and so forth).  Under 4.x,
> /bin/sh tended to have VSIZE of 640K and RSS of ~300K; under 5.x and later,
> /bin/sh has VSIZE of 1.6MB and RSS of ~1MB.
> 
> Creating a 50MB corefile from a 2MB VSIZE process is unreasonable.  Creating a
> 6MB corefile from a 2MB VSIZE process would be reasonable but probably could be
> improved by a factor of two.
> 
> On the other hand, the staticly linked "t" program is:
> 
> chuck  62964  0.0  0.1   232   128  p0  TX    8:09AM   0:00.03 /tmp/t
> 
> 128K RSS, 232 VSIZE.  A 6MB corefile is a factor of 25 larger, which strikes me
> as something that could be improved by more than a factor of ten.
> 
> What else is being dumped that's not counted by VSIZE, aside from a header and
> maybe some per-thread state, (although aren't the per-thread stacks already part
> of VSIZE)...?
> 
> Can jemalloc only create per-CPU arenas only for processes which are themselves
> multithreaded, when it's running on a multi-CPU system?  Would that help reduce
> the amount of allocated but unreferenced memory that is involved for the common
> case of /bin/sh and friends?

I think the better path is to provide sparse coredumps.  I.e., when dumping a
core, leave the parts of the process map that are mapped but have no backing
store yet (b/c the pages haven't been touched) sparse by not writing to them,
but just seeking past them.  This doesn't require complicating the malloc
implementation just for the sake of a core dump on a CF device.

-- 
John Baldwin <jhb@FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200603221019.43713.jhb>