Date: Sun, 7 Sep 2003 23:42:48 -0700 From: David Schultz <das@FreeBSD.ORG> To: Anders Nordby <anders@fix.no>, Mike Harding <mvh@ix.netcom.com>, stable@FreeBSD.ORG, ports@FreeBSD.ORG, adrian@FreeBSD.ORG, tjr@FreeBSD.ORG, phk@FreeBSD.ORG Subject: Re: Squid memory leaks in -stable using libc malloc Message-ID: <20030908064248.GA7522@HAL9000.homeunix.com> In-Reply-To: <20030907152051.GA44042@totem.fix.no> References: <20030906141108.GA13082@totem.fix.no> <20030906151719.A83B65350@netcom1.netcom.com> <20030907152051.GA44042@totem.fix.no>
index | next in thread | previous in thread | raw e-mail
On Sun, Sep 07, 2003, Anders Nordby wrote: > Hi, > > On Sat, Sep 06, 2003 at 08:17:19AM -0700, Mike Harding wrote: > > Squid uses more memory than you assign to cache_mem, this is > > documented in the Squid FAQ, section 8. cache_mem is sort of a > > 'suggested' value, it's normal for squid to use a lot more memory than > > cache_mem. > > I know this, and I know the contents of the FAQ. However, there's a huge > difference between what top reports as memory used (SIZE) and what cache > manager reports as total allocated KB. In a running process I have just > now, top says Squid uses 1235 MB of memory, while cache manager reports > 663 MB allocated (contents of /proc/<pid>/map attached). That's after > running for around 10 hours. If I use dlmalloc (I do on one system), the > numbers are 2291 MB according to top and 2221 MB according to cache > manager after running for 30 hours. Do you see the difference? To me, > there seems to be a leak, or at least very unfavourable results with > phkmalloc and Squid. phkmalloc satisfies allocations smaller than the system page size from power-of-two buckets, and larger allocations are rounded up to a multiple of the page size. While simple, this scheme works poorly for some applications. For example, postgresql makes some allocations that are slightly larger than a power of two due to its use of inline buffer tags. If you use malloc(3)'s 'U' option in combination with ktrace(1), you can get a good idea of exactly what sizes Squid is trying to allocate. Note also that there's a difference between heap size and physical memory usage. In a demand-paged virtual memory system, you can allocate just about as much heap space as you want practically for free; you only pay for the pages you actually touch. A better metric for the memory pressure that an application is exerting on the system is its resident set size---'RES' in top(1). phkmalloc by default is not aggressive about returning memory to the system, but it usually exhibits very good behavior from the point of view of the VM system.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030908064248.GA7522>