From owner-freebsd-current@FreeBSD.ORG Thu Mar 6 19:22:16 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E42AC1065671 for ; Thu, 6 Mar 2008 19:22:16 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from outbound0.mx.meer.net (outbound0.mx.meer.net [209.157.153.23]) by mx1.freebsd.org (Postfix) with ESMTP id C5B5E8FC14 for ; Thu, 6 Mar 2008 19:22:16 +0000 (UTC) (envelope-from gnn@neville-neil.com) Received: from mail.meer.net (mail.meer.net [209.157.152.14]) by outbound0.mx.meer.net (8.12.10/8.12.6) with ESMTP id m26JM0i3093832; Thu, 6 Mar 2008 11:22:16 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from mail2.meer.net (mail2.meer.net [64.13.141.16]) by mail.meer.net (8.13.3/8.13.3/meer) with ESMTP id m26JLtNX039687; Thu, 6 Mar 2008 11:21:55 -0800 (PST) (envelope-from gnn@neville-neil.com) Received: from gnnbsd.hudson-trading.com.neville-neil.com (hudson-trading.com [66.150.84.160] (may be forged)) (authenticated bits=0) by mail2.meer.net (8.14.1/8.14.1) with ESMTP id m26JLtZZ038223; Thu, 6 Mar 2008 11:21:55 -0800 (PST) (envelope-from gnn@neville-neil.com) Date: Thu, 06 Mar 2008 14:21:53 -0500 Message-ID: <7iejanmze6.wl%gnn@neville-neil.com> From: gnn@freebsd.org To: Jason Evans In-Reply-To: <47CD9F87.4000509@freebsd.org> References: <677e3b3e0802280915x3f29e79cqe6093b5d7bfba975@mail.gmail.com> <7ifxv7pnei.wl%gnn@neville-neil.com> <47CD9F87.4000509@freebsd.org> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.8 (=?ISO-8859-4?Q?Shij=F2?=) APEL/10.6 Emacs/21.3 (amd64--freebsd) MULE/5.0 (SAKAKI) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII Cc: current@freebsd.org Subject: Re: Differences in malloc between 6 and 7? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Mar 2008 19:22:17 -0000 At Tue, 04 Mar 2008 11:14:15 -0800, Jason Evans wrote: > > gnn@freebsd.org wrote: > > One of the folks I'm working with found this. The following code, > > which yes, is just an example, is 1/2 as fast on 7.0-RELEASE as on > > 6.3. Where should I look to find out why? > > There is a definite performance problem an arena_run_alloc(), but I'm > happy to report that it was fixed in -current a while back. I plan to > MFC malloc to RELENG_7 within the next few weeks. > Great! > In a nutshell, the arena_run_alloc() performance problem is due to > using a linear search to find sufficiently large runs of mapped (but > currently unused) pages. There are caching mechanisms that speed up > the searches to some degree, but there are still some linear aspects > to the algorithm, so as memory usage increases, the searches take > progressively longer. In -current, this problem is solved by > maintaining red-black trees, so that arena_run_alloc() does a O(lg > n) tree search, rather than a O(n) iterative search. > > It's worth mentioning that the benchmark is of marginal use, due to > a simple (but common) flaw. At a minimum, a malloc benchmark should > touch all allocated memory at least once. Otherwise, the benchmark > is IMO too far removed from reality to measure anything of value, > since memory access patterns look nothing like those of an actual > application that dynamically allocates memory. Both phkmalloc and > jemalloc use data structures that are mostly disjunct from the > allocations (no headers), so the benchmark never even faults most > pages in. This is especially true for phkmalloc, so jemalloc is > unjustly penalized. If we were to include, say, dlmalloc in this > comparison, it would be even more heavily penalized due to touching > the pages while modifying allocation headers. Fair enough, I'll pass that on. Best, George