From owner-freebsd-hackers Tue Oct 22 13:57: 0 2002 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 900DA37B401 for ; Tue, 22 Oct 2002 13:56:58 -0700 (PDT) Received: from snipe.mail.pas.earthlink.net (snipe.mail.pas.earthlink.net [207.217.120.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id E6FF443E42 for ; Tue, 22 Oct 2002 13:56:56 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0482.cvx21-bradley.dialup.earthlink.net ([209.179.193.227] helo=mindspring.com) by snipe.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 184651-0001EO-00; Tue, 22 Oct 2002 13:56:47 -0700 Message-ID: <3DB5BB47.EFF55D5D@mindspring.com> Date: Tue, 22 Oct 2002 13:55:35 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Tony Finch Cc: hackers@freebsd.org Subject: Re: malloc References: <3DB5A07F.AA118FA6@mindspring.com> <20021022202533.A29425@chiark.greenend.org.uk> <3DB5AB73.E334629@mindspring.com> <20021022213917.B29425@chiark.greenend.org.uk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Tony Finch wrote: > > You are arguing that there is nothing that can account for the > > performance difference, when in fact there is a measured > > performance difference. > > No, I'm saying that some of what you said is either wrong or > misleading, and the comment about security was especially stupid. Yes, it was misleading, in that I didn't provide full implementation details, so a naieve reader looking to reimplement malloc() as it was implemented on FreeBSD, would get it wrong. The security comment had to do with the fact that zeroing occurs in the kernel in the idle loop, and can account for a large latency in the case of a big demand in user space. It's a philosophy issue that led to the implementation, and it has a performance impact that's higher in FreeBSD than in Linux, under some usage patterns. I don't think it's necessary to discuss the implementation details to the level of documenting them for people wanting to reimplement the code. The context of the current discussion is a FreeBSD admin with Linux users bitching at him about core dumps in an overcommit case, where he's hitting an administrative limit, and then trying to dereference a pointer to a page that has an established mapping, but for which there is no page available to act as backing store. Saying this, though, really doesn't help, when you have a user that's pretending to not understand your explanations, and wants an explanation in layman's terms, which is what the original poster is facing. If it weren't for the fact that we were dealing with a programmer that was pretending to be dumb enough to put allocations into an interior loop in order to give FreeBSD a bad "performance" number relative to Linux, this wouldn't have ever been an issue in the first place: for the "perfectly correct", we would just refer the person to the malloc() source code. Instead, we have to deal with a fictional microbenchmark, and discuss why decisions were made. There's "perfectly correct", and then there's "speaking to the level of your audience". > > > PHK malloc uses MAP_ANON on FreeBSD, not /dev/zero -- it uses the > > > latter only if compiled for Solaris. > > > > And tell me, what does the Linux malloc use? > > Exactly the same, and it uses MAP_ANONYMOUS on Linux. It uses mmap for > large allocations whereas phk malloc does not. Since both mmap and > sbrk get zero-filled pages from the kernel this shoudld make little > difference, except in the calloc case explained above. The allocations in this case are lots of very small allocations; it makes no real difference, unless there is a preallocation of space by the malloc() subsystem in order to refill a free pool, to deal with expected future malloc() requests. FreeBSD's "overallocation pool size" is one 4K page, which is relatively small for modern systems with 4G of memory. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message