From owner-freebsd-hackers@FreeBSD.ORG Fri Jun 15 01:10:53 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 690BD16A41F for ; Fri, 15 Jun 2007 01:10:53 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout7.cac.washington.edu (mxout7.cac.washington.edu [140.142.32.178]) by mx1.freebsd.org (Postfix) with ESMTP id 4A2C213C468 for ; Fri, 15 Jun 2007 01:10:53 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from hymn01.u.washington.edu (hymn01.u.washington.edu [140.142.8.55]) by mxout7.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.05) with ESMTP id l5F1AnoN002145 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 Jun 2007 18:10:49 -0700 Received: from localhost (localhost [127.0.0.1]) by hymn01.u.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l5F1AnAo024267; Thu, 14 Jun 2007 18:10:49 -0700 X-Auth-Received: from [192.55.52.1] by hymn01.u.washington.edu via HTTP; Thu, 14 Jun 2007 18:10:49 PDT Date: Thu, 14 Jun 2007 18:10:49 -0700 (PDT) From: youshi10@u.washington.edu To: Matthew Dillon In-Reply-To: <200706150104.l5F14RjG001010@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-PMX-Version: 5.3.1.294258, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.6.14.175233 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='SUPERLONG_LINE 0.05, NO_REAL_NAME 0, __CT 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0' Cc: hackers@freebsd.org Subject: Re: Reason for doing malloc / bzero over calloc (performance)? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jun 2007 01:10:53 -0000 On Thu, 14 Jun 2007, Matthew Dillon wrote: > I'm going to throw a wrench in the works, because it all gets turned > around the moment you find yourself in a SMP environment where several > threads are running on different cpus at the same time, using the > same shared VM space. > > The moment you have a situation like that where you are futzing with > the page tables, i.e. using mmap() for demand-zero and munmap() to > free, the operation becomes extremely expensive verses anything > else because any update to the page table (specifically any removal > of page table entries from the page table) requires a SMP synchronization > to occur between all the cpu's actively sharing that VM space, and > that's on top of the overhead of taking the page fault(s). > > This is true of any memory mapping the kernel has to do in kernel > virtual memory (must be synchronized with ALL cpus) and any mapping > the kernel does on behalf of userland for user memory (must be > synchronized with any cpu's actively using that VM space, i.e. threaded > user programs). The synchronization is required to properly invalidate > stale mappings on other cpus and it must be done synchronously due > to bugs in Intel/AMD related to changing page table entries on one > cpu when instructions are executing using that memory on another cpu. > There is no way to avoid it without tripping up on the Intel/AMD hardware > bugs. > > From this point of view it is much, much better to bzero() memory that > is already mapped then it is to map/unmap new memory. I recently > audited DragonFly and found an insane number of IPIs flying about due > to PAGE_SIZE'd kernel mallocs using the VM trick via kernel_map & > kmem_alloc(). They all went away when I made the kernel malloc use > the slab cache for allocations up to and including PAGE_SIZE*2 bytes. > > Fun, eh? > > -Matt > Matthew Dillon > I have no intention of using malloc/calloc with free, and then repeating the same procedure. It's better just to use the memory allocated, if possible, size permitting this. I wasn't thinking that closely though (ISA/hardware config versus OS implementation), but I had my suspicions since the AMD64 architecture is very different from the PowerPC architecture, in terms of word size, sychronization schemes, instruction count, etc. Interesting insight though. Thanks :). -Garrett