From owner-freebsd-performance@FreeBSD.ORG Tue Apr 24 19:27:31 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D59401065677; Tue, 24 Apr 2012 19:27:31 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 790A08FC18; Tue, 24 Apr 2012 19:27:31 +0000 (UTC) Received: by yhgm50 with SMTP id m50so837833yhg.13 for ; Tue, 24 Apr 2012 12:27:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=j7pYn4TZNu8sAus7s0FzDna2/UGLIbYsiYyPeuYQkDY=; b=nB+qET8SyJ7aZfD3qWF8JdMPTaSBWx7Z04MWqaJk2JJCdY34tX5Z9H7eWOvSz0Blxj HCjCTIOXFi/qV+Leu+24aYZF6XN9nlB7Ry7U+acdmYxKsad0HobiUuh5g5YZPca7TXcY u00g47EQ0sv5PIDRmH3FY0cIzGp8ywTi2Bn81jxdAW4toP0lwW0aB2b8cSxJnYZ9pWLz HLJ0sSx5bYF8LE8eOVoDP+Om8PUab8FPDer3YYaUAMP32rQBwDjsCUlOZ3mg/2Wm3hmY 50PLhAVAYgkA01zYnQA7lolUAkJOfNtfzSOrTKtWEr6SemiM/+rRzrUbp8brYz5dXubK 8iIQ== MIME-Version: 1.0 Received: by 10.50.168.67 with SMTP id zu3mr2989983igb.28.1335295650508; Tue, 24 Apr 2012 12:27:30 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.50.129.39 with HTTP; Tue, 24 Apr 2012 12:27:30 -0700 (PDT) In-Reply-To: <20120424191415.GE76983@zxy.spb.ru> References: <20120424191415.GE76983@zxy.spb.ru> Date: Tue, 24 Apr 2012 21:27:30 +0200 X-Google-Sender-Auth: Mvt-9psx88jaB_iz56_7J8splg0 Message-ID: From: "K. Macy" To: Slawa Olhovchenkov Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org, freebsd-current@freebsd.org Subject: Re: pmap and mtx scalability problem X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2012 19:27:31 -0000 Known problem. There is an open disagreement about how to improve the granularity of locking in pmap. -Kip On Tue, Apr 24, 2012 at 9:14 PM, Slawa Olhovchenkov wrote: > I treid make -j 30 build{world,kernel} (latest -CURRENT) on 24-core machi= ne and see poor > scalability of pmap/mtx -- more then 50% cpu spend on system time. > > pmcstat: > > @ CPU_CLK_UNHALTED_CORE [194841 samples] > > 42.65% =A0[83102] =A0 =A0_mtx_lock_sleep @ /boot/kernel/kernel > =A040.97% =A0[34051] =A0 =A0 pmap_enter > =A0100.0% =A0[34051] =A0 =A0 =A0vm_fault_hold > =A0 100.0% =A0[34051] =A0 =A0 =A0 trap_pfault > =A030.40% =A0[25262] =A0 =A0 vm_page_activate > =A0100.0% =A0[25262] =A0 =A0 =A0vm_fault_hold > =A0 100.0% =A0[25262] =A0 =A0 =A0 trap_pfault > =A018.41% =A0[15300] =A0 =A0 vm_pageq_remove > =A073.63% =A0[11266] =A0 =A0 =A0vm_page_free_toq > =A0 70.69% =A0[7964] =A0 =A0 =A0 =A0vm_object_terminate > =A0 29.31% =A0[3302] =A0 =A0 =A0 =A0vm_object_page_remove > =A026.37% =A0[4034] =A0 =A0 =A0 vm_fault_hold > =A0 100.0% =A0[4034] =A0 =A0 =A0 =A0trap_pfault > > make -j 8: > > 15.44% =A0[10740] =A0 =A0_mtx_lock_sleep @ /boot/kernel/kernel > =A038.77% =A0[4164] =A0 =A0 =A0pmap_enter > =A099.93% =A0[4161] =A0 =A0 =A0 vm_fault_hold > =A0 100.0% =A0[4161] =A0 =A0 =A0 =A0trap_pfault > =A000.07% =A0[3] =A0 =A0 =A0 =A0 =A0kmem_back > =A0 100.0% =A0[3] =A0 =A0 =A0 =A0 =A0 kmem_malloc > =A027.98% =A0[3005] =A0 =A0 =A0vm_page_activate > =A0100.0% =A0[3005] =A0 =A0 =A0 vm_fault_hold > =A0 100.0% =A0[3005] =A0 =A0 =A0 =A0trap_pfault > =A020.64% =A0[2217] =A0 =A0 =A0vm_pageq_remove > =A064.73% =A0[1435] =A0 =A0 =A0 vm_page_free_toq > =A0 63.41% =A0[910] =A0 =A0 =A0 =A0 vm_object_terminate > > make -j 4 > > 06.86% =A0[4222] =A0 =A0 pagezero @ /boot/kernel/kernel > =A098.39% =A0[4154] =A0 =A0 =A0trap_pfault > =A0100.0% =A0[4154] =A0 =A0 =A0 trap > =A001.11% =A0[47] =A0 =A0 =A0 =A0vm_fault_hold > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd= .org" --=20 =A0 =A0=93The real damage is done by those millions who want to 'get by.' The ordinary men who just want to be left in peace. Those who don=92t want their little lives disturbed by anything bigger than themselves. Those with no sides and no causes. Those who won=92t take measure of their own strength, for fear of antagonizing their own weakness. Those who don=92t like to make waves=97or enemies. =A0 =A0Those for whom freedom, honour, truth, and principles are only literature. Those who live small, love small, die small. It=92s the reductionist approach to life: if you keep it small, you=92ll keep it under control. If you don=92t make any noise, the bogeyman won=92t find you. =A0 =A0But it=92s all an illusion, because they die too, those people who roll up their spirits into tiny little balls so as to be safe. Safe?! >From what? Life is always on the edge of death; narrow streets lead to the same place as wide avenues, and a little candle burns itself out just like a flaming torch does. =A0 =A0I choose my own way to burn.=94 =A0 =A0Sophie Scholl