From owner-freebsd-arch@FreeBSD.ORG Mon Jan 16 04:43:36 2006 Return-Path: X-Original-To: arch@freebsd.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5D8316A41F; Mon, 16 Jan 2006 04:43:36 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7A83C43D45; Mon, 16 Jan 2006 04:43:36 +0000 (GMT) (envelope-from ssouhlal@FreeBSD.org) Received: from [192.168.0.2] (c-24-6-173-198.hsd1.ca.comcast.net [24.6.173.198]) by elvis.mu.org (Postfix) with ESMTP id 5CCAE1A3C1B; Sun, 15 Jan 2006 20:43:36 -0800 (PST) Message-ID: <43CB246D.1050705@FreeBSD.org> Date: Sun, 15 Jan 2006 20:43:25 -0800 From: Suleiman Souhlal User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051204) X-Accept-Language: en-us, en MIME-Version: 1.0 To: arch@freebsd.org, current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: [RFC/Benchmarks] Per-CPU freelists X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Jan 2006 04:43:36 -0000 Hello, I implemented per-CPU page freelists this weekend, in the hopes that it would improve performance on SMP machines, as it should save a spinlock acquisition in vm_page_alloc(), in most cases (except when VM_ALLOC_INTERRUPT is set, and when the current cpu's free list is empty), and reduce contention. However, I was only able to test it on machine with two CPUs, where it didn't seem to make any difference. You can set the number of pages that get added to the freelists each time it gets refilled in vm.pcpu.refill_num, and the maximum length of the freelists in vm.pcpu.max_len. Some stats are viewable in vm.pcpu.stats. The patch is available at http://people.freebsd.org/~ssouhlal/testing/pcpu-freelists-20060115.diff I would really appreciate if someone could benchmark/test this on a machine with more processors. Here's the output of ministat(1) for buildkernel: x refill_num=32, max_len=64 + refill_num=4, max_len=-1 (effectively disabling the percpu freelists) +--------------------------------------------------------------------------+ |+ + x x+ x + + x x| | |_____________|_______M_MA____________A____________|__________| | +--------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 5 171.64 172.09 171.69 171.816 0.21220273 + 5 171.44 171.97 171.67 171.702 0.22928149 No difference proven at 95.0% confidence -- Suleiman