From owner-freebsd-arch@FreeBSD.ORG Sat Mar 15 05:43:02 2008 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 190D31065673 for ; Sat, 15 Mar 2008 05:43:02 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.158]) by mx1.freebsd.org (Postfix) with ESMTP id 814BC8FC23 for ; Sat, 15 Mar 2008 05:43:01 +0000 (UTC) (envelope-from joseph.koshy@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so3696780fgg.35 for ; Fri, 14 Mar 2008 22:43:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=1LDe7embEZsf2PAgnps0sas/wEp08h11p+VgJvmqArs=; b=Nh2/gSdvSQ7UetRjd3jhNFO65Pe0X9ZHuGgOfOPWxy8ncDkLelbIMVcVpMcxvGcYg1BePkr7SxuUYysJW+tFEsvLFUbL/edAGsy2SIz5+kX6Gal6B2b0aKAYkYWRnKjhsc6ZRwcolo3t9G/6ax+RD0U8TCUlmix9jGMtdwuxnmU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=esk+2crhO2k4Ut9bpuCn2sMeVD2x7ntzexv2PRAWLjxNd2U/HO3O8lsuzWv5glkWTcpSkVc0vjbxGQfYrLxFi6YI0M3w7UsR9RmEJ6Wns+eXCH5/d5gOHxNFlqSj0mtZvu76AReTLax1BNxJA/It+nQ3a+teoOh6NweFm8oQSFk= Received: by 10.86.26.11 with SMTP id 11mr11266492fgz.74.1205559780337; Fri, 14 Mar 2008 22:43:00 -0700 (PDT) Received: by 10.86.99.18 with HTTP; Fri, 14 Mar 2008 22:43:00 -0700 (PDT) Message-ID: <84dead720803142243r6c8cc68dm325e7fb925189fd@mail.gmail.com> Date: Sat, 15 Mar 2008 11:13:00 +0530 From: "Joseph Koshy" To: "John Baldwin" In-Reply-To: <200803141431.53846.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080313180805.GA83406@dragon.NUXI.org> <200803131516.12284.jhb@freebsd.org> <84dead720803132232k15c3aad7pe59875f0c84e0c27@mail.gmail.com> <200803141431.53846.jhb@freebsd.org> Cc: freebsd-arch@freebsd.org Subject: Re: [PATCH] hwpmc(4) changes to use 'mp_maxid' instead of 'mp_ncpus'. X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 15 Mar 2008 05:43:02 -0000 > FreeBSD has been trying to not be quite as i386-centric as it used to be. If > you look at other code in the kernel that handles per-cpu data such as UMA > you will see that it uses mp_maxid and CPU_ABSENT(). There are other places > in the kernel that are broken though (such as ndis(4)). HWPMC is very x86 centric, for obvious reasons. > Yes, we can certainly export more stuff to userland. The all_cpus mask would > be good as would a MI online_cpus mask, though at this point they would be > cpusets to handle > 32 rather than cpumask_t. Note that machdep.hlt_cpus is > x86-only and would be superseded by a MI online_cpus mask. Sure, an MI counter is a good idea. > > - Will sysctl hw.ncpus represent the count of present CPUs or will it > > represent the maximum CPU id? > > hw.ncpus is always mp_ncpus > kern.smp.cpus is also mp_ncpus > kern.smp.maxcpus is MAX_CPUS. > Userland can just iterate from 0 to kern.smp.maxcpus while handling absent > CPUs. (For example, the kern.cp_time[] sysctl just writes out all 0's for > absent CPUs so that is how userland can determine an absent CPU in that > case.) I thought of that. For PMCTools use, using the proposed 'online_cpus' mask would be a better option. MAX_CPUS is a compile time value and could be large, whereas most machines will have far fewer CPUs than that limit. Why waste cycles needlessly? Now it appears to me that in the scheme of things described above one of mp_maxid and mp_ncpus is superfluous. Here is the reasoning: 0) We need a compile time limit for the kernel; this is kern.smp.maxcpus. 1) A given machine has a maximum number of CPUs that can fit in it. This is usually <<= MAXCPUS. Let us call this {MACHINE-MAX}. We need to scale kernel data structures based on {MACHINE-MAX} since using {MAXCPUS} is probably wasteful. We cannot just count the current number of CPUS, as we do today, because more could be hotplugged in later. 2) At any given instant a subset of CPUs 0..{MACHINE_MAX} will be online. This would be tracked by the kern.smp.online_cpus/all_cpus bitmask. Therefore we can use either a count (mp_ncpus) or a maximum id (mp_maxid) to represent {MACHINE-MAX}, but either one would do. However, x86 MD code uses both, with newer code seeming to prefer mp_maxid. So I am puzzled. There are far more uses of mp_ncpus there though. jk> Changing HWPMC and its userland before the base kernel itself jk> changes does not seem to be the right thing to do. jb> While the userland intIerface is somewhat lacking, all of the in-kernel jb> infrastructure has been in place for at least the past 4 years, and there is jb> no excuse for any in-kernel code not properly handling sparse CPU IDs. I try keep userland, kernel and documentation associated with PmcTools in sync. Looking around, there appear to be lots of nits that need correction. For one, the kern.smp sysctl hierarchy is undocumented. Thanks, Koshy