From owner-svn-src-all@FreeBSD.ORG Wed Jul 27 17:19:05 2011 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACA1E1065672; Wed, 27 Jul 2011 17:19:05 +0000 (UTC) (envelope-from seanbru@yahoo-inc.com) Received: from mrout2.yahoo.com (mrout2.yahoo.com [216.145.54.172]) by mx1.freebsd.org (Postfix) with ESMTP id 8900C8FC14; Wed, 27 Jul 2011 17:19:05 +0000 (UTC) Received: from [127.0.0.1] (cheese.corp.yahoo.com [216.145.50.99]) by mrout2.yahoo.com (8.14.4/8.14.4/y.out) with ESMTP id p6RH8EIt007926; Wed, 27 Jul 2011 10:08:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yahoo-inc.com; s=cobra; t=1311786494; bh=7k9rjDsUHxBRzWrHRpYqZPvOX4hsL9QmSJRa9bVb+zA=; h=Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date: Message-ID:Mime-Version:Content-Transfer-Encoding; b=fxe1d0Hp4mFZEr4DvqXrWTjd4kapsxniq7bmEx0F6VVW/SG0v1Bd9LEGKKLkF4Cns Bz5CETPTykFKRAc2/I1ee8bB2M8xzhk0O8EYDXVGvBAooTCvtHUR7KEeFijixmWHo5 G+WuYdHzmbo8Ywen/0z1V52Jiv7lqtDKS5rdt01U= From: Sean Bruno To: John Baldwin In-Reply-To: <201107270845.39419.jhb@freebsd.org> References: <201107191300.p6JD0UU7073447@svn.freebsd.org> <201107210920.09604.jhb@freebsd.org> <201107270845.39419.jhb@freebsd.org> Content-Type: text/plain; charset="UTF-8" Date: Wed, 27 Jul 2011 10:08:14 -0700 Message-ID: <1311786494.2592.5.camel@hitfishpass-lx.corp.yahoo.com> Mime-Version: 1.0 X-Mailer: Evolution 2.32.2 (2.32.2-1.fc14) Content-Transfer-Encoding: 7bit Cc: "src-committers@freebsd.org" , "svn-src-all@freebsd.org" , Attilio Rao , Sergey Kandaurov , Robert Watson , Pan Tsu , "svn-src-head@freebsd.org" Subject: Re: svn commit: r224217 - in head/sys: amd64/include ia64/include mips/conf sys X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jul 2011 17:19:05 -0000 On Wed, 2011-07-27 at 05:45 -0700, John Baldwin wrote: > On Wednesday, July 27, 2011 2:24:21 am Sergey Kandaurov wrote: > > On 21 July 2011 17:20, John Baldwin wrote: > > > On Thursday, July 21, 2011 8:37:26 am Sergey Kandaurov wrote: > > >> On 21 July 2011 14:14, Attilio Rao wrote: > > >> > 2011/7/20 Pan Tsu : > > >> >> Attilio Rao writes: > > >> >> > > >> >>> Author: attilio > > >> >>> Date: Tue Jul 19 13:00:30 2011 > > >> >>> New Revision: 224217 > > >> >>> URL: http://svn.freebsd.org/changeset/base/224217 > > >> >>> > > >> >>> Log: > > >> >>> Bump MAXCPU for amd64, ia64 and XLP mips appropriately. > > >> >>> From now on, default values for FreeBSD will be 64 maxiumum supported > > >> >>> CPUs on amd64 and ia64 and 128 for XLP. All the other architectures > > >> >>> seem already capped appropriately (with the exception of sparc64 which > > >> >>> needs further support on jalapeno flavour). > > >> >>> > > >> >>> Bump __FreeBSD_version in order to reflect KBI/KPI brekage introduced > > >> >>> during the infrastructure cleanup for supporting MAXCPU > 32. This > > >> >>> covers cpumask_t retiral too. > > >> >>> > > >> >>> The switch is considered completed at the present time, so for > > > whatever > > >> >>> bug you may experience that is reconducible to that area, please > > > report > > >> >>> immediately. > > >> >>> > > >> >>> Requested by: marcel, jchandra > > >> >>> Tested by: pluknet, sbruno > > >> >>> Approved by: re (kib) > > >> >>> > > >> >>> Modified: > > >> >>> head/sys/amd64/include/param.h > > >> >>> head/sys/ia64/include/param.h > > >> >>> head/sys/mips/conf/XLP > > >> >>> head/sys/mips/conf/XLP64 > > >> >>> head/sys/mips/conf/XLPN32 > > >> >>> head/sys/sys/param.h > > >> >>> > > >> >>> Modified: head/sys/amd64/include/param.h > > >> >>> > > > ============================================================================== > > >> >>> --- head/sys/amd64/include/param.h Tue Jul 19 12:41:57 2011 > > > (r224216) > > >> >>> +++ head/sys/amd64/include/param.h Tue Jul 19 13:00:30 2011 > > > (r224217) > > >> >>> @@ -65,7 +65,7 @@ > > >> >>> > > >> >>> #if defined(SMP) || defined(KLD_MODULE) > > >> >>> #ifndef MAXCPU > > >> >>> -#define MAXCPU 32 > > >> >>> +#define MAXCPU 64 > > >> >>> #endif > > >> >>> #else > > >> >>> #define MAXCPU 1 > > >> >> > > >> >> Do you plan to bump MEMSTAT_MAXCPU, too? > > >> >> > > >> >> $ vmstat -z > > >> >> vmstat: memstat_sysctl_uma: Too many CPUs > > >> >> $ vmstat -m > > >> >> vmstat: memstat_sysctl_malloc: Too many CPUs > > >> >> > > >> >> $ sysctl kern. | grep smp.\*cpus > > >> >> kern.smp.maxcpus: 64 > > >> >> kern.smp.cpus: 2 > > >> >> > > >> > > > >> > Jeeeez, we seriously need to fix this getting rid of the static values. > > >> > > > >> > Anyway, can you try the following patch?: > > >> > http://www.freebsd.org/~attilio/memstat_maxcpu.diff > > >> > > > >> > It is going to add some memory overhead for i386 case. > > >> > > > >> > > >> Something like this should work (vmstat -z, vmstat -m both work). > > >> It gets rid of MEMSTAT_MAXCPU at the expense of malloc() at runtime. > > >> http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.diff > > >> > > >> Probably it should work with maxid, instead of maxcpu to save some memory. > > >> Though, using maxcpu is more safe. > > > > > > Actually, I would prefer that it use mp_maxid as that is the general variable > > > things should use. mp_maxcpus is a concession for the few places that may > > > need to know the MAXCPUS value (e.g. if using libkvm to access a structure in > > > a crashdump or live kernel that has a member array with MAXCPU elements). > > > Code that just wants to allocate memory to hold per-CPU data should use > > > mp_maxid whenever possible. > > > > > > > Hi, > > > > I changed the patch to use mp_maxid wherever possible. > > http://plukky.net/~pluknet/patches/libmemstat_nomaxcpu.2.diff > > > > To summarize: > > > > 1) malloc stats > > kern.malloc_stats uses internally MAXCPU, and we have to query MAXCPU > > from kernel, too. See kern/kern_malloc.c:sysctl_kern_malloc_stats(): > > 849 mtsh.mtsh_maxcpus = MAXCPU; > > > > 1a) memstat_sysctl_malloc() > > left unchanged, sysctl kern.smp.maxcpus > > 1b) memstat_kvm_malloc() > > left unchanged, _mp_maxcpus symbol > > > > 2) uma stats > > vm.zone_stats uses (mp_maxid + 1), vm/uma_core.c:sysctl_vm_zone_stats(): > > 3247 ush.ush_maxcpus = (mp_maxid + 1); > > > > 2a) memstat_sysctl_uma() > > Switched to query sysctl kern.smp.maxid > > 2b) memstat_kvm_uma() > > left unchanged, _mp_maxid symbol > > > > So, there's only one change in memstat_sysctl_uma(). > > A bad side of things is that libmemstat() now knows these vm_zone > > and malloc_stats internals. > > As Robert suggested me on IRC to query maxcpu value from > > uma_stream_header and malloc_type_stream_header structures > > respectively to be independent from kernel details. > > Looks good to me, thanks! > Aye +1 here. I'll run this over to peter@ just to be sure though. I'm pretty sure that this will just work. I'll ask HP for their big machine again to validate. Sean