From owner-freebsd-bugs@FreeBSD.ORG Sat May 24 10:30:01 2014 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 04922781 for ; Sat, 24 May 2014 10:30:01 +0000 (UTC) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD1832F1D for ; Sat, 24 May 2014 10:30:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.8/8.14.8) with ESMTP id s4OAU04d096778 for ; Sat, 24 May 2014 10:30:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.8/8.14.8/Submit) id s4OAU0W4096777; Sat, 24 May 2014 10:30:00 GMT (envelope-from gnats) Resent-Date: Sat, 24 May 2014 10:30:00 GMT Resent-Message-Id: <201405241030.s4OAU0W4096777@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Tim Bishop Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9BA54AE for ; Sat, 24 May 2014 10:20:00 +0000 (UTC) Received: from mx1.kent.ac.uk (mx1.kent.ac.uk [129.12.21.39]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6B69A2E71 for ; Sat, 24 May 2014 10:19:59 +0000 (UTC) Received: from apophis.ukc.ac.uk ([129.12.4.11]) by mx1.kent.ac.uk with esmtps (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1Wo93W-0006fU-Fd for FreeBSD-gnats-submit@freebsd.org; Sat, 24 May 2014 11:19:58 +0100 Received: from buzzard.kent.ac.uk ([129.12.3.201]) by apophis.ukc.ac.uk with esmtps (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.80.1) (envelope-from ) id 1Wo93V-0001uv-Rc for FreeBSD-gnats-submit@freebsd.org; Sat, 24 May 2014 11:19:57 +0100 Received: from buzzard.kent.ac.uk (localhost [127.0.0.1]) by buzzard.kent.ac.uk (8.14.9/8.14.9) with ESMTP id s4OAJu9i001380 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Sat, 24 May 2014 11:19:56 +0100 (BST) (envelope-from tdb@buzzard.kent.ac.uk) Received: (from tdb@localhost) by buzzard.kent.ac.uk (8.14.9/8.14.9/Submit) id s4OAJuUA001379; Sat, 24 May 2014 11:19:56 +0100 (BST) (envelope-from tdb) Message-Id: <201405241019.s4OAJuUA001379@buzzard.kent.ac.uk> Date: Sat, 24 May 2014 11:19:56 +0100 (BST) From: Tim Bishop Reply-To: Tim Bishop To: FreeBSD-gnats-submit@freebsd.org X-Send-Pr-Version: 3.114 Subject: kern/190169: Increase MAXCPU on amd64 systems X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 May 2014 10:30:01 -0000 >Number: 190169 >Category: kern >Synopsis: Increase MAXCPU on amd64 systems >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Sat May 24 10:30:00 UTC 2014 >Closed-Date: >Last-Modified: >Originator: Tim Bishop >Release: FreeBSD 11.0-CURRENT amd64 >Organization: >Environment: System: FreeBSD buzzard 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r266603M: Sat May 24 03:25:35 BST 2014 tdb@buzzard:/usr/obj/usr/src/sys/GENERIC amd64 4x Intel(R) Xeon(R) CPU E7-4830 v2 @ 2.20GHz (2200.05-MHz K8-class CPU) This gives 40 cores with a total of 80 threads. >Description: In param.h MAXCPU is set to 64. This results in 16 of the 80 threads being disabled at boot time, and errors like: FreeBSD/SMP: Multiprocessor System Detected: 64 CPUs FreeBSD/SMP: 3 package(s) x 10 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ... cpu62 (AP): APIC ID: 98 cpu63 (AP): APIC ID: 99 cpu (AP): APIC ID: 100 (disabled) cpu (AP): APIC ID: 101 (disabled) cpu (AP): APIC ID: 102 (disabled) cpu (AP): APIC ID: 103 (disabled) cpu (AP): APIC ID: 104 (disabled) cpu (AP): APIC ID: 105 (disabled) cpu (AP): APIC ID: 112 (disabled) cpu (AP): APIC ID: 113 (disabled) cpu (AP): APIC ID: 114 (disabled) cpu (AP): APIC ID: 115 (disabled) cpu (AP): APIC ID: 116 (disabled) cpu (AP): APIC ID: 117 (disabled) cpu (AP): APIC ID: 118 (disabled) cpu (AP): APIC ID: 119 (disabled) cpu (AP): APIC ID: 120 (disabled) cpu (AP): APIC ID: 121 (disabled) WARNING: Non-uniform processors. WARNING: Using suboptimal topology. >How-To-Repeat: Boot any recent FreeBSD version, including HEAD, on a system with more than 64 CPU cores/threads. >Fix: The current top end processors have 15 cores and 30 threads. So a quad socket system would have a total of 120 threads. I believe this should be supported in GENERIC so FreeBSD works out of the box. The patch below does this. It'd be nice to see this MFCed to stable/10 so it makes the 10.1 release. It'd be even nicer to see it MFCed to stable/9 for 9.3, but I suspect it'll be too late for that. For what it's worth I tested the most recent Ubuntu release and it detected all the threads out of the box. Some consideration should also be given to kern/185831. It has been committed but I don't think it has been MFCed yet. I can confirm that I don't receive any panic on HEAD with MAXCPU set to 128 and with witness enabled. --- diff begins here --- Index: sys/amd64/include/param.h =================================================================== --- sys/amd64/include/param.h (revision 266603) +++ sys/amd64/include/param.h (working copy) @@ -65,7 +65,7 @@ #if defined(SMP) || defined(KLD_MODULE) #ifndef MAXCPU -#define MAXCPU 64 +#define MAXCPU 128 #endif #else #define MAXCPU 1 --- diff ends here --- >Release-Note: >Audit-Trail: >Unformatted: