From owner-freebsd-current@freebsd.org Wed Jan 4 19:58:35 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4478C9F954 for ; Wed, 4 Jan 2017 19:58:35 +0000 (UTC) (envelope-from ian.freislich@capeaugusta.com) Received: from mail-yw0-x22c.google.com (mail-yw0-x22c.google.com [IPv6:2607:f8b0:4002:c05::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 855591AF7 for ; Wed, 4 Jan 2017 19:58:35 +0000 (UTC) (envelope-from ian.freislich@capeaugusta.com) Received: by mail-yw0-x22c.google.com with SMTP id a10so324306840ywa.3 for ; Wed, 04 Jan 2017 11:58:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=capeaugusta-com.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=/ORaWFOWruRTbr6lb0ms9O8qeOgzcnKAYjuCACZUMNQ=; b=cnu8vphv2vBOdGyXrUa0bggBRyxUJ7zeJfaedp847rRxyWxt8UD0ADIqVrIZFtowyt oWtoj3Mftrx0ZUfi/KusdxxHmgytNxGMdYSJozBj83/UjrmZwMoWJsiwXzrrRz+vYH+E 36ZCeQ5NugVG6s58kdKzN1P6NxyAy/5DQTYQ9xmKCvDFRv5GwpN/9eqsRN6AvdwxF1zi jbi+/GLurwBiPMOzcyR9Jjip9bMoKFRen2difLwaHyGVsMXaSxlWLvUIDYU42c5SNiwT 1Tc9K74dIaKpR2mCE5gyniW5k6I8dIT6LnhA3I5PsJbx73oNynzruCI4KNiUoubVwewW a6Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=/ORaWFOWruRTbr6lb0ms9O8qeOgzcnKAYjuCACZUMNQ=; b=S70fVTloEKV1dG/CNyljm4ZhsduL3MVgDPw687RTtMBNYsqioiOR+PsRm53+4OvXlq my9iPlGQ0LYa1LdK1aKBMd2P76FbaNmrhecfurXi0iNwzlI6aC+XEuOq8MwO93yQ5HoO /GSdbanOMgDrvyfwrBmuDs45WWsEO49P2+eeX+6JBOHsRzcU05OURLceSByVHvOs6odW b/fBpyi5GaelXLC8squd0klRva4jusNsfYu74JxGefiHUzEdtwE8X9T7xxskSFpYez8a Bs1tlHw1Qz4si46Z91jxhuGwyyJJ+CxLxq711E2EpUi7WNkvfG1fgZdLiXcIvpsrQJs/ T7Qg== X-Gm-Message-State: AIkVDXK9UySBT95N/UTJBK8AVoqK/z0NT8FsUKbD2Qvl2wrU9YEKRsluHi06bsysZ6zfKz5sULMBlxbTn+5HexTZ31rVGNASaZEOXdWWviTVjH45QD7jp9lXs3iwO6w+9qN9pCOcuhU= X-Received: by 10.13.197.68 with SMTP id h65mr65272935ywd.84.1483559914352; Wed, 04 Jan 2017 11:58:34 -0800 (PST) Received: from zen.clue.co.za ([64.53.114.237]) by smtp.gmail.com with ESMTPSA id b140sm30652684ywh.33.2017.01.04.11.58.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 04 Jan 2017 11:58:33 -0800 (PST) Subject: x2APIC support (Re: How many CPU cores does FreeBSD support?) To: Eric Joyner , FreeBSD Current , Andriy Gapon , Konstantin Belousov References: From: Ian FREISLICH Message-ID: Date: Wed, 4 Jan 2017 14:58:33 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2017 19:58:35 -0000 Hi I think your issue relates to x2APIC support and I don't think that full support for parsing x2APIC MADT entries and doing interrupts remapping has been committed yet from my very brief search of the source and commit messages. I've copied Andriy Gapon and Konstantin Belousov who participated in another x2APIC thread in September last year. They will probably have a more intelligible response. sys/x86/include/apicvar.h limits APIC IDs as follows: #define MAX_APIC_ID 0xfe #define APIC_ID_ALL 0xff This is why CPUs with an APIC ID above 255 are being ignored in acpica/madt.c sys/x86/x86/mptable.c has this choice piece of code: if (intr->dst_apic_id == 0xff) apic_id = APIC_ID_ALL; else apic_id = intr->dst_apic_id; which for the current value of APIC_ID_ALL can simply be rewritten as: apic_id = intr->dst_apic_id; I do not think it is safe to increase MAX_APIC_ID yet. Ian -- Ian Freislich On 01/04/17 13:53, Eric Joyner wrote: > Adding freebsd-current, because that's a good idea. > > I see these lines in the beginning of dmesg: > > MADT: Ignoring local APIC ID 256 (too high) > > > [907/911] > MADT: Ignoring local APIC ID 260 (too high) > > > [906/911] > MADT: Ignoring local APIC ID 264 (too high) > > > [905/911] > MADT: Ignoring local APIC ID 268 (too high) > > > [904/911] > MADT: Ignoring local APIC ID 272 (too high) > > > [903/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > > > [901/911] > MADT: Ignoring local APIC ID 272 (too high) > > > [903/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > > > [901/911] > MADT: Ignoring local APIC ID 276 (too high) > > > [902/911] > MADT: Ignoring local APIC ID 280 (too high) > MADT: Ignoring local APIC ID 284 (too high) > MADT: Ignoring local APIC ID 288 (too high) > MADT: Ignoring local APIC ID 292 (too high) > MADT: Ignoring local APIC ID 296 (too high) > MADT: Ignoring local APIC ID 300 (too high) > MADT: Ignoring local APIC ID 257 (too high) > MADT: Ignoring local APIC ID 261 (too high) > MADT: Ignoring local APIC ID 265 (too high) > MADT: Ignoring local APIC ID 269 (too high) > MADT: Ignoring local APIC ID 273 (too high) > MADT: Ignoring local APIC ID 277 (too high) > MADT: Ignoring local APIC ID 281 (too high) > MADT: Ignoring local APIC ID 285 (too high) > MADT: Ignoring local APIC ID 289 (too high) > MADT: Ignoring local APIC ID 293 (too high) > MADT: Ignoring local APIC ID 297 (too high) > MADT: Ignoring local APIC ID 301 (too high) > MADT: Ignoring local APIC ID 258 (too high) > MADT: Ignoring local APIC ID 262 (too high) > MADT: Ignoring local APIC ID 266 (too high) > MADT: Ignoring local APIC ID 270 (too high) > MADT: Ignoring local APIC ID 274 (too high) > MADT: Ignoring local APIC ID 278 (too high) > MADT: Ignoring local APIC ID 282 (too high) > MADT: Ignoring local APIC ID 286 (too high) > MADT: Ignoring local APIC ID 290 (too high) > MADT: Ignoring local APIC ID 294 (too high) > MADT: Ignoring local APIC ID 298 (too high) > MADT: Ignoring local APIC ID 302 (too high) > MADT: Ignoring local APIC ID 255 (too high) > MADT: Ignoring local APIC ID 259 (too high) > MADT: Ignoring local APIC ID 263 (too high) > MADT: Ignoring local APIC ID 267 (too high) > MADT: Ignoring local APIC ID 271 (too high) > MADT: Ignoring local APIC ID 275 (too high) > MADT: Ignoring local APIC ID 279 (too high) > MADT: Ignoring local APIC ID 283 (too high) > MADT: Ignoring local APIC ID 287 (too high) > MADT: Ignoring local APIC ID 291 (too high) > MADT: Ignoring local APIC ID 295 (too high) > MADT: Ignoring local APIC ID 299 (too high) > MADT: Ignoring local APIC ID 303 (too high) > Copyright (c) 1992-2016 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016 > root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 > FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM > 3.8.0) > SRAT: No CPU found for memory domain 1 > VT(efifb): resolution 800x600 > CPU: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz (1396.80-MHz K8-class CPU) > Origin="GenuineIntel" Id=0x50671 Family=0x6 Model=0x57 Stepping=1 > > Features=0xbfebfbff > > Features2=0x7ff8f39f > AMD Features=0x2c100800 > AMD Features2=0x121 > Structured Extended > Features=0x1c0d23ab > Structured Extended Features2=0x1 > XSAVE Features=0x1 > TSC: P-state invariant, performance statistics > real memory = 223332007936 (212986 MB) > avail memory = 216976429056 (206924 MB) > Event timer "LAPIC" quality 600 > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 223 CPUs > FreeBSD/SMP: Non-uniform topology > > It sounds like those MADT errors point to the problem? > > - Eric > > On Tue, Jan 3, 2017 at 11:41 PM Shane Ambler wrote: > >> On 04/01/2017 08:08, Eric Joyner wrote: >>> I have an 11.0-RELEASE-p1 system with: >>> >>> hw.model: Intel(R) Xeon Phi(TM) CPU 7250 @ 1.40GHz >>> >>> It's got 68 cores, but with 4-way SMT, I think it should show up with 272 >>> logical cores on FreeBSD. >>> >>> However, in both kern.sched.topology_spec and hw.ncpu, it shows there are >>> only 223 cores. >>> >>> Is the limit effectively 223 cores, even though I see some #define MAXCPU >>> 256 lines in the source? >> You will probably find freebsd-current a better mailing list for this. >> >> How many show in dmesg? >> >> From https://svnweb.freebsd.org/base?view=revision&revision=270223 >> Bump MAXCPU on amd64 from 64 to 256. In practice APIC only permits 255 >> CPUs (IDs 0 through 254). Getting above that limit requires x2APIC. >> >> Also note that changing MAXCPU breaks ABI compatability so you will >> need to recompile. >> https://svnweb.freebsd.org/base?view=revision&revision=284720 >> >> -- >> FreeBSD - the place to B...Software Developing >> >> Shane Ambler >> >> > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -- Cape Augusta Digital Properties, LLC a Cape Augusta Company *Breach of confidentiality & accidental breach of confidentiality * This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.