From owner-freebsd-hackers@freebsd.org Thu Apr 19 21:46:07 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 096FBFA5634 for ; Thu, 19 Apr 2018 21:46:07 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5E8C189C6C; Thu, 19 Apr 2018 21:46:06 +0000 (UTC) (envelope-from kib@freebsd.org) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w3JLjoAK045012 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 20 Apr 2018 00:45:54 +0300 (EEST) (envelope-from kib@freebsd.org) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w3JLjoAK045012 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w3JLjoxK045008; Fri, 20 Apr 2018 00:45:50 +0300 (EEST) (envelope-from kib@freebsd.org) X-Authentication-Warning: tom.home: kostik set sender to kib@freebsd.org using -f Date: Fri, 20 Apr 2018 00:45:50 +0300 From: Konstantin Belousov To: Conrad Meyer Cc: Colin Percival , "freebsd-hackers@freebsd.org" Subject: Re: RFC: Hiding per-CPU kernel output behind bootverbose Message-ID: <20180419214550.GF6887@kib.kiev.ua> References: <01000162df15f856-1e5d2641-2a72-4250-8d8e-adcd47bc5db4-000000@email.amazonses.com> <20180419204405.GE6887@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.5 (2018-04-13) X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2018 21:46:07 -0000 On Thu, Apr 19, 2018 at 02:37:56PM -0700, Conrad Meyer wrote: > On Thu, Apr 19, 2018 at 1:44 PM, Konstantin Belousov > wrote: > > On Thu, Apr 19, 2018 at 06:06:21PM +0000, Colin Percival wrote: > >> On large systems (e.g., EC2's x1e.32xlarge instance type, with 128 vCPUs) > >> the boot time console output contains a large number of lines of the forms > >> > >> SMP: AP CPU #N Launched! > >> cpuN: on acpi0 > >> estN: on cpuN > >> > >> Having 128 almost-identical lines of output doesn't seem very useful, and > >> it actually has a nontrivial impact on the time spent booting. > >> > >> Does anyone mind if I hide these by default, having them only show up if > >> boot verbosity is requested? > > +1. For the device attaches, perhaps it makes sense to add a device > 'spammy' flag, and set it for per-CPU devices like cpuN or estN. Then > the generic attach code can choose whether to print spammy attaches > based on bootverbose. dmesg for device attaches seems mostly > redundant with devinfo(8) for persistent devices like ACPI CPU and > est(4). > > > The 'CPU XX Launched' messages are very useful for initial diagnostic > > of the SMP startup failures. You need to enable bootverbose to see the > > hang details, but for initial hint they are required. Unfortunately, AP > > startup hangs occur too often to pretend that this can be delegated to > > very specific circumstances. > > Really? I don't know that I've ever seen an AP startup hang. How > often do they occur? It was epidemic with Sandy Bridge, mostly correlated to specific BIOS supplier and its interaction with the x2APIC enablement, see madt.c:170 and below. There were several recent reports of the issue with Broadwell Xeon machines, no additional data or resolution. There are sporadic reports of the problem, where I do not see a clear commonality.