From owner-freebsd-hackers@freebsd.org Thu Apr 19 23:05:02 2018 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B067DFAA489 for ; Thu, 19 Apr 2018 23:05:02 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: from mail-oi0-x230.google.com (mail-oi0-x230.google.com [IPv6:2607:f8b0:4003:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3D6D872704; Thu, 19 Apr 2018 23:05:02 +0000 (UTC) (envelope-from sblachmann@gmail.com) Received: by mail-oi0-x230.google.com with SMTP id n65-v6so6416417oig.6; Thu, 19 Apr 2018 16:05:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=+7sZq/3fviy3H2qG4I87BEOVGzcnamYX2fXFX3TGX6k=; b=EgYRvvEVMCKQ4UijTyR4UY55SWZ8PLbMe1lwG7LcJBhuZaUIJz+mzY+AVQysZZJtwN oTp5zVSk7HLHT707e6QxpOxq7DU0sO7zmnhqdk54QfJODJcwKVdrx7BTrQNtaWw6RQAq KOQzSM5K4tX94Ho1yWUAqisPwwIxHOuCL4EOm+VDdm0uAw4O/UF4mFksNZfhoxRZVuSs h80IoysfcF5fG07gRhpBWKHD1PJ89oo+RSy80ICQU6EYJ3Xtrq+d8ghoyE7VmUnKbvSm kk7F9mvGswWEbwd+eqGvpvRVNzb59XmQ839gD4VV3MBbxilwbbPSiOWmbMr+ASlvVBGu kPfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=+7sZq/3fviy3H2qG4I87BEOVGzcnamYX2fXFX3TGX6k=; b=ucrkJWzvVwaMSxi/HEfIaXuhaYAg2BivrGd1LjLGDUYnTo09VCYi33UsN+YnbPHpQE FVGa8Scr2Y2JgX80IkeUXx3JDpG9NDEI9tvCW3TRo+aHvjzGiT0Zhyzw2+Fe1mhEZSFa JFo5DfwbiMzS9TkZrpAf3bjmtYod23REYxy4TWS6EI5p4tawqFP7vQXqjzhlmgA2OlEv aV3yam9Z1T4dFZu9ZRRw2Yf9aLvD90HoDeGmSBmhnbjEqQZzXSCAjm96BJeGWr7EFE34 K1e+SLs+TrQtjLGvvYg0/PPwhup/4yYZSGfeBq8xg1lAmqkWW1S+/Ls4+K8RBKKNMBl8 Ub6g== X-Gm-Message-State: ALQs6tCx8tRmMmZuK+HSA9IczBf9p+BW88cthSz4fDdNOlqPRyKFfdbp WdHq2Xe15qOy9EaYrLkhJOuFoRyYNjxK9Kn6Ksw= X-Google-Smtp-Source: AIpwx48FiZeGrI5KokHRK7qJJKRbe2b9IzFT58irawhF9yRVDtvl+ZDPrxbbuQFXnKkxgb5aq/HObWA2vCShoWOSgGs= X-Received: by 2002:aca:5751:: with SMTP id l78-v6mr4641685oib.171.1524179101407; Thu, 19 Apr 2018 16:05:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.201.60.135 with HTTP; Thu, 19 Apr 2018 16:05:00 -0700 (PDT) In-Reply-To: <20180419214550.GF6887@kib.kiev.ua> References: <01000162df15f856-1e5d2641-2a72-4250-8d8e-adcd47bc5db4-000000@email.amazonses.com> <20180419204405.GE6887@kib.kiev.ua> <20180419214550.GF6887@kib.kiev.ua> From: Stefan Blachmann Date: Fri, 20 Apr 2018 01:05:00 +0200 Message-ID: Subject: Re: RFC: Hiding per-CPU kernel output behind bootverbose To: Konstantin Belousov Cc: Conrad Meyer , "freebsd-hackers@freebsd.org" , Colin Percival Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Apr 2018 23:05:02 -0000 I would also like a solution to this issue, which is looking highly unprofessional and uselessly wasting space in the dmesg ring buffer. In the first version of my sysutils/cpupdate tool I also had spammy output like being complained about. This was hard-to-read. Thus I changed code so that identical cores output was collected up in blocks of single output, i.e.: "core #n to #m: ". If all cpus are identical, cpu stats will be output only once this way, else in subsequent blocks comprising all identical cores on different cpus. To avoid the spamming at bootup and microcode updating, the kernel would need such a function to read/evaluate *all* cores in a row, like I did in cpupdate. This would be only a few lines of code. On 4/19/18, Konstantin Belousov wrote: > On Thu, Apr 19, 2018 at 02:37:56PM -0700, Conrad Meyer wrote: >> On Thu, Apr 19, 2018 at 1:44 PM, Konstantin Belousov >> wrote: >> > On Thu, Apr 19, 2018 at 06:06:21PM +0000, Colin Percival wrote: >> >> On large systems (e.g., EC2's x1e.32xlarge instance type, with 128 >> >> vCPUs) >> >> the boot time console output contains a large number of lines of the >> >> forms >> >> >> >> SMP: AP CPU #N Launched! >> >> cpuN: on acpi0 >> >> estN: on cpuN >> >> >> >> Having 128 almost-identical lines of output doesn't seem very useful, >> >> and >> >> it actually has a nontrivial impact on the time spent booting. >> >> >> >> Does anyone mind if I hide these by default, having them only show up >> >> if >> >> boot verbosity is requested? >> >> +1. For the device attaches, perhaps it makes sense to add a device >> 'spammy' flag, and set it for per-CPU devices like cpuN or estN. Then >> the generic attach code can choose whether to print spammy attaches >> based on bootverbose. dmesg for device attaches seems mostly >> redundant with devinfo(8) for persistent devices like ACPI CPU and >> est(4). >> >> > The 'CPU XX Launched' messages are very useful for initial diagnostic >> > of the SMP startup failures. You need to enable bootverbose to see the >> > hang details, but for initial hint they are required. Unfortunately, AP >> > startup hangs occur too often to pretend that this can be delegated to >> > very specific circumstances. >> >> Really? I don't know that I've ever seen an AP startup hang. How >> often do they occur? > > It was epidemic with Sandy Bridge, mostly correlated to specific BIOS > supplier and its interaction with the x2APIC enablement, see madt.c:170 > and below. > > There were several recent reports of the issue with Broadwell Xeon > machines, no additional data or resolution. > > There are sporadic reports of the problem, where I do not see > a clear commonality. > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" >