From nobody Fri Jun 18 13:57:51 2021 X-Original-To: arch@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id AB5237EC5AA for ; Fri, 18 Jun 2021 13:58:01 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4G60qs1Z5vz4ZFh for ; Fri, 18 Jun 2021 13:58:00 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 15IDvpNc027075 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 18 Jun 2021 16:57:54 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 15IDvpNc027075 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 15IDvp1o027074; Fri, 18 Jun 2021 16:57:51 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 18 Jun 2021 16:57:51 +0300 From: Konstantin Belousov To: Warner Losh Cc: Poul-Henning Kamp , "freebsd-arch@freebsd.org" Subject: Re: It's time to kill statistical profiling Message-ID: References: <202106180736.15I7aYmk068064@critter.freebsd.dk> List-Id: Discussion related to FreeBSD architecture List-Archive: https://lists.freebsd.org/archives/freebsd-arch List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arch@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4G60qs1Z5vz4ZFh X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Fri, Jun 18, 2021 at 07:35:44AM -0600, Warner Losh wrote: > On Fri, Jun 18, 2021 at 1:37 AM Poul-Henning Kamp > wrote: > > > Warners work to document the kernel timers in D30802 brought stathz up > > again. > > > > To give a representative result, statistical profiling needs to > > sample no less than approx 0.1% of instructions. > > > > On a VAX that meant running the statistical profiling at O(1kHz). > > > > On my 4 CPU, two thread, 2GHz laptop that means statistical profiling > > needs to run at O(10 MHz), which is barely doable. > > > > But it is worse: > > > > The samples must be unbiased with respect to the system activity, > > which was already a problem on the VAX and which is totally impossible > > on modern hardware, with message based interrupts, deep pipelines > > and telegraphic distance memory[1]. > > > > Therefore statistical profiling is worse than useless: it is downright > > misleading, which is why modern CPUs have hardware performance counters. > > > > Instead of documenting stathz, I suggest we retire statistical > > profiling and convert the profiled libraries to code-coverage > > profiling (-fprofile-arcs and -ftest-coverage) > > > > Poul-Henning > > > > [1] One could *possibly* approch unbiased samples, by locking the > > stathz code path in L1 cache and disable L1 updates, but then > > the results would be from an entirely different system. > > > > I've documented them as obsolete in favor of hwpmc (though I know that's not > quite the whole story). I'll remove that when we remove the feature, since > it > also documents the clockinfo sysctl a bit. > > stathz and profhz drive the setitimer ITIMER_PROF functionality. While this > was in POSIX.1-2001, it was marked deprecated in POSIX.1-2008. There's > sysctls > that return these values to userland as well, but those could return bogus > values. > Our current docs for this are vague enough to give some cover :) > > Deleting the code from the kernel is the easy bit, I guess. I generally > support > this notion, but don't have the time to hunt down all the stuff in base and > ports that uses these and make appropriate changes (up to and including git > rm). You cannot remove a kernel feature without breaking ABI. I do not see why do we need to remove it. The userspace bits, specifically build system glue to build _p.a libraries, is different, of course. We already have profiling static libraries build disabled. I also killed (as removed from the kernel build and config(8)) the support for -pg profiling). But for userspace, instead of removing, it could be quite useful to generalize it, instead of removing. In particular, to allow user to build some 'flavor' of system libraries, with intent to support either code coverage analysis, or PGO, or whatever toolchains authors implement. User would specify which additional toolchain options to pass instead of -pg, and probably should be able to provide his own crt1.o flavor.