From owner-freebsd-current@freebsd.org Mon Mar 8 15:58:38 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A1F52576085 for ; Mon, 8 Mar 2021 15:58:38 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from maybe.home.utahime.org (gate.home.utahime.org [183.180.29.210]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DvNL52pGcz4nqj for ; Mon, 8 Mar 2021 15:58:36 +0000 (UTC) (envelope-from yasu@utahime.org) Received: from eastasia.home.utahime.org (eastasia.home.utahime.org [192.168.174.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by maybe.home.utahime.org (Postfix) with ESMTPS id 3EA622DDEB for ; Tue, 9 Mar 2021 00:58:26 +0900 (JST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=utahime.org; s=maybe2019112701; t=1615219106; bh=MZ05giahQ8TaulwincpGLCOtW+Xx5bH45+mqeU1LaR4=; h=Date:To:Subject:From:In-Reply-To:References; b=dsOA5wAjn3CK2o5y9Vms4b+EDP/NqsBFYrucj+pAuRO+qNE+y1L6w2lURJO18utpX VdUNtiMATKfbsMH0rL6iv+vMvwXCxcRsej27KhO5TW+eQHd9QEqVTHVI222JvNH0jr 8XrqOfpmJDah4fYv0CNdcM9SAMnKG1DjpvvLuUHY66Xe3ReA3V2BPo0NXIoaWvV0jw hdJBPY4Ix+J/dHK7t10ECUJuK44qFkkUOrWo4SLi9P4aOCW/I0LdEtObkfDexu30Nm eHGU3pEURNlPeuOsC5Py30TUlP5YsjRQq/KAyapFPXQC76ecMmPOAgp3iUFyOf5WGn NOA+Inmagb/UA== Received: from localhost (rolling.home.utahime.org [192.168.174.11]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-384) server-digest SHA384) (No client certificate requested) by eastasia.home.utahime.org (Postfix) with ESMTPSA id E655231077; Tue, 9 Mar 2021 00:58:24 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.1 at eastasia.home.utahime.org Date: Tue, 09 Mar 2021 00:57:32 +0900 (JST) Message-Id: <20210309.005732.1808108188909983665.yasu@utahime.org> To: freebsd-current@freebsd.org Subject: Re: Waiting for bufdaemon From: Yasuhiro Kimura In-Reply-To: References: <20210306.185955.1096959917131550098.yasu@utahime.org> X-Mailer: Mew version 6.8 on Emacs 27.1 Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DvNL52pGcz4nqj X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=utahime.org header.s=maybe2019112701 header.b=dsOA5wAj; dmarc=none; spf=pass (mx1.freebsd.org: domain of yasu@utahime.org designates 183.180.29.210 as permitted sender) smtp.mailfrom=yasu@utahime.org X-Spamd-Result: default: False [-0.70 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+a:spf-authorized.utahime.org]; TO_DN_NONE(0.00)[]; HFILTER_HELO_IP_A(1.00)[maybe.home.utahime.org]; HFILTER_HELO_NORES_A_OR_MX(0.30)[maybe.home.utahime.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[utahime.org:+]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[183.180.29.210:from]; ASN(0.00)[asn:2519, ipnet:183.180.0.0/16, country:JP]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[utahime.org:s=maybe2019112701]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[utahime.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[183.180.29.210:from:127.0.2.255]; MID_CONTAINS_FROM(1.00)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2021 15:58:38 -0000 From: Konstantin Belousov Subject: Re: Waiting for bufdaemon Date: Mon, 8 Mar 2021 02:03:13 +0200 > On Sun, Mar 07, 2021 at 02:25:45PM -0600, Kyle Evans wrote: >> I'm going to be pedantic here, but note that this isn't a tunable. You >> cannot, AFAIK, influence the timecounter chosen with kenv; it has to >> be set post-boot via sysctl, and if you're really unlucky it could be >> that bufdaemon wedges before you manage to change it (though it seems >> unlikely). >> >> I've had some success with setting it to ACPI-fast in sysctl.conf here. > Right, I forgot that timecounter.hardware is not tunable, I thought that > it was fixed. > > https://reviews.freebsd.org/D29122 I applied the patch of D29122 to 705d06b289e of main, do `make kernel` and reboot. Then I tried the test that consists of following 6 steps. 1. Edit /boot/loader.conf.local 2. shutdow -r now 3. Login as root 4. cd /usr/src 5. make -j 4 -s buildworld 6. shutdown -r now Though I didn't intend it at first, I repeated this test 3 times changing how /boot/loader.conf.local was edited at step 1. And I got the result that was surprising and confusing, at least for me. At first trial I changed /boot/loader.conf.local so it includes 'kern.timecounter.hardware=i8254' in it. Then timeout of bufdaemon didn't happen at step 6. This means adding the line works as workaound for the problem. Next I replaced 'kern.timecounter.hardware=i8254' with 'kern.timecounter.hardware=ACPI-fast'and did test. As I wrote previous mail, 'ACPI-fast' is the initial value of kern.timecounter.hardware on the system in question. So I tried it for comparison purpose expecting the problem happens. But the result is unexpected one. Timeout of bufdaemon didn't happen at step 6 with this case either. So tried yet another case. I removed 'kern.timecounter.hardware=ACPI-fast' from /boot/loader.conf.local and did test again. In this case 'kern.timecounter.hardware' isn't touched at all. So the result should have been same as 2nd case. But it was different. Timeout of bufdeamon happend at step 6 in this case. The result surprised and confused me with following 2 points. (a) Setting initial value (2nd case) should be same as not touching at all (3rd case). But actually they caused different results. (b) 2nd case should have caused same result as one without patch. But actually it caused different one. I found the reason when I execute `sysctl kernel.timecount` and checked the output. If I do it with unpatched kernel, I get following result. ---------------------------------------------------------------------- yasu@rolling-vm-freebsd1[1005]% sysctl kern.timecounter kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 0 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(-100) dummy(-1000000) kern.timecounter.hardware: ACPI-fast kern.timecounter.alloweddeviation: 5 kern.timecounter.timehands_count: 2 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 1311282421 kern.timecounter.tc.ACPI-fast.mask: 4294967295 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 9931 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.TSC-low.quality: -100 kern.timecounter.tc.TSC-low.frequency: 1500035122 kern.timecounter.tc.TSC-low.counter: 4026452692 kern.timecounter.tc.TSC-low.mask: 4294967295 yasu@rolling-vm-freebsd1[1006]% ---------------------------------------------------------------------- But if I do it with patched kernel, I get a bit different one. ---------------------------------------------------------------------- yasu@rolling-vm-freebsd1[1001]% sysctl kern.timecounter kern.timecounter.tsc_shift: 1 kern.timecounter.smp_tsc_adjust: 0 kern.timecounter.smp_tsc: 1 kern.timecounter.invariant_tsc: 1 kern.timecounter.fast_gettime: 1 kern.timecounter.tick: 1 kern.timecounter.choice: ACPI-fast(900) i8254(0) TSC-low(1000) dummy(-1000000) kern.timecounter.hardware: TSC-low kern.timecounter.alloweddeviation: 5 kern.timecounter.timehands_count: 2 kern.timecounter.stepwarnings: 0 kern.timecounter.tc.ACPI-fast.quality: 900 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.counter: 401131388 kern.timecounter.tc.ACPI-fast.mask: 4294967295 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 9975 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.TSC-low.quality: 1000 kern.timecounter.tc.TSC-low.frequency: 1500035491 kern.timecounter.tc.TSC-low.counter: 581590125 kern.timecounter.tc.TSC-low.mask: 4294967295 yasu@rolling-vm-freebsd1[1002]% ---------------------------------------------------------------------- Comparing two output there are two notable differences. The first is the value of kern.timecounter.hardware. With unpatched kernel it is 'ACPI-fast'. On the other hand, with patched kernel it is 'TSC-low'. It means the initial value is changed by applying the patch. it explains why results of 2nd case and 3rd one are different. The second is the value of kern.timecounter.tc.ACPI-fast.counter. With unpatched kernel it is '1311282421' and with patch one it is '401131388'. I don't know what this value means. But I guess the difference is the reason that the result of 2nd case is different from the one of unpatched kernel. So now I know why unexpected result (a) and (b) happen. Furthermore, I comfirmed that setting kern.timecounter.hardware to either 'i8254' or 'ACPI-fast' works as workround of the problem. It is good news anyway. But still one question remains. Why have the value of kern.timecounter.hardware and kern.timecounter.tc.ACPI-fast.counter changed by applying the patch? My understanding is that it only makes 'kern.timecounter.hardware' loader tunable that can be configured with loader.conf. Is it unexpected side effect? Or is everything expected result? --- Yasuhiro Kimura