From nobody Wed Oct 25 01:19:41 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SFWLd0r4sz4xn0v for ; Wed, 25 Oct 2023 01:19:53 +0000 (UTC) (envelope-from erickleuschke@protonmail.com) Received: from mail-4318.protonmail.ch (mail-4318.protonmail.ch [185.70.43.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SFWLc2dgXz3Z2N for ; Wed, 25 Oct 2023 01:19:52 +0000 (UTC) (envelope-from erickleuschke@protonmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=protonmail.com header.s=protonmail3 header.b=XoS0EZb4; spf=pass (mx1.freebsd.org: domain of erickleuschke@protonmail.com designates 185.70.43.18 as permitted sender) smtp.mailfrom=erickleuschke@protonmail.com; dmarc=pass (policy=quarantine) header.from=protonmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1698196788; x=1698455988; bh=pvU9kX5fMQtpoL+BpKcZsnHPagJ8u1aOV2803FdzJsk=; h=Date:To:From:Subject:Message-ID:Feedback-ID:From:To:Cc:Date: Subject:Reply-To:Feedback-ID:Message-ID:BIMI-Selector; b=XoS0EZb4NAGaWxA/a2srAaFVjhQxiDg1iaEOr4qboPRQDT32XMzj5J2KYcjc13QXW ZaaJY68RUacN1/bVJwGYlCQGmdUNrcjZ4Xk+r0CI3uHIfcAh/ZW+qIOl1bYQqDoZ67 DjwPqjR6kPoBItimcDLaYjQUaSRE+8dMOPsNpnerDQ3xfY1nGTjH3k9gbuMK5xGvqF fIXgvvEACu/9SQUX9Vb2izwsngyQQcWW8LBjvJ0pJIaszhZCVlTSeNt3pdFV59OGGW cfcV7nS5+YYFeEZqOjQulVRe4iSzHEhZR4fCVtGW+4/4DhoMrhV6bO31BSgpz5smsZ gMVhuYFOk3BuA== Date: Wed, 25 Oct 2023 01:19:41 +0000 To: "freebsd-stable@freebsd.org" From: erickleuschke Subject: FreeBSD 14.0-RC1 and RC2 dvd1 packages Message-ID: Feedback-ID: 30907790:user:proton List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="b1_h08zAjyl3PYKyEQu72Mv8qdpr8UK1UbERIpGWNB8vA" X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.23 / 15.00]; MIME_BASE64_TEXT_BOGUS(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.82)[-0.823]; DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine]; RWL_MAILSPIKE_EXCELLENT(-0.40)[185.70.43.18:from]; R_SPF_ALLOW(-0.20)[+ip4:185.70.43.0/24]; R_DKIM_ALLOW(-0.20)[protonmail.com:s=protonmail3]; NEURAL_HAM_LONG(-0.11)[-0.110]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_BASE64_TEXT(0.10)[]; TO_DN_EQ_ADDR_ALL(0.00)[]; FREEMAIL_ENVFROM(0.00)[protonmail.com]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ZERO(0.00)[0]; ARC_NA(0.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH]; FREEMAIL_FROM(0.00)[protonmail.com]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[protonmail.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; HAS_PHPMAILER_SIG(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SFWLc2dgXz3Z2N This is a multi-part message in MIME format. --b1_h08zAjyl3PYKyEQu72Mv8qdpr8UK1UbERIpGWNB8vA Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 SGVsbG8sCgpJIGp1c3Qgbm90aWNlZCBzb21lIG9mIHRoZSBwcmVidWlsdCBwYWNrYWdlcyBsaWtl IEtERSB3ZXJlIHJlbW92ZWQgZnJvbSB0aGUgUkMyIGR2ZDEsIGFuZCB0aGVzZSBwYWNrYWdlcyB3 ZXJlIGF2YWlsYWJsZSBpbiBSQzEsIGlzIHRoZXJlIGFueW9uZSBrbm93IHRoZSByZWFzb24gb2Yg dGhpcyBjaGFuZ2U/CgpGcmVlQlNELTE0LjAtUkMxLWFtZDY0LWR2ZDEuaXNvIHNpemU6IDQ1Mzkw NTQwODAKRnJlZUJTRC0xNC4wLVJDMi1hbWQ2NC1kdmQxLmlzbyBzaXplOiAyNzQxNjY3ODQwCgpU aGFua3M= --b1_h08zAjyl3PYKyEQu72Mv8qdpr8UK1UbERIpGWNB8vA Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGRpdiBzdHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0 cHg7Ij5IZWxsbyw8L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTogQXJpYWwsIHNhbnMtc2Vy aWY7IGZvbnQtc2l6ZTogMTRweDsiPjxicj48L2Rpdj48ZGl2IHN0eWxlPSJmb250LWZhbWlseTog QXJpYWwsIHNhbnMtc2VyaWY7IGZvbnQtc2l6ZTogMTRweDsiPkkganVzdCBub3RpY2VkIHNvbWUg b2YgdGhlIHByZWJ1aWx0IHBhY2thZ2VzIGxpa2UgS0RFIHdlcmUgcmVtb3ZlZCBmcm9tIHRoZSBS QzIgZHZkMSwgYW5kIHRoZXNlIHBhY2thZ2VzIHdlcmUgYXZhaWxhYmxlIGluIFJDMSwgaXMgdGhl cmUgYW55b25lIGtub3cgdGhlIHJlYXNvbiBvZiB0aGlzIGNoYW5nZT88YnI+PC9kaXY+PGRpdiBz dHlsZT0iZm9udC1mYW1pbHk6IEFyaWFsLCBzYW5zLXNlcmlmOyBmb250LXNpemU6IDE0cHg7Ij48 YnI+PC9kaXY+DQo8ZGl2IGNsYXNzPSJwcm90b25tYWlsX3NpZ25hdHVyZV9ibG9jayIgc3R5bGU9 ImZvbnQtZmFtaWx5OiBBcmlhbCwgc2Fucy1zZXJpZjsgZm9udC1zaXplOiAxNHB4OyI+DQogICAg PGRpdiBjbGFzcz0icHJvdG9ubWFpbF9zaWduYXR1cmVfYmxvY2stdXNlciBwcm90b25tYWlsX3Np Z25hdHVyZV9ibG9jay1lbXB0eSI+DQogICAgICAgIA0KICAgICAgICAgICAgPC9kaXY+DQogICAg DQogICAgICAgICAgICA8ZGl2IGNsYXNzPSJwcm90b25tYWlsX3NpZ25hdHVyZV9ibG9jay1wcm90 b24iPjxzcGFuPkZyZWVCU0QtMTQuMC1SQzEtYW1kNjQtZHZkMS5pc28gc2l6ZTogNDUzOTA1NDA4 MDwvc3Bhbj48YnI+PHNwYW4+RnJlZUJTRC0xNC4wLVJDMi1hbWQ2NC1kdmQxLmlzbyBzaXplOiAy NzQxNjY3ODQwPC9zcGFuPjwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1haWxfc2lnbmF0dXJlX2Js b2NrLXByb3RvbiI+PGJyPjwvZGl2PjxkaXYgY2xhc3M9InByb3Rvbm1haWxfc2lnbmF0dXJlX2Js b2NrLXByb3RvbiI+VGhhbmtzPGJyPjwvZGl2Pg0KPC9kaXY+DQo= --b1_h08zAjyl3PYKyEQu72Mv8qdpr8UK1UbERIpGWNB8vA-- From nobody Fri Oct 27 02:56:27 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SGnPH0Rd6z4xcwj for ; Fri, 27 Oct 2023 02:56:35 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SGnPG6rgrz4Vkm; Fri, 27 Oct 2023 02:56:34 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698375395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=STdIPmo/ASBBkYHQzPRmakWF/uOMRUUNIODI6Uq1u4w=; b=TnU8dtNUBebx3BGIj9o4DG9HlTi6neYKGpFOiAsvgUcFYPJHQwRm6hWB+5AR34Kz0d8BBA 92h7X6viXhT7N9xo4cd05OqAxAQ/HkauErGpeWewgSvFEe2uW09ZUf0R9iM7wgeb5ASTNX QcJd3mrqcIubCcX2eufKhFkFpRev+36tXWN6cBBMz6d0ousVgN19RKif1b4oJctAQ3/+gX h0gNQtb0T9MHk4i/lbfPF7OxdWBeS7+5pB1NTrlzV78gf2wH2s+wKut+YT9rNQqCd84v+9 rNXJXZ8ZU4fbUBtvtOOwPCwTIpKjDNUFBI9WgJrQg/Gsx+FnukVYeMD5lvFtvg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698375395; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=STdIPmo/ASBBkYHQzPRmakWF/uOMRUUNIODI6Uq1u4w=; b=INKjTMVdtDAEWmXlB7dlX1iW9x83kHVrdekNWaS1grqwgiaW5HFzhUBidSCd93DSuSqptu Yx9wjyepKQJ6nv4Kk8n3r+3vNcEdvwVgekKtRT0dLtwVS2Kj0qouxrlf09hXVqmpCyvhVi 4WyLHxFyAEkQdoD58a8iACh/cTE3VDO5l9yOcVeF1jqvSL3sE+i0HS4r/GlbeIUUQl3tc6 wzD1z1AlgL6Btpl+HAtMX+DaoQIywCztHaNr9cjXzKNKjCDrMpa3/CyddhqM+zwhynOGPY 33+3NS3crHezZF+Er6w70J/gkKZLlFdG71WhVH45fOJvxmR3k44MO9GU6vF7kg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1698375395; a=rsa-sha256; cv=none; b=mMN9cqegX5SQ4av720TFCXNvxqdMJ738K+zH/zvMH14NBvrPOBoPk1zo17Qr1f4t3szqgh 032SpYH760saurKBhNrIqxw347qEWgXJeImQciVWI3nWUsDPkFB5EC7EhoUlFB6BMS1UmB U7pmGRy+n6gVHWo/ywQ2n3Mlw/hnLMutYi3LZrsbv9W5kThd/2Cd/ynL8mpSvQEarnvmdU Ld5PG7Gj5WdDMlV/mt0bOSNkb1XZzk1uSgOwtmbq8Z1A/QgJAiO7QZ9Cj9IoLjIJfC9mLw cSTlyOpgZ5oGCTLo/kiCz9M9WR6opifvF5rC3hjUBC2NwsSOJDo5X6uYyYWbYg== Received: from smtpclient.apple (ns1.oxydns.net [45.32.91.63]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SGnPF4CJ0z6TR; Fri, 27 Oct 2023 02:56:33 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: <7E920253-8CEF-4F2A-9C0A-9B4984059949@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_2C015787-5282-4D98-868D-7CBC564DACBB" List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.4\)) Subject: Re: FreeBSD 14.0-RC1 and RC2 dvd1 packages Date: Fri, 27 Oct 2023 10:56:27 +0800 In-Reply-To: Cc: "freebsd-stable@freebsd.org" , release FreeBSD Engineering Team To: erickleuschke References: X-Mailer: Apple Mail (2.3696.120.41.1.4) --Apple-Mail=_2C015787-5282-4D98-868D-7CBC564DACBB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii > On Oct 25, 2023, at 9:19 AM, erickleuschke = wrote: >=20 > Hello, >=20 > I just noticed some of the prebuilt packages like KDE were removed = from the RC2 dvd1, and these packages were available in RC1, is there = anyone know the reason of this change? >=20 > FreeBSD-14.0-RC1-amd64-dvd1.iso size: 4539054080 > FreeBSD-14.0-RC2-amd64-dvd1.iso size: 2741667840 Forwarded to re@. >=20 > Thanks --Apple-Mail=_2C015787-5282-4D98-868D-7CBC564DACBB Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii

On Oct 25, 2023, at 9:19 AM, erickleuschke <erickleuschke@protonmail.com> wrote:

Hello,

I = just noticed some of the prebuilt packages like KDE were removed from = the RC2 dvd1, and these packages were available in RC1, is there anyone = know the reason of this change?

=20
=20
FreeBSD-14.0-RC1-amd64-dvd1.iso size: 4539054080
FreeBSD-14.0-RC2-amd64-dvd1.iso size: = 2741667840

Forwarded to re@.


Thanks



= --Apple-Mail=_2C015787-5282-4D98-868D-7CBC564DACBB-- From nobody Fri Oct 27 03:02:17 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SGnWw5cMzz4xdtV for ; Fri, 27 Oct 2023 03:02:20 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SGnWw1GDdz4XNV; Fri, 27 Oct 2023 03:02:20 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (50.29.233.174.res-cmts.swb2.ptd.net [50.29.233.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id B46114E720; Fri, 27 Oct 2023 03:02:18 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us B46114E720 Content-Type: multipart/alternative; boundary=Apple-Mail-C2A4352D-77C6-4D42-85E1-3A914ECB7046 Content-Transfer-Encoding: 7bit List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 14.0-RC1 and RC2 dvd1 packages From: Glen Barber In-Reply-To: <7E920253-8CEF-4F2A-9C0A-9B4984059949@FreeBSD.org> Date: Thu, 26 Oct 2023 23:02:17 -0400 Cc: erickleuschke , freebsd-stable@freebsd.org, release FreeBSD Engineering Team Message-Id: <8CA8A7CB-CFE4-49E5-B892-6BBB6758CCBA@freebsd.org> References: <7E920253-8CEF-4F2A-9C0A-9B4984059949@FreeBSD.org> To: Zhenlei Huang X-Mailer: iPhone Mail (20G81) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:208.86.224.0/22, country:US] X-Rspamd-Queue-Id: 4SGnWw1GDdz4XNV --Apple-Mail-C2A4352D-77C6-4D42-85E1-3A914ECB7046 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Does the KDE set fail to build on quarterly= ?  If so, that would explain this.

Nothing else has c= hanged.

Glen
Sent from my phone.
Please excuse my brevity and/or typos.=

On Oct 26, 2023, a= t 10:56 PM, Zhenlei Huang <zlei@freebsd.org> wrote:

=EF=BB=BF
<= div>
On O= ct 25, 2023, at 9:19 AM, erickleuschke <erickleuschke@protonmail.com> wrote:
<= br class=3D"Apple-interchange-newline">
Hello,

I just noticed some of the prebuilt packages like KDE were removed f= rom the RC2 dvd1, and these packages were available in RC1, is there anyone k= now the reason of this change?

=20
=20
FreeBSD-14.0-RC1-amd64-dvd1.iso size: 4539054080
FreeBSD-14.0-RC2-amd64-dvd1.iso size: 2741667840

Forwarded to re@.

Thanks



= --Apple-Mail-C2A4352D-77C6-4D42-85E1-3A914ECB7046-- From nobody Fri Oct 27 10:15:25 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SGz7n2FCwz4y51m for ; Fri, 27 Oct 2023 10:15:33 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SGz7m3XzFz3gGm for ; Fri, 27 Oct 2023 10:15:32 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=aaMYLtPz; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=f5Wn+uV7; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.27 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 62C3F5C01B3 for ; Fri, 27 Oct 2023 06:15:30 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 27 Oct 2023 06:15:30 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :message-id:mime-version:reply-to:sender:subject:subject:to:to; s=fm3; t=1698401730; x=1698488130; bh=WZdjxCp3bh23wXzyYWJMOrvGO 73c2yml5QdgM9BXObQ=; b=aaMYLtPzyefWpHlzR8vUGSKPkcOW6h39mDLPq3Pe/ p/jbU3dmkEJJPsfHTp1Nw7LZRUz4iC34ddmB3/wgl14RvyHV/kw9CFMR+Vy8rDVD nA7y0pYkcgnGDxDXVXYiYG7VNwgp1/BX7FetdMYaTA1/ryu4YR81ztuBqoZfBlKv r5jAg6yRfRbZGKd3XAMAbp+4UdzLve56N/+TxUB8GX2L+bqpXor0gxIh6oYHDKii 0V/97k5LwyYw0A/+pQs6CgOEWvLRSbdF3giHBshAJCdX9QyelCkb12HHmbOXrzOu T4btzeK/Jtb0rS5U7qSetT2G7Ibvz/sjIbBKjim5oic+g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:message-id :mime-version:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1698401730; x=1698488130; bh=WZdjxCp3bh23wXzyYWJMOrvGO73c2yml5Qd gM9BXObQ=; b=f5Wn+uV7ZWzjJr0urgEmaoJpGSlVn+aQTVcZR+1ruuhI9HtNzQ5 py1SgP2Pdm0jWbSKLSySrOHjWBlH7T0NCJgqop33PJPqq1G6R1QLrNjGLSoat04V UkaiG9+Ek76Y/XUmJoFexVgijnERf5GewSKc4ZhGISBiIrzx3Vz4qmA4ZkSOuVpE jFQjR+Wp9TyrRlKRFL1WkbmrccqykkFnoatVGV378rppzGllrFK9Nsyj5gDSFdl6 jAL/q9J0i7F1W66dlXF1iXkKY9ZvibGX8Z8bV87bwVhHid2Wj08yGKq+o/+zAlNa dGh1NR+PS13cOjT2skmWe4g9Hscv4DZwrtQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggddvjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfggtggusehttdertddttd dvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrghtthgv rhhnpeevteeivdfffffhleeggeeugeevtdeihfetheetheehteeigfetudfgheelveekhf enucffohhmrghinheptghrrgiflhdrtggrthenucevlhhushhtvghrufhiiigvpedtnecu rfgrrhgrmhepmhgrihhlfhhrohhmpehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 06:15:29 -0400 (EDT) Date: Fri, 27 Oct 2023 11:15:25 +0100 From: void To: freebsd-stable@freebsd.org Subject: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline X-Spamd-Result: default: False [-4.12 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.52)[-0.517]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.27:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SGz7m3XzFz3gGm X-Spamd-Bar: ---- Hello list, I'm asking this in order to determine whether I've got something misconfigured or maybe my expectations need recalibrating, or maybe both. context is rpi4b with 8GB and usb3-connected 1tb zfs hard disk. /etc/daily.local has been moved out of the way to try to fix the issue. When periodic daily runs, it seems to take a long time. It's running now and has been running for overan hour so far. top shows PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 21649 root 1 20 0 18M 6668K zio->i 1 0:40 0.53% find There's plenty of resources 93 processes: 1 running, 92 sleeping CPU: 0.1% user, 0.0% nice, 0.5% system, 0.0% interrupt, 99.4% idle Mem: 202M Active, 688M Inact, 69M Laundry, 3079M Wired, 216K Buf, 3780M Free ARC: 975M Total, 552M MFU, 244M MRU, 808K Anon, 21M Header, 158M Other 229M Compressed, 905M Uncompressed, 3.95:1 Ratio Swap: 12G Total, 12G Free but the 'find' thing does heavy disk i/o and everything else reading from or writing to disk slows to a crawl. # cat /etc/periodic.conf daily_status_smart_devices="da0" daily_status_include_submit_mailq="NO" daily_clean_hoststat_enable="NO" daily_queuerun_enable="NO" daily_submit_queuerun="NO" daily_scrub_zfs_enable="YES" daily_scrub_zfs_pools="" daily_scrub_zfs_default_threshold="7" daily_status_ntpd_enable="YES" daily_clean_disks_enable="YES" zpool last scrubbed on the 21st; it's not happening right now there's the following in /etc/sysctl.conf relevant to the context: vfs.zfs.min_auto_ashift=12 # # filesystem vm.pageout_oom_seq=120 vm.pfault_oom_attempts=-1 vm.pageout_update_period=0 # vfs.zfs.resilver_min_time_ms=5000 vfs.zfs.arc.min=536870912 vfs.zfs.arc_max=1073741824 The OOM settings are there for poudriere. # zdb | grep ashift ashift: 12 # ps xx | grep periodic 68824 11 S+ 0:00.00 grep periodic 16115 14 I+ 0:00.00 /bin/sh - /usr/sbin/periodic daily 17425 14 I+ 0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily 17747 14 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily <=== why are there two of these? 19460 14 I+ 0:00.00 /bin/sh /usr/sbin/periodic LOCKED daily <=== 19921 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== also here 21312 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== I'm not sure if it's relevant, but the disk, when the zroot/zfs was initialised, was set to enforce 4k blocks, geli-encrypted swap, geli-encrypted fs. Because it's an rpi4, there's no aes-ni equivalent hardware. But the slowdown doesn't show much in other areas, only a couple, like this problem and when git updates src or ports. smartctl shows no remapped or pending blocks. What else can I do to debug the issue? -- From nobody Fri Oct 27 11:45:24 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH17c1vmZz4y93g for ; Fri, 27 Oct 2023 11:45:32 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH17b4LTLz4J0h for ; Fri, 27 Oct 2023 11:45:31 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 27 Oct 2023 13:45:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1698407124; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=oiCDRdZItvUqUOXwgxfyYalsx2t7TKAzuZYA7S6WGGM=; b=zvsvGfpa3aITFsIhDBvX8qznT0+BhMuYjVLxZgoDXBm2L6SggnWk7hVTM1YEtB2gHAVJhu 1aBkl87iAZ9DpkPDLE061ycLohnTmvQnYNcmfcuJSb8Q3Z0hn2hEDoexRGMMoxe2x3fr0k v9PdNGldF0Y5oze/SoHOgaRZbI6MOU9QdkAnV2XykmnfNHp2Ijmlgk8VNFUCkYMcwAOsco WJsE8YmDYk1QtEHWpUcfCA7QTc9Nrkbhi2MtQis4ECp9rtkGNSAP2QmawuzynkEc4xhyhP 24N3dpyMjVQD+H/Zw2pKu/wBQD9Zn77qcdj+mie2lyhkjNPl8haHONbe5KqFdg== From: Ronald Klop To: void Cc: freebsd-stable@freebsd.org Message-ID: <1122335317.4913.1698407124469@localhost> In-Reply-To: References: Subject: Re: periodic daily takes a very long time to run (14-stable) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4912_1359293768.1698407124396" X-Mailer: Realworks (677.7) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4SH17b4LTLz4J0h ------=_Part_4912_1359293768.1698407124396 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: void Datum: vrijdag, 27 oktober 2023 12:15 Aan: freebsd-stable@freebsd.org Onderwerp: periodic daily takes a very long time to run (14-stable) > > Hello list, > > I'm asking this in order to determine whether I've got something misconfigured or > maybe my expectations need recalibrating, or maybe both. > > context is rpi4b with 8GB and usb3-connected 1tb zfs hard disk. > /etc/daily.local has been moved out of the way to try to fix the issue. > > When periodic daily runs, it seems to take a long time. It's running now and has > been running for overan hour so far. > > top shows > PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 21649 root 1 20 0 18M 6668K zio->i 1 0:40 0.53% find > > There's plenty of resources > > 93 processes: 1 running, 92 sleeping > CPU: 0.1% user, 0.0% nice, 0.5% system, 0.0% interrupt, 99.4% idle > Mem: 202M Active, 688M Inact, 69M Laundry, 3079M Wired, 216K Buf, 3780M Free > ARC: 975M Total, 552M MFU, 244M MRU, 808K Anon, 21M Header, 158M Other > 229M Compressed, 905M Uncompressed, 3.95:1 Ratio > Swap: 12G Total, 12G Free > > but the 'find' thing does heavy disk i/o and everything else reading from or writing to disk slows to a crawl. > > # cat /etc/periodic.conf > daily_status_smart_devices="da0" > daily_status_include_submit_mailq="NO" > daily_clean_hoststat_enable="NO" > daily_queuerun_enable="NO" > daily_submit_queuerun="NO" > daily_scrub_zfs_enable="YES" > daily_scrub_zfs_pools="" > daily_scrub_zfs_default_threshold="7" > daily_status_ntpd_enable="YES" > daily_clean_disks_enable="YES" > > zpool last scrubbed on the 21st; it's not happening right now > > there's the following in /etc/sysctl.conf relevant to the context: > > vfs.zfs.min_auto_ashift=12 > # > # filesystem > vm.pageout_oom_seq=120 > vm.pfault_oom_attempts=-1 > vm.pageout_update_period=0 > # vfs.zfs.resilver_min_time_ms=5000 > vfs.zfs.arc.min=536870912 > vfs.zfs.arc_max=1073741824 > > The OOM settings are there for poudriere. > > # zdb | grep ashift > ashift: 12 > > # ps xx | grep periodic > 68824 11 S+ 0:00.00 grep periodic > 16115 14 I+ 0:00.00 /bin/sh - /usr/sbin/periodic daily > 17425 14 I+ 0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily > 17747 14 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily <=== why are there two of these? > 19460 14 I+ 0:00.00 /bin/sh /usr/sbin/periodic LOCKED daily <=== > 19921 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== also here > 21312 14 I+ 0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== > > I'm not sure if it's relevant, but the disk, when the zroot/zfs was initialised, was set > to enforce 4k blocks, geli-encrypted swap, geli-encrypted fs. Because it's an rpi4, there's > no aes-ni equivalent hardware. But the slowdown doesn't show much in other areas, only a couple, > like this problem and when git updates src or ports. smartctl shows no remapped or pending blocks. > > What else can I do to debug the issue? > -- > > > > Hi, Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? And how much bandwidth is uses. The output of "zpool status", "zpool list" and "zfs list" can also be interesting. ZFS is known to become slow when the zpool is full almost full. Regards, Ronald. ------=_Part_4912_1359293768.1698407124396 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: void <void@f-m.fm>
Datum: vrijdag, 27 oktober 2023 12:15
Aan: freebsd-stable@freebsd.org
Onderwerp: periodic daily takes a very long time to run (14-stable)

Hello list,

I'm asking this in order to determine whether I've got something misconfigured or
maybe my expectations need recalibrating, or maybe both.

context is rpi4b with 8GB and usb3-connected 1tb zfs hard disk.
/etc/daily.local has been moved out of the way to try to fix the issue.

When periodic daily runs, it seems to take a long time. It's running now and has
been running for overan hour so far.

top shows
   PID   USERNAME    THR  PRI   NICE SIZE RES   STATE    C   TIME    WCPU COMMAND
   21649 root          1  20    0    18M  6668K zio->i   1   0:40   0.53% find

There's plenty of resources

93 processes:  1 running, 92 sleeping
CPU:  0.1% user,  0.0% nice,  0.5% system,  0.0% interrupt, 99.4% idle
Mem: 202M Active, 688M Inact, 69M Laundry, 3079M Wired, 216K Buf, 3780M Free
ARC: 975M Total, 552M MFU, 244M MRU, 808K Anon, 21M Header, 158M Other
      229M Compressed, 905M Uncompressed, 3.95:1 Ratio
      Swap: 12G Total, 12G Free

but the 'find' thing does heavy disk i/o and everything else reading from or writing to disk slows to a crawl.

# cat /etc/periodic.conf
daily_status_smart_devices="da0"
daily_status_include_submit_mailq="NO"
daily_clean_hoststat_enable="NO"
daily_queuerun_enable="NO"
daily_submit_queuerun="NO"
daily_scrub_zfs_enable="YES"
daily_scrub_zfs_pools=""
daily_scrub_zfs_default_threshold="7"
daily_status_ntpd_enable="YES"
daily_clean_disks_enable="YES"

zpool last scrubbed on the 21st; it's not happening right now

there's the following in /etc/sysctl.conf relevant to the context:

vfs.zfs.min_auto_ashift=12
#
# filesystem
vm.pageout_oom_seq=120
vm.pfault_oom_attempts=-1
vm.pageout_update_period=0
# vfs.zfs.resilver_min_time_ms=5000
vfs.zfs.arc.min=536870912
vfs.zfs.arc_max=1073741824

The OOM settings are there for poudriere.

# zdb | grep ashift
             ashift: 12            

# ps xx | grep periodic
68824 11  S+      0:00.00 grep periodic
16115 14  I+      0:00.00 /bin/sh - /usr/sbin/periodic daily
17425 14  I+      0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily
17747 14  I+      0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily  <=== why are there two of these?
19460 14  I+      0:00.00 /bin/sh /usr/sbin/periodic LOCKED daily  <===
19921 14  I+      0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <=== also here
21312 14  I+      0:00.00 /bin/sh /etc/periodic/daily/100.clean-disks <===

I'm not sure if it's relevant, but the disk, when the zroot/zfs was initialised, was set
to enforce 4k blocks, geli-encrypted swap, geli-encrypted fs. Because it's an rpi4, there's
no aes-ni equivalent hardware. But the slowdown doesn't show much in other areas, only a couple,
like this problem and when git updates src or ports. smartctl shows no remapped or pending blocks.

What else can I do to debug the issue?
-- 
 


Hi,

Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? And how much bandwidth is uses.

The output of "zpool status", "zpool list" and "zfs list" can also be interesting.

ZFS is known to become slow when the zpool is full almost full.

Regards,
Ronald.
  ------=_Part_4912_1359293768.1698407124396-- From nobody Fri Oct 27 12:30:06 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH2786l5Gz4yCTZ for ; Fri, 27 Oct 2023 12:30:12 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH2781xt7z4Mrq for ; Fri, 27 Oct 2023 12:30:12 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=lUNoiA+C; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=N90FIq6n; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 65D4A5C025E for ; Fri, 27 Oct 2023 08:30:11 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Fri, 27 Oct 2023 08:30:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698409811; x=1698496211; bh=eE m82sXjzKb5lo2/uwgny8fkKd/hatQ3cu9BLPfDxac=; b=lUNoiA+CU1KJ7qZ6sp AsjtQMykUIfthJk/B8yqMd1u6bbewmWm8nX4VIxM+JO5evGMF764rQ5bbnGl+uqC zqz4TbhE5u/frhTOnfDlhphM9uUukXhZjF3BklFRKAAaHqoZ8NvRPY4zhFGYZw6d KLacjczHY251VhtaPFWvKQ3EmHzPLMCL5o4owKEDEmg77VT3v0IHii182kApqh0+ DEV8hMOIk3QhoYeWg5SzqTB6NndvoO808rKfvmH9xFGn8SBCNHTJxdwspD6Kxbn9 +2Hw9FvE+/kGBbhgNmFUBEWHqqXScP3s2U7cQy7+bX9dhQFuFMBPuCAstWV4CF93 dIZw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698409811; x=1698496211; bh=eEm82sXjzKb5l o2/uwgny8fkKd/hatQ3cu9BLPfDxac=; b=N90FIq6nkFAP+NfrhJEwbffS5Yk/C 7OcQzdG2mKsjCAIfZFw5hx/o4lI6EXjhljnV6YRT6ZAdbvJ2dwF72b+bbYBwUE7x HtO6jzhfzN50OyQzHJKLKjLMRpBg6PdjI/AX1n3sLH41Gsf8xNHvTlqv6PHJl5gN Ds5yVGe1v7nTxqnWj2KxilvAag5LN/Fuc8ZmxfsF7ixIsyUL7CE0CGLr8XxHvs4U Sa4DYccgtc2e27e8ogKAQyvLgWaPQjRWjg0vsVdNNsZSUJ38LqBGX3fwh0Pi0IR9 XKHEJAIsZiSW5IdN7MLoRwDZx491KZbVXBwx/0p5X8Wi6aNA48ZbOBoWQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdehfecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeekleduvdelhfeileefgffghfffkedtheellefgudfgvdegkeejjedutdehhe fgueenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehv ohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 08:30:10 -0400 (EDT) Date: Fri, 27 Oct 2023 13:30:06 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <1122335317.4913.1698407124469@localhost> X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-1.00)[-0.997]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.25:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH2781xt7z4Mrq X-Spamd-Bar: ---- Hi, On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote: >Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses. > >The output of "zpool status", "zpool list" and "zfs list" can also >be interesting. > >ZFS is known to become slow when the zpool is full almost full. OK. It's just finished the periodic daily I wrote about initially # date && periodic daily && date Fri Oct 27 10:12:23 BST 2023 Fri Oct 27 13:12:09 BST 2023 so almost exactly 3 hrs. Regarding gstat/iostat - do you mean when periodic is running, not running, both? Regarding space used: NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD zroot 790G 93.6G 0B 96K 0B 93.6G zroot/ROOT 790G 64.1G 0B 96K 0B 64.1G zpool status -v # zpool status -v pool: zroot state: ONLINE scan: scrub repaired 0B in 03:50:52 with 0 errors on Sat Oct 21 20:53:27 2023 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 da0p3.eli ONLINE 0 0 0 errors: No known data errors # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 912G 93.6G 818G - - 21% 10% 1.00x ONLINE - # zfs list NAME USED AVAIL REFER MOUNTPOINT zroot 93.6G 790G 96K /zroot zroot/ROOT 64.1G 790G 96K none zroot/ROOT/13.1-RELEASE-p4_2022-12-01_063800 8K 790G 11.3G / zroot/ROOT/13.1-RELEASE-p5_2023-02-03_232552 8K 790G 27.4G / zroot/ROOT/13.1-RELEASE-p5_2023-02-09_153529 8K 790G 27.9G / zroot/ROOT/13.1-RELEASE-p6_2023-02-18_024922 8K 790G 33.4G / zroot/ROOT/13.1-RELEASE_2022-11-17_165717 8K 790G 791M / zroot/ROOT/default 64.1G 790G 14.4G / zroot/distfiles 2.96G 790G 2.96G /usr/ports/distfiles zroot/postgres 96K 790G 96K /var/db/postgres zroot/poudriere 4.17G 790G 104K /zroot/poudriere zroot/poudriere/jails 3.30G 790G 96K /zroot/poudriere/jails zroot/poudriere/jails/140R-rpi2b 1.03G 790G 1.03G /usr/local/poudriere/jails/140R-rpi2b zroot/poudriere/jails/localhost 1.13G 790G 1.13G /usr/local/poudriere/jails/localhost zroot/poudriere/jails/testvm 1.14G 790G 1.13G /usr/local/poudriere/jails/testvm zroot/poudriere/ports 891M 790G 96K /zroot/poudriere/ports zroot/poudriere/ports/testing 891M 790G 891M /usr/local/poudriere/ports/testing zroot/usr 22.1G 790G 96K /usr zroot/usr/home 13.5G 790G 13.5G /usr/home zroot/usr/home/tmp 144K 790G 144K /usr/home/void/tmp zroot/usr/obj 3.83G 790G 3.83G /usr/obj zroot/usr/ports 2.30G 790G 2.30G /usr/ports zroot/usr/src 2.41G 790G 2.41G /usr/src zroot/var 28.9M 790G 96K /var zroot/var/audit 96K 790G 96K /var/audit zroot/var/crash 96K 790G 96K /var/crash zroot/var/log 27.8M 790G 27.8M /var/log zroot/var/mail 688K 790G 688K /var/mail zroot/var/tmp 112K 790G 112K /var/tmp thank you for looking at my query. -- From nobody Fri Oct 27 12:46:39 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH2VH5M8gz4yDB1 for ; Fri, 27 Oct 2023 12:46:47 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH2VH0pdWz4Ppn for ; Fri, 27 Oct 2023 12:46:46 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Received: from rwvirtual373.colo.realworks.nl (rwvirtual373.colo.realworks.nl [10.0.10.73]) by mailrelayint1.colo2.realworks.nl (Postfix) with ESMTP id 4SH2V73M8Rz3xVQ; Fri, 27 Oct 2023 14:46:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1698410799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=tnRu3x34/EXs2WxjGf3S1jtoBr5uPVMH2+d5SkWdr4s=; b=RQvJYwBUa77D7nFtvslhBpw5bH3EfKuPCP1qPmrONN/RxOQIZolyY+1f45Ulqe9bHyDBfJ PwyI3D2hfVh0BqCYb37kg/k+Kx+YxkoxcDKGOWG48NRkyQruq67WSLtChRG+T5Ne2qoYJy KO2McsEotkkZnH3u+ENpCP5+hny1OyOZqng6IdvwmlMCBKhnOzr+Hwm52XN1N7OgxlvMU9 KmnalarW+BFK2aU/Q3dFH+9M/tJlE47CEm6YMg9up4+xQdm8mvzEy7g86ZjOuSP2CZndCW rjpCQNKnv6UB7HjGAb5ds4lMdF7BDVkwFevrTeLMCQulhXV1N0s3c9ufkBde5Q== Received: from rwvirtual373.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual373.colo.realworks.nl (Postfix) with ESMTP id 5322D140E12; Fri, 27 Oct 2023 14:46:39 +0200 (CEST) Date: Fri, 27 Oct 2023 14:46:39 +0200 (CEST) From: Ronald Klop To: void Cc: freebsd-stable@freebsd.org Message-ID: <2146377145.5323.1698410799235@localhost> In-Reply-To: References: <1122335317.4913.1698407124469@localhost> Subject: Re: periodic daily takes a very long time to run (14-stable) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5322_450711279.1698410799210" X-Mailer: Realworks (677.7) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual373 [10.0.10.73] with HTTP; Fri, 27 Oct 2023 14:46:39 +0200 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL] X-Rspamd-Queue-Id: 4SH2VH0pdWz4Ppn ------=_Part_5322_450711279.1698410799210 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: void Datum: vrijdag, 27 oktober 2023 14:30 Aan: freebsd-stable@freebsd.org Onderwerp: Re: periodic daily takes a very long time to run (14-stable) > > Hi, > > On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote: > > >Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses. > > > >The output of "zpool status", "zpool list" and "zfs list" can also >be interesting. > > > >ZFS is known to become slow when the zpool is full almost full. > > OK. It's just finished the periodic daily I wrote about initially > > # date && periodic daily && date > Fri Oct 27 10:12:23 BST 2023 > Fri Oct 27 13:12:09 BST 2023 > > so almost exactly 3 hrs. > > Regarding gstat/iostat - do you mean when periodic is running, not running, > both? > > Regarding space used: > > NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD > zroot 790G 93.6G 0B 96K 0B 93.6G > zroot/ROOT 790G 64.1G 0B 96K 0B 64.1G > > zpool status -v > > # zpool status -v > pool: zroot > state: ONLINE > scan: scrub repaired 0B in 03:50:52 with 0 errors on Sat Oct 21 20:53:27 2023 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > da0p3.eli ONLINE 0 0 0 > > errors: No known data errors > > # zpool list > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 912G 93.6G 818G - - 21% 10% 1.00x ONLINE - > > # zfs list > NAME USED AVAIL REFER MOUNTPOINT > zroot 93.6G 790G 96K /zroot > zroot/ROOT 64.1G 790G 96K none > zroot/ROOT/13.1-RELEASE-p4_2022-12-01_063800 8K 790G 11.3G / > zroot/ROOT/13.1-RELEASE-p5_2023-02-03_232552 8K 790G 27.4G / > zroot/ROOT/13.1-RELEASE-p5_2023-02-09_153529 8K 790G 27.9G / > zroot/ROOT/13.1-RELEASE-p6_2023-02-18_024922 8K 790G 33.4G / > zroot/ROOT/13.1-RELEASE_2022-11-17_165717 8K 790G 791M / > zroot/ROOT/default 64.1G 790G 14.4G / > zroot/distfiles 2.96G 790G 2.96G /usr/ports/distfiles > zroot/postgres 96K 790G 96K /var/db/postgres > zroot/poudriere 4.17G 790G 104K /zroot/poudriere > zroot/poudriere/jails 3.30G 790G 96K /zroot/poudriere/jails > zroot/poudriere/jails/140R-rpi2b 1.03G 790G 1.03G /usr/local/poudriere/jails/140R-rpi2b > zroot/poudriere/jails/localhost 1.13G 790G 1.13G /usr/local/poudriere/jails/localhost > zroot/poudriere/jails/testvm 1.14G 790G 1.13G /usr/local/poudriere/jails/testvm > zroot/poudriere/ports 891M 790G 96K /zroot/poudriere/ports > zroot/poudriere/ports/testing 891M 790G 891M /usr/local/poudriere/ports/testing > zroot/usr 22.1G 790G 96K /usr > zroot/usr/home 13.5G 790G 13.5G /usr/home > zroot/usr/home/tmp 144K 790G 144K /usr/home/void/tmp > zroot/usr/obj 3.83G 790G 3.83G /usr/obj > zroot/usr/ports 2.30G 790G 2.30G /usr/ports > zroot/usr/src 2.41G 790G 2.41G /usr/src > zroot/var 28.9M 790G 96K /var > zroot/var/audit 96K 790G 96K /var/audit > zroot/var/crash 96K 790G 96K /var/crash > zroot/var/log 27.8M 790G 27.8M /var/log > zroot/var/mail 688K 790G 688K /var/mail > zroot/var/tmp 112K 790G 112K /var/tmp > > thank you for looking at my query. > > -- > > > > Mmm. Your pool has a lot of space left. So that is good. About gstat / iostat, yes during the daily scan would be nice. The numbers outside of the daily scan can also help as a reference. NB: There were talks on the ML about vnode re-use problems. But I think that was under a much higher load on the FS. Like 20 find processes in parallel on millions of files. Like this: https://cgit.freebsd.org/src/commit/?id=054f45e026d898bdc8f974d33dd748937dee1d6b and https://cgit.freebsd.org/src/log/?qt=grep&q=vnode&showmsg=1 These improvements also ended up in 14. Regards, Ronald. ------=_Part_5322_450711279.1698410799210 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: void <void@f-m.fm>
Datum: vrijdag, 27 oktober 2023 14:30
Aan: freebsd-stable@freebsd.org
Onderwerp: Re: periodic daily takes a very long time to run (14-stable)

Hi,

On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote:

>Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses.
>
>The output of "zpool status", "zpool list" and "zfs list" can also >be interesting.
>
>ZFS is known to become slow when the zpool is full almost full.

OK. It's just finished the periodic daily I wrote about initially

# date && periodic daily && date
Fri Oct 27 10:12:23 BST 2023
Fri Oct 27 13:12:09 BST 2023

so almost exactly 3 hrs.

Regarding gstat/iostat - do you mean when periodic is running, not running,
both?

Regarding space used:

NAME                                          AVAIL   USED  USEDSNAP  USEDDS  USEDREFRESERV  USEDCHILD
zroot                                          790G  93.6G        0B     96K             0B      93.6G
zroot/ROOT                                     790G  64.1G        0B     96K             0B      64.1G

zpool status -v

# zpool status -v
   pool: zroot
state: ONLINE
   scan: scrub repaired 0B in 03:50:52 with 0 errors on Sat Oct 21 20:53:27 2023
config:

       NAME         STATE     READ WRITE CKSUM
       zroot        ONLINE       0     0     0
         da0p3.eli  ONLINE       0     0     0

errors: No known data errors

# zpool list
NAME    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
zroot   912G  93.6G   818G        -         -    21%    10%  1.00x    ONLINE  -

# zfs list
NAME                                           USED  AVAIL  REFER  MOUNTPOINT
zroot                                         93.6G   790G    96K  /zroot
zroot/ROOT                                    64.1G   790G    96K  none
zroot/ROOT/13.1-RELEASE-p4_2022-12-01_063800     8K   790G  11.3G  /
zroot/ROOT/13.1-RELEASE-p5_2023-02-03_232552     8K   790G  27.4G  /
zroot/ROOT/13.1-RELEASE-p5_2023-02-09_153529     8K   790G  27.9G  /
zroot/ROOT/13.1-RELEASE-p6_2023-02-18_024922     8K   790G  33.4G  /
zroot/ROOT/13.1-RELEASE_2022-11-17_165717        8K   790G   791M  /
zroot/ROOT/default                            64.1G   790G  14.4G  /
zroot/distfiles                               2.96G   790G  2.96G  /usr/ports/distfiles
zroot/postgres                                  96K   790G    96K  /var/db/postgres
zroot/poudriere                               4.17G   790G   104K  /zroot/poudriere
zroot/poudriere/jails                         3.30G   790G    96K  /zroot/poudriere/jails
zroot/poudriere/jails/140R-rpi2b              1.03G   790G  1.03G  /usr/local/poudriere/jails/140R-rpi2b
zroot/poudriere/jails/localhost               1.13G   790G  1.13G  /usr/local/poudriere/jails/localhost
zroot/poudriere/jails/testvm                  1.14G   790G  1.13G  /usr/local/poudriere/jails/testvm
zroot/poudriere/ports                          891M   790G    96K  /zroot/poudriere/ports
zroot/poudriere/ports/testing                  891M   790G   891M  /usr/local/poudriere/ports/testing
zroot/usr                                     22.1G   790G    96K  /usr
zroot/usr/home                                13.5G   790G  13.5G  /usr/home
zroot/usr/home/tmp                             144K   790G   144K  /usr/home/void/tmp
zroot/usr/obj                                 3.83G   790G  3.83G  /usr/obj
zroot/usr/ports                               2.30G   790G  2.30G  /usr/ports
zroot/usr/src                                 2.41G   790G  2.41G  /usr/src
zroot/var                                     28.9M   790G    96K  /var
zroot/var/audit                                 96K   790G    96K  /var/audit
zroot/var/crash                                 96K   790G    96K  /var/crash
zroot/var/log                                 27.8M   790G  27.8M  /var/log
zroot/var/mail                                 688K   790G   688K  /var/mail
zroot/var/tmp                                  112K   790G   112K  /var/tmp

thank you for looking at my query.

-- 
 


Mmm. Your pool has a lot of space left. So that is good.

About gstat / iostat, yes during the daily scan would be nice. The numbers outside of the daily scan can also help as a reference.

NB: There were talks on the ML about vnode re-use problems. But I think that was under a much higher load on the FS. Like 20 find processes in parallel on millions of files. Like this: https://cgit.freebsd.org/src/commit/?id=054f45e026d898bdc8f974d33dd748937dee1d6b and https://cgit.freebsd.org/src/log/?qt=grep&q=vnode&showmsg=1
These improvements also ended up in 14.

Regards,
Ronald.
  ------=_Part_5322_450711279.1698410799210-- From nobody Fri Oct 27 12:51:40 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH2c13d3dz4yDJ4 for ; Fri, 27 Oct 2023 12:51:45 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH2c04qppz4Qjr for ; Fri, 27 Oct 2023 12:51:44 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=pb6W99U+; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=elaoC6nB; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 623475C0170 for ; Fri, 27 Oct 2023 08:51:44 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 27 Oct 2023 08:51:44 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698411104; x=1698497504; bh=AI /C3T1iFK9kcimyORRat3s5/0loieiil1Fu9JFKrn4=; b=pb6W99U+g2bfNID9v3 Gfog/ZoNnbsIYGV67SQklQhYRuXcLmrxuIpDI3afMXPFdRLW7DRG+DRCjx7KoLOM th3Ufy7jEiTmQwLBQM7vn0rpfvogo7ZVnpbUzw8qqxfTiLwxifDFujAAienMUt66 nPUoAJ4flwEW+XHhf0Ox4VSKtg22Ad0E3HwydorMSykgIJoPnhB97Pe/Sag8y5S7 wT0Jj1CYI4sjWcFdxsxVaUzEnDrfxDJWLbnE3t+D8IcFIJWbcbh82ddQlRzrwB1j FOO+CA7boQyA6s1bIS13//DD4+JPQpJYmeXgODcngeo1e1reIc4IXZoKMdjrCPB+ pzkA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698411104; x=1698497504; bh=AI/C3T1iFK9kc imyORRat3s5/0loieiil1Fu9JFKrn4=; b=elaoC6nBjRN6ZYVWiomXj7sDAF00R T3rVKyGKT9jpq916XAEMs9jbcWgcto+y9U75DIqZ+Yoq7U52znB+H5U+VpzBCkXS dIGiE1AGYK09meDerY0MMHaveZC9DroNSJQTcJxHu+SPU1goAwEI+zi5Jx6gQP/Y gCgeVgrTj4GYO/v/2HuwU3XhrvZzIVRxgWkUt/3OI/mG6a9ztnZ1j5qYfu6EeD4H +g0CuELRZoPnvrYQOpJZsoof9Z1DkHj/T20aSV9/nS0BTJgEI4P559XVdPddEQlo 1hDht4LjhuELCaejWmPiTKodwpw8zufl9ZZvVYslHy6I7h8oLIEnoCNDA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdehjecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeekleduvdelhfeileefgffghfffkedtheellefgudfgvdegkeejjedutdehhe fgueenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehv ohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 08:51:43 -0400 (EDT) Date: Fri, 27 Oct 2023 13:51:40 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <1122335317.4913.1698407124469@localhost> X-Spamd-Result: default: False [-4.69 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; NEURAL_HAM_SHORT(-1.00)[-0.997]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25:c]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.25:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH2c04qppz4Qjr X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote: >Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses. when periodic is *not* running, gstat numbers fluctuate between whats indicated below and all zeros: dT: 1.011s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 69 0 0 0.0 67 732 0.6 24.3| da0 0 0 0 0 0.0 0 0 0.0 0.0| da0p1 0 0 0 0 0.0 0 0 0.0 0.0| da0p2 0 69 0 0 0.0 67 732 0.6 24.3| da0p3 0 0 0 0 0.0 0 0 0.0 0.0| da0p2.eli 0 69 0 0 0.0 67 732 0.8 24.4| da0p3.eli # iostat -x -d 1 extended device statistics device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 15 18 189.4 565.1 47 7 144 27 0 23 pass0 0 0 0.0 0.0 54 0 154 79 0 0 extended device statistics device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 3 70 172.6 698.0 26 1 129 5 0 33 pass0 0 0 0.0 0.0 0 0 0 0 0 0 extended device statistics device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 0 0 0.0 0.0 0 0 0 0 0 0 pass0 0 0 0.0 0.0 0 0 0 0 0 0 (then all 0s repeatedly) When periodic *is* running: dT: 1.047s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 2 89 34 138 88.1 54 783 33.9 95.0| da0 0 0 0 0 0.0 0 0 0.0 0.0| da0p1 0 0 0 0 0.0 0 0 0.0 0.0| da0p2 2 89 34 138 88.1 54 783 34.0 95.0| da0p3 0 0 0 0 0.0 0 0 0.0 0.0| da0p2.eli 2 90 35 141 86.3 54 783 34.2 96.9| da0p3.eli the kBps number varies wildly between 100 and 1200. %busy is 92-102% # iostat -x -d 1 extended device statistics device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 15 18 189.6 564.1 47 7 144 27 10 23 pass0 0 0 0.0 0.0 54 0 154 79 0 0 Numbers vary wildly here, too. I wonder if theres a switch for iostat that lets it run for say 1000 seconds then produces an average of the values gathered. I'll look for one -- From nobody Fri Oct 27 13:09:45 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH30s5Hl4z4yF8v for ; Fri, 27 Oct 2023 13:09:49 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH30r4kCvz4S9P for ; Fri, 27 Oct 2023 13:09:48 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b="n/iZ92VB"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=B+Nnw6r9; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.25 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 458795C029B for ; Fri, 27 Oct 2023 09:09:48 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 27 Oct 2023 09:09:48 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698412188; x=1698498588; bh=y9 WYeNA+srVLGRru5R5XN1hrambhPT/FHQWQusCl1EM=; b=n/iZ92VB/15mFTTcxF uEYLc1NtTU2LxhFWez5PIyCTcW32X75mTkB8OVDRSR7RTAj5kPyIHfUqF7thpJFC uDcHLIrUqBaAXw5tKQuyY0gU5BUNyvCpM+KaPds3arQ9CcRotyyavCauYzGPTHM/ I+Ic85H/6z7w+yeRxmqxupiZaVqKDT+5Q3ZuWjhYRBZlC7iw9+M2Dk4sorpHj0B1 hu1DCzbh68gw0DDS8XPnVr70DB/80yTvbZYPT/oGr77NwY8ezB5vct0F3vI4edfK eH5MucBkcE7GfibH8EtYEx96exRkXWTKB/h9nuxhACetfukiImKGjPPJnIvzl1wG P0uA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698412188; x=1698498588; bh=y9WYeNA+srVLG Rru5R5XN1hrambhPT/FHQWQusCl1EM=; b=B+Nnw6r9GXKk2/lJEXdS0IkCOts/b mGW0cGugeHI44Avg+i3IjBB5qrEq3xxNpdItpuDKKm5ojQzDyojLL8t7nGmhrscB U/j/KqCGQVzq3lFRogmMSVzF9VLzKeO6Esxyn5YQJgCQSEUjKvtjhqUtK3ehdzp0 0uOrUD/hYyqVBMkQVNMUpeh47jJJltsT2/y+FM7bH4lTgIehlLPzBNd6Xh6yaX+d e8qBEJ6D9GSsI4GIxLw1qVWPdXhNk8R8DwBvCR1+DUH5Fyp+L6Icc/wH3+3aJJef ANIJHvpFU+CdlyCjR3RzesuAv6lv3ToQlZ/KgVSI1fo2L5xaruu1nU3FQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdeivdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpefhueffheeuhedvjeeuffegffdvueejuddvudfgueeutdeitdetleekvedufe eludenucffohhmrghinhepfhhrvggvsghsugdrohhrghdpvhhnlhhruhdrughirhgvtght pdhsthgrthhsrdhfrhgvvgenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmh grihhlfhhrohhmpehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 09:09:47 -0400 (EDT) Date: Fri, 27 Oct 2023 14:09:45 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <2146377145.5323.1698410799235@localhost> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <2146377145.5323.1698410799235@localhost> X-Spamd-Result: default: False [-4.67 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.966]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.25:c]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.25:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.25:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH30r4kCvz4S9P X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 02:46:39PM +0200, Ronald Klop wrote: >Mmm. Your pool has a lot of space left. So that is good. > >About gstat / iostat, yes during the daily scan would be nice. The numbers outside of the daily scan can also help as a reference. > >NB: There were talks on the ML about vnode re-use problems. But I think that was under a much higher load on the FS. Like 20 find processes in parallel on millions of files. Like this: https://cgit.freebsd.org/src/commit/?id=054f45e026d898bdc8f974d33dd748937dee1d6b and https://cgit.freebsd.org/src/log/?qt=grep&q=vnode&showmsg=1 >These improvements also ended up in 14. OK. This is on stable/14-n265566-4533fa42ad91 arm64 built on Oct 21st and I've verified vfs_subr.o was built on that date. Nothing jumps out at me from sysctl output, but I hardly know a thing about this in particular. What do you think? # sysctl -a | grep vnode kern.maxvnodes: 212370 kern.ipc.umtx_vnode_persistent: 0 vm.vnode_pbufs: 256 vm.stats.vm.v_vnodepgsout: 1825669 vm.stats.vm.v_vnodepgsin: 8165147 vm.stats.vm.v_vnodeout: 59017 vm.stats.vm.v_vnodein: 795355 vfs.wantfreevnodes: 53092 vfs.freevnodes: 95043 vfs.vnodes_created: 40292208 vfs.numvnodes: 102660 vfs.vnode.vnlru.uma_reclaim_calls: 0 vfs.vnode.vnlru.kicks: 0 vfs.vnode.vnlru.max_free_per_call: 10000 vfs.vnode.vnlru.failed_runs: 0 vfs.vnode.vnlru.direct_recycles_free: 12193254 vfs.vnode.vnlru.recycles_free: 13211571 vfs.vnode.vnlru.recycles: 0 vfs.vnode.stats.alloc_sleeps: 0 vfs.vnode.stats.free: 95043 vfs.vnode.stats.skipped_requeues: 1769431 vfs.vnode.stats.created: 40292208 vfs.vnode.stats.count: 102660 vfs.vnode.param.wantfree: 53092 vfs.vnode.param.limit: 212370 vfs.cache.debug.vnodes_cel_3_failures: 0 vfs.cache.stats.heldvnodes: 6383 debug.vnode_domainset: debug.sizeof.vnode: 448 debug.fail_point.status_fill_kinfo_vnode__random_path: off debug.fail_point.fill_kinfo_vnode__random_path: off I let iostat run a while and the following seems representative: device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 100 1 401.4 4.0 70 126 0 70 10 97 pass0 0 0 0.0 0.0 0 0 0 0 0 0 kw/s gets to 700, kr/s 500. Very bursty -- From nobody Fri Oct 27 13:34:35 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH3YT2dLHz4yGNq for ; Fri, 27 Oct 2023 13:34:37 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH3YT1Mj8z4VdS for ; Fri, 27 Oct 2023 13:34:36 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Received: from rwvirtual375.colo.realworks.nl (rwvirtual375.colo.realworks.nl [10.0.10.75]) by mailrelayint2.colo2.realworks.nl (Postfix) with ESMTP id 4SH3YR3qWQz2xYC; Fri, 27 Oct 2023 15:34:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1698413675; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vK2j89Q4p2nTgw/ePekjVkA/4FaOIH2Er1QfcBrBJAo=; b=Remd/T8NqNVWnJzePyAshUxlDmDyM6gXofFqajM05jbEFqLH/l1HgNDr/ttioO9a/flFuQ ozOnKrs8WbG63h9a9UzOgNTLbGXhqUgDDyG/CkjmJ+tNTBOp9yTWj+KzJpk4cxUI8gRguw Ld1tta8Z8jDNuUWH9BT7SO43PynYyqeoPEGxrFMsM72gH0zjsq4SjH4bvA645KM17IT30h j8k4P3bRFjQD2+6Bk+X5Q9rXjF5HaxlE6xVkm5yny84jJOzr50ieaBxPnaIFg99M7vYM05 tXOAVoppLgWPEY5OT5ZGf2xm0umf42MWN7+8MvUhcp7aE8ptikJq3IsgdW0JGQ== Received: from rwvirtual375.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual375.colo.realworks.nl (Postfix) with ESMTP id 749FC406C9; Fri, 27 Oct 2023 15:34:35 +0200 (CEST) Date: Fri, 27 Oct 2023 15:34:35 +0200 (CEST) From: Ronald Klop To: void Cc: freebsd-stable@freebsd.org Message-ID: <794932758.6659.1698413675475@localhost> In-Reply-To: References: <1122335317.4913.1698407124469@localhost> Subject: Re: periodic daily takes a very long time to run (14-stable) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_6658_1428730899.1698413675449" X-Mailer: Realworks (677.7) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual375 [10.0.10.75] with HTTP; Fri, 27 Oct 2023 15:34:35 +0200 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL] X-Rspamd-Queue-Id: 4SH3YT1Mj8z4VdS ------=_Part_6658_1428730899.1698413675449 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: void Datum: vrijdag, 27 oktober 2023 14:51 Aan: freebsd-stable@freebsd.org Onderwerp: Re: periodic daily takes a very long time to run (14-stable) > > On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote: > > >Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses. > > when periodic is *not* running, gstat numbers fluctuate between whats indicated below and all zeros: > > dT: 1.011s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 0 69 0 0 0.0 67 732 0.6 24.3| da0 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p1 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p2 > 0 69 0 0 0.0 67 732 0.6 24.3| da0p3 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p2.eli > 0 69 0 0 0.0 67 732 0.8 24.4| da0p3.eli > > # iostat -x -d 1 > extended device statistics > device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b > da0 15 18 189.4 565.1 47 7 144 27 0 23 pass0 0 0 0.0 0.0 54 0 154 79 0 0 extended device statistics > device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b > da0 3 70 172.6 698.0 26 1 129 5 0 33 pass0 0 0 0.0 0.0 0 0 0 0 0 0 extended device statistics > device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b > da0 0 0 0.0 0.0 0 0 0 0 0 0 pass0 0 0 0.0 0.0 0 0 0 0 0 0 > (then all 0s repeatedly) > > When periodic *is* running: > > dT: 1.047s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name > 2 89 34 138 88.1 54 783 33.9 95.0| da0 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p1 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p2 > 2 89 34 138 88.1 54 783 34.0 95.0| da0p3 > 0 0 0 0 0.0 0 0 0.0 0.0| da0p2.eli > 2 90 35 141 86.3 54 783 34.2 96.9| da0p3.eli > > the kBps number varies wildly between 100 and 1200. %busy is 92-102% > > # iostat -x -d 1 > extended device statistics > device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b > da0 15 18 189.6 564.1 47 7 144 27 10 23 pass0 0 0 0.0 0.0 54 0 154 79 0 0 > Numbers vary wildly here, too. I wonder if theres a switch for iostat that lets it > run for say 1000 seconds then produces an average of the values gathered. > > I'll look for one > -- > > > > What stands out to me is that you do quite a lot of writes on the disk. (I might be mistaken.) The max. number of IOPS for HDD is around 80 for consumer grade harddisks. I think this counts for USB connected disks. https://en.wikipedia.org/wiki/IOPS#Mechanical_hard_drives >From the stats you posted it looks like you are almost always doing 50+ writes/second already. That does not leave much IOPS for the find process. ZFS tries to bundle the writes every 5 seconds to leave room for the reads. See "sysctl vfs.zfs.txg.timeout". Unless it has too much data to write or a sync request comes in. #device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b $ iostat -x -w 1 da1 | grep da1 da1 6 19 94.0 525.0 15 2 123 8 1 12 da1 0 4 0.0 15.2 0 1 122 41 0 23 da1 0 0 0.0 0.0 0 0 0 0 0 0 da1 0 2 0.0 738.7 0 4 61 23 0 7 da1 0 0 0.0 0.0 0 0 0 0 0 0 da1 0 98 0.0 2850.9 0 1 0 1 1 7 da1 0 4 0.0 16.0 0 1 139 47 0 28 da1 0 0 0.0 0.0 0 0 0 0 0 0 da1 0 0 0.0 0.0 0 0 0 0 0 0 da1 0 0 0.0 0.0 0 0 0 0 0 0 da1 0 80 0.0 2361.3 0 1 0 1 1 6 da1 0 4 0.0 15.3 0 1 125 42 0 24 Every 5th row is a spike in writes on my RPI4. "zpool iostat 1" should give a similar pattern. So if my observation is right it might be interesting to find out what is writing. I had similar issues after the number of jails on my RPI4 increased and they all were doing a little bit of writing which accumulated in quite a lot of writing. My solution was to add an SSD. Regards, Ronald. ------=_Part_6658_1428730899.1698413675449 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: void <void@f-m.fm>
Datum: vrijdag, 27 oktober 2023 14:51
Aan: freebsd-stable@freebsd.org
Onderwerp: Re: periodic daily takes a very long time to run (14-stable)

On Fri, Oct 27, 2023 at 01:45:24PM +0200, Ronald Klop wrote:

>Can you run "gstat" or "iostat -x -d 1" to see how busy your disk is? >And how much bandwidth is uses.

when periodic is *not* running, gstat numbers fluctuate between whats indicated below and all zeros:

dT: 1.011s  w: 1.000s
  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
     0     69      0      0    0.0     67    732    0.6   24.3| da0
     0      0      0      0    0.0      0      0    0.0    0.0| da0p1
     0      0      0      0    0.0      0      0    0.0    0.0| da0p2
     0     69      0      0    0.0     67    732    0.6   24.3| da0p3
     0      0      0      0    0.0      0      0    0.0    0.0| da0p2.eli
     0     69      0      0    0.0     67    732    0.8   24.4| da0p3.eli

# iostat -x -d 1
        extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
da0           15      18    189.4    565.1    47     7   144    27    0  23 pass0          0       0      0.0      0.0    54     0   154    79    0   0         extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
da0            3      70    172.6    698.0    26     1   129     5    0  33 pass0          0       0      0.0      0.0     0     0     0     0    0   0         extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
da0            0       0      0.0      0.0     0     0     0     0    0   0 pass0          0       0      0.0      0.0     0     0     0     0    0   0
(then all 0s repeatedly)

When periodic *is* running:

dT: 1.047s  w: 1.000s
  L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
     2     89     34    138   88.1     54    783   33.9   95.0| da0
     0      0      0      0    0.0      0      0    0.0    0.0| da0p1
     0      0      0      0    0.0      0      0    0.0    0.0| da0p2
     2     89     34    138   88.1     54    783   34.0   95.0| da0p3
     0      0      0      0    0.0      0      0    0.0    0.0| da0p2.eli
     2     90     35    141   86.3     54    783   34.2   96.9| da0p3.eli

the kBps number varies wildly between 100 and 1200. %busy is 92-102%

# iostat -x -d 1
                         extended device statistics  
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b  
da0           15      18    189.6    564.1    47     7   144    27   10  23 pass0          0       0      0.0      0.0    54     0   154    79    0   0
Numbers vary wildly here, too. I wonder if theres a switch for iostat that lets it
run for say 1000 seconds then produces an average of the values gathered.

I'll look for one
-- 
 


What stands out to me is that you do quite a lot of writes on the disk. (I might be mistaken.)
The max. number of IOPS for HDD is around 80 for consumer grade harddisks. I think this counts for USB connected disks.
https://en.wikipedia.org/wiki/IOPS#Mechanical_hard_drives
>From the stats you posted it looks like you are almost always doing 50+ writes/second already. That does not leave much IOPS for the find process.

ZFS tries to bundle the writes every 5 seconds to leave room for the reads. See "sysctl vfs.zfs.txg.timeout". Unless it has too much data to write or a sync request comes in. 
#device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
$ iostat -x  -w 1 da1 | grep da1
da1            6      19     94.0    525.0    15     2   123     8    1  12
da1            0       4      0.0     15.2     0     1   122    41    0  23
da1            0       0      0.0      0.0     0     0     0     0    0   0
da1            0       2      0.0    738.7     0     4    61    23    0   7
da1            0       0      0.0      0.0     0     0     0     0    0   0
da1            0      98      0.0   2850.9     0     1     0     1    1   7
da1            0       4      0.0     16.0     0     1   139    47    0  28
da1            0       0      0.0      0.0     0     0     0     0    0   0
da1            0       0      0.0      0.0     0     0     0     0    0   0
da1            0       0      0.0      0.0     0     0     0     0    0   0
da1            0      80      0.0   2361.3     0     1     0     1    1   6
da1            0       4      0.0     15.3     0     1   125    42    0  24

Every 5th row is a spike in writes on my RPI4. "zpool iostat 1" should give a similar pattern.

So if my observation is right it might be interesting to find out what is writing. I had similar issues after the number of jails on my RPI4 increased and they all were doing a little bit of writing which accumulated in quite a lot of writing. My solution was to add an SSD.

Regards,
Ronald.
  ------=_Part_6658_1428730899.1698413675449-- From nobody Fri Oct 27 14:42:00 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH53K5g0Wz4yKJw for ; Fri, 27 Oct 2023 14:42:05 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH53J6Jwfz4b0B for ; Fri, 27 Oct 2023 14:42:04 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b="mzFA/Yyd"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="cUcJ5W7/"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailout.nyi.internal (Postfix) with ESMTP id 780225C0166 for ; Fri, 27 Oct 2023 10:42:04 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 27 Oct 2023 10:42:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698417724; x=1698504124; bh=7u dlC+ES+ZjTsKvnc84mcPQs/pi/mrMqAX6gYtFaEbI=; b=mzFA/YydVN87nguqhv Lwi2nBVEABMqoe68/gaxtXzt7kSX3Fp3IHlZl663bt9aV2AlhpGVNPUrJCWbh+Mu 92ODLJ40GCWUwhcphIQC5pPyHHGZRkye+8gmSQbrRVRKIL31AsFh6M2AzLRujAoV IOSUMntN1qKXZySIhdQMGiSxV7jFQZYCsUjZzuZIvBoXlMBDvbLWq2ASypFNK558 vMnV9HyVAG5ngm19Utbh6+w+Zd+Rt2ywlcr4JUJQPX5axgeMMw4hmoMofuLSlQmP TJk8Q6DN2FqVqPZrVpH3geTJiKR9J+Lq55wHdRlJ9830nOO6tQsp7wZUVjn0wYv5 9bQg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698417724; x=1698504124; bh=7udlC+ES+ZjTs Kvnc84mcPQs/pi/mrMqAX6gYtFaEbI=; b=cUcJ5W7/CeTn9KeKttTHe0QIkx6Lm 7KpJWPp3VFpYoZIvyl29zLBqTA2vyyTJvlkLhnKwscevUrrY642tl74UcbHqL1lD ClTUer8W/3M5ajc7JTf4iPS5N+oDgyNIfzMzOm/nI0/7hCRcG5LGEwhC0Ji85WpB hNw+bHWHUyIRlnt5QimIazIgS5xpgiwjHEX1WRM8qNA6A1gyKseCJv3T0MnZbRkx r6Vb03xk6RJmQoXccdDx8V7hmOjFb35GnzKyanzKF53u9uCT818MoFJdwnfb18Y7 SszZTVcHKYKa/CYXASzC7y6ncVtaoTml18ryosKnpyfLuL+KrGvNZfAUg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdektdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeejudeuiefffedtteehudehgeeukeetffdvfeevieefhefhgfffteduueehte eikeenucffohhmrghinhepfihikhhiphgvughirgdrohhrghenucevlhhushhtvghrufhi iigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehvohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 10:42:03 -0400 (EDT) Date: Fri, 27 Oct 2023 15:42:00 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <794932758.6659.1698413675475@localhost> X-Spamd-Result: default: False [-4.69 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.987]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH53J6Jwfz4b0B X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 03:34:35PM +0200, Ronald Klop wrote: > >What stands out to me is that you do quite a lot of writes on the disk. (I might be mistaken.) >The max. number of IOPS for HDD is around 80 for consumer grade harddisks. I think this counts for USB connected disks. >https://en.wikipedia.org/wiki/IOPS#Mechanical_hard_drives >>From the stats you posted it looks like you are almost always doing 50+ writes/second already. That does not leave much IOPS for the find process. > >ZFS tries to bundle the writes every 5 seconds to leave room for the reads. See "sysctl vfs.zfs.txg.timeout". Unless it has too much data to write or a sync request comes in. % sysctl vfs.zfs.txg.timeout vfs.zfs.txg.timeout: 5 do I need to tune this? Here's equivalent output from my setup (I ran periodic daily again) #device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b da0 16 18 191.9 557.9 50 8 144 29 10 24 da0 107 0 699.7 0.0 52 0 0 52 1 99 da0 102 0 409.2 0.0 71 0 0 71 2 98 da0 65 6 259.6 49.4 101 143 0 105 12 101 da0 57 14 227.7 123.9 153 163 0 155 12 100 da0 40 19 158.8 285.8 205 103 0 172 12 98 da0 46 30 191.1 441.9 180 58 0 132 11 91 da0 63 4 261.6 16.1 162 250 239 170 6 112 da0 67 10 273.7 83.6 99 66 0 95 12 91 da0 32 21 129.4 177.9 223 102 0 175 5 97 da0 48 16 191.9 261.3 173 130 0 162 9 109 da0 38 19 152.2 191.3 168 61 292 139 8 104 da0 92 0 366.9 0.0 104 0 0 104 4 100 da0 73 10 291.7 87.9 76 99 0 79 12 97 da0 49 15 195.2 270.9 156 129 0 150 11 103 da0 53 15 212.3 248.3 139 128 0 137 12 92 da0 54 22 216.1 272.1 151 81 92 130 8 107 da0 80 4 320.9 16.0 74 201 125 80 3 100 da0 55 10 218.8 72.9 89 73 0 87 11 82 ^C % zpool iostat 1 capacity operations bandwidth pool alloc free read write read write ---------- ----- ----- ----- ----- ----- ----- zroot 93.6G 818G 13 16 161K 506K zroot 93.6G 818G 91 0 367K 0 zroot 93.6G 818G 113 0 454K 0 zroot 93.6G 818G 102 0 411K 0 zroot 93.6G 818G 98 0 422K 0 zroot 93.6G 818G 67 18 271K 171K zroot 93.6G 818G 43 16 173K 252K zroot 93.6G 818G 43 28 173K 376K zroot 93.6G 818G 78 3 315K 15.9K zroot 93.6G 818G 94 0 378K 0 zroot 93.6G 818G 103 0 414K 0 zroot 93.6G 818G 102 0 658K 0 zroot 93.6G 818G 98 0 396K 0 zroot 93.6G 818G 109 0 438K 0 zroot 93.6G 818G 101 0 404K 0 zroot 93.6G 818G 47 13 191K 91.4K zroot 93.6G 818G 52 11 209K 126K zroot 93.6G 818G 50 20 202K 301K zroot 93.6G 818G 46 12 186K 128K zroot 93.6G 818G 86 0 346K 3.93K zroot 93.6G 818G 45 18 183K 172K zroot 93.6G 818G 42 15 172K 343K zroot 93.6G 818G 43 24 173K 211K zroot 93.6G 818G 87 0 596K 0 ^C >So if my observation is right it might be interesting to find out what is writing. would ktrace and/or truss be useful? something else? The truss -p output of the find PID produces massive amounts of output, all like this: fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ mode=-rw-r--r-- ,inode=367004,size=10312,blksize=10752 },AT_SYMLINK_NOFOLLOW) = 0 (0x0) with the filename changing each time (later...) that file is in ccache!!! locate 5e70d5f895ccc92af6a7d5226f818b-81464.o /var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o maybe if I can exclude that dir (and /usr/obj) it'll lessen the periodic runtime. But i don't know yet whats calling find(1) when periodic daily runs. If I can, I might be able to tell it not to walk certain heirarchies. >I had similar issues after the number of jails on my RPI4 increased and they all >were doing a little bit of writing which accumulated in quite a lot of writing. I'm at a loss as to what's doing the writing. The system runs the following: poudriere-devel # for aarch64 and armv7 apcupsd # for ups monitoring vnstat # bandwidth use, writes to its db in /var/db/vnstat sshd exim (local) pflogd # right now it's behind a firewall, on NAT so it's not doing much pf # same ntpd powerd nginx # this serves the poudriere web frontend, and that's it (http-only) syslogd >My solution was to add an SSD. I have an nfs alternative. The LAN is 1GB. But I think the fix will be to tell find to not search some paths. just need to work out how to do it. What would the effect be of increasing or decreasing the txg delta with system performance? -- > From nobody Fri Oct 27 15:01:03 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH5TT0qbXz4yLbK for ; Fri, 27 Oct 2023 15:01:17 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Received: from gromit.dlib.vt.edu (gromit.dlib.vt.edu [128.173.126.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH5TS5pcWz4ckK for ; Fri, 27 Oct 2023 15:01:16 +0000 (UTC) (envelope-from paul@gromit.dlib.vt.edu) Authentication-Results: mx1.freebsd.org; none Received: from itso-dell.gromit23.net (c-174-179-68-208.hsd1.va.comcast.net [174.179.68.208]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by gromit.dlib.vt.edu (Postfix) with ESMTPSA id E1B6875DA2; Fri, 27 Oct 2023 11:01:04 -0400 (EDT) Message-ID: <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> Subject: Re: periodic daily takes a very long time to run (14-stable) From: Paul Mather To: void , freebsd-stable@freebsd.org Date: Fri, 27 Oct 2023 11:01:03 -0400 In-Reply-To: References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable User-Agent: Evolution 3.44.4-0ubuntu2 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:1312, ipnet:128.173.0.0/16, country:US] X-Rspamd-Queue-Id: 4SH5TS5pcWz4ckK On Fri, 2023-10-27 at 15:42 +0100, void wrote: > would ktrace and/or truss be useful? something else? The truss -p > output of the > find PID produces massive amounts of output, all like this: >=20 > fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ mode=3D-rw- > r--r--=20 > ,inode=3D367004,size=3D10312,blksize=3D10752 },AT_SYMLINK_NOFOLLOW) =3D 0 > (0x0) >=20 > with the filename changing each time >=20 > (later...) >=20 > that file is in ccache!!! >=20 > locate 5e70d5f895ccc92af6a7d5226f818b-81464.o > /var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o >=20 > maybe if I can exclude that dir (and /usr/obj) it'll lessen the > periodic runtime. > But i don't know yet whats calling find(1) when periodic daily runs. > If I can,=20 > I might be able to tell it not to walk certain heirarchies. This doesn't affect the periodic daily run (to my knowledge) but if you are using ccache then it will slow down the periodic weekly run when it executes the /etc/periodic/weekly/310.locate script. I use ccache, too, and my ccache cache file system has millions of files in it. I found no practical use for me to have the weekly locate indexing to include ephemeral ccache files in the locate index. It just slows down the indexing process. You can tinker with the settings in /etc/locate.rc and add your ccache file system/directory to PRUNEPATHS in order to get the /usr/libexec/locate.updatedb indexing script to skip that hierarchy. Cheers, Paul. From nobody Fri Oct 27 15:07:13 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH5cZ0S6Xz4yLjX for ; Fri, 27 Oct 2023 15:07:26 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH5cX2bz3z4fJx for ; Fri, 27 Oct 2023 15:07:24 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="Q TmLKV8"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SH5cP1S3xz6ddR for ; Fri, 27 Oct 2023 17:07:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698419234; x= 1700233635; bh=WfVUpkQIh0b18urEHWVkBPCaSQ2bA04Ln2a8YuX5iec=; b=Q TmLKV8ZwHgWTmROdimwFUDL0tjHt83eICDNfGaG1rmo6n012Jk4PXFnotxFWxYZB x58v1BpTt0kem3LvSxiu3a3NGMPeZ5aXCaxFsSsrSYZsKlmpKvXt22sGV5In8X20 jmHR/UAhe92CwRN4cJzZ56LeB4T4uOIdjTK6OB5r5MuF5hGFLHO2RsjcDVaBp9sc I/QE2RjRfiRo89os8Ult5Kp4ds+IqUxfv+4/QjwWMim1km5Zzz0JvFmnROfA2U6y jfe4y2CQeTPh83Q5qHUf/kJwIznAalZo1rlTWlSidx9rvnXVdI59M57JunSyXOdm 4piyLxJGl0fjITBXTxLhw== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id 9uGvCFsML7XW for ; Fri, 27 Oct 2023 17:07:14 +0200 (CEST) Message-ID: Date: Fri, 27 Oct 2023 17:07:13 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US, it To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; BLOCKLISTDE_FAIL(0.00)[159.69.1.99:server fail]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SH5cX2bz3z4fJx X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 27/10/23 16:42, void wrote: > On Fri, Oct 27, 2023 at 03:34:35PM +0200, Ronald Klop wrote: >> So if my observation is right it might be interesting to find out what >> is writing. > > would ktrace and/or truss be useful? something else? The truss -p output > of the > find PID produces massive amounts of output, all like this: > > fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ > mode=-rw-r--r-- ,inode=367004,size=10312,blksize=10752 > },AT_SYMLINK_NOFOLLOW) = 0 (0x0) > > with the filename changing each time > > (later...) > > that file is in ccache!!! > > locate 5e70d5f895ccc92af6a7d5226f818b-81464.o > /var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o > > maybe if I can exclude that dir (and /usr/obj) it'll lessen the periodic > runtime. > But i don't know yet whats calling find(1) when periodic daily runs. If > I can, I might be able to tell it not to walk certain heirarchies. I stumbled on a similar problem. Periodic trying to scan huge ccache directories. The script doing that was locate database indexing in weekly. Not sure if you're having the same issue, but this could give you ideas. My fix was: weekly_locate_enable="YES" PRUNEPATHS="/tmp, /usr/tmp, /var/tmp, /var/db/portsnap, /var/db/freebsd-update, /dumpster/ccache, /poudriere" There is no advantage in locate indexing ccache, and I don't want it to cache poudriere either. (mhmm, that portsnap directory looks like a leftover from the past...) Also, unluckily poudriere in modern times does a lot of disk IO (old plain ports building too BTW) and it is really best, especially for SSD media, to have enough ram to run the full build there. Will really save on SSD writes. -- Guido Falsi From nobody Fri Oct 27 15:08:18 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH5df4S2mz4yLpc for ; Fri, 27 Oct 2023 15:08:22 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH5dd4G52z3C28 for ; Fri, 27 Oct 2023 15:08:21 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="V 6nM0VF"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SH5dc5NKPz6fqH for ; Fri, 27 Oct 2023 17:08:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :references:from:from:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698419299; x= 1700233700; bh=HsfmhH8HwYS6iLjlDZiXMUPkBkM3yvI4g/XUPU3wvKE=; b=V 6nM0VFCSdfWErpOXp+cnyEJy3NjrrdhDfhY8pZ86vDqhGKfZ7wYEy3XPS9Pwrzjy UaWMviN518SRz15yvMhljGaQ8bqV8lwJUD0BsDKENTrIGcaX0Q7sXsN5/q+CyVZd ODmaFI7ITQCltcwx3oBAcvSjgr/6s3/mP+zTQagDNN5sh7/99AT7KoJOpEqhfP9V kz3wIxOM7sst/5XySN2PaANGXG5qRErzMvDbtMuH0Yhs05ePEkxaFtArfdusEGwQ 21PSVxQ4zjxDN7rBISmoTvQ8/GRzM2kPg9VkqklOj2nPk4oSdc/tRGeEldGwFh76 ncuZ/02iJOBi9vzruP9Aw== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id Y_uDyNELekru for ; Fri, 27 Oct 2023 17:08:19 +0200 (CEST) Message-ID: <7968dd3b-5a79-475a-9edb-fd2cc7cb8ea4@madpilot.net> Date: Fri, 27 Oct 2023 17:08:18 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US, it From: Guido Falsi To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; BLOCKLISTDE_FAIL(0.00)[159.69.1.99:server fail]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SH5dd4G52z3C28 X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 27/10/23 17:07, Guido Falsi wrote: > On 27/10/23 16:42, void wrote: >> On Fri, Oct 27, 2023 at 03:34:35PM +0200, Ronald Klop wrote: >>> So if my observation is right it might be interesting to find out >>> what is writing. >> >> would ktrace and/or truss be useful? something else? The truss -p >> output of the >> find PID produces massive amounts of output, all like this: >> >> fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ >> mode=-rw-r--r-- ,inode=367004,size=10312,blksize=10752 >> },AT_SYMLINK_NOFOLLOW) = 0 (0x0) >> >> with the filename changing each time >> >> (later...) >> >> that file is in ccache!!! >> >> locate 5e70d5f895ccc92af6a7d5226f818b-81464.o >> /var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o >> >> maybe if I can exclude that dir (and /usr/obj) it'll lessen the >> periodic runtime. >> But i don't know yet whats calling find(1) when periodic daily runs. >> If I can, I might be able to tell it not to walk certain heirarchies. > > > I stumbled on a similar problem. Periodic trying to scan huge ccache > directories. > > The script doing that was locate database indexing in weekly. Not sure > if you're having the same issue, but this could give you ideas. > > My fix was: > > weekly_locate_enable="YES" > PRUNEPATHS="/tmp, /usr/tmp, /var/tmp, /var/db/portsnap, > /var/db/freebsd-update, /dumpster/ccache, /poudriere" > Forgot to mention, this goes in /etc/periodic.conf. -- Guido Falsi From nobody Fri Oct 27 15:23:49 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH5zX6GZsz4yMcr for ; Fri, 27 Oct 2023 15:23:52 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH5zX0xFWz3Fst for ; Fri, 27 Oct 2023 15:23:52 +0000 (UTC) (envelope-from SRS0=D2w5=GJ=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Fri, 27 Oct 2023 17:23:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1698420230; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SUsACkhxKzepr0g/93zbLD76IK6I4qzx8vU/RkaxFcc=; b=UKkOT/N0q3omokSfI61LGDq6o7fVqLFxfLiBZxHBnCouIjikG0WPqEOuhfbYlRg+ud4DwH NxsA3/32ViW96Z/NWQVC+wgVyvAqNFP03FCBkQJd9B01/cDIf2xHjRL1iGWg9dM4UHZDlJ tAdmNdkzRlQHmgGItuf1ae91Kn38ro0h8tSvsnwy81D56sLaBfzDLgv87HKSv2i7CXyQX+ 3BXNunoAwEMBIOouyHA4vd/KCuPu9784Bs/iODYrADvQwGJvAqa7WNph5GZUqd4ow2cRSJ 5Hiyz/KFJG/jgQ1XHCxDFNB8ZFVzIOey5YNJ850dLGeumBwPg2hWpe7rs8Sznw== From: Ronald Klop To: void Cc: freebsd-stable@freebsd.org Message-ID: <1210534753.8409.1698420229888@localhost> In-Reply-To: References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> Subject: Re: periodic daily takes a very long time to run (14-stable) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_8408_2001362475.1698420229883" X-Mailer: Realworks (677.7) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4SH5zX0xFWz3Fst ------=_Part_8408_2001362475.1698420229883 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: quoted-printable Van: void Datum: vrijdag, 27 oktober 2023 16:42 Aan: freebsd-stable@freebsd.org Onderwerp: Re: periodic daily takes a very long time to run (14-stable) >=20 > On Fri, Oct 27, 2023 at 03:34:35PM +0200, Ronald Klop wrote: > > > >What stands out to me is that you do quite a lot of writes on the disk. = (I might be mistaken.) > >The max. number of IOPS for HDD is around 80 for consumer grade harddisk= s. I think this counts for USB connected disks. > >https://en.wikipedia.org/wiki/IOPS#Mechanical_hard_drives > >From the stats you posted it looks like you are almost always doing 50+ = writes/second already. That does not leave much IOPS for the find process. > > > >ZFS tries to bundle the writes every 5 seconds to leave room for the rea= ds. See "sysctl vfs.zfs.txg.timeout". Unless it has too much data to write = or a sync request comes in. >=20 > % sysctl vfs.zfs.txg.timeout > vfs.zfs.txg.timeout: 5 >=20 > do I need to tune this? >=20 > Here's equivalent output from my setup (I ran periodic daily again) >=20 > #device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen = %b > da0 16 18 191.9 557.9 50 8 144 29 10 = 24 da0 107 0 699.7 0.0 52 0 0 52 1 = 99 da0 102 0 409.2 0.0 71 0 0 71 2= 98 da0 65 6 259.6 49.4 101 143 0 105 1= 2 101 da0 57 14 227.7 123.9 153 163 0 155 = 12 100 da0 40 19 158.8 285.8 205 103 0 172 = 12 98 da0 46 30 191.1 441.9 180 58 0 132 = 11 91 da0 63 4 261.6 16.1 162 250 239 170= 6 112 da0 67 10 273.7 83.6 99 66 0 9= 5 12 91 da0 32 21 129.4 177.9 223 102 0 1= 75 5 97 da0 48 16 191.9 261.3 173 130 0 = 162 9 109 da0 38 19 152.2 191.3 168 61 292 = 139 8 104 da0 92 0 366.9 0.0 104 0 0 = 104 4 100 da0 73 10 291.7 87.9 76 99 0= 79 12 97 da0 49 15 195.2 270.9 156 129 = 0 150 11 103 da0 53 15 212.3 248.3 139 128 = 0 137 12 92 da0 54 22 216.1 272.1 151 81 = 92 130 8 107 da0 80 4 320.9 16.0 74 201 = 125 80 3 100 da0 55 10 218.8 72.9 89 73 = 0 87 11 82 ^C >=20 > % zpool iostat 1 > capacity operations bandwidth > pool alloc free read write read write > ---------- ----- ----- ----- ----- ----- ----- > zroot 93.6G 818G 13 16 161K 506K > zroot 93.6G 818G 91 0 367K 0 > zroot 93.6G 818G 113 0 454K 0 > zroot 93.6G 818G 102 0 411K 0 > zroot 93.6G 818G 98 0 422K 0 > zroot 93.6G 818G 67 18 271K 171K > zroot 93.6G 818G 43 16 173K 252K > zroot 93.6G 818G 43 28 173K 376K > zroot 93.6G 818G 78 3 315K 15.9K > zroot 93.6G 818G 94 0 378K 0 > zroot 93.6G 818G 103 0 414K 0 > zroot 93.6G 818G 102 0 658K 0 > zroot 93.6G 818G 98 0 396K 0 > zroot 93.6G 818G 109 0 438K 0 > zroot 93.6G 818G 101 0 404K 0 > zroot 93.6G 818G 47 13 191K 91.4K > zroot 93.6G 818G 52 11 209K 126K > zroot 93.6G 818G 50 20 202K 301K > zroot 93.6G 818G 46 12 186K 128K > zroot 93.6G 818G 86 0 346K 3.93K > zroot 93.6G 818G 45 18 183K 172K > zroot 93.6G 818G 42 15 172K 343K > zroot 93.6G 818G 43 24 173K 211K > zroot 93.6G 818G 87 0 596K 0 > ^C >=20 > >So if my observation is right it might be interesting to find out what i= s writing. > would ktrace and/or truss be useful? something else? The truss -p output = of the > find PID produces massive amounts of output, all like this: >=20 > fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ mode=3D-rw-r-= -r-- ,inode=3D367004,size=3D10312,blksize=3D10752 },AT_SYMLINK_NOFOLLOW) = =3D 0 (0x0) >=20 > with the filename changing each time >=20 > (later...) >=20 > that file is in ccache!!! >=20 > locate 5e70d5f895ccc92af6a7d5226f818b-81464.o > /var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o >=20 > maybe if I can exclude that dir (and /usr/obj) it'll lessen the periodic = runtime. > But i don't know yet whats calling find(1) when periodic daily runs. If I= can, I might be able to tell it not to walk certain heirarchies. >=20 > >I had similar issues after the number of jails on my RPI4 increased and = they all >were doing a little bit of writing which accumulated in quite a l= ot of writing. >=20 > I'm at a loss as to what's doing the writing. The system runs the followi= ng: >=20 > poudriere-devel # for aarch64 and armv7 > apcupsd # for ups monitoring > vnstat # bandwidth use, writes to its db in /var/db/vnstat > sshd > exim (local) > pflogd # right now it's behind a firewall, on NAT so it's not do= ing much > pf # same > ntpd > powerd > nginx # this serves the poudriere web frontend, and that's it (h= ttp-only) syslogd >=20 > >My solution was to add an SSD. >=20 > I have an nfs alternative. The LAN is 1GB. But I think the fix will be to= tell find > to not search some paths. just need to work out how to do it. >=20 > What would the effect be of increasing or decreasing the txg delta with s= ystem > performance? > --=20 > > > =20 >=20 >=20 >=20 Well. You could remove daily_clean_disks_enable=3D"YES" from /etc/periodic.= conf. That saves you the "find". I have never used it before. The default i= s "off". $ grep clean_disks /etc/defaults/periodic.conf =20 daily_clean_disks_enable=3D"NO" # Delete files daily daily_clean_disks_files=3D"[#,]* .#* a.out *.core *.CKP .emacs_[0-9]*" daily_clean_disks_days=3D3 # If older than this The list of files it checks for doesn't look very useful to me in 2023. Thi= s does do a full find over *all* directories and files. *every day* ??? If you have a lot of *.core files you are better of putting this in sysctl.= conf: kern.corefile=3D/var/tmp/%U.%N.%I.%P.core . So you know where to look= to delete them. Actually my RPI3 has this in cron: @daily find /var/tmp/ -name "*.core" -m= time +7 -ls -delete . About the vfs.zfs.txg.timeout. Don't mess with it if you don't have a solid= reason. It used to be 30 seconds when ZFS was new if I remember correctly.= But it could give too big write bursts which paused the system noticeably.= This is all from memory. I might be totally wrong here. :-) The stats of your system look pretty reasonable. Also the zpool iostat 1 in= dicates periods of no writes. So that is ok. You just have a lot of IOPS fo= r 1 spinning disk when everything runs together. Poudriere & ccache indicat= e that you are using it for compiling pkgs or other stuff. That is pretty h= eavy for you setup if you manage to run things in parallel as the RPI4 has = 4CPUs. It doesn't help to run daily_cleanup together. ;-) Regards, Ronald. =20 ------=_Part_8408_2001362475.1698420229883 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: quoted-printable

Van: void <void@f-m.fm>
Datum: vrijdag, 27 oktober 2023 16:42
Aan: freebsd-stable@freebsd.org
Onderwerp: Re: periodic daily takes a very long time to ru= n (14-stable)

On Fri, Oct 27, 2023 at 03:34:35P= M +0200, Ronald Klop wrote:
>
>What stands out to me is that you do quite a lot of writes on the disk.= (I might be mistaken.)
>The max. number of IOPS for HDD is around 80 for consumer grade harddis= ks. I think this counts for USB connected disks.
>h= ttps://en.wikipedia.org/wiki/IOPS#Mechanical_hard_drives
>From the stats you posted it looks like you are almost always doing 50+= writes/second already. That does not leave much IOPS for the find process.=
>
>ZFS tries to bundle the writes every 5 seconds to leave room for the re= ads. See "sysctl vfs.zfs.txg.timeout". Unless it has too much data to write= or a sync request comes in.

% sysctl vfs.zfs.txg.timeout
vfs.zfs.txg.timeout: 5

do I need to tune this?

Here's equivalent output from my setup (I ran periodic daily again)

#device       r/s     w/s=     kr/s     kw/s  ms/r  = ;ms/w  ms/o  ms/t qlen  %b
da0           16  &n= bsp;   18    191.9    557.9 &n= bsp;  50     8   144   &n= bsp;29   10  24 da0       &nbs= p;  107       0    6= 99.7      0.0    52   &nb= sp; 0     0    52    = ;1  99 da0          102 &= nbsp;     0    409.2   &n= bsp;  0.0    71     0  &n= bsp;  0    71    2  98 da0 &nb= sp;         65   &nb= sp;   6    259.6     49.4=   101   143     0   105 =   12 101 da0         &nbs= p; 57      14    227.7  &= nbsp; 123.9   153   163     0 =   155   12 100 da0       =     40      19   &nb= sp;158.8    285.8   205   103  &nbs= p;  0   172   12  98 da0   &nb= sp;       46     &nb= sp;30    191.1    441.9   180  = ;  58     0   132   11 &n= bsp;91 da0           63 &= nbsp;     4    261.6   &n= bsp; 16.1   162   250   239   = 170    6 112 da0        &= nbsp;  67      10    273.= 7     83.6    99    66 &n= bsp;   0    95   12  91 da0 &n= bsp;         32   &n= bsp;  21    129.4    177.9  &n= bsp;223   102     0   175  &nb= sp; 5  97 da0         &nb= sp; 48      16    191.9  =   261.3   173   130     0=   162    9 109 da0      =      38      19  &nb= sp; 152.2    191.3   168    61=   292   139    8 104 da0   &n= bsp;       92     &n= bsp; 0    366.9      0.0  = ; 104     0     0   = 104    4 100 da0        &= nbsp;  73      10    291.= 7     87.9    76    99 &n= bsp;   0    79   12  97 da0 &n= bsp;         49   &n= bsp;  15    195.2    270.9  &n= bsp;156   129     0   150  &nb= sp;11 103 da0           5= 3      15    212.3   &nbs= p;248.3   139   128     0  &nb= sp;137   12  92 da0       &nbs= p;   54      22    2= 16.1    272.1   151    81  &nb= sp; 92   130    8 107 da0    &= nbsp;      80      &= nbsp;4    320.9     16.0   &nb= sp;74   201   125    80   &nbs= p;3 100 da0           55 =      10    218.8    =  72.9    89    73    &nbs= p;0    87   11  82 ^C

% zpool iostat 1
capacity     operations     bandwid= th
pool        alloc   free  = ; read  write   read  write
----------  -----  -----  -----  -----  ----- &nbs= p;-----
zroot       93.6G   818G  &nbs= p;  13     16   161K   50= 6K
zroot       93.6G   818G  &nbs= p;  91      0   367K  &nb= sp;   0
zroot       93.6G   818G  &nbs= p; 113      0   454K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p; 102      0   411K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p;  98      0   422K  &nb= sp;   0
zroot       93.6G   818G  &nbs= p;  67     18   271K   17= 1K
zroot       93.6G   818G  &nbs= p;  43     16   173K   25= 2K
zroot       93.6G   818G  &nbs= p;  43     28   173K   37= 6K
zroot       93.6G   818G  &nbs= p;  78      3   315K  15.= 9K
zroot       93.6G   818G  &nbs= p;  94      0   378K  &nb= sp;   0
zroot       93.6G   818G  &nbs= p; 103      0   414K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p; 102      0   658K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p;  98      0   396K  &nb= sp;   0
zroot       93.6G   818G  &nbs= p; 109      0   438K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p; 101      0   404K   &n= bsp;  0
zroot       93.6G   818G  &nbs= p;  47     13   191K  91.4K zroot       93.6G   818G  &nbs= p;  52     11   209K   12= 6K
zroot       93.6G   818G  &nbs= p;  50     20   202K   30= 1K
zroot       93.6G   818G  &nbs= p;  46     12   186K   12= 8K
zroot       93.6G   818G  &nbs= p;  86      0   346K  3.9= 3K
zroot       93.6G   818G  &nbs= p;  45     18   183K   17= 2K
zroot       93.6G   818G  &nbs= p;  42     15   172K   34= 3K
zroot       93.6G   818G  &nbs= p;  43     24   173K   21= 1K
zroot       93.6G   818G  &nbs= p;  87      0   596K  &nb= sp;   0
^C

>So if my observation is right it might be interesting to find out what = is writing.
would ktrace and/or truss be useful? something else? The truss -p output of= the
find PID produces massive amounts of output, all like this:

fstatat(AT_FDCWD,"5e70d5f895ccc92af6a7d5226f818b-81464.o",{ mode=3D-rw-r--r= -- ,inode=3D367004,size=3D10312,blksize=3D10752 },AT_SYMLINK_NOFOLLOW) =3D = 0 (0x0)

with the filename changing each time

(later...)

that file is in ccache!!!

locate 5e70d5f895ccc92af6a7d5226f818b-81464.o
/var/cache/ccache/f/5/5e70d5f895ccc92af6a7d5226f818b-81464.o

maybe if I can exclude that dir (and /usr/obj) it'll lessen the periodic ru= ntime.
But i don't know yet whats calling find(1) when periodic daily runs. If I c= an, I might be able to tell it not to walk certain heirarchies.

>I had similar issues after the number of jails on my RPI4 increased and= they all >were doing a little bit of writing which accumulated in quite= a lot of writing.

I'm at a loss as to what's doing the writing. The system runs the following= :

poudriere-devel # for aarch64 and armv7
apcupsd         # for ups monitorin= g
vnstat          # bandwidth us= e, writes to its db in /var/db/vnstat
sshd
exim (local)
pflogd          # right now it= 's behind a firewall, on NAT so it's not doing much
pf             =  # same
ntpd
powerd
nginx          # this serves t= he poudriere web frontend, and that's it (http-only) syslogd

>My solution was to add an SSD.

I have an nfs alternative. The LAN is 1GB. But I think the fix will be to t= ell find
to not search some paths. just need to work out how to do it.

What would the effect be of increasing or decreasing the txg delta with sys= tem
performance?
-- 
>
 




Well. You could remove daily_clean_disks_enable=3D"YES" from /etc/periodic.= conf. That saves you the "find". I have never used it before. The default i= s "off".

$ grep clean_disks /etc/defaults/periodic.conf  
daily_clean_disks_enable=3D"NO"         =        # Delete files daily
daily_clean_disks_files=3D"[#,]* .#* a.out *.core *.CKP .emacs_[0-9]*"
daily_clean_disks_days=3D3          = ;      # If older than this

The list of files it checks for doesn't look very useful to me in 2023. Thi= s does do a full find over *all* directories and files. *every day* ???
If you have a lot of *.core files you are better of putting this in sysctl.= conf: kern.corefile=3D/var/tmp/%U.%N.%I.%P.core . So you know where to look= to delete them.
Actually my RPI3 has this in cron: @daily  find /var/tmp/ -name "*.cor= e" -mtime +7 -ls -delete .

About the vfs.zfs.txg.timeout. Don't mess with it if you don't have a solid= reason. It used to be 30 seconds when ZFS was new if I remember correctly.= But it could give too big write bursts which paused the system noticeably.= This is all from memory. I might be totally wrong here. :-)

The stats of your system look pretty reasonable. Also the zpool iostat 1 in= dicates periods of no writes. So that is ok. You just have a lot of IOPS fo= r 1 spinning disk when everything runs together. Poudriere & ccache ind= icate that you are using it for compiling pkgs or other stuff. That is pret= ty heavy for you setup if you manage to run things in parallel as the RPI4 = has 4CPUs. It doesn't help to run daily_cleanup together. ;-)

Regards,
Ronald.


  ------=_Part_8408_2001362475.1698420229883-- From nobody Fri Oct 27 15:24:58 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH60t17fPz4yMl9 for ; Fri, 27 Oct 2023 15:25:02 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH60s4DXnz3GrD for ; Fri, 27 Oct 2023 15:25:01 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=b05Mo0ld; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=RDrCwVfe; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute7.internal (compute7.nyi.internal [10.202.2.48]) by mailout.nyi.internal (Postfix) with ESMTP id 79B3C5C0244 for ; Fri, 27 Oct 2023 11:25:01 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute7.internal (MEProxy); Fri, 27 Oct 2023 11:25:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698420301; x=1698506701; bh=d7 3AP8NBnN/dWZUeC8Q07GJUAcM9e47gEN28OR6K9bI=; b=b05Mo0ldYwbkg3xc00 mJpTFCSOZl67actS2p4rrsyxmzX1G2SUDuvevRjK4ywfhaFCrdo62/WaDuLcnesV prWkpDrezmhBS+Pr3Yy+fWjGv/OXyRj6Nf1x5vL+O5xcorrdwrH8W8+w1QJQcGYt g4yp9hlo+uChDbTvDF9WQGsF3iP3HtZcWcx4+gBPkJX8E9EBQ1I1E81JIAuJEeKo TVSVlsd0NMbwxzhpteT+Oz3q0ZE55/9HZ7yVlIgJZcPtqP8visCXWRYzlVOJq4WX 9HxJAwhEn63PUXV5nMLMX6FEcc0008v+LgsT+OhkR0alHKUXk73UOy8hEyTPtFkn HkZw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698420301; x=1698506701; bh=d73AP8NBnN/dW ZUeC8Q07GJUAcM9e47gEN28OR6K9bI=; b=RDrCwVfeRAaWgL8YnaGTU5TJRDwU9 smg51ESuD1vbyGoqXyBS3KFyFelw4kkLAodRZCWOdFBcCx/2lkhlJrQ7Lk3DimOa BL4YvvwSV+D3gZjHwJYoraixnd6lMqbGGpGLxGVj7GZg0OvYx7HtcDfBU7rQGJ4h 8Gi/hGdMlfZoPj4X3j9P0HVKeQJ0LQjr2DBTEZUy5RhvBtvOV7XmQKmEOOG/Tn8u 9GVtAwzPWdZXgHkuDfNTqKiBgBQNBNes0SFQKTt4utEzhl+sp9qSZClRrDDj+jxt 3mx0qf3aovXmfNG2CLxzSXjPXvivh2pkk2cH3syuRCVCoNI4eTWpcCsWA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdekkecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdortd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeeivefgfeegfffhgffgiedtjeejhfetfeefgfelfeeiveelveejueeiudegle fggfenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehv ohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 11:25:00 -0400 (EDT) Date: Fri, 27 Oct 2023 16:24:58 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> X-Spamd-Result: default: False [-4.68 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.978]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28:c]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; BLOCKLISTDE_FAIL(0.00)[66.111.4.28:server fail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH60s4DXnz3GrD X-Spamd-Bar: ---- Hi, On Fri, Oct 27, 2023 at 11:01:03AM -0400, Paul Mather wrote: >This doesn't affect the periodic daily run (to my knowledge) # ps x | grep periodic 59319 14 S+ 0:00.00 grep periodic 27376 17 I+ 0:00.00 /bin/sh - /etc/periodic/security/110.neggrpperm 27987 17 I+ 0:00.00 /bin/sh - /etc/periodic/security/110.neggrpperm 43295 17 I+ 0:00.00 /bin/sh - /usr/sbin/periodic daily 44018 17 I+ 0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock /bin/sh /usr/sbin/periodic LOCKED daily 44447 17 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily 47348 17 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily 47882 17 I+ 0:00.00 /bin/sh /etc/periodic/daily/450.status-security 48386 17 I+ 0:00.00 /bin/sh - /usr/sbin/periodic security 49217 17 I+ 0:00.00 lockf -s -t 0 /var/run/periodic.security.lock /bin/sh /usr/sbin/periodic LOCKED security 49566 17 I+ 0:00.01 /bin/sh /usr/sbin/periodic LOCKED security 51783 17 I+ 0:00.00 /bin/sh /usr/sbin/periodic LOCKED security It's been like this (110.newgrpperm) for 2hrs 16 mins /etc/periodic/security/110.neggrpperm has the following line, which I've put in quote marks (the quote marks are not present in the actual file) "n=$(find -sx $MP /dev/null \( ! -fstype local \) -prune -o -type f \" although I'm not certain, it suggests to me it's looking through the entire disk, not excluding anything. >You can tinker with the settings in /etc/locate.rc and add your ccache >file system/directory to PRUNEPATHS in order to get the >/usr/libexec/locate.updatedb indexing script to skip that hierarchy. thank you for that tip I didn't know there was a locate.rc I used to run 310.locate in daily.local long ago ;) In this context though, there's no daily.local it's all standard. -- From nobody Fri Oct 27 15:38:58 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH6K25xYPz4yMtL for ; Fri, 27 Oct 2023 15:39:02 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH6K218Nnz3JHB for ; Fri, 27 Oct 2023 15:39:02 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="j +E9ipJ"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SH6K11t9Wz6ddR for ; Fri, 27 Oct 2023 17:39:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698421139; x= 1700235540; bh=UZasNgtLEpyNFGriItBPvJktHL0H3vqO1n3fFtSyTWs=; b=j +E9ipJIjV5nv6udPSAVobRn778qw+CdClPtqHUN1GLbo00Rj8c5JOlp2kBQ5ZVrl vODKtN6V2tflXrqwgeovI/Ja9bJmk0hNS+L18zeK8EtLfO30CymBz64JOslyZNNN gKpKrfcporCXSY8MSexymzQCYMWECS8CdxeipR8qEXn2qCzA/c4cRv/elXMepXqO xG7JU1Nk86ZCl+iGg2yMYUQb/a7gSFdCbquTo22WActBEvM2037R//zKyzC/JzBy hUjOsssy11jgM8eQK7CjVestTgkIvZ3RWFMJHh+17RzZDUFE7yK/ncujiWnkhK5z 0waOxxs1E6YLr9I/DKzoA== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id 92qR2B32YUnj for ; Fri, 27 Oct 2023 17:38:59 +0200 (CEST) Message-ID: <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> Date: Fri, 27 Oct 2023 17:38:58 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US, it To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SH6K218Nnz3JHB X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 27/10/23 17:24, void wrote: > Hi, > > On Fri, Oct 27, 2023 at 11:01:03AM -0400, Paul Mather wrote: > >> This doesn't affect the periodic daily run (to my knowledge) > > # ps x | grep periodic > > 59319 14  S+       0:00.00 grep periodic > 27376 17  I+       0:00.00 /bin/sh - /etc/periodic/security/110.neggrpperm > 27987 17  I+       0:00.00 /bin/sh - /etc/periodic/security/110.neggrpperm > 43295 17  I+       0:00.00 /bin/sh - /usr/sbin/periodic daily > 44018 17  I+       0:00.00 lockf -s -t 0 /var/run/periodic.daily.lock > /bin/sh /usr/sbin/periodic LOCKED daily > 44447 17  I+       0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily > 47348 17  I+       0:00.01 /bin/sh /usr/sbin/periodic LOCKED daily > 47882 17  I+       0:00.00 /bin/sh /etc/periodic/daily/450.status-security > 48386 17  I+       0:00.00 /bin/sh - /usr/sbin/periodic security > 49217 17  I+       0:00.00 lockf -s -t 0 /var/run/periodic.security.lock > /bin/sh /usr/sbin/periodic LOCKED security > 49566 17  I+       0:00.01 /bin/sh /usr/sbin/periodic LOCKED security > 51783 17  I+       0:00.00 /bin/sh /usr/sbin/periodic LOCKED security > > It's been like this (110.newgrpperm) for 2hrs 16 mins > > /etc/periodic/security/110.neggrpperm has the following line, which I've > put > in quote marks (the quote marks are not present in the actual file) > > "n=$(find -sx $MP /dev/null \( ! -fstype local \) -prune -o -type f \" > > although I'm not certain, it suggests to me it's looking through the > entire disk, > not excluding anything. If this script is the culprit is easily tested, disable it and see. I guess you can stay a few days without checking for negative permissions. Also, $MP is defined in such a way to exclude filesystems that set "nosuid/noexec". I do happen to have those setup for my ccache directory so it is skipping that. With zfs it is especially easy to set these flags, and I think having noexec/nosuid/nodev for ccache cache is a good practice anyway. I have this for ccache FS (zfs): dumpster/ccache compression lz4 inherited from dumpster dumpster/ccache atime on local dumpster/ccache devices off local dumpster/ccache exec off local dumpster/ccache setuid off inherited from dumpster -- Guido Falsi From nobody Fri Oct 27 16:38:34 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH7dr25y0z4yRHY for ; Fri, 27 Oct 2023 16:38:40 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH7dq3vY6z3QLN for ; Fri, 27 Oct 2023 16:38:39 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=Hj5rtu3O; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=IDd+pRyT; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id F08145C01DA for ; Fri, 27 Oct 2023 12:38:38 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Fri, 27 Oct 2023 12:38:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698424718; x=1698511118; bh=Dm 5dIRB5HkLCe+x7bbyBxe4odAOHz1zlBMlvjhIwqk4=; b=Hj5rtu3OHoy2MQyvJP 1L/9QNp8k6y8/BuBQGQMe773i1MO/3aw75GFx50cstQ1aTuqUJAGuLoJ1OBYLgFI yBJ2xQM0ab6SuF2AZQMfdA/tUVQBXxY9B5htGpSGGWWwtq0DZ3lbzxg5QMxV2eDu kwNH+8LU2Q3+CNCYfItI6LUsS4IYbp7YTZem4KwyXBzd6ETRPYU7RCBU9HhgrJyy iAlL00V5Pr26Z+snGijy9Yw39ELe2wvMcsea6J+r2ez+O0d+X6mxqLZLTxuTnokz cfFttHzHLljn1kfF4dwBDeGITYIn1qznERD7tmzPxpqMHiRgXQz9/mEkJFvjDogg swWA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698424718; x=1698511118; bh=Dm5dIRB5HkLCe +x7bbyBxe4odAOHz1zlBMlvjhIwqk4=; b=IDd+pRyTgpqGx35Hcr7/l1Mb1Ws7x scI+zv/8ms6Me8iEubF9l+aHwQjtW0PDnxcuQCrg+K17xlPj1avL3MMJrFVLZjuo DVELmy/tNPRFKWL1xy/ryRB5Pvn2TKWu8sOkrC79pljAhoMINO+Bmorhh6/J3J8+ Jn6dYLQZhz2dS9gptyrn6vu66Hp2K1YhEGgysoQIx10cZ3CiMaPkn390DLhWvqju og8kbjCBATUvA+dTCQyOuGo7FlUBm+htDnjzCWrx03QAVzgjotXDSdYzbJLMNtw8 Lpc6J1ybDiNVMvOCmBJkQZKdflVz7/03d7sRWwjzgyb3aQo0oMxtURyTw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggddutdegucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepteetvdeihfegueevheefveduleelvdelvdffvdegheekhefhheetgfdtud fgkeejnecuffhomhgrihhnpegtohhrvgdrshhonecuvehluhhsthgvrhfuihiivgeptden ucfrrghrrghmpehmrghilhhfrhhomhepvhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 12:38:38 -0400 (EDT) Date: Fri, 27 Oct 2023 17:38:34 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <1210534753.8409.1698420229888@localhost> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <1210534753.8409.1698420229888@localhost> X-Spamd-Result: default: False [-4.68 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_HAM_SHORT(-0.98)[-0.980]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; BLOCKLISTDE_FAIL(0.00)[66.111.4.28:server fail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH7dq3vY6z3QLN X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 05:23:49PM +0200, Ronald Klop wrote: >Well. You could remove daily_clean_disks_enable="YES" from /etc/periodic.conf. >That saves you the "find". I have never used it before. The default is "off". Yes, I'll try that, but it's a very recent addition. The periodic daily problem is something thats been happening since even before the machine went from 13-stable to 14-stable. The addition of daily_clean_disks has made a bad problem worse, because rather than the problem happening for ~1 hour it happens for 3. periodic daily should be finished soon, it's moved onto /usr/local/etc/periodic/security/460.pkg-checksum edit: it finished, took exactly 3 hrs: # date && periodic daily && date Fri Oct 27 13:40:15 BST 2023 Fri Oct 27 16:40:14 BST 2023 >The list of files it checks for doesn't look very useful to me in 2023. >This does do a full find over *all* directories and files. *every day* ??? yeah that was sort of my reaction. I've not looked yet for a monthly-clean_disks where I could define an exclude pattern for things like ccache. That to me would be useful. The reason it was enabled was no more than a "sounds like a good idea" variable to enable on a machine that is presently used mainly for poudriere, which can tend to generate a lot of core consequential to pkg build failures etc. >If you have a lot of *.core files you are better of putting this in >sysctl.conf: kern.corefile=/var/tmp/%U.%N.%I.%P.core . >So you know where to look to delete them. >Actually my RPI3 has this in cron: @daily find /var/tmp/ -name "*.core" -mtime +7 -ls -delete . thanks for these, have implemented both >That is pretty heavy for you setup if you manage to run things in parallel as the RPI4 >has 4CPUs. PARALLEL_JOBS=1 TMPFS=ALL # with excludes for things like llvm & rust & gcc >It doesn't help to run daily_cleanup together. ;-) Before I started asking about the issue, what I was trying to address/work around was that sometimes (if for example the poudriere run went over 24 hrs) it would run into periodic daily. This caused problems before I made it worse by adding daily cleanup LOL Now to test periodic daily without daily cleanup... How long does yours take? -- From nobody Fri Oct 27 17:09:21 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SH8KL0FHcz4yT7H for ; Fri, 27 Oct 2023 17:09:26 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SH8KJ6MnMz3Tdc for ; Fri, 27 Oct 2023 17:09:24 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b="mAtW/05A"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=FoYe28Fe; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.26 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.nyi.internal (Postfix) with ESMTP id 09F4C5C0281 for ; Fri, 27 Oct 2023 13:09:24 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Fri, 27 Oct 2023 13:09:24 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698426564; x=1698512964; bh=nz sAhQdnfE2j2lp0AAEs2xBMDsTSfwMJrZeuybAZORI=; b=mAtW/05AMNrq4Pb6yL 1SoM7YWFODSJDvWRYyirWN+2YR2uS26OfAdhjL19aYeqyynA4bCrz5HGV/LIkjvN 5D1xYNzfCs6xHvYJhA0IUHvvBy9JCk6WmScYX59ZVHYySG65ufwRhOjQ7nzBC/mE h1uTxUiWqZqSxrs7cfORu6nCUP1HPwjVwr5/0wu8sZxpQXX6IX5pAf5QDXK32li5 k6wQ6HYvwgG46O7EvcIcCNyqkp2QYndaGwJ64VBW1R+N/SfrdULC5F7odxga6CDB n6egw8F3ApUdS7LYbwvQDGBhrqlEiyGENRn5CID5qpBe6VelSHJgm8gODWWjNhk5 kEbg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698426564; x=1698512964; bh=nzsAhQdnfE2j2 lp0AAEs2xBMDsTSfwMJrZeuybAZORI=; b=FoYe28FeXpQU4zd9eOumpDAtZCR57 1JiU+J/eiE2FNaDS8J6vGzpUBppoGaZxgUp2/zVkhXsXgTBGLVFtSNxYHsQ2V4R1 vrI1+I7lHBjIlLgWXH1wlix/QJi3e+W9whDFgkI0TghMM6mMQEqFvy/6OxkRrWfP uHJqMmqwwDUuJe3/JDrE+dF6hk9JjTP5Asr/3wAY84NNfXhSnP9pRQI/uQBh6zkH IoymSRFx0c2+u4XhNZ5j/GyzZm8svXZ3HtGbe6buIZAidKo5FXFz4GuZwYyWDFSN vJyBTng8x72AXLhUkfMfM1Fc9VMZzxAlnHAmBkl2ZIriddpYF3w1RpaDw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdduuddtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepkeeluddvlefhieelfefggffhffektdehleelgfdugfdvgeekjeejuddthe ehgfeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep vhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 13:09:23 -0400 (EDT) Date: Fri, 27 Oct 2023 18:09:21 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> X-Spamd-Result: default: False [-4.59 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.89)[-0.892]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.26:c]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.26:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.26:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; BLOCKLISTDE_FAIL(0.00)[66.111.4.26:server fail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SH8KJ6MnMz3Tdc X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: >If this script is the culprit is easily tested, disable it and see. I >guess you can stay a few days without checking for negative >permissions. I'm unsure how to disable that one script from periodic. but I think that what you recommend following it is better: >Also, $MP is defined in such a way to exclude filesystems that set >"nosuid/noexec". I do happen to have those setup for my ccache >directory so it is skipping that. doing this right now. thanks for the tip. BTW the same initial symptoms happening again even after disabling daily_clean. I'll retry periodic daily after setting nosuid/noexec/nodev on ccache dir Is there any benefit from setting a recordsize smaller than 128k there? -- From nobody Fri Oct 27 21:31:41 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHG842w9Bz4xxmR for ; Fri, 27 Oct 2023 21:31:48 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHG831LmVz4QT8 for ; Fri, 27 Oct 2023 21:31:47 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=BTLAsKex; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="BVf/QC3f"; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.29 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 379EE5C01FC for ; Fri, 27 Oct 2023 17:31:46 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Fri, 27 Oct 2023 17:31:46 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698442306; x=1698528706; bh=kY GpMgKYX6zV+JVrFjUkwiCAn+MCRmQNHc729unhNu4=; b=BTLAsKexsz/fusdBrq MZELdYWbuqYqd2OUNQ3X26h//B8f9JDdpl60gzQFFtfbbkaF8q0e9vcn82554ALQ 6xJrCSi4fCOQtv8ZpvSVHzMLxHL1IiU3rrhUC2BLzF7TBgIVIdMWqZjaRYwV4cUg 5IMPrTSILM6mx4qPMryp3JfIjAzEBk3d/4i4QFYuMtUqrVPqIbsf3zJSe33Hg4hl cjlbyXqg7Vkjod+Mktn08bqRL0NdsNC6AjmPfFJP4byQSk17KLcxjbjZSCdZqjZd qB6bbtbCaEZ+WOOvU8RaNHJfT4dMtaOwEwr7ep5q9FuNc8BKxybYTLh1CL0os0ka tqxQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698442306; x=1698528706; bh=kYGpMgKYX6zV+ JVrFjUkwiCAn+MCRmQNHc729unhNu4=; b=BVf/QC3f++Nudgkr15i3UWgaNFrhs cPwXw3lToyqvqI89fqN4WKs1nDOxw6r6SBnxgaa4jaajVl0lvKzrJ3HHGEP6w45Z nWJ/faq80KDR14hPYDfmDMu0JcJuBVDs5GGOI4C6YSsY6Uy5VmWWz6qHMfEvs7cB YHKZQDWGtEx0M5h9DzM+0BkKFKy1IR86U2SD34qLX3yzUD2V23k//rM1UGc4IxIm OUYa4CeG4272d5GIYGzZ2icGr8QyBBoFsgsOBivr6WKampN1U1+JcZcKADelyzT5 DvEPrWQPis+x+Ccf45DRwLscNafwvTMbcTcxREc9rYYBtvbredrgUdHog== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleeggdduiedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujgesthdtre dttddtvdenucfhrhhomhepvhhoihguuceovhhoihgusehfqdhmrdhfmheqnecuggftrfgr thhtvghrnhepkeeluddvlefhieelfefggffhffektdehleelgfdugfdvgeekjeejuddthe ehgfeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhep vhhoihgusehfqdhmrdhfmh X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 27 Oct 2023 17:31:45 -0400 (EDT) Date: Fri, 27 Oct 2023 22:31:41 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> X-Spamd-Result: default: False [-4.60 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.29]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SHG831LmVz4QT8 X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: >Also, $MP is defined in such a way to exclude filesystems that set >"nosuid/noexec". I do happen to have those setup for my ccache >directory so it is skipping that. > >With zfs it is especially easy to set these flags, and I think having >noexec/nosuid/nodev for ccache cache is a good practice anyway. > >I have this for ccache FS (zfs): > >dumpster/ccache compression lz4 >inherited from dumpster >dumpster/ccache atime on local >dumpster/ccache devices off local >dumpster/ccache exec off local >dumpster/ccache setuid off >inherited from dumpster This fixed the problem :D TYVM # date && periodic daily && date Fri Oct 27 20:00:03 BST 2023 Fri Oct 27 20:25:23 BST 2023 25 mins :D it used to be 3 hrs!!! thanks everyone -- From nobody Fri Oct 27 21:50:38 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHGbb6z7sz4y0G7 for ; Fri, 27 Oct 2023 21:52:11 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHGbb0DH4z4TFg for ; Fri, 27 Oct 2023 21:52:11 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="M N+OcW7"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SHGZy66JXz6ddR for ; Fri, 27 Oct 2023 23:51:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698443438; x= 1700257839; bh=njCd6N9yRZJAjbM2PZOH9Ybaav3Y+ERFhiJTrPr5JwI=; b=M N+OcW7Ag1cXc8O3NCq5zJCqU+xZLtj3It4cMDDoqGGct+6LIlHHNds1Idu776yex mMap4xw1Shj4lYdd3Zb4RzUQXQ0MvFgc1GD7duF/a3po/vnAb0kALmFvF1gmtrIJ bZcSMlwslEDfTwIsrKsFP21upPdhkL7mVj/xzloBdi9mWmu07kXgo2Ob64obwV/z OHav2IzvmluWrBu/q5FR7GQwlrL6eKhDAUgPASYxInFWBiwuQJ31UrUKEwwDBkx9 4rbar7wImMTTo9everlK3mTpMTeaYSzIzNbyUBGEWikjEsLgcFYDlA4envYMQPfv eBD5MClLrSjz5RvAOW8Xw== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id ETR0jvz9MTtK for ; Fri, 27 Oct 2023 23:50:38 +0200 (CEST) Message-ID: <39590600-cdec-4a36-807e-b81999481a7f@madpilot.net> Date: Fri, 27 Oct 2023 23:50:38 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_NONE(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHGbb0DH4z4TFg X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 27/10/23 19:09, void wrote: > On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: > >> If this script is the culprit is easily tested, disable it and see. I >> guess you can stay a few days without checking for negative permissions. > > I'm unsure how to disable that one script from periodic. but I think > that what you recommend following it is better: > >> Also, $MP is defined in such a way to exclude filesystems that set >> "nosuid/noexec". I do happen to have those setup for my ccache >> directory so it is skipping that. > > doing this right now. thanks for the tip. > > BTW the same initial symptoms happening again even after > disabling daily_clean. I'll retry periodic daily after > setting nosuid/noexec/nodev on ccache dir > > Is there any benefit from setting a recordsize smaller than 128k there? > As far as I know reducing blocksize helps only for filesystems used by software using a fixed record size, like some databases. I don't think it could be helpful for ccache, that saves files in variable size, some quite big. Also, reducing block size reduces the ability for filesystem level compression (I'm disabling compression in ccache, and letting ZFS do that part), but I don't know for sure if this is the best option actually. -- Guido Falsi From nobody Fri Oct 27 21:54:57 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHGfv4RNNz4y0vp for ; Fri, 27 Oct 2023 21:55:03 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHGft6Q2bz4VhJ for ; Fri, 27 Oct 2023 21:55:02 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="T aIYfwE"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SHGft0HKLz6ddR for ; Fri, 27 Oct 2023 23:55:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:references:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698443698; x= 1700258099; bh=X1NzridKmdBG3PEKrIzbwMDcGtVWOEhwP831AezeCMs=; b=T aIYfwE2HIaLbG9xuZP4puxXCE+gV5+fYJGj3xtP8hYPIagaQNmUfWeAYYXjwkTP1 qjIpFvEUdeGkOweHjtKYDCu5pCuRKeDgHfrQJru/KGlqeiij6crzLlPxg0tEvToq l1cKbnvKOCsPwD35UyAMsRcB7SR2Yrgdhzlw1G4ziORKokEjcJaJjb4HYNGsCqzb XYtlTiMksjphQggoXMA93gPA9o/zHrCKcy61aVsedIwvCeOcyxWvI2JldGoUdaxl eByuJo/qQhocg1gcLcv3UPHH9alWlGPnq2HYw1zGd7e+RuMODWnaLa72nT1yNvJW Vcptn2X/0f+li9z3ypHtg== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id 2TLbhJlrn6ob for ; Fri, 27 Oct 2023 23:54:58 +0200 (CEST) Message-ID: <0d3c4a36-9c40-44f1-baa1-afa960931604@madpilot.net> Date: Fri, 27 Oct 2023 23:54:57 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHGft6Q2bz4VhJ X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 27/10/23 19:09, void wrote: > On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: > >> If this script is the culprit is easily tested, disable it and see. I >> guess you can stay a few days without checking for negative permissions. > > I'm unsure how to disable that one script from periodic. but I think > that what you recommend following it is better: > >> Also, $MP is defined in such a way to exclude filesystems that set >> "nosuid/noexec". I do happen to have those setup for my ccache >> directory so it is skipping that. > > doing this right now. thanks for the tip. > > BTW the same initial symptoms happening again even after > disabling daily_clean. I'll retry periodic daily after > setting nosuid/noexec/nodev on ccache dir > > Is there any benefit from setting a recordsize smaller than 128k there? > You can disable it by adding: security_status_neggrpperm_enable="NO" to /etc/periodic.conf. You can also add: security_status_neggrpperm_period="weekly" for example to make it run weekly. The dafaults that can be overridden in periodic.conf are all listed in /etc/defaults/periodic.conf -- Guido Falsi From nobody Fri Oct 27 23:36:37 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHJw744XMz4y5wb; Fri, 27 Oct 2023 23:36:39 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHJw73mwhz4gFX; Fri, 27 Oct 2023 23:36:39 +0000 (UTC) (envelope-from gjb@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698449799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=xScyQQpLkicJLhWavFfNhaEfR0qNjPoKy5+MVqqRCNo=; b=m9PA71xHIHX6GpjgcVS5AjH67dJe649la2ldI4aykw9IIDOlftR/dlGa8N8/BYRFUPj/Yn inkJfumn8CD0SYRjFhlK2Wa5iQPZawdj1t0vmyFrk2ZgDsD18xrKZw46dQXCFa+EepdUu7 sl7mmR2ofdNYkrSsz7Z78rSQf0rULbJekEvB1hb3ijgUL87Ra3jbLFOwn9kGlzfAF/bITz aaN51AKncsTfwP6vBZaU7TR+W7XIx3LlIhljwKznZb+9dMy1S7I3afvU423efqJGncx1t8 26gJ3tQOSSktfy2JYyIfQPBDJhCxXf6MNM/EYDMKpm7tMcmKFT2svwyMKPiOsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1698449799; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type; bh=xScyQQpLkicJLhWavFfNhaEfR0qNjPoKy5+MVqqRCNo=; b=RIqae7wTjFemY8Be7dAjzIdy2C9zDX+oylclT8/T362dmNt9fA61I4vCjutPanahiGUPO+ oR/HlbNtCTEU50HxellhC0gUPzIPTad8gnkIWQowW76JGC4RFLAq1aI8c5Bmj2OCHU8QZn gN5fXmQOtWAzW9BQ1KgjR2xzDIpO0V6l8xOP5C5diskzpPGDGQStUsQaMsg13PWli7BauQ i+YTN2Cn3NegkLKyrUilDdrBvq7emfhr2fICMg11AhV8KFh0ZtSlS+mkjldhK2BorvxknM ScaG+flWl4ozmxbFZXyzO5l+w2Jlqu0tpfvF43NVnkWRd256mWxWWlTrwiWseg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1698449799; a=rsa-sha256; cv=none; b=Gao3qjtOj3sq1BgphEpcOkGnoPR/hOvK++zzMGpnjvhlX0w8dRVVtdq9d0RY9PU4M6AMIY ZyKRX1b6yok2NI4iezGhbzMkN0tqd1wz3T/FKmu9F+0SZoYFw8CPARja8Yr7T/fDiLne+S OGiEETlPV2oi9RWgUIa3uMkt1+xT+u1p0qUPt1Vn7CEvtTK3Ddqzqx/lMBgDpxgKVW4FoA oil1tM6N1IxchMKy3V4P892O92n2aVaVQ8G3YzZK3mz6yWmW6FXeim98vjbOI+EywRoqfu /MRgRdGjjQTuuDDHUTPZLXQxBBpWpQy5S0aDQjK2paChoxR+X5wmoTJWPhAC+A== Received: from FreeBSD.org (freefall.freebsd.org [IPv6:2610:1c1:1:6074::16:84]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 2CE5B13FFB; Fri, 27 Oct 2023 23:36:39 +0000 (UTC) (envelope-from gjb@freebsd.org) Date: Fri, 27 Oct 2023 23:36:37 +0000 From: Glen Barber To: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org Cc: FreeBSD Release Engineering Team Subject: FreeBSD 14.0-RC3 Now Available Message-ID: <20231027233637.GA1307@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; x-action=pgp-signed -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 The third RC build of the 14.0-RELEASE release cycle is now available. Installation images are available for: o 14.0-RC3 amd64 GENERIC o 14.0-RC3 i386 GENERIC o 14.0-RC3 powerpc GENERIC o 14.0-RC3 powerpc64 GENERIC64 o 14.0-RC3 powerpc64le GENERIC64LE o 14.0-RC3 powerpcspe MPC85XXSPE o 14.0-RC3 armv7 GENERICSD o 14.0-RC3 aarch64 GENERIC o 14.0-RC3 aarch64 RPI o 14.0-RC3 aarch64 PINE64 o 14.0-RC3 aarch64 PINE64-LTS o 14.0-RC3 aarch64 PINEBOOK o 14.0-RC3 aarch64 ROCK64 o 14.0-RC3 aarch64 ROCKPRO64 o 14.0-RC3 riscv64 GENERIC o 14.0-RC3 riscv64 GENERICSD Note regarding arm SD card images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system. Installer images and memory stick images are available here: https://download.freebsd.org/releases/ISO-IMAGES/14.0/ The image checksums follow at the end of this e-mail. If you notice problems you can report them through the Bugzilla PR system or on the -stable mailing list. If you would like to use Git to do a source based update of an existing system, use the "releng/14.0" branch. A summary of changes since 14.0-RC2 includes: o Test suite fixes. o Support for IUTF8 in (s)tty has been added. o Support for Realtek Killer E2600 has been added. o Several kernel TLS updates have been added. o OpenSSL has been updated to version 3.0.12. o And other miscellaneous bug fixes and updates. A list of changes since 13.2-RELEASE is available in the releng/14.0 release notes: https://www.freebsd.org/releases/14.0R/relnotes/ Please note, the release notes page is not yet complete, and will be updated on an ongoing basis as the 14.0-RELEASE cycle progresses. === Virtual Machine Disk Images === VM disk images are available for the amd64, i386, and aarch64 architectures. Disk images may be downloaded from the following URL (or any of the FreeBSD download mirrors): https://download.freebsd.org/releases/VM-IMAGES/14.0-RC3/ BASIC-CI images can be found at: https://download.freebsd.org/releases/CI-IMAGES/14.0-RC3/ The partition layout is: ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) The disk images are available in QCOW2, VHD, VMDK, and raw disk image formats. The image download size is approximately 135 MB and 165 MB respectively (amd64/i386), decompressing to a 21 GB sparse image. Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI loader file is needed for qemu-system-aarch64 to be able to boot the virtual machine images. See this page for more information: https://wiki.freebsd.org/arm64/QEMU To boot the VM image, run: % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ -drive if=none,file=VMDISK,id=hd0 \ -device virtio-blk-device,drive=hd0 \ -device virtio-net-device,netdev=net0 \ -netdev user,id=net0 Be sure to replace "VMDISK" with the path to the virtual machine image. === Amazon EC2 AMI Images === FreeBSD/amd64 EC2 AMI IDs can be retrieved from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/amd64/base/ufs/14.0/RC3 /aws/service/freebsd/amd64/base/zfs/14.0/RC3 /aws/service/freebsd/amd64/cloud-init/ufs/14.0/RC3 /aws/service/freebsd/amd64/cloud-init/zfs/14.0/RC3 FreeBSD/aarch64 EC2 AMI IDs can be retrieved from the Systems Manager Parameter Store in each region using the keys: /aws/service/freebsd/aarch64/base/ufs/14.0/RC3 /aws/service/freebsd/aarch64/base/zfs/14.0/RC3 /aws/service/freebsd/aarch64/cloud-init/ufs/14.0/RC3 /aws/service/freebsd/aarch64/cloud-init/zfs/14.0/RC3 === Vagrant Images === FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can be installed by running: % vagrant init freebsd/FreeBSD-14.0-RC3 % vagrant up === Upgrading === The freebsd-update(8) utility supports binary upgrades of amd64, i386, and aarch64 systems running earlier FreeBSD releases. Systems running earlier FreeBSD releases can upgrade as follows: # freebsd-update upgrade -r 14.0-RC3 During this process, freebsd-update(8) may ask the user to help by merging some configuration files or by confirming that the automatically performed merging was done correctly. # freebsd-update install The system must be rebooted with the newly installed kernel before continuing. # shutdown -r now After rebooting, freebsd-update needs to be run again to install the new userland components: # freebsd-update install It is recommended to rebuild and install all applications if possible, especially if upgrading from an earlier FreeBSD release, for example, FreeBSD 12.x. Alternatively, the user can install misc/compat12x and other compatibility libraries, afterwards the system must be rebooted into the new userland: # shutdown -r now Finally, after rebooting, freebsd-update needs to be run again to remove stale files: # freebsd-update install == ISO CHECKSUMS == o 14.0-RC3 amd64 GENERIC: SHA512 (FreeBSD-14.0-RC3-amd64-bootonly.iso) = aa7829a910a8d06fb427ae2b0618e62458b61983d1bc3f363695040384a412d3e57ac242627f86879f987d59650f1be6cb7909973183fd5d56deeb031fb71ee4 SHA512 (FreeBSD-14.0-RC3-amd64-bootonly.iso.xz) = b2b0b7b188261b42123068ce7662cc3b2a033e31c7ea11b7c50830d1f6a7148ec75f1abae0e0beb7cf407b484926df207beba92ba81c7e3d9e4c54d0ee41b231 SHA512 (FreeBSD-14.0-RC3-amd64-disc1.iso) = 94cca74317734276910bfa9f1909929a5633075df2dee364389b89d439ed893a937af85d8389074bcc72f429e33fd48ab0d801e1b33bc2748594093893e523e1 SHA512 (FreeBSD-14.0-RC3-amd64-disc1.iso.xz) = ba7196e60c39ea65e18368e1680e613b9b178ee98ebcf6c7b19d6ce0cb11a373fe56fc8bc4ccad4667323eea8d298771de2316fe1ed349421ba0c42c012e7f01 SHA512 (FreeBSD-14.0-RC3-amd64-dvd1.iso) = 0d6359880e62c8c5745e1e50bcd53ca11613803cfba7ce0a0ed0273358b2036dcbe99342ae060c1dfccb11852ea75208f6169c116c003a55a144aca996af2cc1 SHA512 (FreeBSD-14.0-RC3-amd64-dvd1.iso.xz) = f8aad9ed21dd78648bce10af9e67af287008b9d0a5b97be09bcd3561ce22070f1357572f8d1981519b8326097aa65fbd87de89917001ebcac41e7bb6a3837c89 SHA512 (FreeBSD-14.0-RC3-amd64-memstick.img) = 7ad404ffa68859fa5fbc20e9aa5e13d70b5934a2d11b4c23f9c7d6ca6816e820d7f6aa19cec9731b5441dc7c223c302d85437ad5437a1a1513f151dfec58b8c9 SHA512 (FreeBSD-14.0-RC3-amd64-memstick.img.xz) = 617938573596074a8aadea6faa7bb70ee0faa298999671381429f58cca489879356dd299e32bbc3a060edca2cac283bff5d94e20294c51541198f3a27703ed48 SHA512 (FreeBSD-14.0-RC3-amd64-mini-memstick.img) = a5e81c1446219c4a764190822aab7b63b5240f6677f7d92a9a25acf919838b330723e8ef4eff2f626e58140a9dc08ab90dea16705d41ae6120f96892a44ad763 SHA512 (FreeBSD-14.0-RC3-amd64-mini-memstick.img.xz) = 6cdbc788876d0a430b474dfcd01a79b169103402af474ac0bea188f11969e63b38c7b7788419db939a90f36016e560823149b17c23b42a13abe38521e83fe378 SHA256 (FreeBSD-14.0-RC3-amd64-bootonly.iso) = 7a617feb5c11e9d0b406a636b1d0b73d8be2802d89496aa41e64c7dc7c58b3d9 SHA256 (FreeBSD-14.0-RC3-amd64-bootonly.iso.xz) = 5922b1308c24dd1b518d8bdf80518995060ad8a34deb4485025c7202d7737404 SHA256 (FreeBSD-14.0-RC3-amd64-disc1.iso) = bffdc0dec7e2b4c65acff0d54bf31592d85fd2ac4443e5e2fa724cc968acd2b8 SHA256 (FreeBSD-14.0-RC3-amd64-disc1.iso.xz) = cb37ce2cdcb0cbe6ad287eac941cfc02dc240f0d30ea03ad86a1ae1d4296c5b1 SHA256 (FreeBSD-14.0-RC3-amd64-dvd1.iso) = 5ca6dffcd849a612a08757c2df0203a074a26c3b655484c130bccb3d864dfce6 SHA256 (FreeBSD-14.0-RC3-amd64-dvd1.iso.xz) = 205f750e2bc81e9d2b523e2a11314ff6ca114c3de60c4777430d7cd9e4428082 SHA256 (FreeBSD-14.0-RC3-amd64-memstick.img) = 2e70a0e27d1c276659093fc567c4e6ee3caa7102542175ea565f78b4b749b716 SHA256 (FreeBSD-14.0-RC3-amd64-memstick.img.xz) = 5f40150201797e13da01602b548338afb3c25693d1cc6366c87262371625bffb SHA256 (FreeBSD-14.0-RC3-amd64-mini-memstick.img) = 5e4440252f754502b2e4bfa6a0015ea6d88f4478f6ee02e2722ec80a4e0f9661 SHA256 (FreeBSD-14.0-RC3-amd64-mini-memstick.img.xz) = fb03e03aa573d2e62d513d6d492d93039b81827a069f7bee5f8c28fd10b0b54d o 14.0-RC3 i386 GENERIC: SHA512 (FreeBSD-14.0-RC3-i386-bootonly.iso) = aedb8c8fcd552690738dfb6f00cac331a824faf1bb9143fc79b911efe537e39fc4b697d34c556472a79fe99fffb845388badbd35832870200de9bc3f2c913392 SHA512 (FreeBSD-14.0-RC3-i386-bootonly.iso.xz) = 07d74868768f6ed1d9271cde2658b3785993dadccf591f96bc8ca3d6492aeec42f8fef372d6232707a2aeff522ce5bdb9f295248aea19068cefdb024f6e29c02 SHA512 (FreeBSD-14.0-RC3-i386-disc1.iso) = 5a8646b24d0f4ecdd7a3dfa639ff232d2e6ba786802d89f14909dee6cd5f40caace0814e71b300c0ed12fbfd5900062642c2058f249d5c6d23c69b77698dc297 SHA512 (FreeBSD-14.0-RC3-i386-disc1.iso.xz) = a3e09da0cb15064b14be77f0a929afcdd3f9596a8f16217898990f64566e40758f70919ce135a4751a72ff1104b8b3612a30790bd67be1816174703e75460a41 SHA512 (FreeBSD-14.0-RC3-i386-dvd1.iso) = daa7ff12255d61515215bdea9762ba0078b9308b964eef7c1de01e5b53a8656dacada954ac261b9b8ec693a80f808dc57c1659a68cec92fbbdc31cae25218b9d SHA512 (FreeBSD-14.0-RC3-i386-dvd1.iso.xz) = e118bdc41bb99ea92734045f2e0f94a4e1fe673b5539ae0ed62be4c21dea39c5a6abebf49acd636076f1184c4b0757e0c02ab5a6d22d1bf47c0814b96391c7bc SHA512 (FreeBSD-14.0-RC3-i386-memstick.img) = baedb2287b53cbd5484fe3d2996a103d7cbc6d0feaa4784bb0adc75eff70e0445d233a741685f6cf41c42560ea683bb69c39adb77d945862d1ddcb671b64df48 SHA512 (FreeBSD-14.0-RC3-i386-memstick.img.xz) = 8a9ad484acf00ff51eaf276155624c5c1cdff1e299fea6360129f10ddd153ad4af7c5fd66610dcd2aecbedfdc58a76e02314067eb8fbb0403834b14ac7506c94 SHA512 (FreeBSD-14.0-RC3-i386-mini-memstick.img) = 6de6d1dc4722b9a3db95607fa4a8cfeb7a2abf8c99d0d90609a1f8e98a41c7c8db59f91b60545bc1f84e3af1946b2b8dcc120e63ab5845f054c4f0a8c9d178f8 SHA512 (FreeBSD-14.0-RC3-i386-mini-memstick.img.xz) = 46dd633193d1aeb8c6e053c34ff1ac7f46ad8ca83145837535b16c4351bdf70fbec30c0df142e859bda9ccb856f9a7fa7df1b2ae532c01faf7da4b26e5d403de SHA256 (FreeBSD-14.0-RC3-i386-bootonly.iso) = e67d55f4ea8320cf800a9933c9db611bf9e788821e457a4b04f9f92bb16b169f SHA256 (FreeBSD-14.0-RC3-i386-bootonly.iso.xz) = cce88097e175ac90b52575da822e04413ab4f33348cb165a97976c818c327363 SHA256 (FreeBSD-14.0-RC3-i386-disc1.iso) = 416c228d312587d802730868424e6366fca5e3a3f2fca4495e559e1367101a13 SHA256 (FreeBSD-14.0-RC3-i386-disc1.iso.xz) = 2508b15297d822b4a1dd564771289dff42fd576db951c614677627f0fd8193a0 SHA256 (FreeBSD-14.0-RC3-i386-dvd1.iso) = 8a1d5547beebb7e313e1b3e7fa621d5adc9dc6b4cb6d785fac66f519416ca915 SHA256 (FreeBSD-14.0-RC3-i386-dvd1.iso.xz) = f5ebb742c2748baccfe9202c5430184eea61cd4ca66cf197f95e756daa902a31 SHA256 (FreeBSD-14.0-RC3-i386-memstick.img) = b9fa936e2fec86e0e41a7a0b4c3abf16965f4e27c7b8958617eb2fec1686308c SHA256 (FreeBSD-14.0-RC3-i386-memstick.img.xz) = ff62879e94dd61ee8df49af84c19a288f01288924c8215a692e834b850c4cc4d SHA256 (FreeBSD-14.0-RC3-i386-mini-memstick.img) = be20a0001decd69fbb32f4d1cce16fc7f871a3bff4fe8142f98ea75d21f3122f SHA256 (FreeBSD-14.0-RC3-i386-mini-memstick.img.xz) = e92edc84f6d57627284dd794dcfedec276e88b4c49f9757232d5a060779ebcb5 o 14.0-RC3 powerpc GENERIC: SHA512 (FreeBSD-14.0-RC3-powerpc-bootonly.iso) = 2f6edaf8dd31dfdb88d1dc28c73c43b4c2e1d64f51bd36b8559ab0932d08966410a6500bde354ed545e45a84ae89243229dd15bf23fc09d2609fd3243ab32c66 SHA512 (FreeBSD-14.0-RC3-powerpc-bootonly.iso.xz) = edaccd645db387ef793ff5a9e3d5cad50b9feea4d6ce8afde52220e12f45c42305b2d8ce22ef74a2d445e330433793d8e05c640be2be8ef4b767657672972fcf SHA512 (FreeBSD-14.0-RC3-powerpc-disc1.iso) = 597a0251b6f65ce69a3d81c11af373763d59b34f86ca7f36806f08f14d56c2a307dcdb1b97bf8e2d0abbccafe04150d869ab7dc7af1d3d7af100cd95e2176784 SHA512 (FreeBSD-14.0-RC3-powerpc-disc1.iso.xz) = ac8d136b3f067882892bd13e849a5146af8addb1f1289483d8fdf0775a9d6ccfb5a8d593bf2d3e82fda0670fa83f02e1767df33d148b06bf30b872228c637200 SHA256 (FreeBSD-14.0-RC3-powerpc-bootonly.iso) = f6b8acd4665ea7fb95ca90af927ef7671df0efe7f1f20850664af8b2e0bd0080 SHA256 (FreeBSD-14.0-RC3-powerpc-bootonly.iso.xz) = 5e015cec60b41789202bfe04ee75bade31479f1c525de069dd116fd41e6035fb SHA256 (FreeBSD-14.0-RC3-powerpc-disc1.iso) = bcf7dd8467f02ffde799f30bb9ab26128fd0cbf1225ca013c6abfc28ba6a0fbf SHA256 (FreeBSD-14.0-RC3-powerpc-disc1.iso.xz) = 4056b215255fe8ca0c0eacf20b61d9dda3cff76cf5d42812ab2eb78142b19771 o 14.0-RC3 powerpc64 GENERIC64: SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64-bootonly.iso) = 6a11c6f9e4f45590510b3d4c172223b87c980959062d49a79644a792049e5209e387e0de20f6c2fc2e374b438eb6f1e32544af25f938f1d974020219f6175b68 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64-bootonly.iso.xz) = a8bf5acf27941f78f544cee28bc59152c110011fdac6329ff2ce3f6b683ac1db0591c468497e6b457332eb44799379ba40ce0f4ac0609544ec34c9cf7e266361 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64-disc1.iso) = eb6cdbd1fd88d71fb6d5d166725840cd647c3cadad5728ad2e209fb5227e81e434435566911e912d737017171e4c3aeda85aad252fa155faa7b13081e9746785 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64-disc1.iso.xz) = ad4dda3d7f90d714cacfdfb9ee641fbfc1e30bf968e12196d0b8a7a325027a4a2d5feccfcfa08d689d5692e9335643aef4a13bd53d8397c5abc25a7c0ec62148 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64-bootonly.iso) = f3cf91018408f1eea62fbd22d9708f402fe72e772b217eb7cc41d89617a6caef SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64-bootonly.iso.xz) = 065c3bfedf313127fc30b1f677b48436560b755d99718b05a8b8a37871054078 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64-disc1.iso) = 5f67f521ae7a7dedb66a8721d363739cf0cedc7d0a7da1cac88b233c9113a000 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64-disc1.iso.xz) = 1c92e7f392df5053810b7436a0ceeacf467dfc901464d0302413c511a62705da o 14.0-RC3 powerpc64le GENERIC64LE: SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64le-bootonly.iso) = 96642f843d9fd4a63d812269b24f085ff73b69f77bdbd48a06c7790f2fd87573b4edb7f0d969627a6f997ca2fd8870c28ddacdf47cdca13a3c59e9b5e3f0bfd4 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64le-bootonly.iso.xz) = 0525c26b88a421f49a4830804993209771fb1064e2e80f39df0d6e47cc87eaa15fa28003baa49bc72f5df1b8205f6459b23ed277cb74881694feebaaa6555b70 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64le-disc1.iso) = 4c9447178e4c4419536638047c15dd8d169972f7241e0f8683d7f1ec9ad7eb27da288d2e86157391227422617971f7efca3c05903ea012e6654d5d4c0acda9bf SHA512 (FreeBSD-14.0-RC3-powerpc-powerpc64le-disc1.iso.xz) = 83019f7c60c3c28371f2c201235f582a95bae36d1a652c7b8a9335b3ce54808c2af05e0ae6eb39e52296332b4bc0647de6134e8ca84b3e6acfd3fa2d6b555181 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64le-bootonly.iso) = 064ba96e9295112b8f6864d3a38d691a9a28fa21166118f8701cef6613957715 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64le-bootonly.iso.xz) = 1e9f9c5571a9b297c2bf68a97066fd2c9d4d3db3a744edb38999a240c2924de9 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64le-disc1.iso) = fff8ed0d98f995ea788dad179fcf06e440ef1a566f4c1be38ea410719cf0542a SHA256 (FreeBSD-14.0-RC3-powerpc-powerpc64le-disc1.iso.xz) = 88b65bbeb2f4313d62dfec94ef91835166417856ed13c94a7f5637f9cff94279 o 14.0-RC3 powerpcspe MPC85XXSPE: SHA512 (FreeBSD-14.0-RC3-powerpc-powerpcspe-bootonly.iso) = 52abb77315970c0267f3865a19fd34ef7cbd0c8acbb662bab1c398d1b4e0858772a4e7629095469ffc51c60f81f936676ee6399a5237395d4cb1f9533b1e6c20 SHA512 (FreeBSD-14.0-RC3-powerpc-powerpcspe-bootonly.iso.xz) = 7edbc7162772e57e67d6250134dabd244b1ef080025a5bf363d8b0c3c65d490dedaa8556d07e440de4db9aabc3ec75594f32073a3486b007841060dcf8307efb SHA512 (FreeBSD-14.0-RC3-powerpc-powerpcspe-disc1.iso) = 6931e192518b3c00eeb8208c087070fe31d5213dcda359f7a8c4aedff82c64bdc9eae40171a4ed3d3e6893dafa3406664bfa79f2afbbc3148161f6c1fd4df8ab SHA512 (FreeBSD-14.0-RC3-powerpc-powerpcspe-disc1.iso.xz) = 143ff203564f6cb2c80a60f5765013d04c60027bbad99d60f927e977885b6bbe67aaa1ee79e6ab1f11837c296e1aa4c9e7069b6b18da6e461ec5f2579ab82479 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpcspe-bootonly.iso) = 4573754f2d16a19696cd1ee8edb5f52854c5a2ad322eb68b1147cebf6ea44e43 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpcspe-bootonly.iso.xz) = d08fcbe00c3bbfdb384c096689009fe0e68a3551566b53211d1a736170ddd768 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpcspe-disc1.iso) = 48fb4f09a6f3a3816d667ca6acc2f6df4e8f2efcddd4a3cdd2b0d692087030e9 SHA256 (FreeBSD-14.0-RC3-powerpc-powerpcspe-disc1.iso.xz) = 09158da4afd2b48d7e35fbbf99da5463ff910844b5956cfd4aefbb25ea5ce0ef o 14.0-RC3 armv7 GENERICSD: SHA512 (FreeBSD-14.0-RC3-arm-armv7-GENERICSD.img.xz) = 248c4e2e32ba347c7287afc7f8d1dcb383de1236a61fe84759af873a051f98dff1f4bb134a55df598f5be27141113b16485e59ace94cd013561202e72a5a41dd SHA256 (FreeBSD-14.0-RC3-arm-armv7-GENERICSD.img.xz) = 0a0b599ffaac422003077a3b09da699529a4f0fb328962e833899fc76c5c79fe o 14.0-RC3 aarch64 GENERIC: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-bootonly.iso) = 249c2bba6c3761a7581fa36abfe3c448640ef0c0f8e01cb183822fe533957215ec0f9a5ee8d2f403069e64449b8bfbca6c690638da9c5343500aa7db19cb01f4 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-bootonly.iso.xz) = 1acf5435bc8ef985102ff856555de3187463cd324d6c4793683358580256da8cc1792aa31833bafe2b7930e4d7d28543a57afbf692731dcd3bda1bc43aa49266 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-disc1.iso) = f86de5245847ae25ae762b43d41d41df4bb855303f5f17c8402f7a110179cfa5aa6c9a70fa0bf22fcd15d3f7d63d11365e9a0079799e7b2612dba862204b9c4f SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-disc1.iso.xz) = d7ea212836c2b703f1758d689edb3084b6fd67c9eb8d4f3c73f6b2690f85e0cb051bdbdbadcfece50cf269112073e0614384d899756a5d24f220eb510f0a5ef6 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-dvd1.iso) = 36fc3863c11e3e61b63532fe806d05f3270b1717f59518dcf41241a455cbcc35a62308965dbfdb657cdf5c83ceba9a69bd2ebed5823a1c96d3e2bac1aae2b2fd SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-dvd1.iso.xz) = 77012247681d49767bc7216a39383a5b828b3670690bf01824b13797263e7660eb9be3b2e2df1d170c0bc1fbbdaf17dfeb396d1d186576879bf1458efde9cf4c SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-memstick.img) = dc07c62dbc09f515f1f85d0c29775bff6c6d05aca5d4b2182b00dff06db2bbce4322d8d8b53d3da5516e3fe484fa01245a62935154af1c761baf8fe79f038e24 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-memstick.img.xz) = b5dbdb40ce6c2fbbe4f3a9f2be201426b53d97b7b40ae040ba9c170cb67f991700f9492a2b31e5e941db71d5834741e109f617e1db9bb6d5d3dcc9ce56e54df8 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-mini-memstick.img) = 2c7c327559293b64eea34eeac74bb72d1b67ae59549583c7d5156a38a8b8a16a58b08cb5bdb4cfa9d7654b4473bbb50664b0f0cc3c4c175459f06da3be10cf57 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-mini-memstick.img.xz) = 4ae5a6cbd003a0a87835456fa062bd2fae7221069a38450dd8e254d267726f3904ab01a126aec675e602c508d650d6374386616101a5301464312bada43a7dae SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-bootonly.iso) = 7242265c90ba798b2dff9e1b1309a754cedc481291ebeecb493bf1a1341d755d SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-bootonly.iso.xz) = ee75296f623ebc90c3dc26283713c95699514b550020e0f48d0874d65aad1fa6 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-disc1.iso) = a3529368ea3ffb060ea5e206c1113ef948c645a7952399b94e7f4afb941ac42b SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-disc1.iso.xz) = f608baa85d2a15bae0fdcb15f77c0a680056b7b774f6b2bb30f31fe7f2af3aba SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-dvd1.iso) = 754d393c3e7b546f863c77597be61162a4b53aade8c9986df6a10cc5063d6c14 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-dvd1.iso.xz) = f231f1221de1c7d23dff25d93a65486030163f809dfb47f014013ba81c48542e SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-memstick.img) = ea1fea45890080710532ceba9dbefcfc7cb721f9575a93657595528c90ab5497 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-memstick.img.xz) = d17077f3178aca60a4fbee3d55bfab46b40668211e71b782cc99fd132693f4eb SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-mini-memstick.img) = fd561921d3b4cc9d364ac063527ee35ea2e5e2f808e55b85d825bed8b0c1abfa SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-mini-memstick.img.xz) = 88ed3e8e8e55bb202b48b76f295d8f28143aa4b55f7e3a63dc36806f833d2e70 o 14.0-RC3 aarch64 RPI: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-RPI.img.xz) = 91919b6f2d71d1eb4e9c011b53249531d1503ed9e140d14dc4599c0e43c8f998796153135241d4a4f29ccd5277b032e92eacce7dfc6ab17ad09ea101e3168c06 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-RPI.img.xz) = b8c1898111d773817af4205baab974210197a3858d3911824c9f8a8908001417 o 14.0-RC3 aarch64 PINE64: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-PINE64.img.xz) = 84a6a96057e3d181d90007b4632730500c143b54095db060c9091e7c000a9ce2121c74d4777dd86e77add98602839c545778e51966914e966d56218614426e83 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-PINE64.img.xz) = ee529055d79e70064696e6c7b7f739dadd0bc368f148aad8e7a4510ffaf25f3a o 14.0-RC3 aarch64 PINE64-LTS: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-PINE64-LTS.img.xz) = 41b7f06e4295d9a689bf1a487053f1f29ad1458180febeafffefeffc7a863fc9542c24e2d6f0be6ac60084bf7d660332bf6a4f2256d50a424c64fffb50ca2536 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-PINE64-LTS.img.xz) = 708e361f44a370c63331be2f5fc98977219fc6bf901d8c9b32a1eacc635e43e1 o 14.0-RC3 aarch64 PINEBOOK: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-PINEBOOK.img.xz) = d6b9c1f801057d802db95d288c6d9ecc97717a829cf732753aea813472778fb82e9f25a044d6457e571b092ed63e982245dbba925132c3da9f32b9dc051ce8f1 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-PINEBOOK.img.xz) = 9f720afff7c0ba0d5a64cc7b5bba3466e3dbf03c890bd30e52ca15c351af93ab o 14.0-RC3 aarch64 ROCK64: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ROCK64.img.xz) = ae2624d1205bcaeda14b8bf48c35f203132b7fbbccb05ea9e47da077db3743379ec40e85da2210acc025ab1153b9c0f578490959bb2d1e90dbc5b6595731f51b SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ROCK64.img.xz) = 3578dab21ab8d660ac059f9d0b1f652243d730e3a43d5fc00939f7dd696d9b96 o 14.0-RC3 aarch64 ROCKPRO64: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ROCKPRO64.img.xz) = a64e793678d8e0c78908dace3a49b2a9031ec6c580f3ba7ae58612eb7e86142536de9b7de1469daa5f5092d2de3848629eb7f46e8e1a6324f4c4939a6cff0734 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ROCKPRO64.img.xz) = 54e1f4df68958fce829d64c2247454ff220442060e7bd8256136d2575c00f6ed o 14.0-RC3 riscv64 GENERIC: SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-bootonly.iso) = be28d9b471429cd9787bae58684f1f51c714dec1bf94e1db0b725b9f85051aa5ec43abe387321341b104837a8a2e2e11d4a87deb0ede93dac61481cb79b0432a SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-bootonly.iso.xz) = 8eb44c635504fb6612daf98b2de42108d8a3343f5699b165615cfb0cc70eda111d39333de7eabee145faa71293e71782e61b395c23da6c5abf8bc5e49a6cd7f3 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-disc1.iso) = 90189689f21a54d7c198b5e51d3a001af928948c506fe0c146c60b280f1558c630d136c9b97cdee5fbfb9ca6d571cc3adc8674ad18a90fd88a06d55e68e1d11a SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-disc1.iso.xz) = 294afac64648a39fc3f9885d6fc8d904b2df34909443bce5f4b0b88aa3463cd222e16f717ed64ea9443498607381725583342f5f7ac3d8a575f350ab8dc335c5 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-memstick.img) = d2a6078677f6c4edec9926dca96f528550d7b4524c1890e7eb8b4c64baab3795c720876943a0e07392d6171f8411b8d3c88b3d8ffaf49c4224b52bf63da80b42 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-memstick.img.xz) = 3a879cffde3ffa6e35b9ba183a97ee008c97849ea7670072b0be80854c9dc1cf5ea47c9300bef2e41222ad7513075bbb96156e33a34264966421eb7222d43e68 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-mini-memstick.img) = 8de68b59d975dd591168ac8b5dcc392b8ba9493d468dd8de4b94d86c38b2566231cf0dae09f22f7eb16e2704966fd9d05ea41188f39d4cc147027166cc9eccd1 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-mini-memstick.img.xz) = 205806e57d7d816e9b8ac6d56bbfc5380621e6374cbedc357c31c95a3c74a6d3e18218605cdb9efa4cc3a679f6320252cd1aa31631629f72806aad766ee4d311 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-bootonly.iso) = 1d79b7027431d009d13115f9b314f10613abecad2fbf4afb09a0773b36a1fd9a SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-bootonly.iso.xz) = 29e4a8ecfbc960a86724671359f37a04baa29fa61188641c132c9dd6c33bdb73 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-disc1.iso) = a8d174b2c0fca114d4f33822da266c9dd5a21dd667dcc3d54a28167b4d605ca0 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-disc1.iso.xz) = 656465557f59818f0490c58cf1d4ad9845ea086ba84c65e467c0a3364e0a2321 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-memstick.img) = 91f3a245357d0b897a5096062b650550fb001ac6dd7c580da2c36eddd359a194 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-memstick.img.xz) = caba6b155a7496289a2602711409cc77091657ed538962f896358c938bbdbfed SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-mini-memstick.img) = a54824985f6053f39278e695749d503bd8f9113078762c169c9d37ec23637da8 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-mini-memstick.img.xz) = 2abc9e10ad8e3c723d934d9584f9e2e8afd8bd5559793822adcac62a849796fe o 14.0-RC3 riscv64 GENERICSD: SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-GENERICSD.img.xz) = f1e95515aec2be55736b7d4f82ffaa281cd4f946092b3f3a798960c2dd2391a126f5a81295ea009200fb2efd3e20eb87cfc5f2306ec7b1db0bf43777849df949 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-GENERICSD.img.xz) = 6e890ac6958be5f22f93e77a6bb2a39285bc89e4545b09aafc1bfed45fe1e388 == VM IMAGE CHECKSUMS == o 14.0-RC3 amd64: SHA512 (FreeBSD-14.0-RC3-amd64-ufs.qcow2.xz) = c48059751e301c6b0c58214dcfe11a5f71223003e124609a1e6978308420fc95888a289eae6404d998977529068fed1eed6bd4be0df25beddbd1bce258dd780d SHA512 (FreeBSD-14.0-RC3-amd64-ufs.raw.xz) = a5967df7766376e9f7b52e2ed1d0ec2c379097bed59ca41481f3c516058cd45c80c18be9e7ca63eb847f23ce6f0a91e3bd970fb62059e0a9548720496d2cb92f SHA512 (FreeBSD-14.0-RC3-amd64-ufs.vhd.xz) = 0107814976f4962493d780fe1ad0e4a83499afc48435a2f85eb1ef040347238c02dc93a5a5065dc6733c7b5f4d78a147faedeab2766e4374bb87d9ddfcf6aced SHA512 (FreeBSD-14.0-RC3-amd64-ufs.vmdk.xz) = 749b7b45868dc24c6090a684b27a9cca32de2ac3361f539d53ac7f95c04e3032590d8af51b29bbb658b6f18fa3eb37f66345d755d8566f67c32e6801ff4de0dd SHA512 (FreeBSD-14.0-RC3-amd64-zfs.qcow2.xz) = 938e39023002203394e007de2c530edbadda620155d91c91e18c4d3400dfd8130e7fd58cf376ad4edd837e9c7ad680d5008fec64989a33c9b5353ec157a78fa0 SHA512 (FreeBSD-14.0-RC3-amd64-zfs.raw.xz) = 4c921529022da29a560015e9224f2a99a387ec9efd0b4f8967b750859b969554150a070e8e5e81bd15cd87b186187017dcca4566141257449cc5784e9eecbd8d SHA512 (FreeBSD-14.0-RC3-amd64-zfs.vhd.xz) = 4fd158807f4f401a3f3e697b44732aa8043bad481e8b0d0529db5217e10d2fbc8309d2067185a9a5d66ffaceb55b5b87afa2c20325023e9f6761c3360273d91c SHA512 (FreeBSD-14.0-RC3-amd64-zfs.vmdk.xz) = 61266788e39d97479e99519ca214a263d58d5bfecdf70c39bc33c96f669deab546d9f30deab74cc82fc038806a98f477bf92b6cd442672720b0508e9ca75e922 SHA512 (FreeBSD-14.0-RC3-amd64.qcow2.xz) = c48059751e301c6b0c58214dcfe11a5f71223003e124609a1e6978308420fc95888a289eae6404d998977529068fed1eed6bd4be0df25beddbd1bce258dd780d SHA512 (FreeBSD-14.0-RC3-amd64.raw.xz) = a5967df7766376e9f7b52e2ed1d0ec2c379097bed59ca41481f3c516058cd45c80c18be9e7ca63eb847f23ce6f0a91e3bd970fb62059e0a9548720496d2cb92f SHA512 (FreeBSD-14.0-RC3-amd64.vhd.xz) = 0107814976f4962493d780fe1ad0e4a83499afc48435a2f85eb1ef040347238c02dc93a5a5065dc6733c7b5f4d78a147faedeab2766e4374bb87d9ddfcf6aced SHA512 (FreeBSD-14.0-RC3-amd64.vmdk.xz) = 749b7b45868dc24c6090a684b27a9cca32de2ac3361f539d53ac7f95c04e3032590d8af51b29bbb658b6f18fa3eb37f66345d755d8566f67c32e6801ff4de0dd SHA256 (FreeBSD-14.0-RC3-amd64-ufs.qcow2.xz) = a361f28259828d9d0beee8dd58c32af3cebd905d284cdbc514d7ae0cc42dc161 SHA256 (FreeBSD-14.0-RC3-amd64-ufs.raw.xz) = 3c8a382f39ed8e8282f5706044a084273e9b652ad24d35a7cd45144f7a4e9c72 SHA256 (FreeBSD-14.0-RC3-amd64-ufs.vhd.xz) = 2499169ff2957e80dcf2f824108701dac19dee5172a7635774c5f830ccf29e9b SHA256 (FreeBSD-14.0-RC3-amd64-ufs.vmdk.xz) = 87a5f1a66981e58d8b62f8c1d1af154932cfd6dca2ce4f8d206e1815b1fc652b SHA256 (FreeBSD-14.0-RC3-amd64-zfs.qcow2.xz) = 42d1d87d94cb1110eb6e1276325336d731a7af39e24d2b68b5949399a10f6c55 SHA256 (FreeBSD-14.0-RC3-amd64-zfs.raw.xz) = 6f1cb56b0286c3497b559d6ffce8e4311ca77fb3f296df7c7577912a76b7484f SHA256 (FreeBSD-14.0-RC3-amd64-zfs.vhd.xz) = 146d366c3c953eecad6cb3c3cf8563fffe0d0e2f4dd363f1cb13840ad016078e SHA256 (FreeBSD-14.0-RC3-amd64-zfs.vmdk.xz) = 64c42aad76e7d5519b08d498a6c6334502900a3229859690a443ac022f152cfd SHA256 (FreeBSD-14.0-RC3-amd64.qcow2.xz) = a361f28259828d9d0beee8dd58c32af3cebd905d284cdbc514d7ae0cc42dc161 SHA256 (FreeBSD-14.0-RC3-amd64.raw.xz) = 3c8a382f39ed8e8282f5706044a084273e9b652ad24d35a7cd45144f7a4e9c72 SHA256 (FreeBSD-14.0-RC3-amd64.vhd.xz) = 2499169ff2957e80dcf2f824108701dac19dee5172a7635774c5f830ccf29e9b SHA256 (FreeBSD-14.0-RC3-amd64.vmdk.xz) = 87a5f1a66981e58d8b62f8c1d1af154932cfd6dca2ce4f8d206e1815b1fc652b o 14.0-RC3 i386: SHA512 (FreeBSD-14.0-RC3-i386-ufs.qcow2.xz) = 88b84f67bcf568603c1435427b1e7cce7e0a5aa934b45139bdfc627cbf495ff36ad977e5c55ff323f0d183d222f4258c5c37af9955f01cf2a07eb7ef10eabe8f SHA512 (FreeBSD-14.0-RC3-i386-ufs.raw.xz) = 29df638cb85bbe8fed6c4b37c8de2bd64944e545a5c94d1f2512a8e08de8f38bc0576520da2d7856add882779231f22cf08fc1d6f3761b041b63ea6fecd00666 SHA512 (FreeBSD-14.0-RC3-i386-ufs.vhd.xz) = 453339c01f773faf2d30225ad110559a91682c1ddc8c6fe10aada8a2bcfa1be6f822c9df14f3de04da08938e257df49cd6f08d1be3d173e62aa6c624fe4f66c8 SHA512 (FreeBSD-14.0-RC3-i386-ufs.vmdk.xz) = 3ac3adb352ef307179e5117a1bc91d237776af1065ed60a71b3b7035753c04d2573d80ff8c86ff2fdf3fe7a42f41a6d4033ec420ffddaf488a8bf8f24b46336a SHA512 (FreeBSD-14.0-RC3-i386-zfs.qcow2.xz) = 1fc951c8192cf4f32edf8ce5269ffab865be6c3e1b3a3209228bf37fa9804e6f637695ec42986547c7726c54491eb72879b8d28f4feaa826d8ebacd7a492248d SHA512 (FreeBSD-14.0-RC3-i386-zfs.raw.xz) = 6d7e3f44fb028564030ba0431b494a36fa0e11f21df6091c2cfc60a2acc7a550df5e45babd83d10ccd13ffb878c5da9117af874d76f9ea53e651b76ce0ebf3c5 SHA512 (FreeBSD-14.0-RC3-i386-zfs.vhd.xz) = 4e2add7012cbeefed4189801946f827fcb65b2579deca957f3748cdb41da6647cbc0183259b04cc1dce9b0a746bd6ebeb38ce6bdcdb511d7577658115377f74a SHA512 (FreeBSD-14.0-RC3-i386-zfs.vmdk.xz) = 7a2113bd32e5cdec7bc4873efeb8057138ec079f475d96791b9159fb2585a2fd8792364f5760b30081243da268ef9b9a5ecab7643686c4d917f398e6fadef449 SHA512 (FreeBSD-14.0-RC3-i386.qcow2.xz) = 88b84f67bcf568603c1435427b1e7cce7e0a5aa934b45139bdfc627cbf495ff36ad977e5c55ff323f0d183d222f4258c5c37af9955f01cf2a07eb7ef10eabe8f SHA512 (FreeBSD-14.0-RC3-i386.raw.xz) = 29df638cb85bbe8fed6c4b37c8de2bd64944e545a5c94d1f2512a8e08de8f38bc0576520da2d7856add882779231f22cf08fc1d6f3761b041b63ea6fecd00666 SHA512 (FreeBSD-14.0-RC3-i386.vhd.xz) = 453339c01f773faf2d30225ad110559a91682c1ddc8c6fe10aada8a2bcfa1be6f822c9df14f3de04da08938e257df49cd6f08d1be3d173e62aa6c624fe4f66c8 SHA512 (FreeBSD-14.0-RC3-i386.vmdk.xz) = 3ac3adb352ef307179e5117a1bc91d237776af1065ed60a71b3b7035753c04d2573d80ff8c86ff2fdf3fe7a42f41a6d4033ec420ffddaf488a8bf8f24b46336a SHA256 (FreeBSD-14.0-RC3-i386-ufs.qcow2.xz) = 40d6e81e3dd08daba7d8284266f825ffc4f75d81ef0b676099a59fd5c1aefa1c SHA256 (FreeBSD-14.0-RC3-i386-ufs.raw.xz) = f55c3d1568d389424b6e12c4e36546766eb6923088113fb6e3ecf8c0ff291443 SHA256 (FreeBSD-14.0-RC3-i386-ufs.vhd.xz) = 954da705a71febb02a2505f2052be8818b992156469e10aa163ce370d4a19ec6 SHA256 (FreeBSD-14.0-RC3-i386-ufs.vmdk.xz) = ad67fdd57c875010ab5bb4bab382a1fe60b2713e82e405197e9559d2e299d0af SHA256 (FreeBSD-14.0-RC3-i386-zfs.qcow2.xz) = aee2bc86005b5e84336bbc813d02ecd4332194a44b9316d97d5f2ca9922fe523 SHA256 (FreeBSD-14.0-RC3-i386-zfs.raw.xz) = 155dc2d1c90f143711e243efed6084c26e76ec530b9b4452890a23f70f6456ba SHA256 (FreeBSD-14.0-RC3-i386-zfs.vhd.xz) = 257116206a56a50bbfbd306840aebb350739bf2dc26877997167ef43027456c4 SHA256 (FreeBSD-14.0-RC3-i386-zfs.vmdk.xz) = 7464458784a32e53dfb6e2978bf9efd76cd3dcdd475f3f81bb12348e4c9531fb SHA256 (FreeBSD-14.0-RC3-i386.qcow2.xz) = 40d6e81e3dd08daba7d8284266f825ffc4f75d81ef0b676099a59fd5c1aefa1c SHA256 (FreeBSD-14.0-RC3-i386.raw.xz) = f55c3d1568d389424b6e12c4e36546766eb6923088113fb6e3ecf8c0ff291443 SHA256 (FreeBSD-14.0-RC3-i386.vhd.xz) = 954da705a71febb02a2505f2052be8818b992156469e10aa163ce370d4a19ec6 SHA256 (FreeBSD-14.0-RC3-i386.vmdk.xz) = ad67fdd57c875010ab5bb4bab382a1fe60b2713e82e405197e9559d2e299d0af o 14.0-RC3 aarch64: SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.qcow2.xz) = 8baffca3966a7d90297ef8173b295ced9458fd6601f0e2613639d6d495545ee7a841cd44087d4de2c261610a5893026528f82aecee1191daf2c6427fe6e90d2a SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.raw.xz) = e8b738eb0c86773ccb24c6bd0385c983537a7866f880e9acd9dcb0e182f262cb6411e0bbaf5582b02eaa8279a8dd35efb7efae0858623bff5c6a0808c1cb52af SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.vhd.xz) = 413112a54ffb97b54b45e0f322de2a7b8e024a5b3f0ecd92344fc7e6d4ce1b2961214342a0859bca0bcf4ec4a6d76f3796402f2de76f68a6da0915f70b1f99bc SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.vmdk.xz) = 87bc7cab5a0ded35ef0082b06217f6462ef80b70aaa30f6913c2560c0e0bcfdad619f5ff4d06db8b191843978cece5d6462b0c30f2ae3e530a971a1d4e0b46a5 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.qcow2.xz) = a46f305d905dcda68deb2990f9b52bf8f5a2f5a93b842da3b63738b12d1815e4d6e0720b539b07175492f5109f815b59aae4c2476c03b26343c2695995a41e43 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.raw.xz) = c37d95d3fe1483f85adbfa7e412260346697d2e313c0efe0585715fa7568eb2cf2f412ac627c7bdb84594e410266a42e54c9e1e4738258d555adcf961bc9faf9 SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.vhd.xz) = 482d1a135539e41bcae96a76f5f1d8df89a67ecd12db872ae9f73f510bd0c086332e7de182c5b196978e51c8c82d46c1e6d7352c54ad756deb694f610b433a4f SHA512 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.vmdk.xz) = 2b26533127bcd66c0d8d8b5d0cc9bb2b01f7ec82c81818f878937a0927d72699f000b08f97272c3870415279311b78c5a48322a536975bab07c4493e704ecf7c SHA512 (FreeBSD-14.0-RC3-arm64-aarch64.qcow2.xz) = 8baffca3966a7d90297ef8173b295ced9458fd6601f0e2613639d6d495545ee7a841cd44087d4de2c261610a5893026528f82aecee1191daf2c6427fe6e90d2a SHA512 (FreeBSD-14.0-RC3-arm64-aarch64.raw.xz) = e8b738eb0c86773ccb24c6bd0385c983537a7866f880e9acd9dcb0e182f262cb6411e0bbaf5582b02eaa8279a8dd35efb7efae0858623bff5c6a0808c1cb52af SHA512 (FreeBSD-14.0-RC3-arm64-aarch64.vhd.xz) = 413112a54ffb97b54b45e0f322de2a7b8e024a5b3f0ecd92344fc7e6d4ce1b2961214342a0859bca0bcf4ec4a6d76f3796402f2de76f68a6da0915f70b1f99bc SHA512 (FreeBSD-14.0-RC3-arm64-aarch64.vmdk.xz) = 87bc7cab5a0ded35ef0082b06217f6462ef80b70aaa30f6913c2560c0e0bcfdad619f5ff4d06db8b191843978cece5d6462b0c30f2ae3e530a971a1d4e0b46a5 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.qcow2.xz) = 159277d3698dbe03fff0cef7bca2633b9884b59fa648e34f73670b9998467d4f SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.raw.xz) = d9d2206f96c99b0caf46aca6d852e6f824f3c223b58f0472c652ef43085b78bf SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.vhd.xz) = e535ec045b1e61063dd14ada95606b4a0763a9ef324cd5d9ae976def448b1fae SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-ufs.vmdk.xz) = 75ff0ddb26bffadeabb31944ff273092c949f8bd83ac4bd333bcfaf1afa8ff13 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.qcow2.xz) = f5b0741294347219f4e16d60a78b6b55672b50fc51a2d15e5d19018421771de4 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.raw.xz) = 8542dafea9cfbf49289c2dc6d348d257279b279e753c50742495d5b7f20ae6b2 SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.vhd.xz) = 25338932d46c0486bb579108890f8052f7764fda4beb4dc00297f02d0f61390e SHA256 (FreeBSD-14.0-RC3-arm64-aarch64-zfs.vmdk.xz) = a7df8ff84f71d7263c90df81cd9af5ea5450aaa15e0798d02aa336fe6f5dd67a SHA256 (FreeBSD-14.0-RC3-arm64-aarch64.qcow2.xz) = 159277d3698dbe03fff0cef7bca2633b9884b59fa648e34f73670b9998467d4f SHA256 (FreeBSD-14.0-RC3-arm64-aarch64.raw.xz) = d9d2206f96c99b0caf46aca6d852e6f824f3c223b58f0472c652ef43085b78bf SHA256 (FreeBSD-14.0-RC3-arm64-aarch64.vhd.xz) = e535ec045b1e61063dd14ada95606b4a0763a9ef324cd5d9ae976def448b1fae SHA256 (FreeBSD-14.0-RC3-arm64-aarch64.vmdk.xz) = 75ff0ddb26bffadeabb31944ff273092c949f8bd83ac4bd333bcfaf1afa8ff13 o 14.0-RC3 riscv64: SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.qcow2.xz) = f895cba48b0cfa3883fce9be4c87869a75239a74d8746c053452fbcfbe817ae519115f02ac3dc1298f4d755890e03f0f48c2be0e67247b42c6288b744357b739 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.raw.xz) = 0760823e4b28ea46a397dd32d70c506896b38bfd0fb692d6e37918d08b61870472bdbe980e776cedc84a55b1b19566a55ce6cb064d81a795fdfeb32d988b257a SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.vhd.xz) = 565d558863d39aa4d336b44e83e8cfde2f532a683cbd8fd398036af967293c40460473e95cac4f0e7b069d0a8ca38ede035ec4b6be9d795d2b097134681517f2 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.vmdk.xz) = ddd2c66a2d58a457fd436b0af44489dc8702ad09b8470bf16f628c5e05dcc0ec488cd24b418a29b723bc48f14a8c659c38bb9b8afbe61860c3cb8218604d7cc8 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.qcow2.xz) = 5e2fe66359db6030db057a5a4e3aca5b25bc5b6add08e994a30d0a4ecb3242b6b5d4a8521b381dce08c5e2741dc782b0dbc49053119fc87d46d1208bad8962dc SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.raw.xz) = 987d1b35c01d0f07a8be5db2e1c5bc95ba2cdecad1318996f54251cd38e67b093072ad986599335527a8766ee096f964cfdde24219eb95325f9b85bfe8a5a5b2 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.vhd.xz) = 286b0ced9a50b8e209c76006661555ebe0e1a796fae64322f2d85fe9b3f8c785693a1db71558932026c6a5917eeb36b61246de0235657668f916a0217018eedb SHA512 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.vmdk.xz) = 851dee6eacbe3559138fd26ce5eaba930c87b8756954b24d93503e493bc499cb2099748d57113d77207b55118af6e306ffafcf44b3ae06587df0ce7c18feacbb SHA512 (FreeBSD-14.0-RC3-riscv-riscv64.qcow2.xz) = f895cba48b0cfa3883fce9be4c87869a75239a74d8746c053452fbcfbe817ae519115f02ac3dc1298f4d755890e03f0f48c2be0e67247b42c6288b744357b739 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64.raw.xz) = 0760823e4b28ea46a397dd32d70c506896b38bfd0fb692d6e37918d08b61870472bdbe980e776cedc84a55b1b19566a55ce6cb064d81a795fdfeb32d988b257a SHA512 (FreeBSD-14.0-RC3-riscv-riscv64.vhd.xz) = 565d558863d39aa4d336b44e83e8cfde2f532a683cbd8fd398036af967293c40460473e95cac4f0e7b069d0a8ca38ede035ec4b6be9d795d2b097134681517f2 SHA512 (FreeBSD-14.0-RC3-riscv-riscv64.vmdk.xz) = ddd2c66a2d58a457fd436b0af44489dc8702ad09b8470bf16f628c5e05dcc0ec488cd24b418a29b723bc48f14a8c659c38bb9b8afbe61860c3cb8218604d7cc8 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.qcow2.xz) = 90fa7e012972747ac14e48340142e93bf9596e0c7505afc26f1f57eba3cf6585 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.raw.xz) = f498c030f76e09989d0b6cf595fff0622611983a4b89185362bd0bcb040da50f SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.vhd.xz) = 8aa51dc45241cb983df0dcd180b7212a442f70e08189ddc157fe89f7e8ee9d84 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-ufs.vmdk.xz) = ce940d0f1a7997bac78a70f6c52f5d50453bba63f448682edd632ed6d8d77c24 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.qcow2.xz) = c3cf87fab1ada28fbaf750d0671dc8687b04761fe6ae304c9edf28c650738e79 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.raw.xz) = 94183cbfb0115e758c4cff9b5540f938540928c07f320b964d6f5d0242a68e2b SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.vhd.xz) = 30679d2392e81d28713dcfc7ee172258d8e038e0ea4b267c87d6b53157867bb4 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64-zfs.vmdk.xz) = d88af7fa8525e97dedc0231a2947a27934c13327a5d8becafa245529aa4a7a28 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64.qcow2.xz) = 90fa7e012972747ac14e48340142e93bf9596e0c7505afc26f1f57eba3cf6585 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64.raw.xz) = f498c030f76e09989d0b6cf595fff0622611983a4b89185362bd0bcb040da50f SHA256 (FreeBSD-14.0-RC3-riscv-riscv64.vhd.xz) = 8aa51dc45241cb983df0dcd180b7212a442f70e08189ddc157fe89f7e8ee9d84 SHA256 (FreeBSD-14.0-RC3-riscv-riscv64.vmdk.xz) = ce940d0f1a7997bac78a70f6c52f5d50453bba63f448682edd632ed6d8d77c24 o 14.0-RC3 amd64 BASIC-CI: SHA512 (FreeBSD-14.0-RC3-amd64-BASIC-CI.raw.xz) = 0fc1fbaf361a342bfb68a9682e87a6caa54325a44783b71e4dd36a88273a87a862edcc69d501eb71365ec7c90682c4e0634cffe731535072c09acfa5a286abef SHA256 (FreeBSD-14.0-RC3-amd64-BASIC-CI.raw.xz) = a8505db7bf3d89c8754d13ebf47ec62afbffe689d041604b7ee83349e874139b Regards, Glen Please consider donating to help support my FreeBSD work: https://www.gofundme.com/f/gjbbsd https://paypal.me/gjbbsd Love FreeBSD? Support this and future releases with a donation to the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEjRJAPC5sqwhs9k2jAxRYpUeP4pMFAmU8SYUACgkQAxRYpUeP 4pNtkQ//UTcS3pnNb3awpfroNH2gBf7PECObI2n1tBh6JxIuAeTpDJaKfhIMSrDr 69GOFE1GtyO4N67+XEeXo8Ql7yuqLIGDHg9B5kx5cO6asL9o1kVdnAo5bIGlHevm Z9PioQfp8kgB5ZvEap79jmbpCU2aMXXhVt3PAd/olcb3RASJ4+FNnSX3PmuX7aLp vTBx65jl6/pbLgckpbZ+6Nq0/s3RH+WCxBCd4tYOwO6Zvd+nlJp5/qtWklppbpLV rd2BlAYvgK/LItAGZtfpBJ0YWIRGccQmhnipvUoNfMfZJE8CbSYIFY0hSg+RHq9n aLSXrshS2/nU+bujfh2QBAamTGVjCSxVxC6mazTE3yzHKtyYmfESBhRYYFdoFDd7 YfehGK0esRH9968PYcizWmt4E3rJGHIU7nV/kQKUNsDANKXU0CI7MrP+6AjQdnrM RMhRT1NJuABK9xhigaa3hW6vp471pbspBuEIqufORsCTd9hLbWx1No9nmKhuI2FT SNkotkTcdOCVC8DdwDuynJsUct7S0Uje0He6iZjdDOauLZLVteKeH8wxIYHEfNyi 2fSF73U3pAISDUBd69gqLbYsT+dl71mSClXzw8EWcAMfUbbmMFpHkErtBiTSDYp7 7GpkCne/yJ1DQ12mkDyHaXUGO6MQ0WrvCwix3dZXA0nZP66pTx4= =DklJ -----END PGP SIGNATURE----- From nobody Sat Oct 28 00:55:13 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHLg970Qtz4y9f0 for ; Sat, 28 Oct 2023 00:55:33 +0000 (UTC) (envelope-from jon@xyinn.org) Received: from mail-4018.proton.ch (mail-4018.proton.ch [185.70.40.18]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHLg94Tqgz4n4Q for ; Sat, 28 Oct 2023 00:55:33 +0000 (UTC) (envelope-from jon@xyinn.org) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xyinn.org; s=protonmail2; t=1698454529; x=1698713729; bh=av7rt5zVu/em/siGiuTWsbBswru/usLI6vTpc67NgOo=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=pKbI9ev6tVkjCebFV546bmw4OK4dsQTyCD27vE+OkeMQl9ZEunBYpM+w4wPPWBs+P k1ur6jwkUq7wXEiWdC/HlC99QL1wj+HVfENjgG5RrHVg7wPEfCy2rm0dyfOgdNdjnz LPgrQex9eJc4U2tyH3tFq9MkTLTMmYq1mnSqsBRYNK2yqUms3MClR+lzy7DoiE6lK8 /j6blqm1SUSB8gWSb9OglLZ9gvyH+nUr7mrXSikzknWvAFsaTEZa9COpt24gbASxY5 kFVG4CBFB4739fvGiI89m/fq99QpMwNEv9JDP3+vwa197cRwbt0HBZvb5oRzwIc/Uz KkntV7+pxvqKQ== Date: Sat, 28 Oct 2023 00:55:13 +0000 To: Glen Barber From: Jonathan Vasquez Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, FreeBSD Release Engineering Team Subject: Re: FreeBSD 14.0-RC3 Now Available Message-ID: <13F587GDH9emqWsMCsYSfJne1sedG9roayQWnqk0S2IJ4aCZLSUh-wQoV3YEEwk6p8iLJq4hHe0OGRZjgJPFQNNm_0qwVQFt7rznm7-AjuI=@xyinn.org> In-Reply-To: <20231027233637.GA1307@FreeBSD.org> References: <20231027233637.GA1307@FreeBSD.org> Feedback-ID: 12351801:user:proton List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] X-Rspamd-Queue-Id: 4SHLg94Tqgz4n4Q Thanks Glen. I just updated my Framework Laptop from RC2 to RC3 and it boot= ed fine. I'll report back if I notice any weirdness. uname: FreeBSD leslie 14.0-RC3 FreeBSD 14.0-RC3 #0 releng/14.0-n265368-c6cfdc13055= 4: Fri Oct 27 05:57:28 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/u= sr/src/amd64.amd64/sys/GENERIC amd64 From nobody Sat Oct 28 02:55:28 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHPKl4fPHz4yHXy; Sat, 28 Oct 2023 02:55:39 +0000 (UTC) (envelope-from gjb@freebsd.org) Received: from mail0.glenbarber.us (mail0.glenbarber.us [208.86.227.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0.glenbarber.us", Issuer "Gandi Standard SSL CA 2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHPKl30Zxz3FdR; Sat, 28 Oct 2023 02:55:39 +0000 (UTC) (envelope-from gjb@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from smtpclient.apple (50.29.233.174.res-cmts.swb2.ptd.net [50.29.233.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: gjb) by mail0.glenbarber.us (Postfix) with ESMTPSA id 365524F7B3; Sat, 28 Oct 2023 02:55:31 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 mail0.glenbarber.us 365524F7B3 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (1.0) Subject: Re: FreeBSD 14.0-RC3 Now Available From: Glen Barber In-Reply-To: <13F587GDH9emqWsMCsYSfJne1sedG9roayQWnqk0S2IJ4aCZLSUh-wQoV3YEEwk6p8iLJq4hHe0OGRZjgJPFQNNm_0qwVQFt7rznm7-AjuI=@xyinn.org> Date: Fri, 27 Oct 2023 22:55:28 -0400 Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org, FreeBSD Release Engineering Team Message-Id: References: <13F587GDH9emqWsMCsYSfJne1sedG9roayQWnqk0S2IJ4aCZLSUh-wQoV3YEEwk6p8iLJq4hHe0OGRZjgJPFQNNm_0qwVQFt7rznm7-AjuI=@xyinn.org> To: Jonathan Vasquez X-Mailer: iPhone Mail (20G81) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36236, ipnet:208.86.224.0/22, country:US] X-Rspamd-Queue-Id: 4SHPKl30Zxz3FdR Great to hear. Glen Sent from my phone. Please excuse my brevity and/or typos. > On Oct 27, 2023, at 8:55 PM, Jonathan Vasquez wrote: >=20 > =EF=BB=BFThanks Glen. I just updated my Framework Laptop from RC2 to RC3 a= nd it booted fine. I'll report back if I notice any weirdness. >=20 > uname: >=20 > FreeBSD leslie 14.0-RC3 FreeBSD 14.0-RC3 #0 releng/14.0-n265368-c6cfdc1305= 54: Fri Oct 27 05:57:28 UTC 2023 root@releng1.nyi.freebsd.org:/usr/obj/u= sr/src/amd64.amd64/sys/GENERIC amd64 >=20 >=20 From nobody Sat Oct 28 08:30:10 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHXln4W5Dz4yb1V for ; Sat, 28 Oct 2023 08:30:13 +0000 (UTC) (envelope-from r.m@rmswch.plus.com) Received: from avasout-peh-001.plus.net (avasout-peh-001.plus.net [212.159.14.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "avasout.plus.net", Issuer "GeoTrust TLS RSA CA G1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHXlm5vHzz4Dd4 for ; Sat, 28 Oct 2023 08:30:12 +0000 (UTC) (envelope-from r.m@rmswch.plus.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plus.com header.s=042019 header.b=ngDTxBkV; spf=none (mx1.freebsd.org: domain of r.m@rmswch.plus.com has no SPF policy when checking 212.159.14.17) smtp.mailfrom=r.m@rmswch.plus.com; dmarc=none Received: from opendev.home ([80.229.151.92]) by smtp with SMTP id wehuqvkY98h5Qwehvqj396; Sat, 28 Oct 2023 09:30:12 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019; t=1698481812; bh=rj49Ko+3PTlSwufCK6InrP2xsxsne1a7wsFSvSKQ6Ik=; h=Date:From:To:Subject; b=ngDTxBkVQpYdCglEYFN1BnfvNRW3h4sP8bfjLL5eCxVlx188lGhopJlGKfTlmLkg7 bB3J3leABmsXEGd0Ry2pG3cAKIHPCbq6obpdo5dGzkxTlzl6sUBt+FcDsL40S+u8z4 576QhlHpKF9pQQI6tZdxiejPumfvhkcSqUJKtlhLB+3G4BXs/dL0c49bQqqtaGU2lT f0jNB6AdYbw3+Yt5WryyADVzAzJw4wW0MGT5UP2R9a+ZvBhXq05D5HVtWE2D2CfRSR ghQtvDmXQg6Hm1uXPeLNEMGDmHGFKPEUAbHgj8WDLASbgDFcV2kJUNGgwhycQLWeKU W5MKR+WBGJhGQ== X-Clacks-Overhead: "GNU Terry Pratchett" X-CM-Score: 0.00 X-CNFS-Analysis: v=2.4 cv=JujiEe0C c=1 sm=1 tr=0 ts=653cc694 a=6xlXYeu7y0jVCiZ/8t1ozg==:117 a=6xlXYeu7y0jVCiZ/8t1ozg==:17 a=kj9zAlcOel0A:10 a=WcxHNsqOAAAA:8 a=CBHx1vjFdkW0cNmGLNIA:9 a=CjuIK1q_8ugA:10 a=O848WxombkMA:10 a=WXeDAsiyfwSRC1lbKA7R:22 Date: Sat, 28 Oct 2023 08:30:10 +0000 From: Roger Marsh To: FreeBSD-stable@FreeBSD.org Subject: Version of OpenSSH in FreeBSD 14.0 Message-ID: <20231028083010.0f880289@opendev.home> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; x86_64-unknown-openbsd7.4) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfC6AXgnAn/rFp/Ej7FBE59Z1fr1it9qyuXfqDOd1DQDOBcc7xOHClY6PmA3PiOvOcLtcqGzgHNONE7s+WVOWZDyJHLoNhnxW6b7zGycgtQXHTcG0SrnO Ci67F2GfSkPYmERjDvrWi4Yc8zbB2DNrdBiuTAzbCNc5Cziud8zyarFzETUXL2nw/02k4BW2XKps1jrOfkHyBvJbNxuJXThaytE= X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.90)[-0.904]; R_DKIM_ALLOW(-0.20)[plus.com:s=042019]; RWL_MAILSPIKE_GOOD(-0.10)[212.159.14.17:from]; RCVD_IN_DNSWL_LOW(-0.10)[212.159.14.17:from]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[plus.com:+]; DMARC_NA(0.00)[plus.com]; ASN(0.00)[asn:6871, ipnet:212.159.0.0/19, country:GB]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_NA(0.00)[no SPF record] X-Rspamd-Queue-Id: 4SHXlm5vHzz4Dd4 X-Spamd-Bar: --- Does 14.0 have OpenSSH 9.3 or OpenSSH 9.4? This thread: Delay in starting programs on FreeBSD via ssh after upgrade OpenBSD from 7.3 to 7.4 on openssh-unix-dev@mindrot.org is why I ask. Thanks Roger From nobody Sat Oct 28 08:39:09 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHXyJ5qKhz4ybNT for ; Sat, 28 Oct 2023 08:39:20 +0000 (UTC) (envelope-from herbert@gojira.at) Received: from mail.bsd4all.net (mail.bsd4all.net [94.130.200.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-384) client-digest SHA384) (Client CN "mail.bsd4all.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHXyG4Qnhz4FxM for ; Sat, 28 Oct 2023 08:39:18 +0000 (UTC) (envelope-from herbert@gojira.at) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gojira.at header.s=mail202005 header.b=YsSWxi5f; spf=pass (mx1.freebsd.org: domain of herbert@gojira.at designates 94.130.200.20 as permitted sender) smtp.mailfrom=herbert@gojira.at; dmarc=none Date: Sat, 28 Oct 2023 10:39:09 +0200 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gojira.at; s=mail202005; t=1698482349; bh=Phsl5ZkOOXY9zCn8DLLfqK3dq3OELpYp6WGttlKgAUQ=; h=Date:From:To:Subject:Message-ID:MIME-Version:Content-Type; b=YsSWxi5fXef2nw8MncbYGLIdinfLtkfqUMqDVoWEu9Ky9YKxGEwzexvmcqsa0j+iC j31kkZp8bmsj7sVCIJiD/LkvROs+INlkt1ArTzKQmbvDK7FlnYMoojP3bh/Ivci9mF LdL3B+7p/TY9Qqf8x/zAeQb6WOZIst6m+73wamTk9bXHcYADF2KzpE3yLubKoinibw +N/oufJWLZ1s/YUqpCBP2xzxbPgHY4VD+Fdft+GSu/sufkHAwMRu2pSXiSX0BPwF/b 6q1HgzjRdYMh3EAyZs8sB4SqrMAVeN7HZTqXf1UKh6qdnk6Omi+cVTn1WROEdolCvd yiYOhUQxdvmHw== From: "Herbert J. Skuhra" To: FreeBSD-stable@freebsd.org Subject: Re: Version of OpenSSH in FreeBSD 14.0 Message-ID: References: <20231028083010.0f880289@opendev.home> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231028083010.0f880289@opendev.home> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; R_SPF_ALLOW(-0.20)[+ip4:94.130.200.20]; R_DKIM_ALLOW(-0.20)[gojira.at:s=mail202005]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[FreeBSD-stable@freebsd.org]; DKIM_TRACE(0.00)[gojira.at:+]; ARC_NA(0.00)[]; DMARC_NA(0.00)[gojira.at]; MIME_TRACE(0.00)[0:+]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Queue-Id: 4SHXyG4Qnhz4FxM X-Spamd-Bar: --- On Sat, Oct 28, 2023 at 08:30:10AM +0000, Roger Marsh wrote: > Does 14.0 have OpenSSH 9.3 or OpenSSH 9.4? OpenSSH 9.5p1. See: https://cgit.freebsd.org/src/commit/?h=releng/14.0&id=676824f5cdf9dfdbfca38c9cfff2398345db9211 -- Herbert From nobody Sat Oct 28 08:40:19 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHXzd1ctRz4ybrr for ; Sat, 28 Oct 2023 08:40:29 +0000 (UTC) (envelope-from pi@freebsd.org) Received: from fc.opsec.eu (fc.opsec.eu [IPv6:2001:14f8:200:4::4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHXzc3xZRz4Gnq for ; Sat, 28 Oct 2023 08:40:28 +0000 (UTC) (envelope-from pi@freebsd.org) Authentication-Results: mx1.freebsd.org; none Received: from pi by fc.opsec.eu with local (Exim 4.96.2 (FreeBSD)) (envelope-from ) id 1qwerj-00033W-1b; Sat, 28 Oct 2023 10:40:19 +0200 Date: Sat, 28 Oct 2023 10:40:19 +0200 From: Kurt Jaeger To: Roger Marsh Cc: FreeBSD-stable@freebsd.org Subject: Re: Version of OpenSSH in FreeBSD 14.0 Message-ID: References: <20231028083010.0f880289@opendev.home> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20231028083010.0f880289@opendev.home> X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:12502, ipnet:2001:14f8::/32, country:DE] X-Rspamd-Queue-Id: 4SHXzc3xZRz4Gnq Hi! > Does 14.0 have OpenSSH 9.3 or OpenSSH 9.4? ssh -V says: OpenSSH_9.5p1, OpenSSL 3.0.12 24 Oct 2023 -- pi@FreeBSD.org +49 171 3101372 Now what ? From nobody Sat Oct 28 09:45:36 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHZR32tbjz4yg1L for ; Sat, 28 Oct 2023 09:45:51 +0000 (UTC) (envelope-from pete@twisted.org.uk) Received: from toybox.twisted.org.uk (toybox.twisted.org.uk [178.250.76.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHZR06YMzz4N3Z for ; Sat, 28 Oct 2023 09:45:48 +0000 (UTC) (envelope-from pete@twisted.org.uk) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=twisted.org.uk header.s=tbx header.b=dxz52Ash; spf=pass (mx1.freebsd.org: domain of pete@twisted.org.uk designates 178.250.76.50 as permitted sender) smtp.mailfrom=pete@twisted.org.uk; dmarc=pass (policy=none) header.from=twisted.org.uk DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=twisted.org.uk; s=tbx; h=Content-Transfer-Encoding:Content-Type:In-Reply-To :From:References:To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=TLLpTfuB3MWTe4VD7sCNAlXJs4zeucrHmyrJmfAdT8Q=; t=1698486348; x=1699350348; b=dxz52AshM6rEVmPy5VOzE+Casg8VHgkgvL9d+ay3n8khVxCKv6cC7ZnLRcula8wG/7m/PA7vDr5 oIWDBWPRH/DdnPgs7fDSKrOTaBrvIEUUA3cAVF3uE9YTIp4NCefB6Ql1OD6xOwpig+DHisiylL51r FoIzB+fngwozpq8zdRL0KYtSAdOrb55thUASGu9fRwC/9TfsLank3y21TcGNbZhdRTgXjMKDEaQh4 n08RuWq1ryDO8JiCpjKHJiB8UHmJhy4ccFeQiPpfqgxOB1Sdm1K4UzXAFw6QkKDoNkEKIfj03J4Fo y/uWXb26Lv6mfbP9Yk6PwgzUS5Frw7mAd4Ug==; Received: from mailnull by toybox.twisted.org.uk with spamc-scanned (Exim 4.95 (FreeBSD)) (envelope-from ) id 1qwfsv-0002Ob-AC for stable@freebsd.org; Sat, 28 Oct 2023 09:45:38 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on toybox.twisted.org.uk X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.5 X-Spam-Score: -1.0 () Received: from balta.twisted.org.uk ([2001:470:6cc4:1::57]) by toybox.twisted.org.uk with esmtpsa (TLS1.3) tls TLS_AES_128_GCM_SHA256 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1qwfsv-0002OX-7x for stable@freebsd.org; Sat, 28 Oct 2023 09:45:37 +0000 Message-ID: <5a16a0e2-34c2-4bcf-bfbe-aec980955fcf@twisted.org.uk> Date: Sat, 28 Oct 2023 10:45:36 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Version of OpenSSH in FreeBSD 14.0 Content-Language: en-GB To: stable@freebsd.org References: <20231028083010.0f880289@opendev.home> From: Pete French In-Reply-To: <20231028083010.0f880289@opendev.home> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-spamc-toybox: true X-transport-toybox: lookuphost X-Spamd-Result: default: False [-3.98 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.989]; DMARC_POLICY_ALLOW(-0.50)[twisted.org.uk,none]; R_SPF_ALLOW(-0.20)[+ip4:178.250.76.50/32]; R_DKIM_ALLOW(-0.20)[twisted.org.uk:s=tbx]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:12290, ipnet:178.250.72.0/21, country:GB]; MLMMJ_DEST(0.00)[stable@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[twisted.org.uk:+]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHZR06YMzz4N3Z X-Spamd-Bar: --- On 28/10/2023 09:30, Roger Marsh wrote: > Does 14.0 have OpenSSH 9.3 or OpenSSH 9.4? > > This thread: > > Delay in starting programs on FreeBSD via ssh after upgrade OpenBSD from 7.3 to 7.4 > > on openssh-unix-dev@mindrot.org This one ? https://lists.mindrot.org/pipermail/openssh-unix-dev/2023-October/041033.html Just looking at that thread and not being able to quite work out what the solution was there (it refers to another thread I cant find). But it seems to be ObscureKeystrokeTiming, yes ? There are no references to that in the config on FreeBSD 14, so I assume it stays on, and at the default. Did this turn out to be the cause of the issue, and did disabling it fix the problem ? I have to admit I hadn't tried ssh'ing *out* of my test 14 box until today, but a quick try doesnt show any delays to anything. However I don't have anything running OpenBSD. -pete. PS: I did see an upgraded ssh cause delays in login a while ago. But that turned out to be dropping support for old key exchange algorithms. If the box you are sshing into is somewhat underpowered then it can take longer to do the key echange, as the newer algorithms are far slower. In my case the delay was 30 seconds to login. From nobody Sat Oct 28 11:42:11 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHd1P5GdHz4ylsx for ; Sat, 28 Oct 2023 11:42:17 +0000 (UTC) (envelope-from r.m@rmswch.plus.com) Received: from avasout-ptp-003.plus.net (avasout-ptp-003.plus.net [84.93.230.244]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "avasout.plus.net", Issuer "GeoTrust TLS RSA CA G1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHd1N3zGCz4XGg for ; Sat, 28 Oct 2023 11:42:16 +0000 (UTC) (envelope-from r.m@rmswch.plus.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plus.com header.s=042019 header.b=C9V1PLPl; spf=none (mx1.freebsd.org: domain of r.m@rmswch.plus.com has no SPF policy when checking 84.93.230.244) smtp.mailfrom=r.m@rmswch.plus.com; dmarc=none Received: from opendev.home ([80.229.151.92]) by smtp with SMTP id whhjqSbdTFuWvwhhkqMZYw; Sat, 28 Oct 2023 12:42:13 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019; t=1698493333; bh=S5C8HmS5r3H+u0nkcwvJStpQ0HCIpLZgsPuMxEfYUbE=; h=Date:From:To:Subject:In-Reply-To:References; b=C9V1PLPlAYKVoYVBd3Z2YEd12UcVGGzhPxBrgn1Lcg0dNjdgGRi+XyZIO1F5/Kxk2 2mgAJLHtQVvxS8SfEuZ3HfdCAc6BXOzHsLrkg+WgXZz+/hagioZyrR4nRYziMREwK/ vikjwtFWG7AbBwaDBREXXri8AmZPIbiyXnW+P39Lsu3pf1hKYASJYHM06uuGq1hGTp oQHGtrZ5ee/AzOYFZzvLg1z/PAZXMSwwLsYmx1xD+V0xI8MygiobcRAxSXx6srBNJ2 zeCH4oxkPJrwbG/97mesTXGMukmSyfmoCZzlCu3TMqmT3Cr9psvunQq3T3nuwI3YLt GA6FdW51wqWOA== X-Clacks-Overhead: "GNU Terry Pratchett" X-CM-Score: 0.00 X-CNFS-Analysis: v=2.4 cv=WcAf1mtX c=1 sm=1 tr=0 ts=653cf395 a=6xlXYeu7y0jVCiZ/8t1ozg==:117 a=6xlXYeu7y0jVCiZ/8t1ozg==:17 a=kj9zAlcOel0A:10 a=iJzMXNBmAAAA:8 a=WcxHNsqOAAAA:8 a=gu6fZOg2AAAA:8 a=JIf7469JWYZH4qYe8m8A:9 a=CjuIK1q_8ugA:10 a=rHaSJRYq46MA:10 a=-FEs8UIgK8oA:10 a=iYdFQb2vDmRvJMxvpxpI:22 a=WXeDAsiyfwSRC1lbKA7R:22 a=2RSlZUUhi9gRBrsHwhhZ:22 Date: Sat, 28 Oct 2023 11:42:11 +0000 From: Roger Marsh To: stable@freebsd.org Subject: Re: Version of OpenSSH in FreeBSD 14.0 Message-ID: <20231028114211.7e0eceaa@opendev.home> In-Reply-To: <5a16a0e2-34c2-4bcf-bfbe-aec980955fcf@twisted.org.uk> References: <20231028083010.0f880289@opendev.home> <5a16a0e2-34c2-4bcf-bfbe-aec980955fcf@twisted.org.uk> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.38; x86_64-unknown-openbsd7.4) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4xfFSuxThi7YRjW4TF2NnPoR9gAf4W0ffulUIk+fUitYWf2WyMMw24XDFma+r1TeTP7Z3NwDlfX3hoaOnSp9XpPkPJfhJ5xwcErJw92D0lo1qk6DzAX6p1 wMdzhH3Jaof0ODo4yjQkSxL3ZnXQXkzMhMtInbRj7u9rEa6nQgifzvwe4DM1Qx177kF5Y07AO74YcA== X-Spamd-Result: default: False [-3.40 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[plus.com:s=042019]; RWL_MAILSPIKE_GOOD(-0.10)[84.93.230.244:from]; RCVD_IN_DNSWL_LOW(-0.10)[84.93.230.244:from]; MIME_GOOD(-0.10)[text/plain]; ONCE_RECEIVED(0.10)[]; MLMMJ_DEST(0.00)[stable@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[plus.com:+]; DMARC_NA(0.00)[plus.com]; ASN(0.00)[asn:6871, ipnet:84.92.0.0/15, country:GB]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_NA(0.00)[no SPF record] X-Rspamd-Queue-Id: 4SHd1N3zGCz4XGg X-Spamd-Bar: --- On Sat, 28 Oct 2023 10:45:36 +0100 Pete French wrote: > On 28/10/2023 09:30, Roger Marsh wrote: > > Does 14.0 have OpenSSH 9.3 or OpenSSH 9.4? > > > > This thread: > > > > Delay in starting programs on FreeBSD via ssh after upgrade OpenBSD from 7.3 to 7.4 > > > on openssh-unix-dev@mindrot.org > > > This one ? > > https://lists.mindrot.org/pipermail/openssh-unix-dev/2023-October/041033.html > Yes. > Just looking at that thread and not being able to quite work out what > the solution was there (it refers to another thread I cant find). But > it seems to be ObscureKeystrokeTiming, yes ? > Thread you cannot find is: https://marc.info/?l=openbsd-misc&m=169824753806131&w=2 > There are no references to that in the config on FreeBSD 14, so I assume > it stays on, and at the default. Did this turn out to be the cause of > the issue, and did disabling it fix the problem ? > On OpenBSD the default for ObscureKeystrokeTiming is on, value 'yes', and it can be turned off, value 'no'. Intervals can be specified too: quoting OpenBSD 7.4 'man ssh_config' for ObscureKeystrokeTiming "The default is to obscure keystrokes using a 20ms packet interval.". Disabling fixed the problem. > I have to admit I hadn't tried ssh'ing *out* of my test 14 box > until today, but a quick try doesnt show any delays to anything. > However I don't have anything running OpenBSD. > One of the posts in the thread on openbsd-misc has a table on this sort of ssh'ing combination (for OpenBSD only). One of the replies on the openbsd-misc thread pointed out why one of the logins took a long time! (It does not get used during upgrades which is why I did not notice the problem till 'too late'.) > -pete. > > PS: I did see an upgraded ssh cause delays in login a while ago. But > that turned out to be dropping support for old key exchange algorithms. > If the box you are sshing into is somewhat underpowered then it can > take longer to do the key echange, as the newer algorithms are far > slower. In my case the delay was 30 seconds to login. > Delay seems to be about five seconds, measured by mutterring "One thousand and one, ..., ... and five" while waiting. The ssh_config entry for ObscureKeystrokeTiming talks about fake keystroke packets after typing ceases. I had not considered the possibility of key exchange problems. > Apologies for the reference to OpenSSH 9.4. On OpenBSD 7.4 both 'ssh -V' and 'sshd -V' give 'OpenSSH_9.5, LibreSSL 3.8.2'. Looks like the problem will go away after I upgrade to FreeBSD 14.0. Roger From nobody Sat Oct 28 12:37:40 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHfFN0WPlz4ynqd for ; Sat, 28 Oct 2023 12:37:44 +0000 (UTC) (envelope-from pete@twisted.org.uk) Received: from toybox.twisted.org.uk (toybox.twisted.org.uk [178.250.76.50]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHfFL2wBkz4bw0 for ; Sat, 28 Oct 2023 12:37:42 +0000 (UTC) (envelope-from pete@twisted.org.uk) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=twisted.org.uk header.s=tbx header.b=GGKkEsrE; spf=pass (mx1.freebsd.org: domain of pete@twisted.org.uk designates 178.250.76.50 as permitted sender) smtp.mailfrom=pete@twisted.org.uk; dmarc=pass (policy=none) header.from=twisted.org.uk DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=twisted.org.uk; s=tbx; h=Content-Transfer-Encoding:Content-Type:In-Reply-To :From:References:To:Subject:MIME-Version:Date:Message-ID:Sender:Reply-To:Cc: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=YDKGVj7wbXAV5WermY5cOFYPQ/nE5/I6vzYUmDLAygc=; t=1698496662; x=1699360662; b=GGKkEsrE2Sl5PND6pMiTWE/xqsjz9BXr3/YWB/ek+pTJW7IG9mL2YhmWJyX/zIg81HdPHcWYQDt muYwXnQ8SmRrecHYV+57wxI96p7tP29bMaLDLxcyhojg5BWkAM8o5vEA2aAHemDhgR84DprNND3mm 7S+wYbGExxKxRz0sP2QhxpzWRvFuGbGPS5mvbKyO5uRZ2fio/hDy6b2Q4bg8QpogFCTKbHLOAuv0v 1j5ZX4cvfUdzlgCqhw6QrmndC+0SWgjVu3wpEJibAsO2NjfHOqlW/rjPZRM248/maOBi8cCgnLU8Q rMB5XK+9D2DYKuiV94NpMUi/ahGJv/cStg3g==; Received: from mailnull by toybox.twisted.org.uk with spamc-scanned (Exim 4.95 (FreeBSD)) (envelope-from ) id 1qwiZQ-000Byq-Pl for stable@freebsd.org; Sat, 28 Oct 2023 12:37:41 +0000 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on toybox.twisted.org.uk X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.5 X-Spam-Score: -1.0 () Received: from balta.twisted.org.uk ([2001:470:6cc4:1::57]) by toybox.twisted.org.uk with esmtpsa (TLS1.3) tls TLS_AES_128_GCM_SHA256 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1qwiZQ-000Bym-Nk for stable@freebsd.org; Sat, 28 Oct 2023 12:37:40 +0000 Message-ID: Date: Sat, 28 Oct 2023 13:37:40 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Version of OpenSSH in FreeBSD 14.0 Content-Language: en-GB To: stable@freebsd.org References: <20231028083010.0f880289@opendev.home> <5a16a0e2-34c2-4bcf-bfbe-aec980955fcf@twisted.org.uk> <20231028114211.7e0eceaa@opendev.home> From: Pete French In-Reply-To: <20231028114211.7e0eceaa@opendev.home> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-spamc-toybox: true X-transport-toybox: lookuphost X-Spamd-Result: default: False [-3.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[twisted.org.uk,none]; R_SPF_ALLOW(-0.20)[+ip4:178.250.76.50/32]; R_DKIM_ALLOW(-0.20)[twisted.org.uk:s=tbx]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:12290, ipnet:178.250.72.0/21, country:GB]; MLMMJ_DEST(0.00)[stable@freebsd.org]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[twisted.org.uk:+]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHfFL2wBkz4bw0 X-Spamd-Bar: --- On 28/10/2023 12:42, Roger Marsh wrote: > Thread you cannot find is: > > https://marc.info/?l=openbsd-misc&m=169824753806131&w=2 Ah, thanks. I tried the OpenBSD site, but it seemed to require a login. > On OpenBSD the default for ObscureKeystrokeTiming is on, value 'yes', and it can be turned off, value 'no'. Intervals can be specified too: quoting OpenBSD 7.4 'man ssh_config' for ObscureKeystrokeTiming "The default is to obscure keystrokes using a 20ms packet interval.". > > Disabling fixed the problem. The default in FreeBSD 14 is for ObscureKeystrokeTiming to be on as well. I just did an 'ssh -G' and checked that. Am glad I know about this though, as I know its something to try if I start seeing issues when I upgrade, so this has been very useful ;) > Looks like the problem will go away after I upgrade to FreeBSD 14.0. Hopefuly! Could you run it up on a VM and check ? I never run point releases, so I will be going to 14-STABLE as soon as 14.0 comes out so my tests are not on exactly the same thing. But should be close enough. Am doing another 14 upgrade today on my test machine to get a minor upgrade to ssh in fact... -pete. From nobody Sat Oct 28 14:26:08 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHhfh3Sr5z4ytkh for ; Sat, 28 Oct 2023 14:26:20 +0000 (UTC) (envelope-from dufresnep@zoho.com) Received: from sender4-pp-o91.zoho.com (sender4-pp-o91.zoho.com [136.143.188.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHhfg1kT4z3GgC for ; Sat, 28 Oct 2023 14:26:19 +0000 (UTC) (envelope-from dufresnep@zoho.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zoho.com header.s=zm2022 header.b=SClvUDNT; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (mx1.freebsd.org: domain of dufresnep@zoho.com designates 136.143.188.91 as permitted sender) smtp.mailfrom=dufresnep@zoho.com; dmarc=pass (policy=reject) header.from=zoho.com ARC-Seal: i=1; a=rsa-sha256; t=1698503175; cv=none; d=zohomail.com; s=zohoarc; b=dAfK9n6tXb4YFWBlrJK7XvuxQ3QVaf2MtmfbQGMB3MnA68pnrmKKCAVrvjCrUMFyCwyXfTms4Oq304u52E27NwOGUIOR/MXXCUzskVtGCU5P2YdRdjfP6qYLGaFQBeiS05SK3vyQOELSbaNGB3TanJwh0AkIHmNzAayna6DvWP0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1698503175; h=Content-Type:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=IVEKHECWM3y6W3/vFWv0XYlPOCDrnBywrowBvoM7DAg=; b=hqWObWpe7vubXE+ojF3B59olssWZVgx7PjmCkATXA3Qr5fCZDchxWumYVB2xP4dLblspsNg/uzTBDgo2hN45ESAtWHLz1BkxFGkhgoEhciWmzyirrz3VJfy5qQk0/gCGZ6n0qTf+Kpa3NIcPlbq9uto/JL0ve1nXfPBKL/g7jnM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=dufresnep@zoho.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1698503175; s=zm2022; d=zoho.com; i=dufresnep@zoho.com; h=Date:Date:From:From:To:To:Message-Id:Message-Id:In-Reply-To:Subject:Subject:MIME-Version:Content-Type:Feedback-ID:Reply-To:Cc; bh=IVEKHECWM3y6W3/vFWv0XYlPOCDrnBywrowBvoM7DAg=; b=SClvUDNThnUaLVfS3AnVwCKHK4B7Npq/Wrh9seiwZwUr5Z3VOPKmfJcyjSe18ePr H7clAGzE9ZEglpkOh4krs8bDsWMzbdRYBP5N1PqhbJwt1k+V+L8xMlZj/3oPzznTZ31 3oibSKk+tPqh/O0BMF8m5m9iFY0EHsrlrxBOrFYI= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1698503168150880.7160391520915; Sat, 28 Oct 2023 07:26:08 -0700 (PDT) Received: from [66.129.153.16] by mail.zoho.com with HTTP;Sat, 28 Oct 2023 07:26:08 -0700 (PDT) Date: Sat, 28 Oct 2023 10:26:08 -0400 From: Paul Dufresne To: "stable" Message-Id: <18b76ad9082.eb31f0a040143.2601069948522976587@zoho.com> In-Reply-To: Subject: So much longer to update than install fresh (+ no install feedback) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_103202_526163099.1698503168130" Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Feedback-ID: rr08011228bfe6fc62bd2447792a799f730000307367b66f45758a0efe55e8b6f2c8a2caea2eec095508b2077e:zu08011227c23e1a3b30a5816ab7ec8b61000023334c660d5baeaff75aa46da70ba7960b7919b6183905590c:rf0801123244def3640a29ebe7c0f71aa40000117be9ca8cd5daa60c5334a3d73a2e6b567ee4c5e8bd1846f746d532699b31ec0dbf4d9e:ZohoMail X-Spamd-Result: default: False [-4.88 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.89)[-0.888]; DMARC_POLICY_ALLOW(-0.50)[zoho.com,reject]; R_SPF_ALLOW(-0.20)[+ip4:136.143.188.0/24]; R_DKIM_ALLOW(-0.20)[zoho.com:s=zm2022]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:2639, ipnet:136.143.188.0/23, country:US]; MLMMJ_DEST(0.00)[stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[zoho.com]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[zoho.com:+]; FREEMAIL_FROM(0.00)[zoho.com]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[136.143.188.91:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[zoho.com:dkim] X-Rspamd-Queue-Id: 4SHhfg1kT4z3GgC X-Spamd-Bar: ---- ------=_Part_103202_526163099.1698503168130 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit At first, I have tried to "freebsd-update -r 14.0-BETA3 upgrade" my 13.2 release (quite experimental install). I did not knew if I was supposed to go from 13.2 to BETA1, then BETA2, then BETA3... I still don't know. But after seeing it not showing anything for about 10 mins while showing "Installing updates..." I concluded something was going wrong, and interrupted it (reboot). I did take back the command, let it go for about 2.5h, and then done the same thing (stop it by force). The thing is that doing an install of 13.2 take about 10-15 mins. And so I am not much expecting so long without feedback for an upgrade. I have tried to "pkg-static -f upgrade" packages before the end of "freebsd-update -r something upgrade" (gone bad, all command missing some file), that I never seen succeed from 13.2. Even after reinstalling 13.2 and trying to upgrade to 14.0-BETA1, I was never patient enough to finish it. I had gone to 13 from 12, where I was upgrading at each BETA release that way. I don't really know if 13.2 take much more time than 12 was to upgrade... At the end I was having a feedback with "ps a" on an other terminal... seeing most of the time an install command,,, sometime ungzip. Wish the upgrader would show something like: "(54/23416 updates) llvm 15.2 (15/4600 files)" or maybe just: "54/23416 updates". The disk is a Barracuda 7200. 10 250 Gbytes (not a SSD). ------=_Part_103202_526163099.1698503168130 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =
At first, I have tried to "freebsd-update -r 14.0-= BETA3 upgrade" my 13.2 release (quite experimental install).
=
I did not knew if I was supposed to go from 13.2 to BETA1, t= hen BETA2, then BETA3... I still don't know.

B= ut after seeing it not showing anything for about 10 mins while showing "In= stalling updates..." I concluded
something was going wrong, a= nd interrupted it (reboot).

I did take back th= e command, let it go for about 2.5h, and then done the same thing (stop it = by force).

The thing is that doing an install = of 13.2 take about 10-15 mins. And so I am not much expecting so long witho= ut
feedback for an upgrade. I have tried to "pkg-static -f up= grade" packages before the end of "freebsd-update -r something upgrade" (go= ne bad, all command missing some file),
that I never seen suc= ceed from 13.2. Even after reinstalling 13.2 and trying to upgrade to 14.0-= BETA1, I was never patient enough to finish it.

I had gone to 13 from 12, where I was upgrading at each BETA release that= way.

I don't really know if 13.2 take much mo= re time than 12 was to upgrade...

At the end I= was having a feedback with "ps a" on an other terminal... seeing most of t= he time an install command,,, sometime ungzip.

Wish the upgrader would show something like: "(54/23416 updates) llvm 15.2= (15/4600 files)" or maybe just: "54/23416 updates".

The disk is a Barracuda 7200. 10 250 Gbytes (not a SSD).

------=_Part_103202_526163099.1698503168130-- From nobody Sat Oct 28 14:36:59 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHhv94971z4yv17 for ; Sat, 28 Oct 2023 14:37:09 +0000 (UTC) (envelope-from dufresnep@zoho.com) Received: from sender4-pp-o90.zoho.com (sender4-pp-o90.zoho.com [136.143.188.90]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHhv84qbmz3J3D for ; Sat, 28 Oct 2023 14:37:08 +0000 (UTC) (envelope-from dufresnep@zoho.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zoho.com header.s=zm2022 header.b=QJbjiesR; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (mx1.freebsd.org: domain of dufresnep@zoho.com designates 136.143.188.90 as permitted sender) smtp.mailfrom=dufresnep@zoho.com; dmarc=pass (policy=reject) header.from=zoho.com ARC-Seal: i=1; a=rsa-sha256; t=1698503824; cv=none; d=zohomail.com; s=zohoarc; b=dxXZ2hQcyLugAp31OhMyenFJWCJYoGJT68kojz+amxXbzPwDybkVmx8UF2E57Odfdm+F4Gc/LJUsw6Q+y+7bdthTcqtin5xp+zOCuCLJmxb8kugRZIooZQxIDjlC3TmDds5ugJZoeDZoku0tztjjUyaqYByjGmi8rxAEkS16Mdg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1698503824; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=3FB/ZDs1TmsNkXCH3OMElKLBtZ+fu6hmpMyB753ukl8=; b=CYyVQ3H7SmKP9b60/8wK0YBw9ghVelgF7t3l+gtvH7fkjHimHgJViU8vo7NwEQ6Es8uNrQ/uLZeoy3Bk8kMl5VA4XU7gaNeFpQKsoO7ApTnWeXmA6LqfKLzZm1mly5XCsBF1hBLZEd2h91I+hFLXXrHaFvHqcyJldW3jHBZsReI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=dufresnep@zoho.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1698503824; s=zm2022; d=zoho.com; i=dufresnep@zoho.com; h=Date:Date:From:From:To:To:Message-ID:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID:Message-Id:Reply-To:Cc; bh=3FB/ZDs1TmsNkXCH3OMElKLBtZ+fu6hmpMyB753ukl8=; b=QJbjiesRGx+lEO52eIHWYrR3nEw5rNVf9eghSPO8XdMUIboR7pWdHLOpjyNcgroS Trrqi4HU1GAvWR0Yy79SOtcxmHM1aM0GBPnFwqmC5eAy6MIZd58XIs++Odi0aCK3AHZ P+nfpSN4jPE/ClWX5IMYeRTwxyi1N/sLF1BodGIo= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1698503819022363.9347912158936; Sat, 28 Oct 2023 07:36:59 -0700 (PDT) Received: from [66.129.153.16] by mail.zoho.com with HTTP;Sat, 28 Oct 2023 07:36:59 -0700 (PDT) Date: Sat, 28 Oct 2023 10:36:59 -0400 From: Paul Dufresne To: "stable" Message-ID: <18b76b77eff.c8f6f84840204.8019000973337529039@zoho.com> In-Reply-To: <18b76ad9082.eb31f0a040143.2601069948522976587@zoho.com> References: <18b76ad9082.eb31f0a040143.2601069948522976587@zoho.com> Subject: =?UTF-8?Q?R=C3=A9=C2=A0:_So_much_longer_to_update_than_i?= =?UTF-8?Q?nstall_fresh_(+_no_install_feedback)?= List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Feedback-ID: rr080112281acf72e76b67afbaf8765f7f0000359bcb85fb9b7e3c078035f630d4348976d45ec869b5e3665196:zu08011227556ba1a6dafc5ebdd671d7bc00001d5f34f0188dc28136816eca07dda06894ee65613f6a4836ac:rf080112319da343bc94f14edf00a7a4870000c695ea69bcbb6690ba399f0abd23e46461aa433b0c0cf1a7357bf639f7d8313aa72f50:ZohoMail X-Spamd-Result: default: False [-4.89 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.90)[-0.898]; DMARC_POLICY_ALLOW(-0.50)[zoho.com,reject]; R_SPF_ALLOW(-0.20)[+ip4:136.143.188.0/24]; R_DKIM_ALLOW(-0.20)[zoho.com:s=zm2022]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:2639, ipnet:136.143.188.0/23, country:US]; MLMMJ_DEST(0.00)[stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[zoho.com]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[zoho.com:+]; FREEMAIL_FROM(0.00)[zoho.com]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[136.143.188.90:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[zoho.com:dkim] X-Rspamd-Queue-Id: 4SHhv84qbmz3J3D X-Spamd-Bar: ---- Sorry, I meant freebsd-update install is missing feedback (and seems much longer than upgrade that have good feedback in my opinion). From nobody Sat Oct 28 19:26:02 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHqJm3hSvz4xwTF for ; Sat, 28 Oct 2023 19:26:16 +0000 (UTC) (envelope-from dufresnep@zoho.com) Received: from sender4-pp-o91.zoho.com (sender4-pp-o91.zoho.com [136.143.188.91]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHqJk1ZJRz4J7p for ; Sat, 28 Oct 2023 19:26:14 +0000 (UTC) (envelope-from dufresnep@zoho.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zoho.com header.s=zm2022 header.b=cC5hlUZv; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (mx1.freebsd.org: domain of dufresnep@zoho.com designates 136.143.188.91 as permitted sender) smtp.mailfrom=dufresnep@zoho.com; dmarc=pass (policy=reject) header.from=zoho.com ARC-Seal: i=1; a=rsa-sha256; t=1698521169; cv=none; d=zohomail.com; s=zohoarc; b=Z0xwXX+2SxVIiWN9tPlX7x+mzyLjaWaAwTLFx0yoJL9uv/wLfb6DiS4dIydHpCpgm9MHQmYq2DD/evUgDUOdvx7O4CmbfykABX5SXE3LDiP2mYLSn6UkHeb/jLN99EdeC/16arTrVd9wZeFr+v2Gwp15kBTaF21BDtJHMQB/P6s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1698521169; h=Content-Type:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=0hhMSlEQjroAF1l2Vxe4DRtYKN9m7EotI0D0XSF2st4=; b=aNxOomTiHZWRgJOqCYUxnlGa6OUg1mB8fekFbmF4W0WTFrkqn9t9lisKAyPk7xNFGJJrTP8o+jQIKZ9iXmz3JPniswtUZpWHp7rpnYosm9TxKFUgvzR4hEdXMK21vJhtnbozQo49W4ixB2m+qKFy/7ng1FTOuTynAEFhK36GMjo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=dufresnep@zoho.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1698521169; s=zm2022; d=zoho.com; i=dufresnep@zoho.com; h=Date:Date:From:From:To:To:Message-Id:Message-Id:Subject:Subject:MIME-Version:Content-Type:Feedback-ID:Reply-To:Cc; bh=0hhMSlEQjroAF1l2Vxe4DRtYKN9m7EotI0D0XSF2st4=; b=cC5hlUZvFxy/XfHZviDBASjQCNSEzOYHj5bOSFXU1rP4UHo+/YWdb2He25a376qA UUmmGf/cOaJ0snQdc54I5osGO1xvZxAAwzBpmFtC8L19I6TfH99OZf31K6KF/n891nW QFjsH183YPSkOADvSHCD5zHLvXXBVttrG9yrY61k= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1698521162542637.739375421396; Sat, 28 Oct 2023 12:26:02 -0700 (PDT) Received: from [66.129.153.16] by mail.zoho.com with HTTP;Sat, 28 Oct 2023 12:26:02 -0700 (PDT) Date: Sat, 28 Oct 2023 15:26:02 -0400 From: Paul Dufresne To: "freebsd-stable" Message-Id: <18b77c0231d.de5e630e41424.270773732049379928@zoho.com> Subject: =?UTF-8?Q?R=C3=A9=C2=A0:_So_much_longer_to_update_than_i?= =?UTF-8?Q?nstall_fresh_(+_no_install_feedback)?= List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_106972_1998043825.1698521162525" Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Feedback-ID: rr0801122858e2da8740443c221d59e33c000012ce382362bf0511ab0bfc20cca2b4acb62a42e9654070e0e919:zu08011227a0293d86ad93841ec206bce1000081440c042b75dd608636a0218938d81df17570e7736eccacf3:rf080112327cf56bb2394fc4e8be3b6c5a000012511762bee0d0a664f63a6ab0566c5cc6c4eb41cda53fa1baebd2d7b802fa99f10f9110:ZohoMail X-Spamd-Result: default: False [-4.64 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.65)[-0.651]; DMARC_POLICY_ALLOW(-0.50)[zoho.com,reject]; R_DKIM_ALLOW(-0.20)[zoho.com:s=zm2022]; R_SPF_ALLOW(-0.20)[+ip4:136.143.188.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:2639, ipnet:136.143.188.0/23, country:US]; FREEMAIL_ENVFROM(0.00)[zoho.com]; RCVD_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[136.143.188.91:from]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[zoho.com:dkim]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[zoho.com:+]; FREEMAIL_FROM(0.00)[zoho.com]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[] X-Rspamd-Queue-Id: 4SHqJk1ZJRz4J7p X-Spamd-Bar: ---- ------=_Part_106972_1998043825.1698521162525 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit I just upgraded from 14.0-RC2 to 14.0-RC3 in only 297 real seconds... so 5 mins (after reboot). That's seems very fast to me. But that's was for about 3k updates... compare to what was (27k * 2 I think) when I was trying to upgrade 13.2 to 14-0. Let's say 54k / 3k * 5 mins =~ 90 mins... I had waited longer than that at some point. I guess I would come back to the idea that feedback (maybe with --verbose) would be nice to have. ------=_Part_106972_1998043825.1698521162525 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: 7bit
I just upgraded from 14.0-RC2 to 14.0-RC3 in only 297 real seconds... so 5 mins (after reboot).
That's seems very fast to me.

But that's was for about 3k updates... compare to what was (27k * 2 I think) when I was trying to upgrade 13.2 to 14-0.

Let's say 54k / 3k * 5 mins =~ 90 mins... I had waited longer than that at some point.

I guess I would come back to the idea that feedback (maybe with --verbose) would be nice to have.

------=_Part_106972_1998043825.1698521162525-- From nobody Sat Oct 28 21:53:47 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHtbC39npz4y560 for ; Sat, 28 Oct 2023 21:53:59 +0000 (UTC) (envelope-from dufresnep@zoho.com) Received: from sender3-pp-o99.zoho.com (sender3-pp-o99.zoho.com [136.143.184.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHtb957YVz4Ybj for ; Sat, 28 Oct 2023 21:53:57 +0000 (UTC) (envelope-from dufresnep@zoho.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zoho.com header.s=zm2022 header.b=OoX2cKA7; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (mx1.freebsd.org: domain of dufresnep@zoho.com designates 136.143.184.99 as permitted sender) smtp.mailfrom=dufresnep@zoho.com; dmarc=pass (policy=reject) header.from=zoho.com ARC-Seal: i=1; a=rsa-sha256; t=1698530033; cv=none; d=zohomail.com; s=zohoarc; b=K76p1kSCaZimgmnhtiCshvYLgj9wSagZn/5ANTQAlzx1r3RUyr+JHG+pf1en/+xXqqd77kDF23gY6lSJ9Z+CMLrSffKsoBkUvX9yQfJOJwoU0d08rzYfDTNJS+eAUMbAleQELRgUz3CScM37udRObYQ3ASrrtSWvM694TGSXlLU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1698530033; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=a1VthXj29lWUTUrOCs7x6dac6JKt+LrLerNITFRE914=; b=REDtJ1ATxZ+Ql2GywMcm6gWgPS25ZBSaJbVYLAl4HWRhzXlGgDiOTJh18q4/0MnkjZUChDAGjMKJtN/rmpvEqeXEzeUNtpeulZGk3uFXUu6Ly9XhyCLxbJ49YwxorNv4NYGiXZAYhX7orQ3mTIkEOEm0sXIgoD2e5TBf3nBR+G4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=dufresnep@zoho.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1698530033; s=zm2022; d=zoho.com; i=dufresnep@zoho.com; h=Date:Date:From:From:To:To:Message-ID:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID:Message-Id:Reply-To:Cc; bh=a1VthXj29lWUTUrOCs7x6dac6JKt+LrLerNITFRE914=; b=OoX2cKA7OFMm2WHOYLffBlkoKEYy27Pyt+h/slGDP7G/eNOeXlJg5Pg2AMjAqq8b 14Zw6PCy9a2UXsW2qvZJp0m3QzMmj82NuwkFaMxoGu91FpD5SyFNUPh8UxvOCfM4lSz 3z0uzF1zDzi+Kb3n2/JUkTx1Xn2BAgogPzq9DYbc= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1698530027813332.67364700337464; Sat, 28 Oct 2023 14:53:47 -0700 (PDT) Received: from [66.129.153.16] by mail.zoho.com with HTTP;Sat, 28 Oct 2023 14:53:47 -0700 (PDT) Date: Sat, 28 Oct 2023 17:53:47 -0400 From: Paul Dufresne To: "stable" Message-ID: <18b78476914.e31ef40044402.789687992348441821@zoho.com> Subject: So much longer to update than install fresh (+ no install feedback) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Feedback-ID: rr080112284368f431aa106f127a1611f90000ef2c7ac97039553330c8697d66f536cb4b8bc97aeb1dc7ca71e9:zu080112274904bebd39a016c625ebc2230000dec0435fd25c6d012d2012d95252e2c1abb9fc989802282352:rf0801123211770d2eef79170fff1fc51e000095c62fcabe8ff5159ef3265bf8d8af5dd3639697b6eebc2196be4f4ff4c74786a4e13742:ZohoMail X-Spamd-Result: default: False [-4.99 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[zoho.com,reject]; R_DKIM_ALLOW(-0.20)[zoho.com:s=zm2022]; R_SPF_ALLOW(-0.20)[+ip4:136.143.184.0/24]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[136.143.184.99:from]; FROM_HAS_DN(0.00)[]; MLMMJ_DEST(0.00)[stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[136.143.184.99:from]; DKIM_TRACE(0.00)[zoho.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[zoho.com]; DWL_DNSWL_NONE(0.00)[zoho.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:2639, ipnet:136.143.184.0/24, country:US]; FREEMAIL_ENVFROM(0.00)[zoho.com]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SHtb957YVz4Ybj X-Spamd-Bar: ---- I realized that my previous message was a bit not precise. So, armed with the knowledge that "freebsd-update install" generally finish, I reinstalled 13.2 (which felt like 5 to 7 mins, did not clocked). Was zfs default install (without ssh server). Then I updated the 44 patches for 13.2, then rebooted. I then used "/usr/bin/time -h freebsd-update -r 14.0-BETA1 upgrade" to clock it (/usr/bin is necessary, else an internal time command is used, -h=humanly readable). It took a bit less than 20 mins to get the 10750 patches, apply them, and then get the 1309 files. Then the "/usr/bin/time -h freebsd-update install" before the reboot took 35.4 seconds. And then the "usr/bin/time -h freebsd-update install" after the reboot took: 42m49s (1m20s user, 1m27s sys). My hardware is a i3-8100 with a Seagate Barracuda 7200.10 of 250 Gbytes. So about 1h to get the first step from 13.2 to 14.0 (still needs Beta2, Beta3, Beta4, Beta5, rc1, rc2 and rc3 to be done). Again... first point is that seems so much slower than a fresh install, second, really not much feedback during install phase of the upgrade. From nobody Sat Oct 28 22:33:36 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHvT12SYnz4y7hM for ; Sat, 28 Oct 2023 22:33:41 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHvT00dVZz4fy2 for ; Sat, 28 Oct 2023 22:33:40 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=JLNugGY6; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=iDfpCzS6; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 61A895C01AA for ; Sat, 28 Oct 2023 18:33:39 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute1.internal (MEProxy); Sat, 28 Oct 2023 18:33:39 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698532419; x=1698618819; bh=7b ZTxO/yNVIr9zwLE9kBEQnAbYaL+/phI1oxuxAcwos=; b=JLNugGY6w3GBsSRL5z A/f080hKBfXSVIAbuktxFHubA08ZN0jkYf6C3majYKSuchOjA+ke92B3BOT+Z8zY dkHYl9mrmfqSKeoMzrSZ55hsh5YC+epmaEOVVQS6BXTHr9/H3jADbABLYktK9clw OVFm/sVzTgn9KiBT7DB9tNf/6AuL8gPWjPqw5HNqZ0rqJtU9ZtharofXQSIavdKJ 8NYMQXSfbs1D4DaJ+nn9MU0ocL8Bua4JzdbjKypWRp0drKiQXnRsvB/ehDKV1sEV JDTd8bL5UiISh9pVrIeJt5c98WkQa153cv0XOAlZTZ8TNGzxHiqKdH7TwXwycSZc NBGQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698532419; x=1698618819; bh=7bZTxO/yNVIr9 zwLE9kBEQnAbYaL+/phI1oxuxAcwos=; b=iDfpCzS6UJxSKzYeT571CXidpfbVG Wt65pqv5hNToMssH1sS3Xv9PDKSitwrFFHWzw4bJBQK9IK8yZUI6oa6f7ts05hmI 4RDUdU8Yq7weqAomHDcTxg4XjUkitpntrwzJIQqJgRg2/VuWcGq1PU2nRWR4kYXN wtL2cTzCH5gy8ZBJRKZiuj/4y+Ms46FQ3c2JLw3VRv82s5dEBLS2yROMMWJT6FXk PCe1XO9ZLZI9lkQQRezw1H530AMD+8dA7cLdppjP5jKm/gFlrg9nHzINUhNYJIXe trYGaOBCEyFgomi399e1kO6JNbllW6CfHYFZZgH2WtdAni/nDBVa0S2zA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleejgdduvdcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeekleduvdelhfeileefgffghfffkedtheellefgudfgvdegkeejjedutdehhe fgueenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehv ohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 28 Oct 2023 18:33:38 -0400 (EDT) Date: Sat, 28 Oct 2023 23:33:36 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SHvT00dVZz4fy2 X-Spamd-Bar: ---- On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: >I have this for ccache FS (zfs): > >dumpster/ccache atime on local I wondered why you have this set to 'on'? I mean I know /var/mail needs it for mail to function. Is it the same wiht ccache ? -- From nobody Sat Oct 28 22:42:14 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHvgC4XMqz4y7jl for ; Sat, 28 Oct 2023 22:42:31 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHvgB0bSbz3CZx for ; Sat, 28 Oct 2023 22:42:29 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="w 8bwUo0"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SHvfz2733z6gTC for ; Sun, 29 Oct 2023 00:42:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :from:from:content-language:references:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698532935; x= 1700347336; bh=kbEXZSA8b60evXSdAVbOABUuoUEp1NU+eP+9E/vvwGE=; b=w 8bwUo0ZRk6Ks6sd580Sd/n7pEcWL9xmCB4SC+2hsJErpDlLWuxhCIZmu9W9osT3v Ag+GybYyWOpchhWAUT4XcAZepkAEzg8LvMOiOG4vYb375C6zUt0eFrvVdbdBF+Xs nte/HfGQ3lWhrzA3qy8uE6Q/uU0rOdM1Da72MwD6siIgvY1obhYo83DcqqPQ04D3 cyzoULSFX+H56pm7NJ8o2N/IzjUd5KHApx88iPA+7VFs8YaGPeUxkTduy1PrNF1b NcVF8yRTO5M60bWYbZGT5xvVC91+i4AtAn5iBcLzApyYyshMrVRE9riSu/2hNA/A SybcbrImAdcoJUuwN5rdQ== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id PCOD2NEj0Yzj for ; Sun, 29 Oct 2023 00:42:15 +0200 (CEST) Message-ID: Date: Sun, 29 Oct 2023 00:42:14 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> Content-Language: en-US From: Guido Falsi Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHvgB0bSbz3CZx X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 29/10/23 00:33, void wrote: > On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: >> I have this for ccache FS (zfs): >> >> dumpster/ccache  atime                 on                     local > > I wondered why you have this set to 'on'? > > I mean I know /var/mail needs it for mail to function. Is it the same > wiht ccache ? > I used to have it off, but recently I turned it on. I have not really properly checked, but I have an hunch that ccache is using atime to decide which files to properly clean up the cache (LRU logic). With atime - off you get atime = mtime, which is not the best option for LRU logic. You want objects to be counted as "hot" if they were created/modified a long time ago, but used yesterday. Recent manual page does talk about atime, also a quick search in the sources reference atime multiple times. So it IS using it. To be on the safe side, I've added this, but as I said I did this recently so I still don't know if it is really helping. -- Guido Falsi From nobody Sat Oct 28 22:49:14 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHvqc28cDz4y81k for ; Sat, 28 Oct 2023 22:49:48 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (vogon.madpilot.net [159.69.1.99]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHvqb45PTz3F3v for ; Sat, 28 Oct 2023 22:49:47 +0000 (UTC) (envelope-from mad@madpilot.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=madpilot.net header.s=bjowvop61wgh header.b="o 4UMpWR"; spf=pass (mx1.freebsd.org: domain of mad@madpilot.net designates 159.69.1.99 as permitted sender) smtp.mailfrom=mad@madpilot.net; dmarc=pass (policy=quarantine) header.from=madpilot.net Received: from mail (mail [IPv6:fd5c:5351:d272::3]) by mail.madpilot.net (Postfix) with ESMTP id 4SHvq04Lpcz6gTC for ; Sun, 29 Oct 2023 00:49:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :references:from:from:content-language:subject:subject:date:date :message-id:received; s=bjowvop61wgh; t=1698533354; x= 1700347755; bh=UIT70dT/y/c8jfSuq7yNWLB28q/npRmc+KMYFGo14Vo=; b=o 4UMpWRL+UzVUgeqqU3++oPx3WYeIKRNLXz0yj5XvlPtm+UKpRkhbyACKPcvyh/ED WL4kF/XIboe2Xmm+BHsG3yAGbkWSKAZCRIF6FtFEhRm9XBcOX30Q7CW5ILjKtP1q HNQ2wCToZlAxSOmHw/XdGs5Z6T1mC5d6MxXNSThFUWDpf5wyrSqLKvLbVdH0nfi7 LfopJnD65HGdriqZ7CSmV+N98xE+V+v3oMtf2wDd+42qU7LpGPmERQ9gVbDRAtD9 CGAYrW3d+ek9ZNxMuVqlxwTuFvEw+qXPdo+4zDbZDBefS+QiSBE1pKpLHMwFM/px RHpr2BC7OP4rOmJLkIP8g== Received: from mail.madpilot.net ([IPv6:fd5c:5351:d272::3]) by mail (mail.madpilot.net [IPv6:fd5c:5351:d272::3]) (amavisd-new, port 10026) with ESMTP id jOPWCDovD-2P for ; Sun, 29 Oct 2023 00:49:14 +0200 (CEST) Message-ID: Date: Sun, 29 Oct 2023 00:49:14 +0200 Subject: Re: periodic daily takes a very long time to run (14-stable) Content-Language: en-US From: Guido Falsi To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> Autocrypt: addr=mad@madpilot.net; keydata= xsBNBE+G+l0BCADi/WBQ0aRJfnE7LBPsM0G3m/m3Yx7OPu4iYFvS84xawmRHtCNjWIntsxuX fptkmEo3Rsw816WUrek8dxoUAYdHd+EcpBcnnDzfDH5LW/TZ4gbrFezrHPdRp7wdxi23GN80 qPwHEwXuF0X4Wy5V0OO8B6VT/nA0ADYnBDhXS52HGIJ/GCUjgqJn+phDTdCFLvrSFdmgx4Wl c0W5Z1p5cmDF9l8L/hc959AeyNf7I9dXnjekGM9gVv7UDUYzCifR3U8T0fnfdMmS8NeI9NC+ wuREpRO4lKOkTnj9TtQJRiptlhcHQiAlG1cFqs7EQo57Tqq6cxD1FycZJLuC32bGbgalABEB AAHNHkd1aWRvIEZhbHNpIDxtYWRAbWFkcGlsb3QubmV0PsLAeAQTAQIAIgUCT4b6XQIbAwYL CQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQGuaGDlbL0pOWigf/YVTVf3+ZRnzeGP7CjGV1 Wrrxzjc8h8W64NZasV0XLHGFjl5MYwtm9jJ9gbL8Ubtqstey7lYpjOk2fG6YDhY5eptWCpR6 1QqYrioukhCfKbodSk6PnIZcx719nJVK2P7ihdFEN78TavpBwqIf9hGEcKkMpbRFQv1mYvXD hKVwQGY+8bkH/a/pAWmIyD4qMfKCMurH5DexxEt5SYWu5BB5hd/DWyZ0wuZ+F79KMPzLBPJW 5cpdLNbrvenSqFZGJEGhtTp7GFJJr6lTy8VLBArxmFHiY5jGyR45eZEGDcz86FfGgvPnnpi7 aNCc/ROdF7fnZYPh8uZGGjQbd4EYK4xMzc7BTQRTEHtBARAAoWGsNx6g90r8gcNKaiPpJBiK y8ztV2FyV5LsT0OgQBW3vIxt/odtsxVNNjpyS/BNZCyzLAsFc1WrGBzhYsmPN9SGB5/5YTvk zf5YViU5VAsZlj/MRWCZrWtpic4c0A7N4csOYReNtk/q8YB4PIFsZ9A+kTuoZhnu5t5PdfBA 74+SVwKu84+PZk9wDEY1LbFVT8vM42oKsmoswlIhwJ2xuJI/gbk+cMUe0yiRpNjo4Svw4RB8 4B6uFwdRr/PtS7xi2Zqoof5AaQT9YSBpGpKJOe/Qk5MP4PF6Fqq+go89n77Y2kJkwcHaLoD/ GJ+ZDASIiMRe1y54FHOQ1RCTGGpnJLXdKuGhwv3J21pU8HNlq0ASNQMMQmYAwtUWzjmp/KEy I1qkcmjafcxb8TmiaoK8SQN1Zf96fc/sIrZN6Z5oOCEyyCQ0prH/PTA2jlRkKQ487PTGk2JS KU5VuS57Nlk2DrnvjWp57aV9eFAhpnrrJPuGmFz83/Pc8gC0t7N7i7VVHYRcC5naxYB2UoI1 OUkyxpT/HvQFXXVZ3/KmdXMzrx191AggCPWIwUAP+VcaURSYpeDk6/ZVAOVOe1ChqcJisCD7 wK20/OOvJ2AtkWreGu1CZ9zSx7nK/VYdLr34GxQ4bT1G+9rBQNnFSNbX2TJ431Mdo1GCjDeR K4CtSnrNKYkAEQEAAcLAXwQYAQgACQUCUxB7QQIbDAAKCRAa5oYOVsvSkw3nCADhsKRf+rAR ULTpOh5HoLam62ZJZAyCkNqqu/rke5uj5AaaDY/h7BNhBDiDqhhZLTeofGpVVaErPsWN+tX5 0fypsIt9KAhy90GFrtrIZlWuyK4wsoZvDfp9yaRk+lIM58dw/Rcfxn670JaPTFSRPECVn/uL qBhJSkbYlY212YT9fxVUTJe6wIvDLQrQEjrQD/h1FMhfcLhAqsndltRd6DPvTKeMd/6VAxn0 hkoBKhEy5LkWjM9CHppu+bBkQ91/kj2uJQSXO8euonwHHS3c+6N2i2H7I0emcHGu07wuRB2t Dnw/RLBxohffdPZT2kbxuG7lhVHzwVDw5DRwSw8GkOdy In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-2.00 / 15.00]; MISSING_MIME_VERSION(2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[madpilot.net,quarantine]; R_SPF_ALLOW(-0.20)[+mx:c]; R_DKIM_ALLOW(-0.20)[madpilot.net:s=bjowvop61wgh]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:24940, ipnet:159.69.0.0/16, country:DE]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[madpilot.net:+]; TO_DN_NONE(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_ONE(0.00)[1]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SHvqb45PTz3F3v X-Spamd-Bar: - List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org On 29/10/23 00:42, Guido Falsi wrote: > On 29/10/23 00:33, void wrote: >> On Fri, Oct 27, 2023 at 05:38:58PM +0200, Guido Falsi wrote: >>> I have this for ccache FS (zfs): >>> >>> dumpster/ccache  atime                 on                     local >> >> I wondered why you have this set to 'on'? >> >> I mean I know /var/mail needs it for mail to function. Is it the same >> wiht ccache ? >> > > I used to have it off, but recently I turned it on. I have not really > properly checked, but I have an hunch that ccache is using atime to > decide which files to properly clean up the cache (LRU logic). > > With atime - off you get atime = mtime, which is not the best option for > LRU logic. You want objects to be counted as "hot" if they were > created/modified a long time ago, but used yesterday. > > Recent manual page does talk about atime, also a quick search in the > sources reference atime multiple times. So it IS using it. > > To be on the safe side, I've added this, but as I said I did this > recently so I still don't know if it is really helping. > Correcting myaelf: The version we currently have in the ports tree (3.7.x) does not care for atime, as far as I can see in the source code. It checks mtime only. Newer versions do use it though! -- Guido Falsi From nobody Sat Oct 28 22:58:11 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHw1L3CtQz4y8kX for ; Sat, 28 Oct 2023 22:58:14 +0000 (UTC) (envelope-from void@f-m.fm) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHw1K3cNjz3GdD for ; Sat, 28 Oct 2023 22:58:13 +0000 (UTC) (envelope-from void@f-m.fm) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=f-m.fm header.s=fm3 header.b=R+O7JQB6; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=GBe5PoB0; spf=pass (mx1.freebsd.org: domain of void@f-m.fm designates 66.111.4.28 as permitted sender) smtp.mailfrom=void@f-m.fm; dmarc=pass (policy=none) header.from=f-m.fm Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id 6386F5C01B7 for ; Sat, 28 Oct 2023 18:58:13 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Sat, 28 Oct 2023 18:58:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=f-m.fm; h=cc :content-type:content-type:date:date:from:from:in-reply-to :in-reply-to:message-id:mime-version:references:reply-to:sender :subject:subject:to:to; s=fm3; t=1698533893; x=1698620293; bh=Ah vBr3MctWjPod5ZeOyFlJ03+CplTHYOCGNHBvkS9+M=; b=R+O7JQB622eKpGKBYR MhukJrA1lzlcu8w9VYhh3oFyHdUVNqwk3xanOctvvo5ASDIKHI2LYjoGOA9fGMZi xDjXMUqTdHjvnYn+y73b4PfR1Lltl2CMOaOGXfcDII52zutXF67Lj8jK3Ii66d2o 6mOMtkbNqV/G+y2ugwDvxi/jbdeyXKpEb5gB10C4LAKBqRaY921yulCLyeeFSAXk 6m2gQg+WGBjKToOxyNebvzuwJVOoakM+6c4t7QvhpF8wG++QG3xFsQDMNLYzb4QR XB1fhbya7pLHOdlMZvkJ39oQ4uPG6WCtH+ztiKbQOipeufg30IGgDpY7xMR3vu1x BfvQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:content-type:date:date :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to :message-id:mime-version:references:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1698533893; x=1698620293; bh=AhvBr3MctWjPo d5ZeOyFlJ03+CplTHYOCGNHBvkS9+M=; b=GBe5PoB0YSOySB911mi1xXuAmDLTP lV+tmBmMb67IDcy2PPfiCNpx1Wv3izrrW9mQd1EGUQ/6HhEZxWpD8CmAa4psQWmw MzWPlSg2wzI7RIPDjxJHEchG93A7irMipT137wbk04zu04XzRTbos++n3npW6eal 1N/dQDUUF7/XeQCFDAecZkPURkaET45LPU6LP1ch/R28VNTUcnAPyvaJGhIzHhjy 8oilbHmhdF6p47kCEQTzg/n/Y3RVIS8HMWhRisot8I59rCXUpZqYoQOlCvYostWE 04ZHAspt0hRF2SRI1Y4USGIXerdOpNPQPbxxUwdn2F7/UmyWO883N9UwQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrleejgddujecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjsehttdertd dttddvnecuhfhrohhmpehvohhiugcuoehvohhiugesfhdqmhdrfhhmqeenucggtffrrght thgvrhhnpeekleduvdelhfeileefgffghfffkedtheellefgudfgvdegkeejjedutdehhe fgueenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpehv ohhiugesfhdqmhdrfhhm X-ME-Proxy: Feedback-ID: i2541463c:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Sat, 28 Oct 2023 18:58:12 -0400 (EDT) Date: Sat, 28 Oct 2023 23:58:11 +0100 From: void To: freebsd-stable@freebsd.org Subject: Re: periodic daily takes a very long time to run (14-stable) Message-ID: Mail-Followup-To: freebsd-stable@freebsd.org References: <1122335317.4913.1698407124469@localhost> <794932758.6659.1698413675475@localhost> <8a5c1f89eb5842f20e463c7d60538e598da3bc46.camel@gromit.dlib.vt.edu> <767774d4-50b5-47c8-9cf1-c0c99fe33e00@madpilot.net> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: X-Spamd-Result: default: False [-4.70 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[f-m.fm,none]; R_DKIM_ALLOW(-0.20)[f-m.fm:s=fm3,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:66.111.4.28:c]; MIME_GOOD(-0.10)[text/plain]; RWL_MAILSPIKE_GOOD(-0.10)[66.111.4.28:from]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.28:from]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; FREEMAIL_FROM(0.00)[f-m.fm]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; DKIM_TRACE(0.00)[f-m.fm:+,messagingengine.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[f-m.fm]; RCVD_VIA_SMTP_AUTH(0.00)[] X-Rspamd-Queue-Id: 4SHw1K3cNjz3GdD X-Spamd-Bar: ---- On Sun, Oct 29, 2023 at 12:49:14AM +0200, Guido Falsi wrote: >Correcting myaelf: > >The version we currently have in the ports tree (3.7.x) does not care >for atime, as far as I can see in the source code. It checks mtime >only. > >Newer versions do use it though! Thanks for checking. I've set it to on and think I'll leave it on as long as there's no e.g performance penalty. One less thing to think about changing again when the port moves versions. -- From nobody Sat Oct 28 23:01:35 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SHw5Q4qB7z4y8v6 for ; Sat, 28 Oct 2023 23:01:46 +0000 (UTC) (envelope-from dufresnep@zoho.com) Received: from sender3-pp-o98.zoho.com (sender3-pp-o98.zoho.com [136.143.184.98]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SHw5P4znlz3J52 for ; Sat, 28 Oct 2023 23:01:45 +0000 (UTC) (envelope-from dufresnep@zoho.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zoho.com header.s=zm2022 header.b=cxw9i1zw; arc=pass ("zohomail.com:s=zohoarc:i=1"); spf=pass (mx1.freebsd.org: domain of dufresnep@zoho.com designates 136.143.184.98 as permitted sender) smtp.mailfrom=dufresnep@zoho.com; dmarc=pass (policy=reject) header.from=zoho.com ARC-Seal: i=1; a=rsa-sha256; t=1698534102; cv=none; d=zohomail.com; s=zohoarc; b=AKLjJt1zT08nc81up19AjI12WHTW6fVRqnUmgDoVfBDdSXIrT/soPaN6svgVrMKw/IlJS58U/nGdNnS7h+zfNpZR/9vNYKTwLGMN5Mg2tr/yuhs98d3sAQHf0v3rnEELxajWwKdZAFUVE+4InKd/iGj3HzPyt5CMhVb5ZvoSekE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1698534102; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=58bYCxFVT3bOGAiXpf8qvPKy4A57VZMVXgk9E+GiUZ4=; b=d6zoZ7bsd+LZHH8MeixLvQZ0ehQPtBGq2kvHIRUXcv4lStvpyiuW17YGnlcZmtxkeal2bL8mII6Utgyy+MKPySOWycWLe0nbbJLeZ1/RUj1+tifnYF81HqG59PP1TYtd0CmLGNC79YNYjWbCzhkbs8tGPkukiQGwAwHNffGTFkg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zoho.com; spf=pass smtp.mailfrom=dufresnep@zoho.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1698534102; s=zm2022; d=zoho.com; i=dufresnep@zoho.com; h=Date:Date:From:From:To:To:Message-ID:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type:Content-Transfer-Encoding:Feedback-ID:Message-Id:Reply-To:Cc; bh=58bYCxFVT3bOGAiXpf8qvPKy4A57VZMVXgk9E+GiUZ4=; b=cxw9i1zw2dNUPQwcQgpFJAzQR2mmKAd9UFB4nMoJbSf4ypUBoRDR3E4+ItJ4O096 mhgJVJyycEkJsdmHSH1DEnuuHWJK+OC8w97wUkLXCEPxZQJ8mo58U+krc5JXdyBBEPv uH12icOUqdi0aWYTdNs24tGOB7jQtGK4jTEqNAxA= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1698534095798707.2900917293659; Sat, 28 Oct 2023 16:01:35 -0700 (PDT) Received: from [66.129.153.16] by mail.zoho.com with HTTP;Sat, 28 Oct 2023 16:01:35 -0700 (PDT) Date: Sat, 28 Oct 2023 19:01:35 -0400 From: Paul Dufresne To: "stable" Message-ID: <18b78857ba5.12376262f44792.1614232842499866698@zoho.com> In-Reply-To: <18b78476914.e31ef40044402.789687992348441821@zoho.com> References: <18b78476914.e31ef40044402.789687992348441821@zoho.com> Subject: =?UTF-8?Q?R=C3=A9=C2=A0:_So_much_longer_to_update_than_i?= =?UTF-8?Q?nstall_fresh_(+_no_install_feedback)?= List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail Feedback-ID: rr08011228f5244449bd17202ed11f803900000cda273627bc2eb30c4711a0502fe2246c852cabb17b91df0dac:zu0801122796eacf7c27eb22445e3cfa3800000ebaa720ca4069ba2346be4f99f093ffa92b56aa313742e55c:rf08011232d46c0db017c29afc97b4c4fa000076f4d7a6e7a387a1e2ae7efd5a7f97d7df0261984d00fc8a09f37e0a3f99f6e344a35ee7:ZohoMail X-Spamd-Result: default: False [-4.99 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[zoho.com,reject]; R_SPF_ALLOW(-0.20)[+ip4:136.143.184.0/24]; R_DKIM_ALLOW(-0.20)[zoho.com:s=zm2022]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; ASN(0.00)[asn:2639, ipnet:136.143.184.0/24, country:US]; MLMMJ_DEST(0.00)[stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[zoho.com]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[zoho.com:+]; FREEMAIL_FROM(0.00)[zoho.com]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[136.143.184.98:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_NONE(0.00)[zoho.com:dkim] X-Rspamd-Queue-Id: 4SHw5P4znlz3J52 X-Spamd-Bar: ---- Turned out that going from 14.0-BETA1 to 14.0-RC3 was significantly faster than from 13.2 to 14.0-BETA1. >From 14.0-BETA1: /usr/bin/time -h freebsd-update -r 14.0-BETA4 upgrade 1810 patchs, 3 files 5m38s real, 2m22 user, 43s sys /usr/bin/time -h freebsd-update install 3m4s [reboot] /usr/bin/time -h freebsd-update install 5m5s real, 22s user, 9s sys [reboot] >From 14.0-BETA4: /usr/bin/time -h freebsd-update -r 14.0-RC3 upgrade 2774 patchs, 1749 files 7m50s real, 2m16s user, 1m47s sys /usr/bin/time -h freebsd-update install 3m9s real, 6s user, 6s sys [reboot] /usr/bin/time -h freebsd-update install 9m4s real, 35s user, 23s sys Now on 14.0-RC3. I am curious (not sure if enough to really try it) if it would be faster on UFS rather than ZFS.