From nobody Sat Aug 19 03:23:00 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RSPG2438mz4q3yC for ; Sat, 19 Aug 2023 03:23:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-23.consmr.mail.gq1.yahoo.com (sonic304-23.consmr.mail.gq1.yahoo.com [98.137.68.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4RSPFw4pnyz3ZCd for ; Sat, 19 Aug 2023 03:23:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=MmF0bnFG; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692415394; bh=Iwys3/kKpqgIBlnEL7nVpI+hcnJXqrE9Q+j0ys91RW0=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=MmF0bnFGD562Pvm0JckNaWvpbM7xwIlsyLc6HVPFTBqNKsdaEhjT/ZzYYKyrMcmDjBgGBvYD4uYPt3Hv4g6WlWcWQJ3kXws1wX8645/DmP1gU14k6Uxav9MoXfUbr2uOKKcPlzeA9Wlo0jeZli2ws6y13nwQpNj3ZeNyKgOqT2u/MB8VaUqE/ixIa1jGyymVDwFLctTljKsJK2c8WwOAKbZRlB9RapQK4RqzcYxhQTr1fL+bwZgzFui/VNCdzIF7ykZcytVopPXhfeTs4BYugsRhkffgBEav4cehVKjFtT7srGQMhBgpxwtBt7ewTN5tU5HhPqpbnXUHgNL+mMDJ/w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1692415394; bh=PbUzvKmIQo38G5D84B3YMCsd1zLZYvMPsWF2jLFvQK7=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=gzTZFGuQhX8Ar2AnSpPAfpvXSX694gqKV1Vbr0dtC6Wm4DyEeT4EbqJMFajOhiffdGdCgF9epc+NdAsxW0z9L59wXFolNQNvIsLX6sEPNuVq0MtO+vS/6EIefAUiniSj6PQ5JtMH65Z0rutwHGdxPS2817xtHG/8COmGk1MaYDZnm67zCTnom2p67vjHeFzBpNOBKXjN9I0dE54wyFyWbEi/b7NDCee04W9wIpEePXoXlRekDO8pZyVTS3gZIPtt224FbIqT/F79wuC7au44KzdhtraOZpG48eHv+tLeChIuTG8PGXs/E9kF7p8bOrEPGToflct3Pvx8zweqJbLhzQ== X-YMail-OSG: BLWeMjQVM1kADw3_2VrfAQiVxeickA.nwE2r2gV1O5GY9DFCO1vgccVTKg.2264 INDMGTCvG9ML9Zv96btNcvEBVGly2_PTLKtwTK5dgoVjDNPVbcuW5M2Xf0PfOHYpI0JPxzFJ.0Qx gQARo7oVF1IZ_aunY9mc8huHbyq1UlRO90XQN.kXSNa_zwv.tHa7UxOoU7BYJfQXU.LwSYB.Jrp2 cnEHewXPmN_pM_CjdwuHRPQXhZlp86oF5G_hhyQ1uoQrbQQtUXZ36RZfNNKc5hAOgUHsDLoRtRgm o5r90spPlAN1OEvQaLvHwbFSoHt8hBhg1LCOLboFsa42aUO42AmuLuswwBdcZ7y.P9OJTA17A6SS i90nY07nMrf2Jx2MeVIJVxRJk8ChgzEWwY3HjdzDl.SrzQz7QQFTaP6EH_N6_0Jm7rxAMJ5nG3A8 Xt9aUXO5huS53vXVVSxv35SbB5jK5hpt4JbgnYOB8LlVBO0OTJ9tpxbc1w43rK7fO6CEwlBU6c2N dMPiaG4BqYX4v2f.wqbg0WyEsyKFd80kaBuBqjOnfSFs1kKFV2yPVTvLag7brN_onQMNO5eTw1jU 2ohzZmFOHLD641ktKTJ_dubFFXdLargpoLK4bBBQUJDzP66rTZV0xCHNG1pl.Bsj6JpJjr2JI3Ii Q8Mr7lKhWHSPyqTBfsm.b1XFQG52doJj.hujmPlnTcWU.nraWsJuusQkmwwb8BOsj49hkKnN42bt F2Kvfems9ISEeMGrEztzDBizzati.0FUum_P6uX8vtJxXE9KxnEmyCHdu_.CB88KdUWmHqhag8eo aEhqjN_Q9k2QjNBWlxsN9Xi.n5uuKkne.cHQGFSxRFjMl.B1QRM939iYpvSAdgfkMFcSUXTVelts 83FJH32N5rTk4F.SlLc2fZQ_51erDlYPTaw3xYXRgrZkX9NqzgHzP0kkdN49.Zwnf74CQQx686Cc 5DoQJAj22UuQ5UFEUoDRj9m.ylXP27fh0I8LR62MkF0i2RGPT614knWwtJ08MLbylnCJcIcuMut7 IePZ_RTnMi8Vn91e.sDAXsMAXgoBN3TSaVwVs8KrZpz4xZFWkUGzYnl_qVCuvgxVdOblzg915gr9 jydmWFcs2oFYWbF3t2Tpax6kG9CnVf_JhvOAT6uxxsdbIJtbLFWCcgvmTRhTe6v77RMWqlhinbBr 2Ncmmp28ikWVc60IyQlZkW0wadx4bEugCF4EA2DN2n80HE7hy6wLK7LH1OT3uaMjPbdLZbsUy4.F E.9kov.dZMIaBuslJHHrrh2qAnukq0TR_gSbdpfP1dIiAUyhEcIWy0FLhzWb8oy5zpUZ.He7xd0k RYsvSJVIAiHCh81sQEB2u4N4HFFO9AyBQ.uRNUWaEpLpXnJupmFA39nEb343wP0D3wQLffnqiwVo 9k4bjjQfxXpK8KJ4nMhB9iwQ21p1OO.x4R1unKYJWr_tGVL1Dp13BtnLOM9wLYC07nxzAgMgIpbY l7iPsJ8nd8qBK7EQ_ai84ZIr4.sWRG1OwZPswFW9t7puHS4Lh6R5MO6K2Mz7hqO4.Ka2quy0g2d3 v0NioTLsBCKe5YbAG_FpHmL1S3UtMBKdTqubchBnRPMKV5H8pNgynJ4MXizdWG__2rwXTLxv2ixb zeoLFHt_TcYL6sdJcgD5VXVkvodsNXYoI4gitG3W0tvNTtjSOMzJ1HzfIAfXjWhqfBWIoO7orBs7 44KZrkmZXiza6PNMz3XLE9G33ZrAAumwmQJSKtKmw_N.6f6lSpPeugE76K37y4321Ywn_xpvLBs. Ux_CxbJZ.6h1k3hfP6UuoTg6Ic7AtISgByFQxWD04tJ3YAfKYJX.1zB0gpvER_n_1ij8Q7rBeL7W 3USIbqijOvFjPTLEpaNhACLSf2PxxDX8ysLxDdBI7nb2QMuHDU1du2204.Aw9EdDuq9e.Ql5HGKz MVG61brWuD.MY3YWjR97iT0kZKtbMAFbWhAWk6Bem94aE_FXwCSt0sOpRUY34Q9m8hi55fYbEn.4 Yawwy4q71OTqDDG1R4TzaMJ6Ls7EK5_MHYo0KVS8St5g7bpH8hasfdJw7j.A3J0p60FaFUf7tLHD uncacxpN3Bd4IBuoxLR7U_lnvVsrLOk3g8IFxqM2.WbgpBLSvgfHuLqoJdIpcY3TdAQLnO0.KQ6m bWPw.jCDcTnPCQEOeUbzzV3ZOeKrkVzd28w6mHDCncc0lyeT2BkB41pCBLNCVb4bY3YunlsfQTeg u8GAF93KGEH2.HJQautlGpbCkhGIlHWktctaOMEkbrmU3RydCejuHGmE51xfIf_x5Vcn32x4skg. xt78- X-Sonic-MF: X-Sonic-ID: 04b5ac7b-db92-48f7-9345-d5223bfd703e Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Sat, 19 Aug 2023 03:23:14 +0000 Received: by hermes--production-ne1-7b767b77cc-62qqp (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 61fec4b730ad27de38541bbbdae322dd; Sat, 19 Aug 2023 03:23:11 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Possible regression in main causing poor performance From: Mark Millard In-Reply-To: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> Date: Fri, 18 Aug 2023 20:23:00 -0700 Cc: Current FreeBSD , FreeBSD-STABLE Mailing List , freebsd-arch Content-Transfer-Encoding: quoted-printable Message-Id: <1B47E578-693D-4690-A577-947E8C9140B5@yahoo.com> References: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> To: Glen Barber X-Mailer: Apple Mail (2.3731.700.6) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BLOCKLISTDE_FAIL(0.00)[98.137.68.204:query timed out]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.204:from]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.204:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_COUNT_TWO(0.00)[2] X-Spamd-Bar: --- X-Rspamd-Queue-Id: 4RSPFw4pnyz3ZCd On Aug 18, 2023, at 19:09, Mark Millard wrote: > Glen Barber wrote on > Date: Sat, 19 Aug 2023 00:10:59 UTC : >=20 >> I am somewhat inclined to look in the direction of ZFS here, as two >> things changed: >>=20 >> 1) the build machine in question was recently (as in a week and a = half >> ago) upgraded to the tip of main in order to ease the transition from >> this machine from building 14.x to building 15.x; >> 2) there is the recent addition of building ZFS-backed virtual = machine >> and cloud images. >>=20 >> . . . >> The first machine runs: >> # uname -a >> FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ >> amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC = 2023 >=20 > I'm confused: >=20 > "the build machine in question was recently (as in a week and a half > ago) upgraded to the tip of main in order to ease the transition from > this machine from building 14.x to building 15.x"? But the above > kernel is from mid July? (-aKU was not used to also get some clue > about world from the pair of 140009? that would show.) >=20 >> Last week's snapshot builds were completed in a reasonable amount of >> time: >>=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc >> 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap >> 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap >> 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap >>=20 >> Overall, 17 hours, including the time to upload EC2, Vagrant, and = GCE. >>=20 >> With no changes to the system, no stale ZFS datasets laying around = from >> last week (everything is a pristine environment, etc.), this week's >> builds are taking forever: >=20 > My confusion may extend to this "no changes" status vs. the uname > output identifying the kernel is from mid July. >=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc >> 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap >> 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap >> 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap >>=20 >> Note, it is just about 4 minutes past 00:00 UTC as of this writing, = so >> we are about to cross well over the 24-hour mark, and cloud provider >> images have not yet even started. >>=20 >> . . . >=20 > In: >=20 > = https://lists.freebsd.org/archives/freebsd-current/2023-August/004314.html= > ("HEADS UP: $FreeBSD$ Removed from main", Wed, 16 Aug 2023) >=20 > Warner wrote: >=20 > QUOTE > . . . , but there's no incremental building > with this change, . . . Also: expect long build times, git fetch = times, etc > after this. > END QUOTE >=20 > Might this be contributing? How long did those two > "Checking out . . ." take? Similar time frames? >=20 The build process and information is not available. So I looked at something I thought might have a chance of being somewhat invariant and have a limited range of types of (parallel) activity: time differences for the CHECKSUM files taht have timestamps after the last *.img* timestamp, as seen via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/14.0/?C=3DM&O=3DD= (so: most recent to oldest as displayed) First today's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1232 2023-Aug-19 00:26 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1744 2023-Aug-19 00:25 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1168 2023-Aug-18 22:59 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1680 2023-Aug-18 22:58 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1232 2023-Aug-18 20:49 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1744 2023-Aug-18 20:48 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 156 2023-Aug-18 18:47 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 220 2023-Aug-18 18:46 = FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013f29d048-264841.i= mg.xz 738514252 2023-Aug-18 18:43 So, in rough terms: = RockPro64->2hrs->riscv64->2hrs->amd64->1.5hrs->aarch64 Last week's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 16:34 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 16:34 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1168 2023-Aug-11 16:28 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1680 2023-Aug-11 16:28 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1160 2023-Aug-11 15:13 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1672 2023-Aug-11 15:12 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 14:46 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 14:46 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 156 2023-Aug-11 13:54 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 220 2023-Aug-11 13:54 = FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc495615f-264678.i= mg.xz 738475300 2023-Aug-11 13:50 So, in rough terms: = RockPro64->1hr->riscv->0.5hr->i386->1hr->amd64->6min->aarch64 Thus, 2hr vs. 1hr in crude terms. I do not see any 2hr like time frames for 13.2 via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/13.2/?C=3DM&O=3DD= =3D=3D=3D Mark Millard marklmi at yahoo.com