Date: Fri, 18 Aug 2023 20:23:00 -0700 From: Mark Millard <marklmi@yahoo.com> To: Glen Barber <gjb@FreeBSD.org> Cc: Current FreeBSD <freebsd-current@freebsd.org>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org>, freebsd-arch <freebsd-arch@freebsd.org> Subject: Re: Possible regression in main causing poor performance Message-ID: <1B47E578-693D-4690-A577-947E8C9140B5@yahoo.com> In-Reply-To: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com> References: <8A267B4A-1B7B-4D61-AA57-8E3156470617@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Aug 18, 2023, at 19:09, Mark Millard <marklmi@yahoo.com> wrote: > Glen Barber <gjb_at_freebsd.org> wrote on > Date: Sat, 19 Aug 2023 00:10:59 UTC : >=20 >> I am somewhat inclined to look in the direction of ZFS here, as two >> things changed: >>=20 >> 1) the build machine in question was recently (as in a week and a = half >> ago) upgraded to the tip of main in order to ease the transition from >> this machine from building 14.x to building 15.x; >> 2) there is the recent addition of building ZFS-backed virtual = machine >> and cloud images. >>=20 >> . . . >> The first machine runs: >> # uname -a >> FreeBSD releng1.nyi.freebsd.org 14.0-CURRENT FreeBSD 14.0-CURRENT \ >> amd64 1400093 #5 main-n264224-c84617e87a70: Wed Jul 19 19:10:38 UTC = 2023 >=20 > I'm confused: >=20 > "the build machine in question was recently (as in a week and a half > ago) upgraded to the tip of main in order to ease the transition from > this machine from building 14.x to building 15.x"? But the above > kernel is from mid July? (-aKU was not used to also get some clue > about world from the pair of 140009? that would show.) >=20 >> Last week's snapshot builds were completed in a reasonable amount of >> time: >>=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/logs >> 20230811-00:03:11 INFO: Creating /releng/scripts-snapshot/chroots >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/release >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/ports >> 20230811-00:03:12 INFO: Creating /releng/scripts-snapshot/doc >> 20230811-00:03:13 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230811-15:11:13 INFO: Staging for ftp: 14-i386-GENERIC-snap >> 20230811-16:27:28 INFO: Staging for ftp: 14-amd64-GENERIC-snap >> 20230811-16:33:43 INFO: Staging for ftp: 14-aarch64-GENERIC-snap >>=20 >> Overall, 17 hours, including the time to upload EC2, Vagrant, and = GCE. >>=20 >> With no changes to the system, no stale ZFS datasets laying around = from >> last week (everything is a pristine environment, etc.), this week's >> builds are taking forever: >=20 > My confusion may extend to this "no changes" status vs. the uname > output identifying the kernel is from mid July. >=20 >> root@releng1.nyi:/releng/scripts-snapshot/scripts # ./thermite.sh -c = ./builds-14.conf ; echo ^G >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/logs >> 20230818-00:15:44 INFO: Creating /releng/scripts-snapshot/chroots >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/release >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/ports >> 20230818-00:15:45 INFO: Creating /releng/scripts-snapshot/doc >> 20230818-00:15:46 INFO: Checking out https://git.FreeBSD.org//src.git = (main) to /releng/scripts-snapshot/release >> [...] >> 20230818-18:46:22 INFO: Staging for ftp: 14-aarch64-ROCKPRO64-snap >> 20230818-20:41:02 INFO: Staging for ftp: 14-riscv64-GENERIC-snap >> 20230818-22:54:49 INFO: Staging for ftp: 14-amd64-GENERIC-snap >>=20 >> Note, it is just about 4 minutes past 00:00 UTC as of this writing, = so >> we are about to cross well over the 24-hour mark, and cloud provider >> images have not yet even started. >>=20 >> . . . >=20 > In: >=20 > = https://lists.freebsd.org/archives/freebsd-current/2023-August/004314.html= > ("HEADS UP: $FreeBSD$ Removed from main", Wed, 16 Aug 2023) >=20 > Warner wrote: >=20 > QUOTE > . . . , but there's no incremental building > with this change, . . . Also: expect long build times, git fetch = times, etc > after this. > END QUOTE >=20 > Might this be contributing? How long did those two > "Checking out . . ." take? Similar time frames? >=20 The build process and information is not available. So I looked at something I thought might have a chance of being somewhat invariant and have a limited range of types of (parallel) activity: time differences for the CHECKSUM files taht have timestamps after the last *.img* timestamp, as seen via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/14.0/?C=3DM&O=3DD= (so: most recent to oldest as displayed) First today's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1232 2023-Aug-19 00:26 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-20230819-77013f29d048-26= 4841 1744 2023-Aug-19 00:25 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1168 2023-Aug-18 22:59 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-amd64-20230818-77013f29d048-264841 = 1680 2023-Aug-18 22:58 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1232 2023-Aug-18 20:49 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-riscv-riscv64-20230818-77013f29d048-26= 4841 1744 2023-Aug-18 20:48 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 156 2023-Aug-18 18:47 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013= f29d048-264841 220 2023-Aug-18 18:46 = FreeBSD-14.0-ALPHA2-arm64-aarch64-ROCKPRO64-20230818-77013f29d048-264841.i= mg.xz 738514252 2023-Aug-18 18:43 So, in rough terms: = RockPro64->2hrs->riscv64->2hrs->amd64->1.5hrs->aarch64 Last week's: = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 16:34 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 16:34 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1168 2023-Aug-11 16:28 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-amd64-20230811-136fc495615f-264678 = 1680 2023-Aug-11 16:28 CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1160 2023-Aug-11 15:13 CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-i386-20230811-136fc495615f-264678 = 1672 2023-Aug-11 15:12 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1232 2023-Aug-11 14:46 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-riscv-riscv64-20230811-136fc495615f-26= 4678 1744 2023-Aug-11 14:46 = CHECKSUM.SHA256-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 156 2023-Aug-11 13:54 = CHECKSUM.SHA512-FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc= 495615f-264678 220 2023-Aug-11 13:54 = FreeBSD-14.0-ALPHA1-arm64-aarch64-ROCKPRO64-20230811-136fc495615f-264678.i= mg.xz 738475300 2023-Aug-11 13:50 So, in rough terms: = RockPro64->1hr->riscv->0.5hr->i386->1hr->amd64->6min->aarch64 Thus, 2hr vs. 1hr in crude terms. I do not see any 2hr like time frames for 13.2 via: http://ftp3.freebsd.org/pub/FreeBSD/snapshots/ISO-IMAGES/13.2/?C=3DM&O=3DD= =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1B47E578-693D-4690-A577-947E8C9140B5>