From nobody Thu Feb 23 21:44:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PN6573S2qz3snvK for ; Thu, 23 Feb 2023 21:45:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PN6572F3Jz3P5x for ; Thu, 23 Feb 2023 21:45:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677188712; bh=34uN9jFTNTtkCET+yDsJuiGjnLTlNdFof/1tbOUh5sI=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=fi2h2iZL0jsT1O3PEXyXcRfvgRTc9xHVTtyInyrKZWHwmJH7/JPbit6j+QcYkBzinumdx9RC27q+Lt4u6tYAZXW00qBDhS91Glp2PD9PtLUKUpRY3ovLIRqXIeY2CIhC9kZQtzcNV5Szlf2yJ5s0k6++2Zywlu88xdyZKH/o+kAOvVOjWDlv8CSSqHEt0g/rK44JTp6iFltbnezzN/CGtN4SYO2GGht6mijywkL6mE/ECR93euoDrj+iVY8YPuHbC/20PHYq2yD3bfrYggW7rZmueEM9JXNch8uPSTaykTgCQAuC/oefrwALfv91LQRttbNbSi6r+6lswcfiRqWeYA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1677188712; bh=URVXU0uyP420WMVPrSpkuZiBRzN9kEwgjJ8RfjOq7jK=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=dXz2i4BX6ElvW2Hk1S9uCsyxPZwbOgnG6RPv4rEmymoO1Q2FKUXg/DgmJdOkO0bEjp+motwa1cDwpiXSA2oMHLIonwEhrXMw/uXlGg3W1Alg5tXUIpOFFkuVEOm/SSY8+dYqDC+bAsePwa53k5SX3eQ72IpoauqwlIroEEV8xUMZhEKiOKj1JQhfoVor6q40/c21w7xNedMu/x6ksUuBvvIFI1iKc4Ytr6LE5jiw75y9Ny2VFcNhJo5lusdagWyi6dnAagtlqzm3IWu28ue7t979JJjAV1foqJI9yzFlUizaJtOeFC5Kuk6GZE15QFjk+UdaLc+0TBORzepAKM6gWg== X-YMail-OSG: 5qCryZIVM1nnMWTUhHCVyHSI51FVmfoERnKiNIGR.DPcVOrQ2RvoT6Gwcb5QzJl bv61Gpf_eeJ8uX_fbdbh8luWmpIY8bw9wlPZxbXZRQFNtKjW7_bs9dv8nIeBWujFP9F7z9H685Ly EXdcAsUMzP0097hfmm1iZpvtbDWQn2mYVWjAvD_23ARyAA_pO7_PkyUN1HI1_ppg8xwhd6k.yag9 iEQ99.iCgJxSsjCicAMSadK2wclNSwJmSH_Bd_zeTA510RL4QZ1MrWSgfvSjTkaR7uTaogQBdnUF rM5E3tYOdboPft3px0n9.aU_0Jog5tzinhdhJCufxATroKrvDCTDUmm7bDDyq_NuvOiDwjhtJPDq gzvHaesSNWbAVxpSD23hDLINgLUOT.64KuCpKSYP9awGJ1kw0yeaz9EuOoi7WAZWg7tphm0u2ucr iJ5aJ_Tetb_YNT7xIe7jcxGmJAy1isRgAZGHtVqXzPtbIYRwOjOaJW_VPtwQNbERSRapX.OZX7wI oWL8vcx_lMbB09QDm1BLXga53ZEJ.94AlGS8eQN4R9lCuOg3QS3W4zzgzBm.MPH6Yu2xscJmCCVa l5GLCxWO5kKqThJ_t3fkHqcxcXWsz5L2IjnmWW28MLTE7iNvC1TsDzKuRoFlyAV3lqFhTcezBu9B dEeUpuvUwpBvvVbbmXr099W3nGDTKOW8BC4qmAjAcnIzE4SAJC6mMBYfAz7krWqTJB3O1C4wL5kR YuRaV9yb9BYqAmkxEiyxeIE4_NJN4ZlXI6PRI8NmiYXiPICrnZMObqPqcSWXUt0TJZXgp_Hzzmc1 8dlXygDOI6WruMxCogauW_jNaqBFNw4TLvEJkHt8wDu4HHXCciRwl7veAQHsp6RETyG2Ywinjp65 TCpynOjTHzKtxM5tXPuiBEt5Q5Jf6iTTFp.IZAL7.9Bmc_aWCR..iBuAM31SWnUplvIr74ep6jXL JfHA6U2sKKCslMR_V3QQLolMaSfbB676e04EUaf1uI8_5WzWUuru3WujG9Qtu0Zv2AhNR9f8jKaA .DSwFCld5mV3M4Pd7IZHEhmxg_y4m1PizVfQrGLZ96HzDVtQWe6S9Z4E9910gO4d91bmd9hVP.7s sorLRetT6AlsjftbNpDd.2gUywIMoDu5WBi2U2jtbHKgFZbvaSK3Ip7IHZLOKzKi_91cPRPs0X9r 3DdrXU7BKxxe3M.ZV8TO4CGZ_IrH55knLH92QQhxLEbvUP95KfiA_rHl5NW0imO_pwHyfg.0D6SG G50I999KeNucO1ck8giLaNV.rGbqMH1Proh6BYIdFyPACKGSVfICZ37DS0RRZn8kXGukWdtcq7sd jtBSEplRnZlKcHYIN8mqvm2UKvVDcVGs87iSgjyaoQJBtspdS2f.F5wY3TN7dBzoLBR0ZJ3RGH8e hwQA3QWdVj3LQVRaKsDPUlI8X8hK_AbJBXC7tZn8VoLfE6_0HK6nfM3njPF7Teu6bEsLaTUnUfFz .Rk_jZJbyugLeBqjjT1fQ9nbupo7129oeaHcNmLV4DqJ22Veu_oLWt._uju3HeZtzjX0CkuiLvuG WDga29LhURxyapM7w_jDRi2Qyzfo.QGKpZjpM9dCahJWwbDXEUOtyOxng4nmFQYYcTMJUWuk1T0c AcUgrsi3lR5vb.KH83Q6d_sgpeFqfLJ8vPS8AH8e46OlcmcDULga5insVerHmZb68QG0C__uoMLE YRGg9gdMsyVSTtgcijvO8u5RRUFk7A0T7rfohexbRAAVwOFHK6bih.jfl.MDBYnWY3aFMUXDCOdP f4YSeF8PPWcpHrHVgMp8WrPBW6IHJZN9ePtPkaYr.v0dtCB6u4OdVg5IUnsnbtX5iSsd8EKcyR0G iiMSTYXSLeo0aWA2BevjkmlRv4dcv7AjaOuKyOScsTcWWxpXe.2ela2gHslXoUbiXRzxC5_cEYZA EsuI0MqvswSQnO4_21DND_Xo369qPhO6XeOD.YGTTy6gtBpdZbE_x6F2diyAdqc0_H4yRc5tbqBu TNU0dcyrc3hq8wV_QcUFoxF2Grifp1.sKbLoxkSnqC.tumpHGajmdaQ9lHZPZSKd7Yxyptv.qLYU 1w7W8tcgzNYrSXEEOqM.8CQq4zbowC3JtZwJ.u4GYzIo7KwMItCR89wMc0SSg7QYuFBXkop3i0_. 4J_6F6voNyyII7wE9tgMALIerRsp2w1hGlafDlt7eWIkTf3mpiSrV7geK6wYg8TE9ROGtsHmpI.E zTOftsnLCenS761KdC_uB9UHiWGkNH0ZolzUtK..6305TySbZqPhyCh.JdYhu0nYA.y6fxCQQnzU wf.JXVw-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Thu, 23 Feb 2023 21:45:12 +0000 Received: by hermes--production-bf1-57c96c66f6-jdmts (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a71aa7c3c85a9f05b1e8d3b655ffd5f2; Thu, 23 Feb 2023 21:45:10 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) From: Mark Millard In-Reply-To: <30.1677183903@kaos.jnpr.net> Date: Thu, 23 Feb 2023 13:44:57 -0800 Cc: Bryan Drewery , Current FreeBSD , Peter Content-Transfer-Encoding: quoted-printable Message-Id: References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> <73088.1611797582@kaos.jnpr.net> <10819.1677108389@kaos.jnpr.net> <76FA98EF-6184-4D7E-A01F-0EE8117D0D10@yahoo.com> <29887.1677115125@kaos.jnpr.net> <27790339-240F-4C97-97C7-38AFD8DE03D5@yahoo.com> <72419.1677133429@kaos.jnpr.net> <30.1677183903@kaos.jnpr.net> To: "Simon J. Gerraty" X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4PN6572F3Jz3P5x X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N [Note for "how many separate bmake instances are in that log?": I do not know how to tell how many submakes are run total. It was with -j32 on the threadripper 1950X, if that is was you were after.] On Feb 23, 2023, at 12:25, Simon J. Gerraty wrote: > Mark Millard wrote: >>>> . . . >=20 >> I got past the issue using :=3D before reading the above. >> (I'm also using MAKEOBJDIR instead of OBJTOP currently.) >=20 > Per my last response, I'd be pretty sure MAKEOBJDIR is incorrect. Which still leaves me experimenting to find a correct reference. Do you know notation will always lead to the same absolute path with the proper /usr/obj/BUILDs/*/usr/main-src/*.*/ prefix ? (I've been showing just the main-amd64-nodbg-clang and amd64.amd64 combination but there are many more.) >>>>> .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP}/tmp/ >>>>=20 >>>> (Ignoring the variability of OBJTOP issue . . .) >>>>=20 >>>> I do not expect that would work: ignoring things >>>> it likely should not. >>>=20 >>> Sure, but it may be useful as an experiment to ensure things are >>> behaving as expected. >>=20 >> As a test: >>=20 >> .if ${.MAKE.LEVEL} =3D=3D 0 >> .MAKE.META.IGNORE_PATHS+=3D ${MAKEOBJDIR:tA}/tmp/ >> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS} >> .endif >=20 > Lose the .if ${.MAKE.LEVEL} =3D=3D 0 > it is almost certainly keeping things from working as expected. What do you want tested instead of MAKEOBJDIR ? I'm taking a guess (no .MAKE.LEVEL use): .MAKE.META.IGNORE_PATHS+=3D ${OBJTOP:tA}/tmp/ .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS:tA} >> I still get things like: >>=20 >> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/zlib/x86.meta: 23: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... >> Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/zlib/x86 >=20 > Because that will not be level 0 and so .MAKE.META.IGNORE_PATHS is not > set. I tried the above and I still get (picking to look for tmp/legacy/usr/sbin/ln examples): = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/aac/opt_scsi.h -- = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/asmc/opt_acpi.h.meta: 22: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/asmc/opt_acpi.h -- . . . and (looking for realpath examples): = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/accf_dns/machine.meta: 23: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/accf_dns/machine = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/accf_data/machine.meta: 23: = file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/accf_http/machine.meta: 23: = file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/accf_http/machine -- = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/ae/machine.meta: 23: file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/ae/machine = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/acl_posix1e/machine.meta: 23: = file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/realpath' is newer than the target... Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/acl_posix1e/machine -- . . . >>=20 >> and: >>=20 >> = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h.meta: 12: = file = '/usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/tmp/legac= y/usr/sbin/ln' is newer than the target... >> Building = /usr/obj/BUILDs/main-amd64-nodbg-clang/usr/main-src/amd64.amd64/sys/GENERI= C-NODBG/modules/usr/main-src/sys/modules/xl/opt_platform.h >>=20 >> for both of a pair of back-to-back runs of buildworld buildkernel. >>=20 >> FYI: The file system is zfs with mounts that look >> like: >>=20 >> zoptb /zoptb >> zoptb/BUILDs /usr/obj/BUILDs >> . . . >> zoptb/BUILDs/main-amd64-nodbg-clang = /usr/obj/BUILDs/main-amd64-nodbg-clang >> . . . >> zoptb/ROOT/main-amd64 / >> . . . >> zoptb/tmp /tmp >> . . . >>=20 >> # bectl list >> BE Active Mountpoint Space Created >> 13S-amd64 - - 4.97G 2021-08-20 16:57 >> 13_0R-amd64 - - 4.30G 2021-08-20 16:56 >> 13_1R-amd64 - - 4.12G 2022-03-10 12:38 >> main-amd64 NR / 7.42G 2023-02-19 15:37 >> old-main-amd64 - - 2.25G 2023-02-09 19:07 >>=20 >> (I use zfs in order to use bectl on a couple of >> systems, not for redundancy.) >>=20 >>=20 >>>> Also, I'd rather grow a smaller set of ignores >>>> gradually to make it easier to detect if an >>>> addition starts causing a problem and can be >>>> backed out. Starting with everything ignored >>>> would make things much harder to figure out >>>> when ignoring creates a problem. >>>=20 >>> Yes. >>>=20 >>>>=20 >>>>> You might need ${OBJTOP:tA}/tmp/ >>>>> or both. >>>=20 >>> I found it necessary in the unit tests to add :tA to both TMPDIR >>> and .OBJDIR to get sane result on one test platform. >>>=20 >>>>>> It is using paths that match the -dM output lines ( sbin >>>>>> use despite sbin -> ../bin being a symbolic link). >>>=20 >>> use :tA if you want to ensure consistent results. >>=20 >> So, for each: >>=20 >> .MAKE.META.IGNORE_PATHS+=3D = ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool} >>=20 >> I need to form an overall :tA on the path? Something >> like: >>=20 >> .if ${.MAKE.LEVEL} =3D=3D 0 >=20 > I think you need to first get rid of that level 0 check before > worrying about anything else. Done. But it appears to make no difference. >> .for ignore_legacy_tool in awk cap_mkdb cat cp crunchgen crunchide dd = egrep env file2c gencat grep gzip jot lex lb ln m4 mkcsmapper mktemp mv = patch realpath rm sed sh touch truncate uudecode uuencode >> xargs >> IGNORELEGACY_${ignore_legacy_tool}=3D = ${MAKEOBJDIR}/tmp/legacy/usr/sbin/${ignore_legacy_tool} >> .MAKE.META.IGNORE_PATHS+=3D ${IGNORELEGACY_${ignore_legacy_tool}:tA} >> .endfor >> .for ignore_other_tool in ctfconvert objcopy nm >> IGNOREOTHER_${ignore_other_tool}=3D = ${MAKEOBJDIR}/tmp/usr/bin/${ignore_other_tool} >> .MAKE.META.IGNORE_PATHS+=3D ${IGNOREOTHER_${ignore_other_tool}:tA} >> .endfor >> .MAKE.META.IGNORE_PATHS:=3D ${.MAKE.META.IGNORE_PATHS} >> .endif >>=20 >> Such seems to make no difference to the text reported via >> -dV -V.MAKE.META.IGNORE_PATHS in my context. >=20 > Yes, right now I think your main problem is only setting > .MAKE.META.IGNORE_PATHS at level 0 Not as far as I can see in the results. (Note: I did not start with explicit use of .MAKE.LEVEL . It was just something I'd added into my experiments recently. Nothing that I've tried has worked. I keep exploring, looking for evidence of some way being able to control the behavior.) =3D=3D=3D Mark Millard marklmi at yahoo.com