From owner-freebsd-current@freebsd.org Wed Jan 27 00:00:16 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CE3C14E4CE3 for ; Wed, 27 Jan 2021 00:00:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-25.consmr.mail.gq1.yahoo.com (sonic311-25.consmr.mail.gq1.yahoo.com [98.137.65.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DQNyl5HCMz4fRn for ; Wed, 27 Jan 2021 00:00:15 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1611705613; bh=cV+HUTwUN2qGJw7S+LbyWkMkhPxHXu6LyDdabAjbD0A=; h=From:Subject:Date:To:From:Subject:Reply-To; b=kl95TQIm8U+gh77zNdaK59ia+UtyXvCs6DenNMSOtZY+CRk9CRuZprc23JABj7/yKJDzixUua16/DdHmqAhiEIaxNe8FP9oEtHWKz9+PN6tRjYfrf7Iv7eN8OxlzxwYFbWHRfKgttJeQ1oUy+6fDTb/t26CNgfx03euHGkF0oZEgxyi6yGcBMdoowFsqoGnGLzT1BYwqiLn0wyyIG80S7aSMGODAEs2+8DNEaN/Cs9VckQ0/izdYbDXIeFWF6JuoiP7a8qHOGSfZlVxkyk4Demrx2XXmTQ2ZJnqIZm/FXoXs1kXOinW2DZWcMX1Z7IEMpvsxkRCI6rbEaEo3se0Qxw== X-YMail-OSG: WQmz9kcVM1nHCCkH.udMJaX2an6322QJpCMcoZ5JP4KV4L5FfWKV2x.Oo5OHf56 4Z1B5BDjDHcFCoN2dBtVyt35ZcqilCXlSZPMvxLYMl9Uh1bcP6rmviGiNRbVL2d_lGG6.ludET7g 0WXVhrpnnuMaiThEfCycWwIrhT2_KZSl7ofzHinWgOnBAHhM.GXFOq8qxw8rWbrZik9I0d3XtC4K npdXWP3I5Dy9aP6JymZkrqZJwy3aLZUi6VD5GcS9.qA8g71.UtOLaK40BBo65jD2i9WtX.OOiUmi oujR1AuXJKP8kz1CQtZtIWzUEs0RZm8S2JLdPNJsp8VstQqD8doZWpooT2AVs8AWTLrcCPQqrNur L2HecG0RBmEW.zGxsxMZl2pqDdlid76lzDGC8H5dS.j3hMugaMObFxH2rcbsS3qqhKMokF1cuGI3 J2b4qHHVSSmJ4oVrzWzWx4phLDLrfIXznspgI5wYpCbDr0A51PUvfGPY82byqi2YXJ7ltmp2sysh ZRVpmH48EdM7_Nk4vjmG6X6YdytxSjxgNsY5_XS0tDjrODTncFz_b6xAwLoVTx2mGipSv4icg6l9 .551jP_DCDyIq2OyWS_KeP6IXxr6dhar1m5PRWGACJE8UsZY_0ddR4LaOwFtECWexQkd4.cFUo0B 0_j3UZvTJQtDof.KOtd._mP.1Yf1w6B0oISvotbuKTOPBo81qBjFOsV40GrY237A0nKsQcWipuV9 yGoZcW09BlyCVFa_k1Mim3uvJTDMYvtpSFjdjd2TyYKhAgaZ12l_5aTiM3ZUwrMzEUmMqOX4_GCP uEnnLBgJLo6JgU1bHlPOpofxztFx.yGUwulLxvfzdwHvMYZC6N_oVpheXBHWU7rL0pVOoynD1fdd OkqW22Ra_LLENY4wHe6zRMH8XFJK3T3bf066Bk6bJ4LLQ2LatVeVDH3Y8GsDk7IWLwGeu1Q0pkwK y6kzbOSMYs0kf1aEQhfKFfTiCcva.d4YMoJUAgOkAiO7sqIx.jWgM1cWPp0ps.7ooTFP3WFNpPS3 ZQhttb5azmBTA3HTgo59yHBMrtYX4781DBK43oM0C9FnMLw9y.Du2Scpqf87cdbYYBmGaRpNk_kd KXX4G4C1ESo_swSHfbsQoHHeeWlgtt6Xc6UShZ8XCXl99WFnd_PYnGYsTPrnh0Gh5S7LmjvWyDfe VhYMKAqWryCJNWOA7i6XuMa5MmX4Oj2JN8G1sL82ZBU4GpwF1grMIkv0VFj4bPek3xTqE7OnF4Nx Eo.bCep2gj1s8lOAF4rwPx2U4eMtaokl_LmCx8Vvm3UtIzokNLL4XoyGdcEWvVErNbGFL_g_2pL3 WzWv.cPEVyW_OeJjXWj5qT2Yw69tzLdDwE6hAyrXwvFsBAt06Q65OWB60RyWkIuNbwBhOs7HknCc mKmFXRiKpkCaI5Myh55OCtTsVXW80AWlO5cnNFOk7J3Ct8.sZkJav0dxg_2.9I4vY2zVHyBeoGqa zTCSMsykGiQ3JdRzGAEHgHzqrRlhtn.2SBNUP1nqN5sMXHbpLyuXMc9YggPgp7aMFLgbUfuM9epq BB3viHY2KdDA0V56BVss13GlskiMNaJpqHLEBqtrkkKM1gNxMn.vjRaoHzkaHkXk_H8seKZKUD.H MNLkHVxnLQuTBd9.ePlUxyVNHVeOTFXle0bEwpTeaB7fotYoEasvjwvIxZ_gKyQ5aW3dCBZyRUCX rQr1JlVcJb_8Q.oLjaqYPuXNKu1hYwBFd_Sls6cfA0wavcO5v0qeo2YW.mUTNRplRZTxGG83Bmdo tYsUORFpcd868.hy.jmNSknI7xHlFuLDYpxbBrorb.DbQ0phi9v.fFLCusVLx_N6ZmWO4K4b8CBT NiW0TBr.r7KsVGnRa1Vdf4n7K3mqJ.hS7h0DjjDFrPjOiubLy74QTFqe.uEMH.u7IAvtz0mSF2Rw .bI.B0oPs0FVJxZFwPdBdZmGGRJpriAjAPs9vvt967poCfSORXiQ1d8GEeRwRKPDDMpMB4wQsZ80 hylST90aLSiskN3T3KmrLjcCd0qKFcNiXGuscUWSOprTp0XvLnFskbuoHet4W3Y9U5eWCQmbPijb tpntdWaCNBUXZk.ScUk4DPS.r5miov_bAAapMyHxdbgaO4aeOKNShyVO0_anycbs_A3TWOGKncqB 6MBDHVlMvtGE2gXs2M4Uhtoh5Cu6LVtHKbndZYBkHZgdzJp1UZT6Ra6EmoNtLFMO9GJavlkrIcXB LxsLo_B0RWtxpLvMlxTllK.ogXaMBIv9jbK4z5yzoHL69YSM9dfuG2_WPd7oc.FnkMN15rudZvaK oZN35zemPWS9wT1qCnlHZgJm2AC4VwzYQqycnGk3Uer85nguaTUYB45qbJRxDRvpIIEHkxf4OEmQ 9eNqsGoPNx1jGTynE.1tdNk6JjelIsRuf5oorUxSpN0MGJ4aRvkD4O7S9Lz83IvWEeFRuGjXjT4J BkwILdl6t9U3xaT5xMFu.v4PT_YLwEGHH3iIT8H1KksHSbitpzL8_qcCtKxxJ2zlgBBB5QorO Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Wed, 27 Jan 2021 00:00:13 +0000 Received: by smtp424.mail.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 90b742bef35f4a48d5177cf92a0c99d6; Wed, 27 Jan 2021 00:00:08 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.40.0.2.32\)) Subject: Re: FYI: Why META_MODE rebuilds so much for building again after installworld (no source changes) Date: Tue, 26 Jan 2021 16:00:05 -0800 References: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> To: Bryan Drewery , Current FreeBSD In-Reply-To: <3345EBA5-A09C-4E3F-B94D-39F57F56BDBB@yahoo.com> Message-Id: X-Mailer: Apple Mail (2.3654.40.0.2.32) X-Rspamd-Queue-Id: 4DQNyl5HCMz4fRn X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RBL_DBL_DONT_QUERY_IPS(0.00)[98.137.65.206:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[98.137.65.206:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.206:from]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.206:from]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-current] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jan 2021 00:00:16 -0000 On 2021-Jan-23, at 21:37, Mark Millard wrote: > On 2021-Jan-22, at 01:45, Mark Millard wrote: >=20 >> Given an already built, installed and booted system version, I've >> noted a big difference for META_MODE in 2 rebuild contexts (no >> source updates involved): >>=20 >> *) Prior buildworld buildkernel, installkernel, installworld, boot >> presumed before (A) and before (B) below. >>=20 >> A) make . . . buildworld buildkernel >> make . . . buildworld buildkernel >> (the 2nd buildworld buildkernel in (A) builds far less than the = first) >> (that means that the first built more than I would have guessed) >>=20 >> vs. >>=20 >> B) make . . . buildworld buildkernel >> make . . . installworld >> make . . . buildworld buildkernel >> (the 2nd buildworld buildkernel in (B) builds far more than it did = in (A)) >> (so, more like the 1st buildworld buildkernel in (A) and (B), given >> the specified prior context) >>=20 >> So I used make -dM for the commented buildworld buildkernel lines, = logging >> the build output and later diff'ing them. >>=20 >> Result that I noticed? Lots of lines uniquely from (B)'s case, ending = with >> one of: >>=20 >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/awk' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/cap_mkdb' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/cat' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/cp' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/crunchgen' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/crunchide' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/dd' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/egrep' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/env' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/file2c' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/gencat' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/grep' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/gzip' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/jot' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/lex' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/ln' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/m4' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/mv' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/patch' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/rm' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/sed' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/sh' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/touch' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/truncate' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/uudecode' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/uuencode' is newer than the target... >> file = '/usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/u= sr/sbin/xargs' is newer than the target... >>=20 >> The lines with these lead to more files being updated and so causing = more >> indirect rebuild activity (that cascades). >>=20 >> Many/most/all(?) of these seem to me to be unlikely to actually need = to >> contribute to what needs to be rebuilt (just based on being newer). = So >> the option to ignore (some of?) them could be useful in making = META_MODE >> builds quicker. >=20 > The following from one of the .meta files makes the point that rm use > in the example is unlikely to be important to needing to rebuild, > despite it actually causing a file rebuild. Nor is the specific echo > command listed relevant. Only the "ar" command is: >=20 > # Meta data file = /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++/li= bc++.a.meta > CMD @echo building static c++ library > CMD @rm -f libc++.a > CMD ar -crsD libc++.a algorithm.o any.o atomic.o barrier.o bind.o = charconv.o chrono.o condition_variable.o condition_variable_destructor.o = debug.o exception.o filesystem/directory_iterator.o filesyste > m/int128_builtins.o filesystem/operations.o functional.o future.o = hash.o ios.o iostream.o locale.o memory.o mutex.o mutex_destructor.o = new.o optional.o random.o random_shuffle.o regex.o shared_mutex.o > stdexcept.o string.o strstream.o system_error.o thread.o typeinfo.o = utility.o valarray.o variant.o vector.o cxxrt_auxhelper.o = cxxrt_dynamic_cast.o cxxrt_exception.o cxxrt_guard.o = cxxrt_libelftc_dem_g > nu3.o cxxrt_memory.o cxxrt_stdexcept.o cxxrt_terminate.o = cxxrt_typeinfo.o > CWD = /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/lib/libc++ > TARGET libc++.a > -- command output -- > building static c++ library >=20 > -- filemon acquired metadata -- > # filemon version 5 > # Target pid 22471 > # Start 1611359217.214996 > V 5 > E 22961 /bin/sh > R 22961 /etc/libmap.conf > R 22961 /var/run/ld-elf.so.hints > R 22961 /lib/libedit.so.7 > R 22961 /lib/libc.so.7 > R 22961 /lib/libncursesw.so.9 > R 22961 /usr/share/locale/C.UTF-8/LC_CTYPE > F 22961 22962 > E 22962 = /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/us= r/sbin/rm > R 22962 /etc/libmap.conf > R 22962 /var/run/ld-elf.so.hints > R 22962 /lib/libc.so.7 > R 22962 /usr/share/locale/C.UTF-8/LC_CTYPE > D 22962 libc++.a > X 22962 0 0 > . . . >=20 > The timestamp on . . ./tmp/legacy/usr/sbin/rm is not > actually relevant to if libc++.a needs to be rebuilt. >=20 > Of course, the structure also point out the judgment > is specific to understanding the sequence of CMD's > listed above. Only a hack of ignoring, not recording, > or commenting out the filemon lines ending in > /tmp/legacy/usr/sbin/rm would seem to avoid the @rm > handling issue. Such might well have its own risks. >=20 > Some other /tmp/legacy/usr/sbin/* filemon line endings > likely would have a similar status of being a reference > to a file that could(/should?) have its timestamp > relationship not checked. Just for reference for more about the sequencing involved: Looks like in my example various . . ./tmp/legacy/. . ./*bin/ actually are links to files in: = /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/bi= n/ and the later after-install buildworld "Rebuilding the temporary build tree" step leads to the updated dates for files in that area, updated via the code that reports: Linking host tools into = /usr/obj/amd64_clang/amd64.amd64/usr/fbsd/mm-src/amd64.amd64/tmp/legacy/bi= n So the prior installworld leaves updated dates and the linking to those installed files makes . . ./tmp/legacy/. . ./*bin/* have the newer dates show up for the legacy paths as well. In turn the dependency tracking via META_MODE uses the new dates on . . ./tmp/legacy/. . ./*bin/* files to cause rebuilds of more materials than if installworld had not been done. It is not obvious if Bryan D. would find the effort to avoid this worthwhile for the contexts that drive FreeBSD's build environment choices. =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)