Date: Wed, 24 Jul 2024 08:56:41 -0600 From: Warner Losh <imp@bsdimp.com> To: "Bjoern A. Zeeb" <bz@freebsd.org> Cc: John Baldwin <jhb@freebsd.org>, "freebsd-arch@freebsd.org" <arch@freebsd.org> Subject: Re: Default NO_CLEAN=yes in 15+ Message-ID: <CANCZdfp7TxOSibpjURJpUFOw6xoEVQ6-k7_1=SKAkxCk4Fuq9g@mail.gmail.com> In-Reply-To: <no8rrp46-5544-0qn2-q70s-706p37p84679@SerrOFQ.bet> References: <9bbb12ee-d5e0-4e9c-a832-bbfe5eea0ba6@FreeBSD.org> <ia2xwmohoomnica7vp7hotd7sknovyyb3hgkfrx6blekbx6qdn@aax3blyqxcer> <71d28fd9-7679-4aaa-981e-81fa70c5da71@FreeBSD.org> <no8rrp46-5544-0qn2-q70s-706p37p84679@SerrOFQ.bet>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000023b8ea061dff7d3a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, Jul 24, 2024, 8:33=E2=80=AFAM Bjoern A. Zeeb <bz@freebsd.org> wrote= : > On Tue, 23 Jul 2024, John Baldwin wrote: > > > On 7/23/24 16:08, Shawn Webb wrote: > >> On Tue, Jul 23, 2024 at 03:58:13PM -0400, John Baldwin wrote: > >>> The buildworld and buildkernel targets include a "clean" step before > >>> building > >>> objects dating back before my time to 'make world' (I haven't looked > to > >>> see > >>> how far back it goes). To permit incremental builds, this step can b= e > >>> skipped > >>> via NO_CLEAN=3Dyes. This step is a bit unusual in build systems > however. > >>> Most > >>> build systems have separate commands for building vs cleaning (e.g. > 'make > >>> all' > >>> vs 'make clean') and over time FreeBSD's build system has gained > dedicated > >>> clean targets as well (cleanworld and cleankernel). For myself, I > always > >>> use NO_CLEAN=3Dyes when building worlds and kernels. If I need a cle= an > >>> build > >>> I use the dedicated clean targets (e.g. cleanworld) first. In > particular, > >>> cleanworld/cleankernel are far more efficient since they use a single > >>> recursive 'rm' whereas the "clean" step involves a full tree walk wit= h > >>> nested make invocations of the 'cleandir' target. > >>> > >>> A few years ago, Ed Maste added a MK_CLEAN option to src.opts.mk to > as a > >>> WITH/WITHOUT knob for the "clean" step similar to NO_CLEAN=3Dyes. To > >>> preserve > >>> existing behavior this knob currently defaults to on, but I know Ed's > goal > >>> was to eventually flip the default so that NO_CLEAN builds would be t= he > >>> default. I would like us to do that starting in 15. > >> > >> It would make sense to me to default MK_CLEAN=3Dno in release branches= . > >> Perhaps stable branches, too. While I don't hold a strong opinion on > >> the matter, I would prefer MK_CLEAN=3Dyes to remain the default on the > >> main branch. > >> > >> I can't give tangible examples, but I remember running into weird > >> issues occasionally when using `make buildworld WITHOUT_CLEAN=3Dyes` i= n > >> main. I probably should do a better job at documenting those > >> (infrequent) issues when they arise. > > > > To be clear, the suggestion is that when you hit an issue, just run > > 'make cleanworld' rather than relying on the omission of NO_CLEAN=3Dyes > > to do this for you (and much slower at that). Have you used any other > > build systems where 'make' does an implicit 'make clean' before it > > builds? I have never encountered another where this is true. 'clean' = is > > always a separate target from 'all' in my experience. > > Can we (if not in 15 but then in 16) simply: > > (1) document clean builds as make cleanblah .. && make blah > (2) document that if we do not run make cleanblah we'll get an > incremental build > (3) if an incremental build fails (CI will detect as well) -- do as Ed > said > No. It won't. Or only sometimes. I've had several times when tree A builds just fine whole tree B won't. 90% of the time it is an accumulation of changes that cause B (the older) to fail. The rest of the time it's build settings differences. We can get a lot of it from CI, but we will miss about 10% or so of the issues. There's maybe one breakage in 20 or 30 that will be some kind of judgement call, so we likely should establish sone guidance. Like how old is too old? On current, this is likely 6-9 months. On stable it's likely the branch point for the oldest supported release on the branch (also 6-9 months), though a good case could be made for one additional release prior since the cost is likely small, though the testing logistics get complicated. Likewise for build options. And for everything there's always a "too hard" reset available, though maybe that should be decided with the forthcoming srcmgr of current and maybe with re too for stable. (4) make sure re and so are aware of the extra make cleanblah needed? > They already start with an empty /usr/obj. (5) remove NO_CLEAN and MK_CLEAN entirely (there's no point in flipping > the default if you can choose to run a make step or not)? > > I think that is basically your suggestion? > Basically, yes. Warner +101 for that from here. > > /bz > > -- > Bjoern A. Zeeb r15:7 > > --00000000000023b8ea061dff7d3a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Wed, Jul 24, 2024, 8:33=E2=80=AFAM Bjoern A. Zeeb &= lt;<a href=3D"mailto:bz@freebsd.org">bz@freebsd.org</a>> wrote:<br></div= ><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1= px #ccc solid;padding-left:1ex">On Tue, 23 Jul 2024, John Baldwin wrote:<br= > <br> > On 7/23/24 16:08, Shawn Webb wrote:<br> >> On Tue, Jul 23, 2024 at 03:58:13PM -0400, John Baldwin wrote:<br> >>> The buildworld and buildkernel targets include a "clean&q= uot; step before <br> >>> building<br> >>> objects dating back before my time to 'make world' (I = haven't looked to <br> >>> see<br> >>> how far back it goes).=C2=A0 To permit incremental builds, thi= s step can be <br> >>> skipped<br> >>> via NO_CLEAN=3Dyes.=C2=A0 This step is a bit unusual in build = systems however. <br> >>> Most<br> >>> build systems have separate commands for building vs cleaning = (e.g. 'make <br> >>> all'<br> >>> vs 'make clean') and over time FreeBSD's build sys= tem has gained dedicated<br> >>> clean targets as well (cleanworld and cleankernel).=C2=A0 For = myself, I always<br> >>> use NO_CLEAN=3Dyes when building worlds and kernels.=C2=A0 If = I need a clean <br> >>> build<br> >>> I use the dedicated clean targets (e.g. cleanworld) first.=C2= =A0 In particular,<br> >>> cleanworld/cleankernel are far more efficient since they use a= single<br> >>> recursive 'rm' whereas the "clean" step invo= lves a full tree walk with<br> >>> nested make invocations of the 'cleandir' target.<br> >>> <br> >>> A few years ago, Ed Maste added a MK_CLEAN option to <a href= =3D"http://src.opts.mk" rel=3D"noreferrer noreferrer" target=3D"_blank">src= .opts.mk</a> to as a<br> >>> WITH/WITHOUT knob for the "clean" step similar to NO= _CLEAN=3Dyes.=C2=A0 To <br> >>> preserve<br> >>> existing behavior this knob currently defaults to on, but I kn= ow Ed's goal<br> >>> was to eventually flip the default so that NO_CLEAN builds wou= ld be the<br> >>> default.=C2=A0 I would like us to do that starting in 15.<br> >> <br> >> It would make sense to me to default MK_CLEAN=3Dno in release bran= ches.<br> >> Perhaps stable branches, too. While I don't hold a strong opin= ion on<br> >> the matter, I would prefer MK_CLEAN=3Dyes to remain the default on= the<br> >> main branch.<br> >> <br> >> I can't give tangible examples, but I remember running into we= ird<br> >> issues occasionally when using `make buildworld WITHOUT_CLEAN=3Dye= s` in<br> >> main. I probably should do a better job at documenting those<br> >> (infrequent) issues when they arise.<br> ><br> > To be clear, the suggestion is that when you hit an issue, just run<br= > > 'make cleanworld' rather than relying on the omission of NO_CL= EAN=3Dyes<br> > to do this for you (and much slower at that).=C2=A0 Have you used any = other<br> > build systems where 'make' does an implicit 'make clean= 9; before it<br> > builds?=C2=A0 I have never encountered another where this is true.=C2= =A0 'clean' is<br> > always a separate target from 'all' in my experience.<br> <br> Can we (if not in 15 but then in 16) simply:<br> <br> (1) document clean builds as make cleanblah .. && make blah<br> (2) document that if we do not run make cleanblah we'll get an<br> =C2=A0 =C2=A0 =C2=A0incremental build<br> (3) if an incremental build fails (CI will detect as well) -- do as Ed<br> =C2=A0 =C2=A0 =C2=A0said<br></blockquote></div></div><div dir=3D"auto"><br>= </div><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gma= il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef= t:1ex"></blockquote></div></div><div dir=3D"auto">No. It won't. Or only= sometimes. I've had several times when tree A builds just fine whole t= ree B won't. 90% of the time it is an accumulation of changes that caus= e B (the older) to fail. The rest of the time it's build settings diffe= rences. We can get a lot of it from CI, but we will miss about 10% or so of= the issues.</div><div dir=3D"auto"><br></div><div dir=3D"auto">There's= maybe one breakage in 20 or 30 that will be some kind of judgement call, s= o we likely should establish sone guidance. Like how old is too old? On cur= rent, this is likely 6-9 months. On stable it's likely the branch point= for the oldest supported release on the branch (also 6-9 months), though a= good case could be made for one additional release prior since the cost is= likely small, though the testing logistics get complicated.=C2=A0</div><di= v dir=3D"auto"><br></div><div dir=3D"auto">Likewise for build options.</div= ><div dir=3D"auto"><br></div><div dir=3D"auto">And for everything there'= ;s always a "too hard" reset available,=C2=A0 though maybe that s= hould be decided with the forthcoming srcmgr of current and maybe with re t= oo for stable.</div><div dir=3D"auto"><br></div><div dir=3D"auto"><br></div= ><div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_qu= ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex= "> (4) make sure re and so are aware of the extra make cleanblah needed?<br></= blockquote></div></div><div dir=3D"auto"><br></div><div dir=3D"auto">They a= lready start with an empty /usr/obj.</div><div dir=3D"auto"><br></div><div = dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quote" s= tyle=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> (5) remove NO_CLEAN and MK_CLEAN entirely (there's no point in flipping= <br> =C2=A0 =C2=A0 =C2=A0the default if you can choose to run a make step or not= )?<br> <br> I think that is basically your suggestion?<br></blockquote></div></div><div= dir=3D"auto"><br></div><div dir=3D"auto">Basically, yes.</div><div dir=3D"= auto"><br></div><div dir=3D"auto">Warner</div><div dir=3D"auto"><br></div><= div dir=3D"auto"><div class=3D"gmail_quote"><blockquote class=3D"gmail_quot= e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> +101 for that from here.<br> <br> /bz<br> <br> -- <br> Bjoern A. Zeeb=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r15:7<br> <br> </blockquote></div></div></div> --00000000000023b8ea061dff7d3a--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfp7TxOSibpjURJpUFOw6xoEVQ6-k7_1=SKAkxCk4Fuq9g>