Date: Tue, 11 Jul 2023 21:05:34 -0700 From: Kevin Oberman <rkoberman@gmail.com> To: Mark Millard <marklmi@yahoo.com> Cc: "Hurling, Rainer" <rhurlin@freebsd.org>, FreeBSD Mailing List <freebsd-ports@freebsd.org> Subject: Re: Problem with the package builds Message-ID: <CAN6yY1uhOTMKkfQZ09Esz4VW-jZ-CNuet_CAKTQ4rTO-pnL2ag@mail.gmail.com> In-Reply-To: <44201182-73C9-46BC-A846-7DC765F5B81F@yahoo.com> References: <2DCC0132-0307-4A73-A473-B749ABF87367.ref@yahoo.com> <2DCC0132-0307-4A73-A473-B749ABF87367@yahoo.com> <1aaeafcc-9812-64bd-a369-bae7b9fc0e5f@gwdg.de> <422DBC3B-D85F-4AFB-ABDE-842A08482EC8@yahoo.com> <58250d08-ce0a-c1ac-ed15-7d55d517218e@gwdg.de> <2ea168aa-3e0e-ee31-d4a3-82b4ce46b330@gwdg.de> <CAN6yY1sqtLCbqW4%2BxtXfduN7WPPQBnd=FbiknnvDFtGMisveaw@mail.gmail.com> <78c1497b-1ec7-08b6-a407-d671e3c4f9dd@gwdg.de> <ED4C0309-19BB-496D-AA48-462A3303412F@yahoo.com> <CAN6yY1sMceeGHOZHECUHTQKa2UXoXt9x5fdm7A5-b-uyaxCzWQ@mail.gmail.com> <44201182-73C9-46BC-A846-7DC765F5B81F@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000da5e4c0600425494 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jul 11, 2023 at 1:10=E2=80=AFPM Mark Millard <marklmi@yahoo.com> wr= ote: > On Jul 11, 2023, at 12:57, Kevin Oberman <rkoberman@gmail.com> wrote: > > > On Mon, Jul 10, 2023 at 9:42=E2=80=AFPM Mark Millard <marklmi@yahoo.com= > wrote: > > On Jul 10, 2023, at 21:27, Rainer Hurling <rhurlin@gwdg.de> wrote: > > > > > As I understand it, the ports-mgmt/pkg of the system running Poudrier= e > must be updated beforehand? > > > > > > At least on my side, this seems to work as expected :) > > > > > > > poudriere builds pkg updates first (if needed) and then uses the pkg it > > built for building the later ports into packages. > > > > But, after the restarts of main-* builds, the FreeBSD build servers are > > still showing examples were, after an 1hr, some builds are still in > > build-depends. Also there was an example I saw were after 1.5 hr it was > > still in run-depends. > > > > It may be that things are improved but not fully fixed relative to > > some performance issues. > > > > =3D=3D=3D > > Mark Millard > > marklmi at yahoo.com > > A new build started this morning at 1:06 UTC and, with pkg-1.20.2, it'= s > better, but not much. It's running at 21 packages/hour, a 100% improvemen= t > on the last attempt which appears to have been killed last night. The log= s > indicate the installation of dependencies, but I don't see any sign of > caching. It's a re-install every time. (I may not understand how poudrier= e > does things, but I am pretty sure that caching is done.) > > Just about "caching" relative to "poudriere bulk" builds . . . > > Nope. At the end of a builder run of a port build the context is destroye= d. > At the start of the builder building its next port the context is recreat= ed > from scratch. The only ports installed are exactly the declared > dependencies, > no more, no less, for the new port to be built. > > Caching installed state would imply access to ports from prior build > activity that do not apply: It would make the build environment polluted > with > irrelevant history. poudriere's purpose is to have a "clean-room" context > for > each port build. Thus its construction of such a context for each port > build. > > Caching vs. not is not the source of the large increase in how long thing= s > take to build. > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > OK. I knew it started in a clean jail. I just thought that it might use some caching technique to speed repeated installs of a single port... not that I have any idea of how this might be done. I think I'll monitor the speed of installs. I iish build logs included timestamps --=20 Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 --000000000000da5e4c0600425494 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><div class=3D"gmail_default" style=3D"fon= t-family:tahoma,sans-serif;font-size:small">On Tue, Jul 11, 2023 at 1:10=E2= =80=AFPM Mark Millard <<a href=3D"mailto:marklmi@yahoo.com">marklmi@yaho= o.com</a>> wrote:</div></div><div class=3D"gmail_quote"><blockquote clas= s=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid r= gb(204,204,204);padding-left:1ex">On Jul 11, 2023, at 12:57, Kevin Oberman = <<a href=3D"mailto:rkoberman@gmail.com" target=3D"_blank">rkoberman@gmai= l.com</a>> wrote:<br> <br> > On Mon, Jul 10, 2023 at 9:42=E2=80=AFPM Mark Millard <<a href=3D"ma= ilto:marklmi@yahoo.com" target=3D"_blank">marklmi@yahoo.com</a>> wrote:<= br> > On Jul 10, 2023, at 21:27, Rainer Hurling <<a href=3D"mailto:rhurli= n@gwdg.de" target=3D"_blank">rhurlin@gwdg.de</a>> wrote:<br> > <br> > > As I understand it, the ports-mgmt/pkg of the system running Poud= riere must be updated beforehand?<br> > > <br> > > At least on my side, this seems to work as expected :)<br> > > <br> > <br> > poudriere builds pkg updates first (if needed) and then uses the pkg i= t<br> > built for building the later ports into packages.<br> > <br> > But, after the restarts of main-* builds, the FreeBSD build servers ar= e<br> > still showing examples were, after an 1hr, some builds are still in<br= > > build-depends. Also there was an example I saw were after 1.5 hr it wa= s<br> > still in run-depends.<br> > <br> > It may be that things are improved but not fully fixed relative to<br> > some performance issues.<br> > <br> > =3D=3D=3D<br> > Mark Millard<br> > marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_= blank">yahoo.com</a><br> >=C2=A0 A new build started this morning at 1:06 UTC and, with pkg-1.20.= 2, it's better, but not much. It's running at 21 packages/hour, a 1= 00% improvement on the last attempt which appears to have been killed last = night. The logs indicate the installation of dependencies, but I don't = see any sign of caching. It's a re-install every time. (I may not under= stand how poudriere does things, but I am pretty sure that caching is done.= )<br> <br> Just about "caching" relative to "poudriere bulk" build= s . . .<br> <br> Nope. At the end of a builder run of a port build the context is destroyed.= <br> At the start of the builder building its next port the context is recreated= <br> from scratch. The only ports installed are exactly the declared dependencie= s,<br> no more, no less, for the new port to be built.<br> <br> Caching installed state would imply access to ports from prior build<br> activity that do not apply: It would make the build environment polluted wi= th<br> irrelevant history. poudriere's purpose is to have a "clean-room&q= uot; context for<br> each port build. Thus its construction of such a context for each port buil= d.<br> <br> Caching vs. not is not the source of the large increase in how long things<= br> take to build.<br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_blank= ">yahoo.com</a><br></blockquote><div><span class=3D"gmail_default" style=3D= "font-family:tahoma,sans-serif;font-size:small"></span><div class=3D"gmail_= default" style=3D"font-family:tahoma,sans-serif;font-size:small">OK. I knew= it started in a clean jail. I just thought that it might use some caching = technique to speed repeated installs of a single port... not that I have an= y idea of how this might be done.</div><div class=3D"gmail_default" style= =3D"font-family:tahoma,sans-serif;font-size:small"><br></div><div class=3D"= gmail_default" style=3D"font-family:tahoma,sans-serif;font-size:small">I th= ink I'll monitor the speed of installs. I iish build logs included time= stamps</div> </div></div><span class=3D"gmail_signature_prefix">-- </span><= br><div dir=3D"ltr" class=3D"gmail_signature"><div dir=3D"ltr"><div><div di= r=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr">Kevin Oberman, Part t= ime kid herder and retired Network Engineer<br>E-mail: <a href=3D"mailto:rk= oberman@gmail.com" target=3D"_blank">rkoberman@gmail.com</a><br></div><div>= PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683</div></div></div>= </div></div></div></div></div></div> --000000000000da5e4c0600425494--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAN6yY1uhOTMKkfQZ09Esz4VW-jZ-CNuet_CAKTQ4rTO-pnL2ag>