Date: Sat, 10 Jul 2021 10:40:10 -0700 From: bob prohaska <fbsd@www.zefox.net> To: Mark Millard <marklmi@yahoo.com> Cc: freebsd-ports@freebsd.org Subject: Re: Too many pythons in poudriere Message-ID: <20210710174010.GA91320@www.zefox.net> In-Reply-To: <0313838D-C54A-4517-A53B-C70B4EDDE296@yahoo.com> References: <044A7E63-2734-41F4-A1A2-AE5096C6A62C@yahoo.com> <CDDBF956-9647-464C-9114-CB1021C14699@yahoo.com> <EE5052EA-0393-4AEB-A6A7-AE24F63F8E46@yahoo.com> <20210708154558.GB60914@www.zefox.net> <E1984653-A703-4B92-A48D-131514C92E92@yahoo.com> <20210708175436.GA70414@www.zefox.net> <24795168-CEBF-4049-A4BD-D529F70E7F8D@yahoo.com> <6D9BAA86-CB85-42E9-B4FC-6272141CFFDB@yahoo.com> <20210709230738.GA83522@www.zefox.net> <0313838D-C54A-4517-A53B-C70B4EDDE296@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Jul 09, 2021 at 05:12:57PM -0700, Mark Millard wrote: > On 2021-Jul-9, at 16:07, bob prohaska <fbsd at www.zefox.net> wrote: [big snip] > > Just to be clear, after updating kernel and world I gather the > > suggestion is to repeat > > > > # cd /usr/src > > # make installworld DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1 > > # make distrib-dirs DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1 > > # make distribution DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1 > > I though you had already synchronized /usr/local/poudriere/poudriere-system > to match /usr/src/ and the host root file system in use. If so, no reason > to repeat until /usr/src/ (and the booted host root file system) is updated > for some reason. Avoid mixes where /usr/src/ is mismatched with the root > file system at the time. > > Of course, you might want to do another update to those for all I know. > I tend to update /usr/src and /usr/ports somewhat randomly, based on elapsed time or in response to trouble of some kind. It looks like that habit may be the source of at least some of my trouble. > One thing you did not show was the command for after the above block of > 4 commands for updates: > > # make -DBATCH_DELETE_OLD_FILES delete-old DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1 > > If you have not done that yet, it would be appropriate. > > For the current context of starting over, another command that > is appropriate (that would normally be delayed to after all the > poudreire bulk commands are done [were prior package builds > were instead being put to use]) is: > > # make -DBATCH_DELETE_OLD_FILES delete-old-libs DESTDIR=/usr/local/poudriere/poudriere-system DB_FROM_SRC=1 > I got burned by a delete-old-something long ago, broke most of the installed ports, and haven't touched it since. In this case it does seem appropriate. Added to the update to-do list. > I'll note that the host updating procedure also involves using > delete-old and, after package updates are all installed, > delete-old-libs . (No explicit DESTDIR= or DB_FROM_SRC= use > involved for the root file system update.) > It resembles the command that bit me.... > > Next, > > cd /usr/local/poudriere > > poudriere jail -d main -C all > > to get rid of the old jail and packages, then re-make the jail with > > > > # poudriere ports -c -m null -M /usr/ports > > # poudriere jail -c -j main -m null -M /usr/local/poudriere/poudriere-system -S /usr/src -v 14.0-CURRENT > > > > Have I got this right? One thing I'm hesitant about is > > the -C all option. > > No reason to delete and recreate the jail, just do a bulk build that > rebuilds the ports. The first "poudriere bulk" of a sequence can use > the "-c" option to clear out the existing packages (and logs) before > the builders start building. > Aye, there's the rub.... It appears that incremental updates are a costly approach (world/kernel, then a port, then world/kernel...). It seemed one could get away with it using make in the ports tree, though perhaps some of my troubles with that approach had the same origin. > As far as I know you eventually want to install: > > # pkg install chromium > > (that will also install things required to run chromium). > > I'm not sure if you have a list of other packages that > you want explicitly installed. So my notes below need > not cover everything that you want to do. > Getting a running version of chromium was the origninal goal. The train seems to have run off the tracks......8-) > (I do not know if ALLOW_MAKE_JOBS= and such is appropriate for the > llvm10 bulk build but it may fit in the resources okay:) > > # poudriere bulk -j main -c devel/llvm10 > bulk_from_scratch_for_llvm10_1st.log > > (I'm unclear if you would want a -J? to limit the builders > count for the above command: if so, use it too.) > > That -c will first clear out the existing package builds (and > the logs according to the documentation). Then it will start > from scratch building the required ports. > > This will build a lot of ports, those needed to in turn build > devel/llvm10 . So having -J1 used and ALLOW_MAKE_JOBS missing > would make for a very long build time. You probably want to > allow some combination of multiple cores to be in use, > possibly also involving MAKE_JOBS_NUMBER= use. You know more > about what combinations worked before. > > Reminder that ALLOW_MAKE_JOBS= control would go in: > > /usr/local/etc/poudriere.conf > > but MAKE_JOBS_NUMBER= control would go in: > > /usr/local/etc/poudriere.d/make.conf > That is a helpful distinction unclear to me till now. > (I do not know if ALLOW_MAKE_JOBS= and such is appropriate for the > rust bulk build but some combination may fit in the resources okay:) > > # poudriere bulk -j main lang/rust > bulk_for_rust_2nd.log > > (I'm unclear if you would want a -J? to limit the builders > count for the above command: if so, use it too. You know > more about what combinations worked before.) > > (Without ALLOW_MAKE_JOBS= is likely appropriate --or at least the > MAKE_JOBS_NUMBER= may be needed-- for:) > > # poudriere bulk -j main www/chromium > bulk_for_chromium_3rd.log > > (I'm unclear if you would want a -J? to limit the builders > count for the above command: if so, use it too. You know > more about what combinations worked before.) > > Note that the management of the limited RPi3B context involves > adjusting things like ALLOW_MAKE_JOBS= and -J? usage for each > huge build to limit resource use to an appropriate scale for > those builds. For smaller builds, more can be allowed (more > adjustments). Since I usually play with ports one at a time, is there any advantage to using testport versus bulk? Better diagnostics, maybe? > > I do not know if there are other ports that you would want > to build and later install that are not automatically built > by the above or if you have a list of such in a file that > can be used with -f . In your context this would probably > be a list of ports that do not involve huge builds, given > that you had already done the above bulk runs (and any others > that might involve more huge builds). > It's clear that by segregating build dependencies from run dependencies poudriere provides a valuable service. Might there be a make target in ports that cleans up no-longer-needed build dependencies during installation of a given port? That would, I think, have been enough to avoid the python27 conflicts. Is there a plain-language description of the logic poudriere follows for building ports? I found this: https://forums.freebsd.org/threads/pkg-package-repository-using-ports-mgmt-poudriere-with-or-without-zfs.38859/ but it doesn't really touch on the topics I'm looking for. Something on the level of a simplified flowchart or (gasp!) PowerPoint, maybe? Thanks for reading, and all your help! bob prohaska
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20210710174010.GA91320>