Date: Fri, 14 May 2021 19:29:15 -0700 From: Mark Millard <marklmi@yahoo.com> To: bob prohaska <fbsd@www.zefox.net>, FreeBSD ports <freebsd-ports@freebsd.org> Subject: Re: Python 37/38 conflict, was Re: Trubles compiling lxqt on RPi4 Message-ID: <0B407A98-E0D4-461E-BFD8-E02019E96757@yahoo.com> References: <0B407A98-E0D4-461E-BFD8-E02019E96757.ref@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
bob prohaska fbsd at www.zefox.net wrote on Fri May 14 01:35:28 UTC 2021 : > On Thu, May 13, 2021 at 01:35:50PM -0700, Mark Millard wrote: > > You have apparently chosen to build/update ports via a > > technique that requires you to manage the dependencies, at > > least some of the time. (Not that when is necessarily > > obvious up front.) > >=20 >=20 > You give me too much credit 8-) >=20 > > Your environment is now based on a mix of python37 and > > python 38 related materials. You are likely going to > > need to rework/rebuild/reinstall things to avoid that. > >=20 > > Hints may come from that UPDATING that I quoted but > > things are more broken overall than what UPDATING is > > intended to cover. You might end up needing to > > uninstall a bunch of stuff until python is unused > > (or only one python is used) and then follow the > > notes if you have only python37 use and want to > > get to python38. Finally rebuild/reinstall what > > was uninstalled, based on python38 as a context. > >=20 > Trying to deinstall both python37 and python38 didn't > suffice. Python38's reinstall fails with the same > conflict. Deleting the offending file doesn't help=20 > If other things need to be deinstalled it's not clear > what they might be. > =20 > > Due to how I build/install ports, I've not had to deal > > with ending up with the mix so I'm not familiar with > > the details for recovering from a messy mix. > >=20 >=20 > Would use of poudriere help with this sort of problem? > It isn't trivial to configure, but this sort of difficulty > has been growing ever worse. I didn't want to deal with the=20 > complexity and overhead, but maybe it's time.=20 >=20 I happen to use ports-mgmt/poudriere-devel and it avoids such issues but does have build-time tradeoffs and the like. (I'll note that I presume the sort of sustem tuning to avoid Out Of Memory kills and I try to scale to avoid a literal out of swap space problems.) I'll start with very overall background for port building because I do not understand your context or goals. Otherwise my material could end-up implicitly be picking from the alternatives in an inappropriate way. Some of this is relevant to (all?) other forms of port building as well. Some basic questions will be: A) ZFS vs. UFS? (There are some configuration setting(s) dependent on which.) I currently have examples of both: I've started experimenting with ZFS again in some contexts, after years of not using it. No individual context is using a mix of both and I use ZFS in contexts with >=3D 8 GiBytes of RAM. I do not try to tune it for small memory contexts (small on ZFS's scale). B) How a builder establishes a world-context to execute in? For reasons of testing patches and such I build and install a world into a directory tree and have poudriere use that tree instead of poudriere installing or even building its own world in a tree. (And I've never done it any other way.) I do this with separate world-trees for aarch64 vs. armv7 on an aarch64 system so I build for armv7 in a faster context with more RAM and then transfer materials to the armv7 system for pkg to use for pkg commands. (I've not set up a server/client context.) You could, of course, just deal with "native" and ignore the RPi* aarch64's supporting doing armv7 builds. I use the same buildworld for updating the running kernel and world and for installing the world used for poudriere when the same OS vintage/variation is to be used for both. If you prebuild, there will be questions of what paths you want to use to reference the for-poudriere build trees. C) How a builder establishes a ports tree? For reasons of testing patches and such I have a /usr/ports tree of my own (sometimes under another name) and have poudriere use that tree instead of making its own. (And I've never done it any other way.) I use the same /usr/ports for both aarch64 and armv7, so only the one copy. You might want a different path than /usr/ports if you pre-establish the ports tree. D) What FreeBSD versions to target? I do not happen to use ports that must track the kernel version in detail so I can target a releng/13's release/13.?.? and use the ports for stable/13 as well. In fact, I can generally get away with using those same ports on main [so: 14], being explicit about the ABI for the pkg commands. You might use ports to drive displays and such in a way that leaves you required to track kernel versions more closely. But if it is only RPi*'s, then may be not for that. But there are other ports around that violate the clean separation vs. kernel details. To some extent this gets into "how many builds to cover all the systems?". That in turn can influence how the systems are set up, such as to eliminate some builds being needed. Your context might be simple, with only one type of context to cover. E) Build as root? As non-root? I happen to only have done build as root but the systems are not used for other activities. There could be ownership/permission issues that I've not run into. Examples of a couple of configuration files . . . You might end up with a file like: # more /usr/local/etc/poudriere.d/make.conf=20 WANT_QT_VERBOSE_CONFIGURE=3D1 (I no longer remember why I enabled that.) For a UFS context an example of the differences with the /usr/local/etc/poudriere.conf.sample file is as follows ( +: poudriere.conf and -: poudriere.conf.sample ). Some of it presumes example answers to the earlier questions. (white space details might not go through unchanged) # diff -u /usr/local/etc/poudriere.conf.sample = /usr/local/etc/poudriere.conf --- /usr/local/etc/poudriere.conf.sample 2021-03-09 = 07:43:07.000000000 -0800 +++ /usr/local/etc/poudriere.conf 2021-05-14 19:09:30.055596000 = -0700 @@ -13,7 +13,7 @@ =20 ### NO ZFS # To not use ZFS, define NO_ZFS=3Dyes -#NO_ZFS=3Dyes +NO_ZFS=3Dyes =20 # root of the poudriere zfs filesystem, by default /poudriere # ZROOTFS=3D/poudriere @@ -27,7 +27,7 @@ # Also note that all protocols supported by fetch(1) are supported = here, even # file:/// # Suggested: https://download.FreeBSD.org -FREEBSD_HOST=3D_PROTO_://_CHANGE_THIS_ +FREEBSD_HOST=3Dhttps://download.FreeBSD.org =20 # By default the jails have no /etc/resolv.conf, you will need to set # RESOLV_CONF to a file on your hosts system that will be copied to @@ -56,7 +56,15 @@ # yes - Enables tmpfs(5) for wrkdir and data # no - Disable use of tmpfs(5) # EXAMPLE: USE_TMPFS=3D"wrkdir data" -USE_TMPFS=3Dyes +#NOTE: Disable (or just "data") for < 8 GiByte RAM environments, +# presuming -j4 for 4 cores. wrkdir can be rather large +# (17+ GiByte for lang/rust) and localbase can be larger. +# Swap covers tmpfs space needs. +# For aarch64 I keep total swap somewhat under 4*RAM. +# For armv7 I keep total swap somewhat under 2*RAM. +# This avoids hard to interpret boot notices about being +# mistuned. +USE_TMPFS=3Dno =20 # How much memory to limit tmpfs size to for *each builder* in GiB # (default: none) @@ -155,6 +163,11 @@ # # Example to define PARALLEL_JOBS to one single job # PARALLEL_JOBS=3D1 +#NOTE: on 2 GiByte RAM contexts I've used: PARALLEL_JOBS=3D2 but +# two llvm*'s are likely the biggest combination that +# could occur in my context. lang/rust or other even +# larger build contexts need not be appropriate. I +# normally use ALLOW_MAKE_JOBS=3Dyes . =20 # How many jobs should be used for preparing the build? These tend to # be more IO bound and may be worth tweaking. Default: PARALLEL_JOBS * = 1.25 @@ -188,12 +201,13 @@ =20 # By default MAKE_JOBS is disabled to allow only one process per cpu # Use the following to allow it anyway -# ALLOW_MAKE_JOBS=3Dyes +ALLOW_MAKE_JOBS=3Dyes =20 # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. #ALLOW_MAKE_JOBS_PACKAGES=3D"pkg ccache py*" +ALLOW_MAKE_JOBS_PACKAGES=3D"pkg ccache py* gcc* llvm* rust* ghc* = *webkit* *office* chromium* iridium* mongodb*" =20 # Timestamp every line of build logs # Default: no @@ -208,11 +222,22 @@ # This defines the max time (in seconds) that a command may run for a = build # before it is killed for taking too long. Default: 86400 #MAX_EXECUTION_TIME=3D86400 +# Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: +MAX_EXECUTION_TIME=3D432000 =20 # This defines the time (in seconds) before a command is considered to # be in a runaway state for having no output on stdout. Default: 7200 #NOHANG_TIME=3D7200 +# Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: +# Also: boost-libs seems to have a long time with no output but it = making +# progress in parallel builds. +NOHANG_TIME=3D28800 =20 +# Cortex-A53 and such are slow for the purpose, allow 4 times the = defaults: +MAX_EXECUTION_TIME_EXTRACT=3D14400 +MAX_EXECUTION_TIME_INSTALL=3D14400 +MAX_EXECUTION_TIME_PACKAGE=3D28800 +MAX_EXECUTION_TIME_DEINSTALL=3D14400 =20 # The repository is updated atomically if set yes. This leaves the # repository untouched until the build completes. This involves using @@ -260,7 +285,7 @@ # set. Note that to use ccache with BUILD_AS_NON_ROOT you will need to # use a non-shared CCACHE_DIR that is only built by PORTBUILD_USER and = chowned # to that user. Then set CCACHE_DIR_NON_ROOT_SAFE to yes. -#BUILD_AS_NON_ROOT=3Dno +BUILD_AS_NON_ROOT=3Dno =20 # Define to the username to build as when BUILD_AS_NON_ROOT is yes. # Default: nobody (uid PORTBUILD_UID) =3D=3D=3D Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0B407A98-E0D4-461E-BFD8-E02019E96757>