Date: Fri, 29 Jan 2021 17:38:31 +0100 From: Henrik Rosenke <rosenke@dssgmbh.de> To: Miroslav Lachman <000.fbsd@quip.cz>, FreeBSD <freebsd-ports@freebsd.org> Subject: Re: poudriere merging multiple ports trees Message-ID: <3f87ebbe-7ecc-9f52-e556-eeb939e7acd2@dssgmbh.de> In-Reply-To: <1f0d7c6c-f97e-32a1-6639-512c804431df@dssgmbh.de> References: <3ea16730-84a1-52ce-2251-bdd808fe5c52@pinyon.org> <4990d009-1962-62c2-3f4e-4f62fd03e26d@madpilot.net> <20210125141051.qheedhp76d2ha6zy@ivaldir.net> <3f2d08b2-2959-dd3e-89e5-895437e44040@quip.cz> <20210125152825.mnd4qhi4k23dy6dw@ivaldir.net> <0e79482c-fd7c-87a2-0052-5d7eeffd3099@dssgmbh.de> <a17ab317-b023-3b35-52a7-4952786742dc@dssgmbh.de> <b0fa1609-2a58-d68e-57c7-337f5101379b@quip.cz> <a6886f94-ed37-8241-3884-5dc56a149835@dssgmbh.de> <e66b47f4-6fa2-70cf-830a-31b717e9ce56@quip.cz> <1f0d7c6c-f97e-32a1-6639-512c804431df@dssgmbh.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Greetings, i created a simple patch for this, this may not be the best solution but my tests seems to be successful to get the desired result: Without NOT_MOVED: =>> Debug: Reading /usr/local/etc/poudriere.conf [00:00:00] Creating the reference jail... done [00:00:02] Mounting system devices for j12p64-default-server [00:00:02] Mounting ports/packages/distfiles [00:00:02] Using packages from previously failed build: /home/poudriere/data/packages/j12p64-default-server/.building [00:00:02] Mounting ccache from: /home/poudriere/ccache [00:00:02] Mounting packages from: /home/poudriere/data/packages/j12p64-default-server [00:00:02] Copying /var/db/ports from: /usr/local/etc/poudriere.d/j12p64-options [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/server-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/default-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/j12p64-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/default-server-make.conf [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/chan_capi [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-kmod [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-utils [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): sysutils/env4801 [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): security/ipsec-tools /etc/resolv.conf -> /home/poudriere/data/.m/j12p64-default-server/ref/etc/resolv.conf [00:00:02] Starting jail j12p64-default-server [00:00:02] Logs: /home/poudriere/data/logs/bulk/j12p64-default-server/2021-01-29_11h14m59s [00:00:02] WWW: http://dsspkg.incore/poudriere//build.html?mastername=j12p64-default-server&build=2021-01-29_11h14m59s [00:00:02] Loading MOVED for /home/poudriere/data/.m/j12p64-default-server/ref/usr/ports [00:00:03] Ports supports: FLAVORS SELECTED_OPTIONS [00:00:03] Gathering ports metadata [00:00:03] Error: MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has expired: Ancient and unsupported release [00:00:03] Error: Fatal errors encountered gathering initial ports metadata [00:00:03] Cleaning up [00:00:03] Unmounting file systems With NOT_MOVED: === root@dsspkg (pts/1) /home/poudriere/ports/dss_overlay 35 -> cat NOT_MOVED sysutils/sge62 === root@dsspkg (pts/1) /home/poudriere/ports/dss_overlay 97 -> poudriere bulk -vv -O dss_overlay -j j12p64 -z server -p default sysutils/sge62 =>> Debug: Reading /usr/local/etc/poudriere.conf [00:00:00] Creating the reference jail... done [00:00:02] Mounting system devices for j12p64-default-server [00:00:02] Mounting ports/packages/distfiles [00:00:02] Using packages from previously failed build: /home/poudriere/data/packages/j12p64-default-server/.building [00:00:02] Mounting ccache from: /home/poudriere/ccache [00:00:02] Mounting packages from: /home/poudriere/data/packages/j12p64-default-server [00:00:02] Copying /var/db/ports from: /usr/local/etc/poudriere.d/j12p64-options [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/server-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/default-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/j12p64-make.conf [00:00:02] Appending to make.conf: /usr/local/etc/poudriere.d/default-server-make.conf [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/chan_capi [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-kmod [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): comms/isdn4bsd-utils [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): sysutils/env4801 [00:00:02] Warning: Blacklisting (from /usr/local/etc/poudriere.d/j12p64-blacklist): security/ipsec-tools /etc/resolv.conf -> /home/poudriere/data/.m/j12p64-default-server/ref/etc/resolv.conf [00:00:02] Starting jail j12p64-default-server [00:00:03] Logs: /home/poudriere/data/logs/bulk/j12p64-default-server/2021-01-29_17h28m52s [00:00:03] WWW: http://dsspkg.incore/poudriere//build.html?mastername=j12p64-default-server&build=2021-01-29_17h28m52s [00:00:03] Loading MOVED for /home/poudriere/data/.m/j12p64-default-server/ref/usr/ports [00:00:03] Loading NOT_MOVED from overlaydir: /home/poudriere/data/.m/j12p64-default-server/ref/overlays/dss_overlay [00:00:03] Ports supports: FLAVORS SELECTED_OPTIONS [00:00:03] Gathering ports metadata [00:00:03] Debug: sysutils/sge62 found in /home/poudriere/data/.m/j12p64-default-server/ref/overlays/dss_overlay/NOT_MOVED, skipped checking MOVED File of master portstree [00:00:03] Debug: queueing sysutils/sge62 into gatherqueue (rdep=listed) [00:00:03] Debug: Processing gatherqueue [00:00:03] Warning: (sysutils/sge62): Debug: gather_port_vars_port (sysutils/sge62): LOOKUP [00:00:03] Warning: (sysutils/sge62): Debug: deps_fetch_vars: discovered sysutils/sge62 is sge-6.2.2.1_6 [00:00:03] Warning: (sysutils/sge62): Debug: WILL BUILD sysutils/sge62 Patch leading to this: --- /usr/local/share/poudriere/common.sh.1st 2021-01-27 15:44:15.000000000 +0100 +++ /usr/local/share/poudriere/common.sh 2021-01-29 17:27:14.810441000 +0100 @@ -6815,8 +6815,11 @@ continue fi origin_listed="${origin}" - if shash_get origin-moved "${origin}" new_origin; then - if [ "${new_origin%% *}" = "EXPIRED" ]; then + # check against the overlay list, skip port if its in there + if (grep -q "${origin}" ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED); then + msg_debug "${origin} found in ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED, skipped checking MOVED File of master portstree" + elif shash_get origin-moved "${origin}" new_origin; then + if [ "${new_origin%% *}" = "EXPIRED" ] ; then msg_error "MOVED: ${origin} ${new_origin}" set_dep_fatal_error continue @@ -7085,6 +7088,10 @@ fi [ -f ${MASTERMNT}${PORTSDIR}/MOVED ] || return 0 msg "Loading MOVED for ${MASTERMNT}${PORTSDIR}" + # Respect overlays, ${OVERLAYS} has a leading whitepace + if [ -f ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }/NOT_MOVED ]; then + msg "Loading NOT_MOVED from overlaydir: ${MASTERMNT}${OVERLAYSDIR}/${OVERLAYS# }" + fi bset status "loading_moved:" awk -f ${AWKPREFIX}/parse_MOVED.awk \ ${MASTERMNT}${PORTSDIR}/MOVED | \ Maybe someone else could test this, seems to work for me. Kind regards, Henrik Rosenke Am 28.01.21 um 21:56 schrieb Henrik Rosenke: > > Am 28.01.21 um 21:14 schrieb Miroslav Lachman: >> On 28/01/2021 17:47, Henrik Rosenke wrote: >>> I tried this with a patched MOVED file, copied from the master >>> portstree, deleted the entry and tried to build but this File seems >>> to be ignored. I dont really understand where the MOVED file is >>> handled, it seems to be bsd.port.subdir.mk but i am not sure. >> >> MOVED is handled by bsd.port.subdir.mk but overlays do not account >> with it. Overlays are handled on more places > > i thought this too but it seems poudriere is canceled before this, i > even tried to move the bsd.port.subdir.mk and commented out the > function load_moved in the common.sh of poudriere and can start a > build. Correct me if iam wrong but it seems that the common.sh from > poudriere needs to be patched to handle this rather than the Mk files > itself. I added set -x in the common.sh to get a better understanding > what is happening and it seems that shash_set sucks in the MOVED file > and this is checked later against the ports we want to build: > > + shash_set origin-moved sysutils/sge62 'EXPIRED 2013-06-05 Has > expired: Ancient and unsupported release' > > later: > > + originspec_decode sysutils/sge62 origin '' flavor > + local - > + set +x -f > + [ -n '' ] > + origin_listed=sysutils/sge62 > + shash_get origin-moved sysutils/sge62 new_origin > + local - > + set +x > + [ EXPIRED '=' EXPIRED ] > + msg_error 'MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has expired: > Ancient and unsupported release' > + local - > + set +x > [00:00:07] Error: MOVED: sysutils/sge62 EXPIRED 2013-06-05 Has > expired: Ancient and unsupported release > > Kind regards, > > Henrik Rosenke > >> >> Scripts/do-depends.sh >> Scripts/depends-list.sh >> >> Next is bsd.port.mk: >> >> .if defined(USE_LOCAL_MK) >> .include "${PORTSDIR}/Mk/bsd.local.mk" >> .endif >> .for odir in ${OVERLAYS} >> .sinclude "${odir}/Mk/bsd.overlay.mk" >> .endfor >> >> But there is no bsd.overlay.mk and I don't know what should be in it >> (file in the overlay ports tree) >> >> The AWK code in bsd.port.subdir.mk can be changed to use MOVED from >> overlay if it exists insead of the default one. >> >>> I think the best way to handle this would be a NOT_MOVED file in the >>> overlay portstree with the ports to ignore from the MOVED file with >>> entries like "sysutils/sge62". >> >> This can be possible too. NOT_MOVED can be used as additional INDEXFILE >> >> Or maybe start with PORTSEARCH_MOVED=0 (not tried yet) >> >> Kind regards >> Miroslav Lachman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3f87ebbe-7ecc-9f52-e556-eeb939e7acd2>