From nobody Fri Feb 14 21:47:19 2025 X-Original-To: freebsd-pkg@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4YvlyP35zVz5njNM for ; Fri, 14 Feb 2025 21:47:25 +0000 (UTC) (envelope-from nxjoseph@protonmail.com) Received: from mail-40130.protonmail.ch (mail-40130.protonmail.ch [185.70.40.130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4YvlyP0pLGz3hMW for ; Fri, 14 Feb 2025 21:47:25 +0000 (UTC) (envelope-from nxjoseph@protonmail.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1739569643; x=1739828843; bh=xDAoJNrSoz2T/tdAIAonODTKDPQSf8WWF8W/5eQw3a4=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=qtkY4CvfCv2SI8kSSZ5GbQtQ5ikA9zKAJLAH6ezc+GrFerMhQkaKRlFp/CuCP77SS No3WjiK8A9197qOjoVHCZGbIgqt2Fm3h6ne94bA+9w0ysSFifn3DeodQoSmZM4nA8/ XoTExmfGltg1l6RbkYehQPrFiVC1mLzOSGqpzXXLiSZhdixAYLHl6cFn9PsESrOQlD uuQePxlx6WKT6CNjPWwm9/9qRLGqY1osdR/V9myWfxY3yQwCYeZGyn2mAIjHBuY+LJ Zc8c+b0FbNWBCscoD4nnr1zm5qNjOXYqUVUON7qHVIu941wHivfX3J/aBKENZahxmH yjWSA/HZYJy4w== Date: Fri, 14 Feb 2025 21:47:19 +0000 To: Mark Millard From: Yusuf Yaman Cc: freebsd-pkg@freebsd.org Subject: Re: Poudriere insists on using tmpfs for packages listed in TMPFS_BLACKLIST Message-ID: In-Reply-To: References: <2e040ffc-f587-40a2-9ad1-9edf80b1862f@protonmail.com> <78cd193e-e60c-4e65-b75b-3848d582fc9d@protonmail.com> Feedback-ID: 21989843:user:proton X-Pm-Message-ID: 838d2b12131fc71c3227e531e36a45d8431504f6 List-Id: Binary package management and package tools discussion List-Archive: https://lists.freebsd.org/archives/freebsd-pkg List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-pkg@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4YvlyP0pLGz3hMW X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH] Thanks for explaining, I got it now. Have a good one. On 2/15/25 00:35, Mark Millard wrote: > On Feb 14, 2025, at 12:37, Yusuf Yaman wrote: > >> root@hale:~ # pkg info -x poudriere >> poudriere-3.4.2 >> root@hale:~ # uname -a >> FreeBSD hale.home.arpa 14.2-RELEASE-p1 FreeBSD 14.2-RELEASE-p1 GENERIC a= md64 >> >> -------- Forwarded Message -------- Subject: Poudriere insists on using = tmpfs for packages listed in TMPFS_BLACKLIST Date: Fri, 14 Feb 2025 20:07:3= 3 +0300 From: Yusuf Yaman To: FreeBSD Ports ML >> >> Hi, >> >> I am having a problem where Poudriere (even -devel) does insist on using= tmpfs for big packages that i listed in TMPFS_BLACKLIST list in configurat= ion, also TMPFS_BLACKLIST_DIR is set. I am using ZFS. It happens on at leas= t lang/rust and devel/llvm15. > The likes of lang/rust and devel/llvm* use large amounts of file > system space (compared to, say, just 2 GiBytes) it is the larger > areas that are redirected into where you have TMPFS_BLACKLIST_DIR > point, avoiding that also being a tmpfs area. These can be like > 17+ GiByte, 25+ GiByte, or more for just one builder in the > TMPFS_BLACKLIST_DIR area. These can be larger than the RAM that > some might have, making having a huge RAM+SWAP be important > absent the TMPFS_BLACKLIST entry, especially if multiple such > builders happen to run in parallel. (There is also a hook for > avoiding any of a list of packages from building in parallel.) > > TMPFS_BLACKLIST is not intended to eliminate all tmpfs use by a > a builder, just what most likely potentially grows to be > huge/massive compared to normal: wrkdirs > > For reference: > > # grep -r TMPFS_BLACKLIST_TMPDIR /usr/local/share/poudriere/ > /usr/local/share/poudriere/common.sh: case "${TMPFS_BLACKLIST_TMPDIR:+set= }" in > /usr/local/share/poudriere/common.sh: if [ -d "${TMPFS_BLACKLIST_TMPDIR:?= }/wrkdirs" ] && > /usr/local/share/poudriere/common.sh: ! rm -rf "${TMPFS_BLACKLIST_TMP= DIR:?}/wrkdirs/"*; then > /usr/local/share/poudriere/common.sh: "${TMPFS_BLACKLIST_TMPDIR:?}/wr= kdirs"/* || : > /usr/local/share/poudriere/common.sh: rm -rf "${TMPFS_BLACKLIST_TMPDIR:?}= /wrkdirs"/* || > /usr/local/share/poudriere/common.sh: mkdir -p "${TMPFS_BLACKLIST_TMPDIR:= ?}/wrkdirs" > /usr/local/share/poudriere/common.sh: TMPDIR=3D"${TMPFS_BLACKLIST_TMPDIR:= ?}/wrkdirs" \ > > An example of normal/small is ports-mgmt/portmaster ends up using > under 2.5 GiBytes of tmpfs for USE_TMPFS=3Dall . > >> Thanks in advance. >> >> yusuf@hale ~ % mount -v | grep llvm >> yusuf@hale ~ % mount -v | grep tmpfs >> tmpfs on /poudriere/data/.m/142x86-default/ref/.p (tmpfs, local, vnodes:= count 36 ) >> tmpfs on /poudriere/data/.m/142x86-default/ref/wrkdirs (tmpfs, local, vn= odes: count 2 ) >> tmpfs on /poudriere/data/.m/142x86-default/ref/var/db/ports (tmpfs, loca= l, vnodes: count 4 ) >> tmpfs on /poudriere/data/.m/142x86-default/01/.p (tmpfs, local, vnodes: = count 7 ) >> tmpfs on /poudriere/data/.m/142x86-default/01/wrkdirs (tmpfs, local, vno= des: count 135024 ) >> yusuf@hale ~ % > I'll note that the (unused): > > /poudriere/data/.m/142x86-default/01/wrkdirs > > is still present when: > > ${TMPFS_BLACKLIST_TMPDIR:?}/wrkdirs > > is used instead. The: > > /poudriere/data/.m/142x86-default/01/wrkdirs > > is not deleted and recreated for each builder > to use slot 01 , just avoided for TMPFS_BLACKLIST > usage and emptied before starting a new builder > in the slot. > >> Queued: 1 Inspected: 0 Ignored: 0 Built: 0 Failed: 0 Skipped: 0 Fetched:= 0 Remaining: 1 >> ID TOTAL ORIGIN PKGNAME PHASE TIME TM= PFS CPU% MEM% >> [01] 00:04:01 devel/llvm15@default | llvm15-15.0.7_10 build 00:03:03 1.5= 5 GiB 99.9% 5% > 1.55 GiB indicates that the large amount of file system space was > not placed in TMPFS. The 1.55 GiB appears to have been in/under: > /poudriere/data/.m/142x86-default/01/.p instead. > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com >