From owner-freebsd-ports@freebsd.org Wed May 12 22:02:54 2021 Return-Path: Delivered-To: freebsd-ports@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5C18864E221; Wed, 12 May 2021 22:02:54 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FgTLQ1xkjz3Nd0; Wed, 12 May 2021 22:02:54 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620856974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nYxXWVjSXYiBZrweERIrvNgS3K4tX4zMxBuFQZ8DvBU=; b=phMJeGI3HwCwpehAfb6ZiI6nWolNLoUHSw509Ix1LO2+F8EpXqy3ffyXZZx7CZegno9RD8 iP8RBbtQxzjo7Aux6W82iYiSh30bjDl/wHs2yeLDihuk+fTDCYdpxVNn6RijA4dyoTw1Ak aDd2oLMKNwOuk8ibn+ms9xpfkpxda5zmSwV+Wr0eAvZK3yibu7wBTQ7kfyFOvns9QnR70Z iJbuRRhqPpVw1Wo6DZW27wJsEkIS+oM6I1DZnm/nvWMolk8+24HgxFAegOT8Ui2SkPmMg/ KVw7ET2NAh7gxTKNuFlihIY5mnWH5e4Z+iB1i4xXjRPtvFdYkDf34TIyaSAVgg== Received: from mail.xzibition.com (unknown [127.0.1.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by freefall.freebsd.org (Postfix) with ESMTPS id 050541A19E; Wed, 12 May 2021 22:02:54 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 1229090F8; Wed, 12 May 2021 22:02:53 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id VA68fHTrP8VK; Wed, 12 May 2021 22:02:50 +0000 (UTC) Subject: Re: ports-mgmt/poudriere-devel, lang/rust (for example), and USE_TMPFS that includes wrkdir (or yes) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.xzibition.com 411AC90EF To: Mark Millard , FreeBSD Toolchain Cc: FreeBSD ports References: <8534A936-B9E6-4BF1-AF26-7F214D359E5C.ref@yahoo.com> <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> From: Bryan Drewery Organization: FreeBSD Message-ID: <1ed96ea6-3886-7a9d-e54b-d47aebfbe5ac@FreeBSD.org> Date: Wed, 12 May 2021 15:02:49 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.10.1 MIME-Version: 1.0 In-Reply-To: <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1620856974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=nYxXWVjSXYiBZrweERIrvNgS3K4tX4zMxBuFQZ8DvBU=; b=OiEpxMtqbVbP41awrA4+7ptJ9Loowv0ztjNYmIZIAHAN1RT8bIepByu4Oy/aVYq2bCMsVG fJZIY4a+mQDIskL+hJS+jhWKI/d9tgqOsWBxmkJxQ0QMcfyHQAu909q/tIWFfN78JPf1vJ FRll7KGw7uTQFXF6He7znPl93DahcSMrgJq85WuNwmAHs03zQgfCSd15XhIVCafR/WEJrf 7Rc163Bsv9fa/GHdqagYKAfi137RDnIs1ymfdBL0LFLfh3ClcI4LnHykHHJjzaN7tpduig W7hBaRDbbBEvanlbVXT3dhyTPVt9E7QYQvMbj9dD62iPsAZvaUnYMWyiaTWBzQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1620856974; a=rsa-sha256; cv=none; b=sqxwfDWaFChmYLnitxK2cRdCieqX/zS9RXCvUFVQo1NT6woSnKHYE5hwsINcE6aZUr8rt9 7tqJ66euu33CaFeY9oM4UBFq2E1yYkXf8jFu9uLnTyJ1CZ3IdQRM4uFXb3Re33fVbb+K8a yTISTK9hGCueR7GVYI4d9aCK2KtGf0BNfkFDQXVEZ0ds0pDISHJzxuz5GRIOKGD5punXEH yD6ji4gvJX1dJx0Cknan7fJBvvWQaO441lxAFbnHpQAWpCiBx3c0b43yCEkTbIq5lK7CmO VRSdp+WPycHPXSU39g9Z0Ql3AXUj+rey41bvYmXc0l+cpHfeIrcjpaSUulHKDQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 May 2021 22:02:54 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v Content-Type: multipart/mixed; boundary="FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9"; protected-headers="v1" From: Bryan Drewery To: Mark Millard , FreeBSD Toolchain Cc: FreeBSD ports Message-ID: <1ed96ea6-3886-7a9d-e54b-d47aebfbe5ac@FreeBSD.org> Subject: Re: ports-mgmt/poudriere-devel, lang/rust (for example), and USE_TMPFS that includes wrkdir (or yes) References: <8534A936-B9E6-4BF1-AF26-7F214D359E5C.ref@yahoo.com> <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> In-Reply-To: <8534A936-B9E6-4BF1-AF26-7F214D359E5C@yahoo.com> --FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable There is no solution at the moment and is a common complaint (about concurrent large builds). I had TMPFS_LIMIT=3D15 and had rust fail on me from that. Quite large... Bryan On 5/10/2021 10:19 PM, Mark Millard wrote: > I've been using USE_TMPFS=3Dyes (so "wrkdir data") on > various systems, both ZFS (recently) and UFS > (generally, even now). Only one system builds rust > (in order for something else to be built), at least > so far. >=20 > An example of the wrkdirs tmpfs use for rust is > (UFS context): >=20 > # df -m | grep tmpfs > Filesystem 1M-blocks Used Avail Capacity Mounted on > . . . > tmpfs 301422 17859 283563 6% /usr/local/poudriere/data/= =2Em/FBSDFSSDjail-default/01/wrkdirs > . . . >=20 > This was near the end but the maximum figure was probably > somewhat higher than the 17 GiByte+ figure above. The > context the example is from is for the only large capacity > build machine that I have access to, an amd64 context. I > have other build contexts as well, but, so far, none have > had to deal with building rust. >=20 > Rust likely would fit the 8 GiByte RAM + 24 GiByte swap > aarch64 build context with USE_TMPFS including wrkdir if > it was the only builder running at the time. But the > existing builds for the context allow 4 builders in > parallel, one per core. [This deals just fine with > llvm10, llvm11, llvm12, and, gcc10 (no bootstrap) being > what happens to build in parallel, even with USE_TMPFS > that includes wrkdir. Rust is just uses more space all > by itself.] >=20 > If I end up with something that requires rust for the > aarch64 builder context, is there a different technique > to deal with the tradeoff other than giving up on > USE_TMPFS spanning wrkdir for all other other > ports/builder-instances as well, presuming the same > media and partitioning (such as total swap space)? >=20 > Imaginary examples could be: >=20 > A) Tell poudriere that lang/rust is to be built by itself > despite the general 4-builder context. >=20 > B) Tell poudriere that USE_TMPFS excludes wrkdir for > lang/rust's specific builder. >=20 > C) . . . (good question) . . . >=20 > So far all I've come up with is explicitly building > lang/rust by itself first, a form of (A): >=20 > # poudriere bulk -jNAME -w lang/rust > # poudriere bulk -jNAME -w -f ~/origins/CA72-origins.txt >=20 > (Hopefully, reliably remembering to do so.) >=20 > Is there any better technique that I've not noticed? >=20 > To some extent here, lang/rust is being used an example > of a more general issue: Other ports could have similar > issues with attempted wrkdir-included USE_TMPFS use. >=20 > Note: If I build using WITH_DEBUG, the one system that > I have access to that can build such a lang/rust with > workdir included in USE_TMPFS shows over 130 GiBytes > in the tmpfs earn the end of the builder's activity. > (This is a amd64 context with 128 GiBytes of RAM and > 192 GiBytes of swapping/paging space.) >=20 > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > ( dsl-only.net went > away in early 2018-Mar) >=20 --=20 Regards, Bryan Drewery --FuQu78JiX0N8Mo9kYY25MX39ecoGpeVn9-- --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEE+Rc8ssOq6npcih8JNddxu25Gl88FAmCcUIkFAwAAAAAACgkQNddxu25Gl8+j JQgAzzE7R9yf5z+A/aA6+2Tr5P0oGkjXPVJ1FGd4+lOW90EVbL/fbAW91OSkh+rAdLAzG/yLm05J J6OAfuIpcdIEWYiU+Bv0DI8qgOqAmK7dSUWFA5hbm5fZZQ58Vp8DVv4NWnFErNtJQj0Blo+wPCre h1pP1E99kjnPawb6BsiCE2GrkmGN/oqBCp6Fjl0KKXVOkQ8Ogxj4nvm4WLZEtFRHoHUTVJMZdr95 kImLdGf7/nmmTsJgAlZMXtwRDKWovSHYfojS9gPHkPCdPol3HvvL+ak8yMhcKcytBqyuc0q/Rkag 8ygQHR4tnoU+iRHTvoD9KOLkHeUbUjkoMsWMeM1ENQ== =IL1t -----END PGP SIGNATURE----- --pboq8Ri8RoeEJsmBenVHuCP4CQ6OcZv7v--