Date: Mon, 15 Aug 2022 06:49:20 -0700 From: Mark Millard <marklmi@yahoo.com> To: Nuno Teixeira <eduardo@freebsd.org> Cc: Tomoaki AOKI <junchoon@dec.sakura.ne.jp>, FreeBSD Mailing List <freebsd-ports@freebsd.org> Subject: Re: Resolved: devel/llvm13 build: "ninja: build stopped: subcommand failed" Message-ID: <D0356042-98AA-4B59-BB82-2526E171F680@yahoo.com> In-Reply-To: <CAFDf7U%2Biu4iZM9UHgff819SbDCbsKMJoMg_t2pbyCWSULoD7cA@mail.gmail.com> References: <1D4C14BD-8955-4B86-9C99-3E58D7603122.ref@yahoo.com> <1D4C14BD-8955-4B86-9C99-3E58D7603122@yahoo.com> <CAFDf7UK-pAFXCrZZA9veASaa-wf9HKMdX52fxmcmDgRFiNOF7A@mail.gmail.com> <7CDC63F3-8B68-420E-8012-B1692667E293@yahoo.com> <CAFDf7UJmBNvfVo3SAenPUk1WkFgvpkqoM6=Riv6pwaovuNnAWg@mail.gmail.com> <21FC1F5E-240E-4A8C-A5D2-6B73494026C0@yahoo.com> <CAFDf7ULAPa-YcYezUDCAVP-cMijYLMpLM10z7Xom%2B1siXvn42g@mail.gmail.com> <3E3F8980-8214-45E9-9530-D78243A29D41@yahoo.com> <CAFDf7U%2BDbO0KFKnS4sx4AiXTsS_CPbKFyPTT-B0o1oZS9rWYAg@mail.gmail.com> <DF02B075-78C1-4142-9ACA-12C4CCAFBCF6@yahoo.com> <20220815120616.0cb809413a551717cab3a2eb@dec.sakura.ne.jp> <4D8AB2F3-6A76-4E3A-9C44-348F1352A5D3@yahoo.com> <CAFDf7U%2Biu4iZM9UHgff819SbDCbsKMJoMg_t2pbyCWSULoD7cA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2022-Aug-15, at 06:05, Nuno Teixeira <eduardo@freebsd.org> wrote: > Hi Mark, >=20 > I use TMPFS=3Dno That is the wrong notation. Quoting /usr/local/etc/poudriere.conf.sample = : QUOTE # Use tmpfs(5) # This can be a space-separated list of options: # wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX # data - Use tmpfs(5) for poudriere cache/temp build data # localbase - Use tmpfs(5) for LOCALBASE (installing ports for = packaging/testing) # all - Run the entire build in memory, including builder jails. # yes - Enables tmpfs(5) for wrkdir and data # no - Disable use of tmpfs(5) # EXAMPLE: USE_TMPFS=3D"wrkdir data" USE_TMPFS=3Dyes END QUOTE Note: involving wrkdir uses a lot of tmpfs for some ports. So a correct notation in /usr/local/etc/poudriere.conf would be to replace the USE_TMPFS=3Dyes assignment with: USE_TMPFS=3Dno This might be why something is competing for RAM+SWAP such that you run out: You might have wrkdir involved via an implicit use of yes if you actualyl used the notation TMPFS=3Dno instead. While RAM+SWAP use is high (but not yet out of space), you can use a command like: # df -m | egrep "^(Filesystem|tmpfs) " Filesystem = 1M-blocks Used Avail Capacity Mounted on tmpfs = 1024 35 988 3% = /usr/local/poudriere/data/.m/main-CA72-bulk_a-default/ref/.p tmpfs = 31790 0 31790 0% = /usr/local/poudriere/data/.m/main-CA72-bulk_a-default/ref/var/db/ports tmpfs = 1024 0 1023 0% = /usr/local/poudriere/data/.m/main-CA72-bulk_a-default/01/.p to get a clue how much tmpfs is in use to see if it is large relative to your RAM+SWAP space. > and I don't have WITH_DEBUG set. Good. > I will test with MAKE_JOBS_NUMBER=3Dn in = /usr/local/etc/poudriere.d/make.conf and see what max n I can get it to = compile and observe used ram+swap. >=20 > Other possibility is to increase swap to <=3D60GB but I'd like to = avoid that because I will need to resize freebsd-zfs partition. > What you think about increasing swap? As I mentioned before, you can add SWAP without modifying your existing media by adding new media that has another freebsd-swap partition and can set up to have the old and new freebsd-swap partitions both be active at the same time, thereby increasing the total SWAP. So, for example, a USB3 NVMe SSD could be used to increase the SWAP space available. (You may well have other reasons to not want to do such a thing. But it is technically possible to do.) But, hopefully, you can find and fix why something extra is competing for your RAM+SWAP and so avoid needing more SWAP space. > Thanks >=20 > Mark Millard <marklmi@yahoo.com> escreveu no dia segunda, 15/08/2022 = =C3=A0(s) 04:26: > On 2022-Aug-14, at 20:06, Tomoaki AOKI <junchoon@dec.sakura.ne.jp> = wrote: >=20 > > On Sun, 14 Aug 2022 12:23:03 -0700 > > Mark Millard <marklmi@yahoo.com> wrote: > >=20 > >> On 2022-Aug-14, at 11:07, Nuno Teixeira <eduardo@freebsd.org> = wrote: > >>=20 > >>> I will follow = https://docs.freebsd.org/en/books/handbook/book/#disks-growing and = resize actual swap, but before that I will have to make sure that = backups are ok in case of something goes wrong. > >>>=20 > >>> I've tooked a note about total swap <=3D60GB > >>>=20 > >>> . . . > >>=20 > >>=20 > >> I forgot an important question relative to the resource use > >> for building devel/llvm13 and later: do you need/want the > >> fortran compiler? > >>=20 > >> If not, you can disable the options: FLANG MLIR > >> (building FLAG would require building MLIR) > >>=20 > >> Then the build will be far less memory intensive > >> and take less time. > >>=20 > >> It had slipped my mind that my builds have these 2 options > >> disabled. > >>=20 > >> =3D=3D=3D > >> Mark Millard > >> marklmi at yahoo.com > >=20 > > Is there any possibility that something alike Bug 264949 [1] for = gcc11 > > is happening? > >=20 > > For amd64, option GOLD (LTO support) is implied by default. > > If it causes LTO to be enabled for llvm13 build itself, and lld act = as > > linker of gcc11, small TMPDIR would overflow. > >=20 > > gcc11 required at minimum 5GiB of free space on TMPDIR. > > (4GiB was insufficient.) > >=20 > > [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D264949 > >=20 >=20 > Nuno had a specific problem others have not reported: >=20 > QUOTE > I've tested it but it still fails: > --- > pid 64502 (c++), jid 7, uid 65534, was killed: failed to reclaim = memory > swap_pager: out of swap space > --- > on a Lenovo Legion 5, 16GB RAM and 4GB swap. > END QUOTE >=20 > "was killed: failed to reclaim memory" and "swap_pager: out of swap > space" are not about "small TMPDIR overflow", at least not directly. >=20 > But if the tmpfs is of a form backed by RAM+SWAP, then the tmpfs > use can grow to contribute to causing reclaim problems and/or out of > swap space problems by competing for RAM+SWAP space. >=20 > Nuno reported having disabled such tmpfs use ( via USE_TMPFS=3Dno in > /usr/local/etc/poudriere.conf ). I do not know if that status is > well confirmed or not. I've also no clue if WITH_DEBUG=3D might be > in use. (WITH_DEBUG needs to not be in use unless huge resources are > available.) >=20 > My amd64 tests indicate that something beyond the compiles/links > themselves was using significant RAM+SWAP in Nuno's context. I've > no clue what. =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?D0356042-98AA-4B59-BB82-2526E171F680>