From nobody Fri Sep 19 14:18:06 2025 X-Original-To: java@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4cSvk13B3yz684ly for ; Fri, 19 Sep 2025 14:18:17 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4cSvk021LQz3ppw for ; Fri, 19 Sep 2025 14:18:16 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=pDcBNCwQ; dmarc=pass (policy=quarantine) header.from=klop.ws; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 87.255.56.188 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws Received: from smtp-relay-int-backup.realworks.nl (crmpreview4.colo2.realworks.nl [10.2.52.34]) by mailrelayint1.colo2.realworks.nl (Postfix) with ESMTP id 4cSvjr4jp8zSD; Fri, 19 Sep 2025 16:18:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1758291488; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3u1YIK8RK5d9Q051aLl+YgTmGDXivoW26f2bDjgilsU=; b=pDcBNCwQ519skFJB19MSVRm0B1Xylh8tUjgJjntCGID1S+PxMknc6Ack9ZkYYXpEGUEEJ9 W+38R4dgbFkloSt1kATJ3PGyRAXAMGr++l6NkS7HHQr1orThIqJMYjoF13e0/Ac9iKEyEn pbKLQCcKAyNeRxiVfLJ34JytY4mDMzVE8aLVlZSlzfzvPl8MOdORw9Db7bQIzgflyNX8p7 3b5R1fdtFQr90EjvJIJmGU0rYdB+xi5CfSJWpm9kx4Bm0q/JgS4f7fGDkrLU7ZwZQEsBmM mkANMKzg5zPQlCs0yCPfiVCGEDdns5meB8MZhxY3ObJpdG2xQqeV6t58IcGccg== Received: from crmpreview4.colo2.realworks.nl (localhost [127.0.0.1]) by crmpreview4.colo2.realworks.nl (Postfix) with ESMTP id 1B3D91C0568; Fri, 19 Sep 2025 16:18:06 +0200 (CEST) Date: Fri, 19 Sep 2025 16:18:06 +0200 (CEST) From: Ronald Klop To: Ronald Klop Cc: Harald Eilertsen , java@freebsd.org Message-ID: <1330499598.9375.1758291486798@localhost> In-Reply-To: <883854905.286987.1758060251528@localhost> References: <1221594030.658.1756976173189@localhost> <1330428831.1750.1756980558066@localhost> <2053927433.1876.1757575928041@localhost> <387357483.2815.1757924099539@localhost> <1457853127.243778.1758058293681@localhost> <883854905.286987.1758060251528@localhost> Subject: Re: (solved, I think) Re: openjdk build failures on "nm" List-Id: Porting Java to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-java List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-java@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9374_626992353.1758291486716" X-Mailer: Realworks (766.91) X-Originating-Host: from (83-81-212-149.cable.dynamic.v4.ziggo.nl [83.81.212.149]) by crmpreview4.colo2.realworks.nl [10.2.52.34] with HTTP; Fri, 19 Sep 2025 16:18:06 +0200 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:143.0) Gecko/20100101 Firefox/143.0 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; R_SPF_ALLOW(-0.20)[+ip4:87.255.56.128/26]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[java@freebsd.org]; HAS_X_PRIO_THREE(0.00)[3]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[klop.ws:+] X-Rspamd-Queue-Id: 4cSvk021LQz3ppw ------=_Part_9374_626992353.1758291486716 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Van: Ronald Klop Datum: woensdag, 17 september 2025 00:04 Aan: Ronald Klop CC: Harald Eilertsen , java@freebsd.org Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm" > > > Van: Ronald Klop > Datum: dinsdag, 16 september 2025 23:31 > Aan: Harald Eilertsen > CC: java@freebsd.org > Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm" >> >> >> Van: Harald Eilertsen >> Datum: dinsdag, 16 september 2025 13:33 >> Aan: Ronald Klop >> CC: java@freebsd.org >> Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm" >>> >>> Hi Ronald, >>> >>> On Mon, Sep 15, 2025 at 10:14:59AM +0200, Ronald Klop wrote: >>> > The current construction in openjdk17 handles MAKE_JOBS_NUMBER >>> > properly. And with that also the MAKE_JOBS_UNSAFE variable. After more >>> > testing we can use this blueprint in other openjdk ports. >>> > Also in openjdk23 and -24. They don't have flapping builds but can >>> > improve handling of MAKE_JOBS_NUMBER. >>> >>> Nice finds! I don't really have a strong opinion either way, but think >>> it should probably be selectable whether to use (and how many) parallel >>> jobs to run. I'm curious if there has been problems with this number of >>> parallel jobs for other platforms as well. I'll try to ask around a bit. >>> >>> The OpenJDK build process does, as you say, determine how many jobs to >>> spawn on it's own, based on available CPU cores and memory. Having the >>> port Makefile trying to parallelize that further would get in the way of >>> this, so we set MAKE_JOBS_UNSAFE to prevent it. As I understand it, this >>> should not prevent the OpenJDK build system to spawn parallel jobs. >>> >>> In addition to setting the --with-jobs config parameter, it's possible >>> to tune this with the --with-num-cores and --with-memory-size to limit >>> the number of jobs to not occupy the whole capacity of the machine. >>> >>> It would be interesting to know if there's any difference if we clamp >>> number of cores it's allowed to use to f.ex. 16 or something. >>> >>> Will you be at EuroBSDCon or the FreeBSD dev summit? Could be >>> interesting to dig into these things a bit there with some of the >>> experts on the port system available too. >>> >>> Take care! >>> Harald >>> >>> >>> >> >> >> Hi Harald, >> >> Thanks for the tip. >> I just found the documentation of --with-num-cores in doc/building.md. I think that is what we need. It makes it possible to set an upper limit on the number of parallel jobs and lets configure still calculate the balance with the amount of memory available. >> >> I'm curious what the pkg build cluster does tomorrow morning. This morning the relevant pkg builds (like 142amd64-default) crashed before any pkg was build. >> >> Unfortunately I'm not at EuroBSDCon this year. I have been there in previous editions. Lots of fun >> >> Regards, >> Ronald. >> > > > BTW, in the future we can add something like 'LOG=info,cmdlines' to MAKE_ENV in the ports Makefile. This displays more information during the build which might show us what is going wrong. > > But, let's first get a stable build on the short term. > > Regards, > Ronald. > A lot of what we are discovering here was already in java/openjdk8. Somehow the --with-jobs=${MAKE_JOBS_NUMBER} configure parameter was lost in java/openjdk11. History repeats itself? :-) I'm starting to apply the changes done to openjdk17 to openjdk11 and -18 now. Will commit after build testing tonight/this weekend. Regards, Ronald. ------=_Part_9374_626992353.1758291486716 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: woensdag, 17 september 2025 00:04
Aan: Ronald Klop <ronald-lists@klop.ws>
CC: Harald Eilertsen <haraldei@anduin.net>, java@freebsd.org
Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm"

 

Van: Ronald Klop <ronald-lists@klop.ws>
Datum: dinsdag, 16 september 2025 23:31
Aan: Harald Eilertsen <haraldei@anduin.net>
CC: java@freebsd.org
Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm"

 

Van: Harald Eilertsen <haraldei@anduin.net>
Datum: dinsdag, 16 september 2025 13:33
Aan: Ronald Klop <ronald-lists@klop.ws>
CC: java@freebsd.org
Onderwerp: Re: (solved, I think) Re: openjdk build failures on "nm"

Hi Ronald,

On Mon, Sep 15, 2025 at 10:14:59AM +0200, Ronald Klop wrote:
> The current construction in openjdk17 handles MAKE_JOBS_NUMBER
> properly. And with that also the MAKE_JOBS_UNSAFE variable. After more
> testing we can use this blueprint in other openjdk ports.
> Also in openjdk23 and -24. They don't have flapping builds but can
> improve handling of MAKE_JOBS_NUMBER.

Nice finds! I don't really have a strong opinion either way, but think
it should probably be selectable whether to use (and how many) parallel
jobs to run. I'm curious if there has been problems with this number of
parallel jobs for other platforms as well. I'll try to ask around a bit.

The OpenJDK build process does, as you say, determine how many jobs to
spawn on it's own, based on available CPU cores and memory. Having the
port Makefile trying to parallelize that further would get in the way of
this, so we set MAKE_JOBS_UNSAFE to prevent it. As I understand it, this
should not prevent the OpenJDK build system to spawn parallel jobs.

In addition to setting the --with-jobs config parameter, it's possible
to tune this with the --with-num-cores and --with-memory-size to limit
the number of jobs to not occupy the whole capacity of the machine.

It would be interesting to know if there's any difference if we clamp
number of cores it's allowed to use to f.ex. 16 or something.

Will you be at EuroBSDCon or the FreeBSD dev summit? Could be
interesting to dig into these things a bit there with some of the
experts on the port system available too.

Take care!
Harald


Hi Harald,

Thanks for the tip.
I just found the documentation of --with-num-cores in doc/building.md. I think that is what we need. It makes it possible to set an upper limit on the number of parallel jobs and lets configure still calculate the balance with the amount of memory available.

I'm curious what the pkg build cluster does tomorrow morning. This morning the relevant pkg builds (like 142amd64-default) crashed before any pkg was build.

Unfortunately I'm not at EuroBSDCon this year. I have been there in previous editions. Lots of fun

Regards,
Ronald.
 


BTW, in the future we can add something like 'LOG=info,cmdlines' to MAKE_ENV in the ports Makefile. This displays more information during the build which might show us what is going wrong.

But, let's first get a stable build on the short term.

Regards,
Ronald.
 



A lot of what we are discovering here was already in java/openjdk8. Somehow the --with-jobs=${MAKE_JOBS_NUMBER} configure parameter was lost in java/openjdk11.

History repeats itself? :-)

I'm starting to apply the changes done to openjdk17 to openjdk11 and -18 now. Will commit after build testing tonight/this weekend.

Regards,
Ronald.
  ------=_Part_9374_626992353.1758291486716--