Date: Wed, 5 Aug 2026 12:05:04 -0700 From: "Edward Sanford Sutton, III" <mirror176@hotmail.com> To: questions@freebsd.org Subject: Re: Chromium availability. Message-ID: <SA1PR11MB8811826D04B453FC7838A84DE6D32@SA1PR11MB8811.namprd11.prod.outlook.com> In-Reply-To: <564e15e8-cca9-4c4b-818b-e19319fcedf0@palaceofretention.ca> References: <4cc52e97-de21-47e2-baac-275d12aed1b5@palaceofretention.ca> <418ae9091e90abaf0f9ea6dbf70b72f45068000f.camel@telaman.net.au> <ccdb2283-7620-454b-bea7-57282816de7e@palaceofretention.ca> <20260805092829.GA3425@jmk1.org> <564e15e8-cca9-4c4b-818b-e19319fcedf0@palaceofretention.ca>
index | next in thread | previous in thread | raw e-mail
On 8/5/26 06:08, Mark G. wrote:
> On 8/5/26 02:28, Johannes-Maria Kaltenbach wrote:
>> Hello,
>>
>> just for your information:
>>
>> On Tue, Aug 04, 2026 at 07:59:17PM -0700, Mark G. wrote:
>>> On 8/4/26 19:35, David wrote:
>>>> On Tue, 2026-08-04 at 19:31 -0700, Mark G. wrote:
>>>>> On 8/4/26 19:26, David wrote:
>>>>>> On Tue, 2026-08-04 at 18:56 -0700, Mark G. wrote:
>>>>>>> On 8/4/26 18:51, David wrote:
>> ...
>>> I am running 14.4-RELEASE-p8, on another system and see
>>> that chromium is missing from "latest" and "quarterly"
>>> (I am just showing quarterly below, switching to latest
>>> gives the same results):
>>
>> same version here.
>> On 14 I was never able to install chromium; it always ended
>> in a timeout (after 24 hours).
>> Perhaps that's why it isn't in the repository?
>>
>> from the poudriere log:
>>
>> [00:01:08] [01] [00:00:00] Building www/chromium | chromium-148.0.7778.96
>> [00:01:08] [01] [00:00:00] Status www/chromium |
>> chromium-148.0.7778.96: check-sanity
>> [00:01:18] [01] [00:00:10] Status www/chromium |
>> chromium-148.0.7778.96: pkg-depends
>> [00:01:19] [01] [00:00:11] Status www/chromium |
>> chromium-148.0.7778.96: fetch-depends
>> [00:01:19] [01] [00:00:11] Status www/chromium |
>> chromium-148.0.7778.96: fetch
>> [00:01:55] [01] [00:00:47] Status www/chromium |
>> chromium-148.0.7778.96: checksum
>> [00:02:00] [01] [00:00:52] Status www/chromium |
>> chromium-148.0.7778.96: extract-depends
>> [00:02:00] [01] [00:00:52] Status www/chromium |
>> chromium-148.0.7778.96: extract
>> [00:03:00] [01] [00:01:52] Status www/chromium |
>> chromium-148.0.7778.96: patch-depends
>> [00:03:01] [01] [00:01:53] Status www/chromium |
>> chromium-148.0.7778.96: patch
>> [00:03:03] [01] [00:01:55] Status www/chromium |
>> chromium-148.0.7778.96: build-depends
>> [00:06:12] [01] [00:05:04] Status www/chromium |
>> chromium-148.0.7778.96: lib-depends
>> [00:06:53] [01] [00:05:45] Status www/chromium |
>> chromium-148.0.7778.96: configure
>> [00:07:27] [01] [00:06:19] Status www/chromium |
>> chromium-148.0.7778.96: build
>> [1D:00:07:42] [01] [1D:00:06:34] Status www/chromium |
>> chromium-148.0.7778.96: timeout
>> [1D:00:44:46] [01] [1D:00:43:38] Finished www/chromium |
>> chromium-148.0.7778.96: Failed: build/timeout
>>
>>
>> I don't depend on chromium, so I ignored it.
>> Is there a built-in timeout in poudriere?
>>
>
>
> There are indeed a couple of knobs that need setting in:
>
> /usr/local/etc/poudriere.conf
>
> In no particular order (I had some file limit fails, so bumped this up):
>
> # How many file descriptors to limit each jail process to (default: 1024)
> # This can also be set per PKGBASE, such as MAX_FILES_RStudio=2048.
> # Package names with hyphens (-) should be replaced with underscores (_).
> MAX_FILES=4096
>
>
> The one you're looking for (I set to 10 days, and chromium finishes):
>
> # This defines the max time (in seconds) that a command may run for a build
> # before it is killed for taking too long. Default: 86400
> MAX_EXECUTION_TIME=864000
>
>
> I also lowered PARALLEL_JOBS from the max (32 in my case) to less,
> so I would have CPUs available for other things than the builder.
>
> # parallel build support.
> #
> # By default poudriere uses hw.ncpu to determine the number of builders.
> # You can override this default by changing PARALLEL_JOBS here, or
> # by specifying the -J flag to bulk/testport.
> #
> # Example to define PARALLEL_JOBS to one single job
> PARALLEL_JOBS=28
>
>
> I also disabled the use of TMPFS (and some failures stopped):
>
> # Use tmpfs(5)
> # This can be a space-separated list of options:
> # wrkdir - Use tmpfs(5) for port building WRKDIRPREFIX
> # data - Use tmpfs(5) for poudriere cache/temp build data
> # localbase - Use tmpfs(5) for LOCALBASE (installing ports for
> packaging/testing)
> # all - Run the entire build in memory, including builder jails.
> # yes - Enables tmpfs(5) for wrkdir and data
> # no - Disable use of tmpfs(5)
> # EXAMPLE: USE_TMPFS="wrkdir data"
>
> # Disable 20240916 see if build problems go away
> USE_TMPFS=no
Consider adjusting USE_TMPFS if your builds don't fit in memory.
Going to swap slows down the build but it may still be faster than doing
filesystem partially or fully on disk. Reducing PARALLEL_JOBS also helps
as multiple ports being built = multiple extracted directories which
adds up to using RAM faster than some people will expect.
Once you have decreased PARALLEL_JOBS, you have decreased how many
work directories are simultaneously extracted which with TMPFS can have
a very large impact on the needed RAM as you get into larger builds. I
find mathematical factors of the CPU count and set PARALLEL_JOBS to one
of them and /usr/local/etc/poudriere.d/make.conf:MAKE_JOBS to the other
to try to create an approximate limit of how many cores are used to
equal those available. I will decrease it for better system
responsiveness and increase it to try to guarantee full core saturation.
I usually favor fewer PARALLEL and more MAKE + keep TMPFS set to get a
more efficient run; seeing that rust requires >32GB of RAM and I only
have 32GB I should make exceptions but for now I just let it force some
swap for a bit which is still nothing like the havoc that occurs with
swap from running Firefox for a while. Fewer PARALLEL_JOBS + more
MAKE_JOBS means each port will finish faster and in the case of
something like chromium it takes a very long time on modern processors
when you give it the default of MAKE_JOBS=1 since you run only one,
generally single threaded, command at a time. Unless all other
PARALLEL_JOBS were active throughout the time you were building
Chromium, you likely just increased package building throughput.
Some ports do not properly respect MAKE_JOBS and will exceed the
number, usually going up to a value of MAKE_JOBS*MAKE_JOBS. The problem
seems to be a result of mixing build systems and having the outer build
system + inner build system both launch MAKE_JOBS number of processes.
The reverse is also going to happen as there are times where a port does
not have as many tasks that can be worked on as MAKE_JOBS is set to and
I assume there are still a few that consider multiple jobs as unsafe
(which poudriere doesn't account for).
Once you apply MAKE_JOBS, you can likely significantly reduce
per-build timeouts. Applying the PARALLEL_JOBS+MAKE_JOBS adjustment to
the official build servers would likely help if nothing else has changed
as I thought I recall seeing amd64 reach at least 98% of swap space use
reported during a build.
Repeat runs that failed may complete the next time if there are less
parallel jobs competing for resources and if you use ccache then the
previously compiled results will be read much faster so without
adjusting the timeout it will get much further along if it doesn't
complete. Ports are normally extracted to a version dependent path so
upgrades of the port's upstream version number will often invalidate
ccache results on that next run. I'd assume ccache settings should be
altered to disregard the path of input files as long as debug is off and
won't have any symbols packaged as subpackages (whenever that becomes a
thing).
For a change that port maintainers are directly responsible for, the
workspaces are further unnecessarily bloated by most ports extracting
everything from their downloads when some things can be filtered out.
When we use other ports instead of bundled copies of dependencies then
the bundled copies don't need to be extracted. Conditional extract
excludes like skipping NLS language files when NLS is off saves a little
bit. Some ports extract some unused files like windows and macos
specific files we don't touch in the build process and text documents
that aren't installed nor used in the build.
A few port maintainers have recognized this and added `rm` steps
sometime after extract and before install (where it happens varies port
to port) but that means we still get wasted I/O, wasted storage space,
and for filesystems like ZFS we get further flagmentation to how data is
laid out and accessed. a more appropriate solution is something like
(adjusted to target directories/files relevant to the port):
#EXTRACT_AFTER_ARGS+= --exclude ${WRKSRC}/*.ods --exclude '*.pdf'\
--exclude '*.doc' --exclude '*.xls*' --exclude '*.xcf'\
--exclude '*.blend' --exclude '*.blend1'\
--exclude '*.yml' --exclude '*.vcproj'
NLS_VARS_OFF= EXTRACT_AFTER_ARGS+="--exclude *.po --exclude *.gmo"
but we also need the following to avoid overwriting default extract args
that will not be set if we define EXTRACT_AFTER_ARGS:
EXTRACT_AFTER_ARGS+="--no-same-owner --no-same-permissions"
I seem to recall exclude syntax was not working as documented; think
it was not working with paths but maybe it was path+wildcard or path
without filename that was not working as I expected. If implementing,
you should test that it really did what you expected.
I think the default should go into an always-appended variable of its
own so it can be directly overridden if needed but will be otherwise be
included by default and not require the port maintainers each re-include
those values since those defaults seemed to be an advised security
adjustment.
The ports framework can be designed to apply things like NLS defined
excludes once NLS becomes involved but should be easy to override if any
conflicts are found.
It would be good if there was a general EXTRACT_EXCLUDES syntax that
maintainers could use which could take a list of directories, files,
wildcard expressions and build the proper syntax of exclude commands to
auto-append to the extractor.
Its been a while since I last tested it but I previously found some
extract steps were slowed down by adding excludes; I would consider that
a bug that should be addressed separately rather than working around it
with an extract+delete.
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?SA1PR11MB8811826D04B453FC7838A84DE6D32>