Date: Sat, 9 Dec 2017 15:15:52 -0800 From: Mark Millard <markmi@dsl-only.net> To: Bryan Drewery <bdrewery@FreeBSD.org>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: parallel_run start_builder and clonefs for UFS in ports-mgmt/poudriere-devel: avoiding elapsed-time spent in the getblk wait channel, sleep a little Message-ID: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net>
next in thread | raw e-mail | index | archive | help
I've been experimenting with using devel/poudriere-devel
in the contexts:
rpi2, bpi-m3 (Cortex-A7's, 4 core effective, 1 GiByte and 2 GiByte RAM)
rpi3, Pine64+ 2GB (Cortex-A53's, 4 core, 1 GiByte and 2 GiByte RAM)
powerpc64 (32-bit) powerpc (so-called Quad-Core PowerMacs, 16GiBYte and =
2 GiByte effective RAM)
amd64 (Ryzen Threadripper 16-core/32-thread-total, 128 GiByte RAM)
As part of this even the large configurations are
currently set up for UFS so that aspect is uniform.
One thing that I've noted (using the amd64 context
as an example) is a non-linear relationship between
builders and starting/cloning time:
[00:00:26] Building 47 packages using 4 builders
[00:00:26] Starting/Cloning builders
[00:00:32] Hit CTRL+t at any time to see build progress and stats
[00:00:22] Building 49 packages using 14 builders
[00:00:22] Starting/Cloning builders
[00:05:07] Hit CTRL+t at any time to see build progress and stats
[00:00:33] Building 38 packages using 29 builders
[00:00:33] Starting/Cloning builders
[00:17:22] Hit CTRL+t at any time to see build progress and stats
Monitoring the longer running ones I see lots of:
getblk=20
(Waiting to access file data that is being accessed by another thread)
So I tried a crude form of staggering the cloning
by adding a 1 second sleep in start_builders
( /usr/local/share/poudriere/common.sh ):
start_builders() {
local arch=3D$(injail uname -p)
msg "Starting/Cloning builders"
bset status "starting_jobs:"
run_hook start_builders start
bset builders "${JOBS}"
bset status "starting_builders:"
parallel_start
for j in ${JOBS}; do
parallel_run start_builder ${j} ${arch}
sleep 1
done
parallel_stop
run_hook start_builders stop
}
The result for 14 was:
[00:00:14] Building 23 packages using 14 builders
[00:00:14] Starting/Cloning builders
[00:00:53] Hit CTRL+t at any time to see build progress and stats
For 23 (I no longer had more to build) the result was:
[00:00:12] Building 23 packages using 23 builders
[00:00:12] Starting/Cloning builders
[00:04:49] Hit CTRL+t at any time to see build progress and stats
Trying 5 seconds instead of 1 second for the
sleep time:
The result for 14 was:
[00:00:12] Building 23 packages using 14 builders
[00:00:12] Starting/Cloning builders
[00:01:25] Hit CTRL+t at any time to see build progress and stats
For 23 the result was:
[00:00:12] Building 23 packages using 23 builders
[00:00:12] Starting/Cloning builders
[00:02:11] Hit CTRL+t at any time to see build progress and stats
It appears that a few second sleep is of significant
help for having lots of builders.
=3D=3D=3D
Mark Millard
markmi at dsl-only.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2F45D0AB-79E8-44B8-9B3C-25B0FF012298>