Date: Wed, 24 Jan 2018 10:07:01 -0800 From: Bryan Drewery <bdrewery@FreeBSD.org> To: Mark Millard <markmi@dsl-only.net>, FreeBSD Toolchain <freebsd-toolchain@freebsd.org> Subject: Re: parallel_run start_builder and clonefs for UFS in ports-mgmt/poudriere-devel: avoiding elapsed-time spent in the getblk wait channel, sleep a little Message-ID: <1882357c-bdfc-31dd-6590-5730e0fbdace@FreeBSD.org> In-Reply-To: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net> References: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA
Content-Type: multipart/mixed; boundary="I5UFMREXw2R7dESlInDRhiWGJKANpKr7L";
protected-headers="v1"
From: Bryan Drewery <bdrewery@FreeBSD.org>
To: Mark Millard <markmi@dsl-only.net>,
FreeBSD Toolchain <freebsd-toolchain@freebsd.org>
Message-ID: <1882357c-bdfc-31dd-6590-5730e0fbdace@FreeBSD.org>
Subject: Re: parallel_run start_builder and clonefs for UFS in
ports-mgmt/poudriere-devel: avoiding elapsed-time spent in the getblk wait
channel, sleep a little
References: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net>
In-Reply-To: <2F45D0AB-79E8-44B8-9B3C-25B0FF012298@dsl-only.net>
--I5UFMREXw2R7dESlInDRhiWGJKANpKr7L
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
On 12/9/2017 3:15 PM, Mark Millard wrote:
> I've been experimenting with using devel/poudriere-devel
> in the contexts:
>=20
> rpi2, bpi-m3 (Cortex-A7's, 4 core effective, 1 GiByte and 2 GiByte RAM)=
> rpi3, Pine64+ 2GB (Cortex-A53's, 4 core, 1 GiByte and 2 GiByte RAM)
> powerpc64 (32-bit) powerpc (so-called Quad-Core PowerMacs, 16GiBYte and=
2 GiByte effective RAM)
> amd64 (Ryzen Threadripper 16-core/32-thread-total, 128 GiByte RAM)
>=20
> As part of this even the large configurations are
> currently set up for UFS so that aspect is uniform.
>=20
> One thing that I've noted (using the amd64 context
> as an example) is a non-linear relationship between
> builders and starting/cloning time:
>=20
>=20
> [00:00:26] Building 47 packages using 4 builders
> [00:00:26] Starting/Cloning builders
> [00:00:32] Hit CTRL+t at any time to see build progress and stats
>=20
> [00:00:22] Building 49 packages using 14 builders
> [00:00:22] Starting/Cloning builders
> [00:05:07] Hit CTRL+t at any time to see build progress and stats
>=20
> [00:00:33] Building 38 packages using 29 builders
> [00:00:33] Starting/Cloning builders
> [00:17:22] Hit CTRL+t at any time to see build progress and stats
>=20
> Monitoring the longer running ones I see lots of:
>=20
> getblk=20
> (Waiting to access file data that is being accessed by another thread)
>=20
> So I tried a crude form of staggering the cloning
> by adding a 1 second sleep in start_builders
> ( /usr/local/share/poudriere/common.sh ):
>=20
> start_builders() {
> local arch=3D$(injail uname -p)
>=20
> msg "Starting/Cloning builders"
> bset status "starting_jobs:"
> run_hook start_builders start
>=20
> bset builders "${JOBS}"
> bset status "starting_builders:"
> parallel_start
> for j in ${JOBS}; do
> parallel_run start_builder ${j} ${arch}
> sleep 1
> done
> parallel_stop
>=20
> run_hook start_builders stop
> }
>=20
> The result for 14 was:
>=20
> [00:00:14] Building 23 packages using 14 builders
> [00:00:14] Starting/Cloning builders
> [00:00:53] Hit CTRL+t at any time to see build progress and stats
>=20
> For 23 (I no longer had more to build) the result was:
>=20
> [00:00:12] Building 23 packages using 23 builders
> [00:00:12] Starting/Cloning builders
> [00:04:49] Hit CTRL+t at any time to see build progress and stats
>=20
> Trying 5 seconds instead of 1 second for the
> sleep time:
>=20
> The result for 14 was:
>=20
> [00:00:12] Building 23 packages using 14 builders
> [00:00:12] Starting/Cloning builders
> [00:01:25] Hit CTRL+t at any time to see build progress and stats
>=20
> For 23 the result was:
>=20
> [00:00:12] Building 23 packages using 23 builders
> [00:00:12] Starting/Cloning builders
> [00:02:11] Hit CTRL+t at any time to see build progress and stats
>=20
> It appears that a few second sleep is of significant
> help for having lots of builders.
>=20
Interesting. I wonder why exactly...
> =3D=3D=3D
> Mark Millard
> markmi at dsl-only.net
>=20
--=20
Regards,
Bryan Drewery
--I5UFMREXw2R7dESlInDRhiWGJKANpKr7L--
--7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBAgAGBQJaaMtFAAoJEDXXcbtuRpfPDeYIAMvJMoKGKsJXhz6y58laToPO
HCWO72p3oGL1ThNN0nD2ZwfKUBTuliBB3tSBYXOivSNVVHV6cWhJpHY6zRFIhzQ0
4YjOjNX5sG0pfnLZCGSO0eukN8E751qjizcNCG5u7pG1JEYYECt6FlGumqy6X1Sa
oB2wwzalxvLcOmGkhlNwBbSaSsGx+/COsHm6UJv65Oc1Yv16xfATn5gD6mqdjJcg
UkKIrgzlnHpOkM+jnR8PU5dNFpIEzanw3tU1PYeZ2jxgQBOre1JZi5WK24bCmKz7
GjznPHwNeJXitR0Rpk/W65kXYooBsgjfmlvOcRsOMMKgyrEVk9l9lO/ZlUyminI=
=7JwS
-----END PGP SIGNATURE-----
--7DRFR2RCYCzgB294hpEE2lepMBKt6WzZA--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1882357c-bdfc-31dd-6590-5730e0fbdace>