Date: Mon, 3 Nov 1997 19:04:13 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: jkh@time.cdrom.com (Jordan K. Hubbard) Cc: wosch@cs.tu-berlin.de, smp@freebsd.org Subject: Re: Some SMP timing tests. Message-ID: <199711031904.MAA20435@usr09.primenet.com> In-Reply-To: <1232.878502052@time.cdrom.com> from "Jordan K. Hubbard" at Nov 2, 97 12:20:52 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > The bottleneck is the SUBDIR target which is not parallelized.
> >
> > Try the patch and run `make -j<n> -I share/mk all' in /usr/src.
> > The patch split the SUBDIR variable into 4 variables SUBDIR1,
> > ... SUBDIR4 and runs 4 subdir targets simultaneously.
>
> Oh *igitt*! There has to be a better way than this, I think.
>
> What about having bsd.subdir.mk take an arbitrary ${SUBDIR} and
> just divide it up internally? Having to manually break your subdir
> list up in to 4 categories really loses as a general mechanism,
> though I will indeed try your patch temporarily just to see what
> effects it has on build time. :)
I think this isn't done because make's topological sort for finding
dependencies only works across a single instance of the workspace.
This means that if it spawned an inferior make, you would have to
fix all implicit dependencies that are currently expressed as the
ordering of the SUBDIR lines.
Specifically, you would have to make a dependency for "libc.so"
that you enforced it getting made before anything which linked with
it, and so on, for each and every library.
The "includes" stuff is also expressed as an implied order (really,
the Makefiles in the source tree are in bad shape as regards having
all of the dependencies explicitly listed).
You could handle this the good way, or you could do it the cheesy
way.
The good way would be to be able to specify directory dependencies
in the makefiles including the subdir.mk so that the order was known
to the make descending, and it could parallelize as much as possible
so long as it preserved dependency order. Of course, libc itself
would not have any ordering requirements, and that would fix the
parallel builds there.
The cheesy way would be to make a "PSUBDIRS", and let the directories
there be descended in parallel only after the SUBDIRS targets had been
made, in order. This would also fix the libc builds the way you want.
This approach is cheesy because it doesn't admit that there might be
a directory A build that depends on a directory B build, and
directories C,D and E,F with the same dependency relationship. The
good soloution would descend A,C,E simultaneously. The cheesy one
could only do A,C,E in order, and then descend B,D,F simultaneously.
This is the simplest failure mode for the cheesy soloution -- for
greater depth with dependency, the failure to parallelize increases
exponentially. Again with the libc, you could conceive of wanting
to build the things which are statically linked after libc.a is built
but before libc.so has finished. Likewise, you could decide to seperate
build and link targets, and build all the sources, but only link the
static once once libc.a has been build, and the dynamic ones when
libc.so has been built, etc..
Terry Lambert
terry@lambert.org
---
Any opinions in this posting are my own and not those of my present
or previous employers.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199711031904.MAA20435>