Date: Tue, 29 May 2007 18:32:05 +1000 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Michel Talon <talon@lpthe.jussieu.fr> Cc: ports@freebsd.org, freebsd-hackers@freebsd.org Subject: Re: Looking for speed increases in "make index" Message-ID: <20070529083205.GN1161@turion.vk2pj.dyndns.org> In-Reply-To: <20070527221528.GA19603@lpthe.jussieu.fr> References: <20070527221528.GA19603@lpthe.jussieu.fr>
next in thread | previous in thread | raw e-mail | index | archive | help
--fWddYNRDgTk9wQGZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2007-May-28 00:15:28 +0200, Michel Talon <talon@lpthe.jussieu.fr> wrote: >Of course a lot of people have thinked about it, and quickly realized >that it was not going to work. In the bsd.ports.mk, evaluation of one >variable may be dependent on some conditional, which may itself be >dependant on some other variable, appearing as some target. This >constantly happens in bsd.ports.mk. =2E.. >To gain some performance, a first idea would be to simplify >bsd.ports.mk. I am convinced that a substantial part of the 4000 lines >are historical crap which serve no useful purpose. There are tons of >variables who have probably purely anecdotical interest. There are >targets which could be happily suppressed. These two paragraphs are contradictory. If bsd.ports.mk really is 4000 lines of historical crap and full full of unused variables than an "evaluate on demand" approach would be a big win. >A second idea would be to multithread make, since modern machines will >have a lot of cores. Not yet. Dual-core machines are still relatively new. Of the seven hosts that I regularly use, only one is dual core. Threading adds significant overheads and it's not clear how it would help in this specific instance. >Anyways, one of the things which cannot be a big factor is reading >bsd.ports.mk from hard disk. Reading 210KB from a modern disk is quite fast. > It is certainly cached in memory when you >run make index. Which is the specific case being discussed in this thread. > On the other hand it is so big that it probably doesn't >fit in cache, or probably only fits in caches of machines generously >endowed. It may not stay in L1 cache but unless your system is extremely memory stressed, it should stay in RAM. I'd suggest that if bsd.ports.mk is not cached then you have more serious problems to address before you worry about the performance of make(1). >(i am thinking Pentium 4 versus Core 2 Duo) is striking. It is less >than 10 minutes on the big cache machine versus 30 minutes on the small >cache one. If the cache effect is indeed dominant, then reducing the >size of bsd.ports.mk by all possible means would be very beneficial. This is likely to be more a combination of the crippled pentium 4 core and a largish memory footprint for make than anything to do with bsd.ports.mk itself. >By the way an alternative system would be to use something other than >make to do the job. This is a possibility but it needs to be something that can be part of the base system. This means you are limited to C/C++, awk, sh and make. In particular, python, ruby, perl and similar scripting languages are out. Whilst make seems the obvious choice for the ports infrastructure, in reality, the infrastructure does not really need or use the sort of implicit dependency tracking and target transformations that make excels at. Of course, any alternative to make needs to provide a language for defining dependencies that is equally powerful and easy to use. --=20 Peter Jeremy --fWddYNRDgTk9wQGZ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGW+UF/opHv/APuIcRAhGyAKC62+7Ap+18egbYrs2OLvpollDxkgCeI8ds daDjRlZq0UdaFn8k1M7fE3c= =2/hK -----END PGP SIGNATURE----- --fWddYNRDgTk9wQGZ--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070529083205.GN1161>