Date: Thu, 17 May 2012 17:49:56 +0200 From: Jilles Tjoelker <jilles@stack.nl> To: Dimitry Andric <dim@FreeBSD.org> Cc: bf1783@gmail.com, freebsd-current@FreeBSD.org, Peter Jeremy <peter@rulingia.com>, "b. f." <bf1783@googlemail.com> Subject: Re: "make delete-old" performance. Message-ID: <20120517154956.GA63826@stack.nl> In-Reply-To: <4FB4EB74.5040009@FreeBSD.org> References: <CAGFTUwPeOGgBqUt2gfeyvYr6W33hu1TQQBekj9_sVpmT7XoT_w@mail.gmail.com> <4FB4EB74.5040009@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, May 17, 2012 at 02:13:40PM +0200, Dimitry Andric wrote: > On 2012-05-17 05:18, b. f. wrote:... > > The slowdown is probably due - at least in part - to two factors: > > - the list of files to be checked for removal has grown substantially, > > because missing entries for old knobs and new entries for new knobs > > have been added; and > > - a new (and slower) method of checking was added in: > > http://svnweb.FreeBSD.org/base?view=revision&revision=220255 > > because the old method broke down with the size of the new lists of files. > Hm, maybe it would have been better to fix make, so it can accept > arbitrarily long lists, without segfaulting? There's such a thing as > malloc(), and I simply don't believe any of those lists could be larger > than a few hundred kilobytes. Alternatively, make could be fixed so that the original code works. Although an invocation like sh -c 'for file in VERY_LONG_LIST; do something; done' will bump into {ARG_MAX}, the shell itself does not have a fixed limitation so longer command lines can be written to a temporary file and passed to sh that way. In some cases (such as with -j), make always uses a temporary file, slowing things down and obscuring ps output. At the cost of needing the temporary file named a bit longer, it is better to pass the pathname to sh rather than feeding the script on standard input: this avoids interfering with terminal input and is potentially faster. The code currently in Makefile.inc1 can probably be sped up by passing the output of the make -V command to something like xargs sh -c 'for file do rm -i "${DESTDIR}/${file}"; done' sh instead of the xargs -n1 | while read file; do ...; done loop. (Note the second "sh" at the end, which serves as a value for $0 so all strings from xargs become positional parameters.) -- Jilles Tjoelker
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120517154956.GA63826>