Date: Sun, 6 Jan 2002 22:20:02 +0200 From: Peter Pentchev <roam@ringlet.net> To: arch@FreeBSD.org Cc: audit@FreeBSD.org Subject: Re: make(1) enhancement - an 'environment processor' option Message-ID: <20020106222002.E314@straylight.oblivion.bg> In-Reply-To: <20011225202925.F304@straylight.oblivion.bg>; from roam@ringlet.net on Tue, Dec 25, 2001 at 08:29:25PM %2B0200 References: <20011225202925.F304@straylight.oblivion.bg>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
No feedback on this proposed change to make(1)? :)
If this means that there are no objections, I'm CC'ing this to -audit
to let the folks there take a look at it too. FWIW, I've been running
three machines with the -stable version of this patch since I posted it,
and builds of both the OS and ports all go just fine, using the appropriate
environment settings for the matched directories.
Or was the introduction a bit too long to read and review? :) If so,
one could safely skip the background and go to the explanations and
the patch itself :)
G'luck,
Peter
--
You have, of course, just begun reading the sentence that you have just finished reading.
On Tue, Dec 25, 2001 at 08:29:25PM +0200, Peter Pentchev wrote:
> Hi,
>
> <BACKGROUND STYLE="shameless plug">
> As some of you may have noticed, I maintain a little utility that
> keeps environment variables on a per-directory basis - the sysutils/penv
> port. Basically, what it does is remember what variables you want set
> for a specific directory and then hand them in the environment of any
> program you want to run. This may be quite useful for ports, especially
> those with lots of tweakable knobs, like vpopmail or MySQL, since it won't
> let you miss one of those knobs the next time you rebuild the port.
>
> However, there is one little problem with penv - it only sets
> the environment based on the data for the current directory. If a port
> has dependencies, and some of them also have knobs, you either have to
> set them for the port you want to build, or you have to build
> the dependency separately. For example, I want Ghostscript to build
> with an A4 output format, but then I want to build Ghostscript as part
> of the textproc/docproj build. If I set 'A4=yes' in the print/ghostscript-gnu
> directory and then I do a 'make' in the textproc/docproj directory,
> penv and make(1) will only pick up the textproc/docproj settings, and
> this A4=yes will be lost. If only there was a way to let make(1) read
> per-directory information for every single directory it recursed into..
> </BACKGROUND>
>
> Here is a little patch to make make(1) do just that - invoke an external
> 'environment processor', read its output and modify its own environment
> before doing anything else. This way, I can keep the A4=yes in the penv
> settings for the print/ghostscript-gnu directory and rest assured that
> a docproj build will pick it right up.
>
> This, incidentally, removes the need for the 'penv' invocation to be
> visible at all. Whereas before one had to run 'penv make all install',
> now all one has to do is run 'make all install', and make(1) will run
> penv all by itself.
>
> For those worried about the overhead of a program invocation, there is
> the MAKEENVDIR environment variable, which, if specified, is treated
> as an extended regular expression for the directory names to run
> the environment processor in. Thus, if MAKEENVDIR is set to something
> like, say, "^(/usr|/home/roam/fbsd)/ports/", then the environment
> processor will only be run in the Ports collection and not at all during
> a buildworld.
>
> Comments? Flames? Instant shootdowns? :)
>
> G'luck,
> Peter
>
> --
> .siht ekil ti gnidaer eb d'uoy ,werbeH ni erew ecnetnes siht fI
>
> Index: src/usr.bin/make/main.c
> ===================================================================
> RCS file: /home/ncvs/src/usr.bin/make/main.c,v
> retrieving revision 1.49
> diff -u -r1.49 main.c
> --- src/usr.bin/make/main.c 25 Apr 2001 14:44:41 -0000 1.49
> +++ src/usr.bin/make/main.c 25 Dec 2001 17:37:25 -0000
> @@ -88,6 +88,7 @@
> #include <stdlib.h>
> #include <errno.h>
> #include <fcntl.h>
> +#include <regex.h>
> #include <stdio.h>
> #include <sysexits.h>
> #ifdef __STDC__
> @@ -139,6 +140,8 @@
> static void MainParseArgs __P((int, char **));
> char * chdir_verify_path __P((char *, char *));
> static int ReadMakefile __P((void *, void *));
> +static void ReadEnvProc __P((const char *, const char *,
> + const char *));
> static void usage __P((void));
>
> static char *curdir; /* startup directory */
> @@ -428,6 +431,81 @@
> return 0;
> }
>
> +/*-
> + * ReadEnvProc --
> + * Read the output of an environment processor program and
> + * set or unset environment variables accordingly.
> + *
> + * Results:
> + * none
> + *
> + * Side effects:
> + * Executes the program specified by the envproc argument and
> + * sets or unsets environment variables according to the program output.
> + */
> +
> +static void
> +ReadEnvProc(const char *envproc, const char *envprocdir, const char *curdir) {
> + regex_t regdir;
> + int r;
> + char regerr[128];
> + FILE *envout;
> + char *s, *new;
> + size_t len;
> +
> + /* First check if the env processor needs to run at all here */
> + if ((envprocdir != NULL) && (envprocdir[0] != '\0')) {
> + memset(®dir, 0, sizeof(regdir));
> + r = regcomp(®dir, envprocdir, REG_EXTENDED | REG_NOSUB);
> + if (r != 0) {
> + regerror(r, ®dir, regerr, sizeof(regerr));
> + errx(1, "parsing regexp %s: %s", envprocdir, regerr);
> + }
> +
> + /* If curdir does not match, return w/o executing anything */
> + r = regexec(®dir, curdir, 0, NULL, 0);
> + if (r == REG_NOMATCH)
> + return;
> + if (r != 0) {
> + regerror(r, ®dir, regerr, sizeof(regerr));
> + errx(1, "matching regexp %s against %s: %s",
> + envprocdir, curdir, regerr);
> + }
> + }
> +
> + if ((envout = popen(envproc, "r")) == NULL)
> + err(1, "invoking environment processor '%s'", envproc);
> +
> + while ((s = fgetln(envout, &len)) != NULL) {
> + /* Null-terminate as needed */
> + if (s[len - 1] == '\n') {
> + s[len - 1] = '\0';
> + new = NULL;
> + } else {
> + if ((new = realloc(s, len + 1)) == NULL)
> + err(1, "reading env processor output");
> + memcpy(new, s, len);
> + new[len] = '\0';
> + s = new;
> + }
> +
> + /*
> + * A 'var=value' specification sets the variable, while
> + * a mere variable name unsets it.
> + */
> + if (strchr(s, '=') != NULL)
> + putenv(s);
> + else
> + unsetenv(s);
> +
> + if (new != NULL)
> + free(new);
> + }
> + if (ferror(envout))
> + err(1, "reading env processor output");
> +
> + pclose(envout);
> +}
>
> /*-
> * main --
> @@ -465,6 +543,8 @@
> char *cp = NULL, *start;
> /* avoid faults on read-only strings */
> static char syspath[] = _PATH_DEFSYSPATH;
> + char *envproc = getenv("MAKEENVPROC");
> + char *envprocdir = getenv("MAKEENVDIR");
>
> #if DEFSHELL == 2
> /*
> @@ -497,6 +577,12 @@
>
> if (stat(curdir, &sa) == -1)
> err(2, "%s", curdir);
> +
> + /*
> + * Look for an environment processor as early as possible
> + */
> + if ((envproc != NULL) && (envproc[0] != '\0'))
> + ReadEnvProc(envproc, envprocdir, curdir);
>
> #if defined(__i386__) && defined(__FreeBSD_version) && \
> __FreeBSD_version > 300003
> Index: src/usr.bin/make/make.1
> ===================================================================
> RCS file: /home/ncvs/src/usr.bin/make/make.1,v
> retrieving revision 1.48
> diff -u -r1.48 make.1
> --- src/usr.bin/make/make.1 10 Aug 2001 13:45:27 -0000 1.48
> +++ src/usr.bin/make/make.1 25 Dec 2001 17:40:39 -0000
> @@ -32,7 +32,7 @@
> .\" from: @(#)make.1 8.4 (Berkeley) 3/19/94
> .\" $FreeBSD$
> .\"
> -.Dd March 19, 1994
> +.Dd December 25, 2001
> .Dt MAKE 1
> .Os
> .Sh NAME
> @@ -373,7 +373,9 @@
> .It Environment variables
> Variables defined as part of
> .Nm Ns 's
> -environment.
> +environment or in the output of the
> +.Ev MAKEENVPROC
> +program, if specified (see the ENVIRONMENT PROCESSORS section below).
> .It Global variables
> Variables defined in the makefile or in included makefiles.
> .It Command line variables
> @@ -716,6 +718,47 @@
> .It Cm U
> Converts variable to upper-case letters.
> .El
> +.Sh ENVIRONMENT PROCESSORS
> +External programs, called
> +.Dq environment processors ,
> +may provide
> +additional environment data before any Makefile parsing is done.
> +If the
> +.Ev MAKEENVPROC
> +environment variable is set before
> +.Nm
> +is invoked, it is interpreted as a command to obtain additional environment
> +variables from.
> +.Nm
> +executes the command using the
> +.Xr popen 3
> +function and examines each line of its output.
> +If the line contains a
> +.Sq =
> +character, it is treated as a
> +.Ar variable Ns No = Ns Ar value
> +assignment operator and
> +.Nm
> +sets the respective variable in its environment to the specified value.
> +If the line does not contain a
> +.Sq =
> +character, it is treated as the name of a variable to be removed from the
> +.Nm
> +environment.
> +.Pp
> +If the
> +.Ev MAKEENVDIR
> +environment variable is also set,
> +.Nm
> +treats it as an extended regular expression (see
> +.Xr re_format 7 )
> +and matches the current directory against it.
> +If there is no match, the environment processor is not executed at all.
> +This allows for running the processor only in certain directory trees, e.g.
> +.Pa /usr/ports ,
> +without the burden of the additional command execution when running
> +.Nm
> +in other directories.
> .Sh DIRECTIVES, CONDITIONALS, AND FOR LOOPS
> Directives, conditionals, and for loops reminiscent
> of the C programming language are provided in
> @@ -1174,6 +1217,8 @@
> uses the following environment variables, if they exist:
> .Ev MACHINE ,
> .Ev MAKE ,
> +.Ev MAKEENVDIR ,
> +.Ev MAKEENVPROC ,
> .Ev MAKEFLAGS ,
> .Ev MAKEOBJDIR ,
> and
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-audit" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020106222002.E314>