From owner-svn-src-head@freebsd.org Thu Jul 5 18:26:50 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBE811045720; Thu, 5 Jul 2018 18:26:49 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F9F98EB7C; Thu, 5 Jul 2018 18:26:49 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id E422D86BA; Thu, 5 Jul 2018 18:26:48 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-lf0-f51.google.com with SMTP id n24-v6so7743967lfh.3; Thu, 05 Jul 2018 11:26:48 -0700 (PDT) X-Gm-Message-State: APt69E20YF1x/HPaGRSuYXDCj39Z+lYqei+5LNd6K1QCuuhYRhK/q5wo u4g5TBjO5xKEsOZyC7Corn00N373hVuPgVjlPEM= X-Google-Smtp-Source: AAOMgpdjspnJvG0zSd3zRkpokmyk+IZX2XItWJo+VW2w/poTEn+BF+k8XSFAh2zGmGj9I0YN5rqvJT7hEDjLJajOsUs= X-Received: by 2002:a19:9481:: with SMTP id o1-v6mr5305900lfk.38.1530815207497; Thu, 05 Jul 2018 11:26:47 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:a2e:5742:0:0:0:0:0 with HTTP; Thu, 5 Jul 2018 11:26:26 -0700 (PDT) In-Reply-To: <20180705195810.12487074@thor.intern.walstatt.dynvpn.de> References: <201807051753.w65HrpQm085589@repo.freebsd.org> <20180705195810.12487074@thor.intern.walstatt.dynvpn.de> From: Kyle Evans Date: Thu, 5 Jul 2018 13:26:26 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r336011 - head/usr.sbin/config To: "O. Hartmann" Cc: src-committers , svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2018 18:26:50 -0000 On Thu, Jul 5, 2018 at 12:57 PM, O. Hartmann wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA512 > > Am Thu, 5 Jul 2018 17:53:51 +0000 (UTC) > Kyle Evans schrieb: > >> Author: kevans >> Date: Thu Jul 5 17:53:51 2018 >> New Revision: 336011 >> URL: https://svnweb.freebsd.org/changeset/base/336011 >> >> Log: >> config(8): De-dupe hint/env vars within a single file >> >> r335653 flipped the order in which hints/env files are concatenated to match >> the order in which vars are processed by the kernel. This is the other >> hammer to drop. >> >> Use nv(9) to de-dupe entries within a single `hint` or `env` file, using the >> latest value specified for a key. This leaves some duplicates if a variable >> is specified in multiple hint/env files or via `envvar` in a kernel config, >> but the reversed order of concatenation (from r335653) makes this a >> non-issue as the latest-specified version will be seen first. >> >> This change also silently rewrote hint bits to use the same sanitization >> process that ian@ wrote for r335642. To the kernel, hints and env vars are >> basically the same thing through early boot, then get merged into the >> dynamic environment once kmem becomes available and the dynamic environment >> is created. They should be subjected to the same restrictions. >> >> MFC after: 1 month >> >> Modified: >> head/usr.sbin/config/Makefile >> head/usr.sbin/config/mkmakefile.c >> >> Modified: head/usr.sbin/config/Makefile >> ============================================================================== >> --- head/usr.sbin/config/Makefile Thu Jul 5 17:28:06 2018 (r336010) >> +++ head/usr.sbin/config/Makefile Thu Jul 5 17:53:51 2018 (r336011) >> @@ -18,7 +18,7 @@ CFLAGS+= -I. -I${SRCDIR} >> >> NO_WMISSING_VARIABLE_DECLARATIONS= >> >> -LIBADD= l sbuf >> +LIBADD= l nv sbuf >> >> CLEANFILES+= kernconf.c >> >> >> Modified: head/usr.sbin/config/mkmakefile.c >> ============================================================================== >> --- head/usr.sbin/config/mkmakefile.c Thu Jul 5 17:28:06 2018 (r336010) >> +++ head/usr.sbin/config/mkmakefile.c Thu Jul 5 17:53:51 2018 (r336011) >> @@ -49,6 +49,8 @@ static const char rcsid[] = >> #include >> #include >> #include >> +#include >> +#include >> #include >> #include "y.tab.h" >> #include "config.h" >> @@ -62,6 +64,10 @@ static void do_objs(FILE *); >> static void do_before_depend(FILE *); >> static int opteq(const char *, const char *); >> static void read_files(void); >> +static void sanitize_envline(char *result, const char *src); >> +static void process_into_file(char *line, FILE *ofp); >> +static void process_into_nvlist(char *line, nvlist_t *nvl); >> +static void dump_nvlist(nvlist_t *nvl, FILE *ofp); >> >> static void errout(const char *fmt, ...) >> { >> @@ -180,64 +186,6 @@ makefile(void) >> moveifchanged(path("Makefile.new"), path("Makefile")); >> } >> >> -/* >> - * Build hints.c from the skeleton >> - */ >> -void >> -makehints(void) >> -{ >> - FILE *ifp, *ofp; >> - char line[BUFSIZ]; >> - char *s; >> - struct hint *hint; >> - >> - ofp = fopen(path("hints.c.new"), "w"); >> - if (ofp == NULL) >> - err(1, "%s", path("hints.c.new")); >> - fprintf(ofp, "#include \n"); >> - fprintf(ofp, "#include \n"); >> - fprintf(ofp, "\n"); >> - fprintf(ofp, "char static_hints[] = {\n"); >> - STAILQ_FOREACH(hint, &hints, hint_next) { >> - ifp = fopen(hint->hint_name, "r"); >> - if (ifp == NULL) >> - err(1, "%s", hint->hint_name); >> - while (fgets(line, BUFSIZ, ifp) != NULL) { >> - /* zap trailing CR and/or LF */ >> - while ((s = strrchr(line, '\n')) != NULL) >> - *s = '\0'; >> - while ((s = strrchr(line, '\r')) != NULL) >> - *s = '\0'; >> - /* remove # comments */ >> - s = strchr(line, '#'); >> - if (s) >> - *s = '\0'; >> - /* remove any whitespace and " characters */ >> - s = line; >> - while (*s) { >> - if (*s == ' ' || *s == '\t' || *s == '"') { >> - while (*s) { >> - s[0] = s[1]; >> - s++; >> - } >> - /* start over */ >> - s = line; >> - continue; >> - } >> - s++; >> - } >> - /* anything left? */ >> - if (*line == '\0') >> - continue; >> - fprintf(ofp, "\"%s\\0\"\n", line); >> - } >> - fclose(ifp); >> - } >> - fprintf(ofp, "\"\\0\"\n};\n"); >> - fclose(ofp); >> - moveifchanged(path("hints.c.new"), path("hints.c")); >> -} >> - >> static void >> sanitize_envline(char *result, const char *src) >> { >> @@ -295,14 +243,96 @@ sanitize_envline(char *result, const char *src) >> *dst = 0; >> } >> >> +static void >> +process_into_file(char *line, FILE *ofp) >> +{ >> + char result[BUFSIZ]; >> + >> + sanitize_envline(result, line); >> + /* anything left? */ >> + if (*result == '\0') >> + return; >> + fprintf(ofp, "\"%s\\0\"\n", result); >> +} >> + >> +static void >> +process_into_nvlist(char *line, nvlist_t *nvl) >> +{ >> + char result[BUFSIZ], *s; >> + >> + sanitize_envline(result, line); >> + /* anything left? */ >> + if (*result == '\0') >> + return; >> + s = strchr(result, '='); >> + *s = 0; >> + if (nvlist_exists(nvl, result)) >> + nvlist_free(nvl, result); >> + nvlist_add_string(nvl, result, s + 1); >> +} >> + >> +static void >> +dump_nvlist(nvlist_t *nvl, FILE *ofp) >> +{ >> + const char *name; >> + void *cookie; >> + >> + if (nvl == NULL) >> + return; >> + >> + while (!nvlist_empty(nvl)) { >> + cookie = NULL; >> + name = nvlist_next(nvl, NULL, &cookie); >> + fprintf(ofp, "\"%s=%s\\0\"\n", name, >> + cnvlist_get_string(cookie)); >> + >> + cnvlist_free_string(nvl, cookie); >> + } >> +} >> + >> /* >> + * Build hints.c from the skeleton >> + */ >> +void >> +makehints(void) >> +{ >> + FILE *ifp, *ofp; >> + nvlist_t *nvl; >> + char line[BUFSIZ]; >> + struct hint *hint; >> + >> + ofp = fopen(path("hints.c.new"), "w"); >> + if (ofp == NULL) >> + err(1, "%s", path("hints.c.new")); >> + fprintf(ofp, "#include \n"); >> + fprintf(ofp, "#include \n"); >> + fprintf(ofp, "\n"); >> + fprintf(ofp, "char static_hints[] = {\n"); >> + nvl = nvlist_create(0); >> + STAILQ_FOREACH(hint, &hints, hint_next) { >> + ifp = fopen(hint->hint_name, "r"); >> + if (ifp == NULL) >> + err(1, "%s", hint->hint_name); >> + while (fgets(line, BUFSIZ, ifp) != NULL) >> + process_into_nvlist(line, nvl); >> + dump_nvlist(nvl, ofp); >> + fclose(ifp); >> + } >> + nvlist_destroy(nvl); >> + fprintf(ofp, "\"\\0\"\n};\n"); >> + fclose(ofp); >> + moveifchanged(path("hints.c.new"), path("hints.c")); >> +} >> + >> +/* >> * Build env.c from the skeleton >> */ >> void >> makeenv(void) >> { >> FILE *ifp, *ofp; >> - char line[BUFSIZ], result[BUFSIZ], *linep; >> + nvlist_t *nvl; >> + char line[BUFSIZ]; >> struct envvar *envvar; >> >> ofp = fopen(path("env.c.new"), "w"); >> @@ -312,27 +342,20 @@ makeenv(void) >> fprintf(ofp, "#include \n"); >> fprintf(ofp, "\n"); >> fprintf(ofp, "char static_env[] = {\n"); >> + nvl = nvlist_create(0); >> STAILQ_FOREACH(envvar, &envvars, envvar_next) { >> if (envvar->env_is_file) { >> ifp = fopen(envvar->env_str, "r"); >> if (ifp == NULL) >> err(1, "%s", envvar->env_str); >> - while (fgets(line, BUFSIZ, ifp) != NULL) { >> - sanitize_envline(result, line); >> - /* anything left? */ >> - if (*result == '\0') >> - continue; >> - fprintf(ofp, "\"%s\\0\"\n", result); >> - } >> + while (fgets(line, BUFSIZ, ifp) != NULL) >> + process_into_nvlist(line, nvl); >> + dump_nvlist(nvl, ofp); >> fclose(ifp); >> - } else { >> - linep = envvar->env_str; >> - sanitize_envline(result, linep); >> - if (*result == '\0') >> - continue; >> - fprintf(ofp, "\"%s\\0\"\n", result); >> - } >> + } else >> + process_into_file(envvar->env_str, ofp); >> } >> + nvlist_destroy(nvl); >> fprintf(ofp, "\"\\0\"\n};\n"); >> fclose(ofp); >> moveifchanged(path("env.c.new"), path("env.c")); >> _______________________________________________ >> svn-src-head@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/svn-src-head >> To unsubscribe, send any mail to "svn-src-head-unsubscribe@freebsd.org" > > This commit might be responsible for a broken buildworld: > > [...] > - --- _bootstrap-tools-usr.sbin/config --- > - --- mkmakefile.o --- > /usr/src/usr.sbin/config/mkmakefile.c:289:28: error: too many arguments to function call, > expected single argument 'cookie', have 2 arguments cnvlist_free_string(nvl, cookie); > ~~~~~~~~~~~~~~~~~~~ ^~~~~~ > /usr/include/sys/cnv.h:106:1: note: 'cnvlist_free_string' declared here > void cnvlist_free_string(void *cookie); > ^ > - --- _bootstrap-tools-kerberos5/tools/asn1_compile --- > ===> kerberos5/tools/asn1_compile (obj,all,install) > - --- _bootstrap-tools-usr.sbin/config --- > 1 error generated. > > Hmm... indeed, r335343 removed the nvlist argument. Taking a look- I guess config(8) needs libnv to be built with bootstrap-tools and link against that- I'll poke at it for a bit and revert in short order if I can't figure it out.