Date: Wed, 16 Sep 2020 22:36:04 +0000 (UTC) From: Kyle Evans <kevans@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r365816 - in stable/12: sys/sys usr.sbin/crunch/crunchgen Message-ID: <202009162236.08GMa4Rs070582@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: kevans Date: Wed Sep 16 22:36:03 2020 New Revision: 365816 URL: https://svnweb.freebsd.org/changeset/base/365816 Log: MFC outstanding crunchgen work: r350152, r364166, r364174, r364234, r364646-r364647, r365605, r365705 r350152: Remove obsolete RELEASE_CRUNCH Remove documentation of RELEASE_CRUNCH here. It's obsolete and no longer a good example. r364166: Fix crunchgen usage of mkstemp() On Glibc systems mkstemp can only be used once with the same template string since it will be modified in-place and no longer contain any 'X' chars. It is fine to reuse the same file here but we need to be explicit and use open() instead of mkstemp() on the second use. While touching this file also avoid a hardcoded /bin/pwd since that may not work when building on non-FreeBSD systems. r364174: Use env pwd instead of pwd in crunchgen.c In r364166 I changed /bin/pwd to pwd, but pwd can be shell builtin that may not correctly return a real path. To ensure that all symlinks are resolved use `env pwd -P` instead (the -P flag is part of POSIX so should be supported everywhere). r364234: crunchgen: use pwd -P without env The -P flag is required by POSIX so we don't have to care whether pwd is a shell builtin or not. This also allows removing pwd from the list of bootstrap tools since all shells we care about for building have a builtin pwd command. This effectively reverts r364190. r364646: Re-indent crunched_main.c in preparation for D25998 r364647: Correctly determine the real executable in crunched binaries This should fix cases like su setting argv[0] to _su for /bin/sh. Previously cheribsdbox (a crunched tool we use in CheriBSD to reduce the size of our minimal disk images to allow loading them onto FPGAs without waiting forever for the transfer) would complain about _su not being compiled in, but now that we also look at AT_EXECPATH it correctly invokes the sh tool. Note: we use use AT_EXECPATH instead of the KERN_PROC_PATHNAME sysctl to get the crunchgen binary name since it seems like KERN_PROC_PATHNAME just returns the last cached path for a given hardlink. When using `su`, instead of invoking /bin/csh this would invoke the last used hardlink to cheribsdbox. This caused weird test failures when running tests due to `id` being executed instead of `echo`: $ id # id is a hardlink to /bin/cheribsdbox $ su postgres -c 'echo 1' # su is also a hardlink uid=1001(postgres) gid=1001(postgres) groups=1001(postgres) r365605: crunchgen(8): fix crunched application build with WARNS=6 This was revealed by the rescue build with a patch I'm working on to default WARNS=6 everywhere. The issues resolved were: - Missing prototype for _crunched_${ident}_stub in the *_stub.c generated bits - Missing prototype for crunched_main - Incomplete prototype for _crunched_${ident}_stub in the generated parts of crunched_main - Literal strings in the stub table must drop const qualifier, unless we const'ify name - f field in struct stub didn't have a proper prototype Most of these issues are minor formalities and easily addressed. I note that if my patch to eventually raise WARNS for the rescue build lands, we'll need to bump the __FreeBSD_version requirement for bootstrapping crunchgen and wipe out the rescue .OBJDIR if it's stale, which we should be able to detect pretty easily from a couple of the issues that have been fixed here. r365705: __FreeBSD_version bump for r365605 (crunchgen producing WARNS-clean) The change in D26397 will need a __FreeBSD_version to base off of for bootstrapping crunchgen, to avoid avoidable build failures just because the host has an outdated crunchgen. Modified: stable/12/sys/sys/param.h stable/12/usr.sbin/crunch/crunchgen/crunched_main.c stable/12/usr.sbin/crunch/crunchgen/crunchgen.1 stable/12/usr.sbin/crunch/crunchgen/crunchgen.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/sys/param.h ============================================================================== --- stable/12/sys/sys/param.h Wed Sep 16 21:49:19 2020 (r365815) +++ stable/12/sys/sys/param.h Wed Sep 16 22:36:03 2020 (r365816) @@ -60,7 +60,7 @@ * in the range 5 to 9. */ #undef __FreeBSD_version -#define __FreeBSD_version 1202501 /* Master, propagated to newvers */ +#define __FreeBSD_version 1202502 /* Master, propagated to newvers */ /* * __FreeBSD_kernel__ indicates that this system uses the kernel of FreeBSD, Modified: stable/12/usr.sbin/crunch/crunchgen/crunched_main.c ============================================================================== --- stable/12/usr.sbin/crunch/crunchgen/crunched_main.c Wed Sep 16 21:49:19 2020 (r365815) +++ stable/12/usr.sbin/crunch/crunchgen/crunched_main.c Wed Sep 16 22:36:03 2020 (r365816) @@ -23,6 +23,38 @@ * Computer Science Department * University of Maryland at College Park */ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright 2020 Alex Richardson <arichardson@FreeBSD.org> + * + * This software was developed by SRI International and the University of + * Cambridge Computer Laboratory (Department of Computer Science and + * Technology) under DARPA contract HR0011-18-C-0016 ("ECATS"), as part of the + * DARPA SSITH research programme. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + */ /* * crunched_main.c - main program for crunched binaries, it branches to a * particular subprogram based on the value of argv[0]. Also included @@ -35,82 +67,142 @@ #include <sys/cdefs.h> __FBSDID("$FreeBSD$"); +#include <sys/param.h> +#include <sys/auxv.h> +#include <sys/sysctl.h> + +#include <err.h> #include <stdio.h> #include <stdlib.h> #include <string.h> +typedef int crunched_stub_t(int, char **, char **); + struct stub { - char *name; - int (*f)(); + const char *name; + crunched_stub_t *f; }; -extern char *__progname; +extern const char *__progname; extern struct stub entry_points[]; static void crunched_usage(void); +crunched_stub_t crunched_main; + +static struct stub * +find_entry_point(const char *basename) +{ + struct stub *ep = NULL; + + for (ep = entry_points; ep->name != NULL; ep++) + if (!strcmp(basename, ep->name)) + break; + + return (ep); +} + +static const char * +get_basename(const char *exe_path) +{ + const char *slash = strrchr(exe_path, '/'); + return (slash ? slash + 1 : exe_path); +} + int main(int argc, char **argv, char **envp) { - char *slash, *basename; - struct stub *ep; + struct stub *ep = NULL; + const char *basename = NULL; - if(argv[0] == NULL || *argv[0] == '\0') - crunched_usage(); + /* + * Look at __progname first (this will be set if the crunched binary is + * invoked directly). + */ + if (__progname) { + basename = get_basename(__progname); + ep = find_entry_point(basename); + } - slash = strrchr(argv[0], '/'); - basename = slash? slash+1 : argv[0]; + /* + * Otherwise try to find entry point based on argv[0] (this works for + * both symlinks as well as hardlinks). However, it does not work when + * su invokes a crunched shell because it sets argv[0] to _su when + * invoking the shell. In that case we look at AT_EXECPATH as a + * fallback. + */ + if (ep == NULL) { + basename = get_basename(argv[0]); + ep = find_entry_point(basename); + } - for(ep=entry_points; ep->name != NULL; ep++) - if(!strcmp(basename, ep->name)) break; + /* + * If we didn't find the entry point based on __progname or argv[0], + * try AT_EXECPATH to get the actual binary that was executed. + */ + if (ep == NULL) { + char buf[MAXPATHLEN]; + int error = elf_aux_info(AT_EXECPATH, &buf, sizeof(buf)); - if(ep->name) - return ep->f(argc, argv, envp); - else { - fprintf(stderr, "%s: %s not compiled in\n", EXECNAME, basename); - crunched_usage(); - } -} + if (error == 0) { + const char *exe_name = get_basename(buf); + /* + * Keep using argv[0] if AT_EXECPATH is the crunched + * binary so that symlinks to the crunched binary report + * "not compiled in" instead of invoking + * crunched_main(). + */ + if (strcmp(exe_name, EXECNAME) != 0) { + basename = exe_name; + ep = find_entry_point(basename); + } + } else { + warnc(error, "elf_aux_info(AT_EXECPATH) failed"); + } + } + if (basename == NULL || *basename == '\0') + crunched_usage(); + if (ep != NULL) { + return ep->f(argc, argv, envp); + } else { + fprintf(stderr, "%s: %s not compiled in\n", EXECNAME, basename); + crunched_usage(); + } +} + int crunched_main(int argc, char **argv, char **envp) { - char *slash; - struct stub *ep; - int columns, len; + if (argc <= 1) + crunched_usage(); - if(argc <= 1) - crunched_usage(); - - slash = strrchr(argv[1], '/'); - __progname = slash? slash+1 : argv[1]; - - return main(--argc, ++argv, envp); + __progname = get_basename(argv[1]); + return main(--argc, ++argv, envp); } - static void crunched_usage() { - int columns, len; - struct stub *ep; + int columns, len; + struct stub *ep; - fprintf(stderr, "usage: %s <prog> <args> ..., where <prog> is one of:\n", - EXECNAME); - columns = 0; - for(ep=entry_points; ep->name != NULL; ep++) { - len = strlen(ep->name) + 1; - if(columns+len < 80) - columns += len; - else { - fprintf(stderr, "\n"); - columns = len; + fprintf(stderr, + "usage: %s <prog> <args> ..., where <prog> is one of:\n", EXECNAME); + columns = 0; + for (ep = entry_points; ep->name != NULL; ep++) { + len = strlen(ep->name) + 1; + if (columns + len < 80) + columns += len; + else { + fprintf(stderr, "\n"); + columns = len; + } + fprintf(stderr, " %s", ep->name); } - fprintf(stderr, " %s", ep->name); - } - fprintf(stderr, "\n"); - exit(1); + fprintf(stderr, "\n"); + exit(1); } /* end of crunched_main.c */ Modified: stable/12/usr.sbin/crunch/crunchgen/crunchgen.1 ============================================================================== --- stable/12/usr.sbin/crunch/crunchgen/crunchgen.1 Wed Sep 16 21:49:19 2020 (r365815) +++ stable/12/usr.sbin/crunch/crunchgen/crunchgen.1 Wed Sep 16 22:36:03 2020 (r365816) @@ -137,9 +137,7 @@ Set the name of a file to be included at the beginning .Pa Makefile Ns s generated by .Nm . -This is useful to define some make variables such as -.Va RELEASE_CRUNCH -or similar, which might affect the behavior of +This is useful to define some make variables which might affect the behavior of .Xr make 1 and are annoying to pass through environment variables. .It Fl m Ar makefile-name Modified: stable/12/usr.sbin/crunch/crunchgen/crunchgen.c ============================================================================== --- stable/12/usr.sbin/crunch/crunchgen/crunchgen.c Wed Sep 16 21:49:19 2020 (r365815) +++ stable/12/usr.sbin/crunch/crunchgen/crunchgen.c Wed Sep 16 22:36:03 2020 (r365816) @@ -39,10 +39,13 @@ __FBSDID("$FreeBSD$"); #include <ctype.h> #include <err.h> +#include <fcntl.h> #include <paths.h> +#include <stdbool.h> #include <stdio.h> #include <stdlib.h> #include <string.h> +#include <sysexits.h> #include <unistd.h> #define CRUNCH_VERSION "0.2" @@ -91,6 +94,7 @@ prog_t *progs = NULL; char confname[MAXPATHLEN], infilename[MAXPATHLEN]; char outmkname[MAXPATHLEN], outcfname[MAXPATHLEN], execfname[MAXPATHLEN]; char tempfname[MAXPATHLEN], cachename[MAXPATHLEN], curfilename[MAXPATHLEN]; +bool tempfname_initialized = false; char outhdrname[MAXPATHLEN] ; /* user-supplied header for *.mk */ char *objprefix; /* where are the objects ? */ char *path_make; @@ -216,6 +220,7 @@ main(int argc, char **argv) snprintf(cachename, sizeof(cachename), "%s.cache", confname); snprintf(tempfname, sizeof(tempfname), "%s/crunchgen_%sXXXXXX", getenv("TMPDIR") ? getenv("TMPDIR") : _PATH_TMP, confname); + tempfname_initialized = false; parse_conf_file(); if (list_mode) @@ -648,8 +653,7 @@ fillin_program(prog_t *p) /* Determine the actual srcdir (maybe symlinked). */ if (p->srcdir) { - snprintf(line, MAXLINELEN, "cd %s && echo -n `/bin/pwd`", - p->srcdir); + snprintf(line, MAXLINELEN, "cd %s && pwd -P", p->srcdir); f = popen(line,"r"); if (!f) errx(1, "Can't execute: %s\n", line); @@ -721,14 +725,26 @@ fillin_program_objs(prog_t *p, char *path) /* discover the objs from the srcdir Makefile */ - if ((fd = mkstemp(tempfname)) == -1) { - perror(tempfname); - exit(1); + /* + * We reuse the same temporary file name for multiple objects. However, + * some libc implementations (such as glibc) return EINVAL if there + * are no XXXXX characters in the template. This happens after the + * first call to mkstemp since the argument is modified in-place. + * To avoid this error we use open() instead of mkstemp() after the + * call to mkstemp(). + */ + if (tempfname_initialized) { + if ((fd = open(tempfname, O_CREAT | O_EXCL | O_RDWR, 0600)) == -1) { + err(EX_OSERR, "open(%s)", tempfname); + } + } else if ((fd = mkstemp(tempfname)) == -1) { + err(EX_OSERR, "mkstemp(%s)", tempfname); } + tempfname_initialized = true; if ((f = fdopen(fd, "w")) == NULL) { - warn("%s", tempfname); + warn("fdopen(%s)", tempfname); goterror = 1; - return; + goto out; } if (p->objvar) objvar = p->objvar; @@ -763,14 +779,14 @@ fillin_program_objs(prog_t *p, char *path) if ((f = popen(line, "r")) == NULL) { warn("submake pipe"); goterror = 1; - return; + goto out; } while(fgets(line, MAXLINELEN, f)) { if (strncmp(line, "OBJS= ", 6)) { warnx("make error: %s", line); goterror = 1; - continue; + goto out; } cp = line + 6; @@ -793,7 +809,7 @@ fillin_program_objs(prog_t *p, char *path) warnx("make error: make returned %d", rc); goterror = 1; } - +out: unlink(tempfname); } @@ -918,7 +934,9 @@ gen_output_cfile(void) fprintf(outcf, "%s\n", *cp); for (p = progs; p != NULL; p = p->next) - fprintf(outcf, "extern int _crunched_%s_stub();\n", p->ident); + fprintf(outcf, + "extern crunched_stub_t _crunched_%s_stub;\n", + p->ident); fprintf(outcf, "\nstruct stub entry_points[] = {\n"); for (p = progs; p != NULL; p = p->next) { @@ -1106,9 +1124,10 @@ prog_makefile_rules(FILE *outmk, prog_t *p) fprintf(outmk, "%s_stub.c:\n", p->name); fprintf(outmk, "\techo \"" "extern int main(int argc, char **argv, char **envp); " + "int _crunched_%s_stub(int argc, char **argv, char **envp);" "int _crunched_%s_stub(int argc, char **argv, char **envp)" "{return main(argc,argv,envp);}\" >%s_stub.c\n", - p->ident, p->name); + p->ident, p->ident, p->name); fprintf(outmk, "%s.lo: %s_stub.o $(%s_OBJPATHS)", p->name, p->name, p->ident); if (p->libs)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202009162236.08GMa4Rs070582>