From owner-svn-src-head@freebsd.org Thu Aug 20 19:28:20 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 923653C6996; Thu, 20 Aug 2020 19:28:20 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BXZSN3Q3Lz3clN; Thu, 20 Aug 2020 19:28:20 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5987F18E73; Thu, 20 Aug 2020 19:28:20 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 07KJSKXU088882; Thu, 20 Aug 2020 19:28:20 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 07KJSJfP088877; Thu, 20 Aug 2020 19:28:19 GMT (envelope-from markj@FreeBSD.org) Message-Id: <202008201928.07KJSJfP088877@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Thu, 20 Aug 2020 19:28:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r364438 - in head/cddl: contrib/opensolaris/lib/libdtrace/common usr.sbin/dtrace/tests/tools X-SVN-Group: head X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in head/cddl: contrib/opensolaris/lib/libdtrace/common usr.sbin/dtrace/tests/tools X-SVN-Commit-Revision: 364438 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2020 19:28:20 -0000 Author: markj Date: Thu Aug 20 19:28:19 2020 New Revision: 364438 URL: https://svnweb.freebsd.org/changeset/base/364438 Log: Enable creation of static userspace probes in incremental builds. To define USDT probes, dtrace -G makes use of relocations for undefined symbols: the target address is overwritten with NOPs and the location is recorded in the DOF section of the output object file. To avoid link errors, the original relocation is destroyed. However, this means that the same input object file cannot be processed multiple times, as happens during incremental rebuilds. Instead, only set the relocation type to NONE, so that all information required to reconstruct USDT probes is preserved. Reported by: bdrewery MFC after: 3 weeks Sponsored by: The FreeBSD Foundation Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.c head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.h head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c Thu Aug 20 19:27:49 2020 (r364437) +++ head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_link.c Thu Aug 20 19:28:19 2020 (r364438) @@ -773,6 +773,7 @@ dt_symtab_lookup(Elf_Data *data_sym, int start, int en #define DT_OP_RET 0xd65f03c0 #define DT_OP_CALL26 0x94000000 #define DT_OP_JUMP26 0x14000000 +#define DT_REL_NONE R_AACH64_NONE static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, @@ -831,7 +832,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, return (0); } #elif defined(__arm__) -/* XXX */ +#define DT_REL_NONE R_ARM_NONE + static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) @@ -841,7 +843,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, return (-1); } #elif defined(__mips__) -/* XXX */ +#define DT_REL_NONE R_MIPS_NONE + static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) @@ -861,7 +864,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, #define DT_IS_BRANCH(inst) ((inst & 0xfc000000) == 0x48000000) #define DT_IS_BL(inst) (DT_IS_BRANCH(inst) && (inst & 0x01)) -/* XXX */ +#define DT_REL_NONE R_PPC_NONE + static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) @@ -878,7 +882,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, * We only know about some specific relocation types. */ if (GELF_R_TYPE(rela->r_info) != R_PPC_REL24 && - GELF_R_TYPE(rela->r_info) != R_PPC_PLTREL24) + GELF_R_TYPE(rela->r_info) != R_PPC_PLTREL24 && + GELF_R_TYPE(rela->r_info) != R_PPC_NONE) return (-1); /* @@ -932,7 +937,7 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, return (0); } #elif defined(__riscv) -/* XXX */ +#define DT_REL_NONE R_RISCV_NONE static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) @@ -952,6 +957,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, #define DT_OP_XOR_EAX_0 0x33 #define DT_OP_XOR_EAX_1 0xc0 +#define DT_REL_NONE R_386_NONE + static int dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, GElf_Rela *rela, uint32_t *off) @@ -974,7 +981,8 @@ dt_modtext(dtrace_hdl_t *dtp, char *p, int isenabled, * x86 architectures. */ if (GELF_R_TYPE(rela->r_info) != R_386_PC32 && - GELF_R_TYPE(rela->r_info) != R_386_PLT32) + GELF_R_TYPE(rela->r_info) != R_386_PLT32 && + GELF_R_TYPE(rela->r_info) != R_386_NONE) return (-1); /* @@ -1273,6 +1281,11 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e * We take a first pass through all the relocations to * populate our string table and count the number of extra * symbols we'll require. + * + * We also handle the case where the object has already been + * processed, to support incremental rebuilds. Relocations + * of interest are converted to type NONE, but all information + * needed to reconstruct the output DOF is retained. */ strtab = dt_strtab_create(1); nsym = 0; @@ -1280,7 +1293,6 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e istr = data_str->d_size; for (i = 0; i < shdr_rel.sh_size / shdr_rel.sh_entsize; i++) { - if (shdr_rel.sh_type == SHT_RELA) { if (gelf_getrela(data_rel, i, &rela) == NULL) continue; @@ -1345,7 +1357,12 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e objkey, s); if (dt_strtab_index(strtab, p) == -1) { - nsym++; + /* + * Do not add new symbols if this object file + * has already been processed. + */ + if (GELF_R_TYPE(rela.r_info) != DT_REL_NONE) + nsym++; (void) dt_strtab_insert(strtab, p); } @@ -1353,13 +1370,14 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e } /* - * If any probes were found, allocate the additional space for - * the symbol table and string table, copying the old data into - * the new buffers, and marking the buffers as dirty. We inject - * those newly allocated buffers into the libelf data + * If any new probes were found, allocate the additional space + * for the symbol table and string table, copying the old data + * into the new buffers, and marking the buffers as dirty. We + * inject those newly allocated buffers into the libelf data * structures, but are still responsible for freeing them once * we're done with the elf handle. */ + osym = isym; if (nsym > 0) { /* * The first byte of the string table is reserved for @@ -1411,9 +1429,8 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e shdr_sym.sh_size += nsym * symsize; (void) gelf_update_shdr(scn_sym, &shdr_sym); - osym = isym; nsym += isym; - } else { + } else if (dt_strtab_empty(strtab)) { dt_strtab_destroy(strtab); continue; } @@ -1423,7 +1440,6 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e * modifications described above. */ for (i = 0; i < shdr_rel.sh_size / shdr_rel.sh_entsize; i++) { - if (shdr_rel.sh_type == SHT_RELA) { if (gelf_getrela(data_rel, i, &rela) == NULL) continue; @@ -1490,32 +1506,51 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e bind = GELF_ST_BIND(fsym.st_info) == STB_WEAK ? STB_WEAK : STB_GLOBAL; - - /* - * Emit an alias for the symbol. It needs to be - * non-preemptible so that .SUNW_dof relocations - * may be resolved at static link time. Aliases - * of weak symbols are given a non-unique name - * so that they may be merged by the linker. - */ - dsym = fsym; - dsym.st_name = istr; - dsym.st_info = GELF_ST_INFO(bind, STT_FUNC); - dsym.st_other = GELF_ST_VISIBILITY(STV_HIDDEN); - (void) gelf_update_sym(data_sym, isym, &dsym); - r = (char *) data_str->d_buf + istr; s = (char *) data_str->d_buf + fsym.st_name; - if (bind == STB_WEAK) - istr += sprintf(r, dt_weaksymfmt, - dt_symprefix, s); - else - istr += sprintf(r, dt_symfmt, - dt_symprefix, objkey, s); - istr++; - isym++; - assert(isym <= nsym); - } else + if (GELF_R_TYPE(rela.r_info) != DT_REL_NONE) { + /* + * Emit an alias for the symbol. It + * needs to be non-preemptible so that + * .SUNW_dof relocations may be resolved + * at static link time. Aliases of weak + * symbols are given a non-unique name + * so that they may be merged by the + * linker. + */ + dsym = fsym; + dsym.st_name = istr; + dsym.st_info = GELF_ST_INFO(bind, + STT_FUNC); + dsym.st_other = + GELF_ST_VISIBILITY(STV_HIDDEN); + (void) gelf_update_sym(data_sym, isym, + &dsym); + isym++; + assert(isym <= nsym); + + r = (char *) data_str->d_buf + istr; + if (bind == STB_WEAK) { + istr += sprintf(r, + dt_weaksymfmt, dt_symprefix, + s); + } else { + istr += sprintf(r, dt_symfmt, + dt_symprefix, objkey, s); + } + istr++; + } else { + if (bind == STB_WEAK) { + (void) asprintf(&r, + dt_weaksymfmt, dt_symprefix, + s); + } else { + (void) asprintf(&r, dt_symfmt, + dt_symprefix, objkey, s); + } + } + } else { goto err; + } if ((pvp = dt_provider_lookup(dtp, pname)) == NULL) { return (dt_link_error(dtp, elf, fd, bufs, @@ -1544,24 +1579,18 @@ process_obj(dtrace_hdl_t *dtp, const char *obj, int *e } /* - * Our linker doesn't understand the SUNW_IGNORE ndx and - * will try to use this relocation when we build the - * final executable. Since we are done processing this - * relocation, mark it as inexistant and let libelf - * remove it from the file. - * If this wasn't done, we would have garbage added to - * the executable file as the symbol is going to be - * change from UND to ABS. + * We are done with this relocation, but it must be + * preserved in order to support incremental rebuilds. */ if (shdr_rel.sh_type == SHT_RELA) { - rela.r_offset = 0; - rela.r_info = 0; - rela.r_addend = 0; + rela.r_info = + GELF_R_INFO(GELF_R_SYM(rela.r_info), 0); (void) gelf_update_rela(data_rel, i, &rela); } else { GElf_Rel rel; - rel.r_offset = 0; - rel.r_info = 0; + rel.r_offset = rela.r_offset; + rela.r_info = + GELF_R_INFO(GELF_R_SYM(rela.r_info), 0); (void) gelf_update_rel(data_rel, i, &rel); } @@ -1613,19 +1642,6 @@ dtrace_program_link(dtrace_hdl_t *dtp, dtrace_prog_t * char *cmd, tmp; size_t len; int eprobes = 0, ret = 0; - - if (access(file, R_OK) == 0) { - fprintf(stderr, "dtrace: target object (%s) already exists. " - "Please remove the target\ndtrace: object and rebuild all " - "the source objects if you wish to run the DTrace\n" - "dtrace: linking process again\n", file); - /* - * Several build infrastructures run DTrace twice (e.g. - * postgres) and we don't want the build to fail. Return - * 0 here since this isn't really a fatal error. - */ - return (0); - } /* * A NULL program indicates a special use in which we just link Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.c ============================================================================== --- head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.c Thu Aug 20 19:27:49 2020 (r364437) +++ head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.c Thu Aug 20 19:28:19 2020 (r364438) @@ -206,6 +206,13 @@ err: return (-1); } +boolean_t +dt_strtab_empty(dt_strtab_t *sp) +{ + /* Always contains "\0". */ + return (sp->str_nstrs == 1); +} + ssize_t dt_strtab_index(dt_strtab_t *sp, const char *str) { Modified: head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.h ============================================================================== --- head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.h Thu Aug 20 19:27:49 2020 (r364437) +++ head/cddl/contrib/opensolaris/lib/libdtrace/common/dt_strtab.h Thu Aug 20 19:28:19 2020 (r364438) @@ -58,6 +58,7 @@ typedef ssize_t dt_strtab_write_f(const char *, size_t extern dt_strtab_t *dt_strtab_create(size_t); extern void dt_strtab_destroy(dt_strtab_t *); +extern boolean_t dt_strtab_empty(dt_strtab_t *); extern ssize_t dt_strtab_index(dt_strtab_t *, const char *); extern ssize_t dt_strtab_insert(dt_strtab_t *, const char *); extern size_t dt_strtab_size(const dt_strtab_t *); Modified: head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh ============================================================================== --- head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh Thu Aug 20 19:27:49 2020 (r364437) +++ head/cddl/usr.sbin/dtrace/tests/tools/exclude.sh Thu Aug 20 19:28:19 2020 (r364438) @@ -201,10 +201,6 @@ exclude EXFAIL common/usdt/tst.eliminate.ksh # Generated headers include , so _DTRACE_VERSION is always defined. exclude EXFAIL common/usdt/tst.nodtrace.ksh -# The second dtrace -G invocation returns an error with "no probes found," which -# makes sense to me. Not yet sure what the expected behaviour is here. -exclude EXFAIL common/usdt/tst.static2.ksh - # Uses the Solaris-specific ppriv(1). exclude EXFAIL common/usdt/tst.user.ksh