From owner-dev-commits-src-main@freebsd.org Mon Aug 2 13:35:06 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 64D3D674697; Mon, 2 Aug 2021 13:35:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GdfBf1bDRz4RQk; Mon, 2 Aug 2021 13:35:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EEC7D20458; Mon, 2 Aug 2021 13:35:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 172DZ5G9050548; Mon, 2 Aug 2021 13:35:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 172DZ5xV050547; Mon, 2 Aug 2021 13:35:05 GMT (envelope-from git) Date: Mon, 2 Aug 2021 13:35:05 GMT Message-Id: <202108021335.172DZ5xV050547@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alex Richardson Subject: git: 7bc797e3f380 - main - Add build system support for ASAN+UBSAN instrumentation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: arichardson X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7bc797e3f3807660cf98e5b1bd63545cafe820f8 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Aug 2021 13:35:06 -0000 The branch main has been updated by arichardson: URL: https://cgit.FreeBSD.org/src/commit/?id=7bc797e3f3807660cf98e5b1bd63545cafe820f8 commit 7bc797e3f3807660cf98e5b1bd63545cafe820f8 Author: Alex Richardson AuthorDate: 2021-08-02 08:48:21 +0000 Commit: Alex Richardson CommitDate: 2021-08-02 13:33:24 +0000 Add build system support for ASAN+UBSAN instrumentation This adds two new options WITH_ASAN/WITH_UBSAN that can be set to enable instrumentation of all binaries with AddressSanitizer and/or UndefinedBehaviourSanitizer. This current patch is almost sufficient to get a complete buildworld with sanitizer instrumentation but in order to actually build and boot a system it depends on a few more follow-up commits. Reviewed By: brooks, kib, markj Differential Revision: https://reviews.freebsd.org/D31043 --- Makefile.inc1 | 12 ++++++++++++ Makefile.libcompat | 1 + lib/Makefile | 2 ++ lib/csu/Makefile.inc | 3 +++ lib/libclang_rt/Makefile.inc | 2 ++ lib/libgcc_eh/Makefile.inc | 8 ++++++++ lib/libgcc_s/Makefile | 3 +++ libexec/rtld-elf/Makefile | 3 +++ share/mk/bsd.README | 2 +- share/mk/bsd.compat.mk | 1 + share/mk/bsd.lib.mk | 8 ++++++-- share/mk/bsd.opts.mk | 4 +++- share/mk/bsd.prog.mk | 2 ++ share/mk/bsd.progs.mk | 2 +- share/mk/bsd.sanitizer.mk | 43 +++++++++++++++++++++++++++++++++++++++++++ share/mk/sys.mk | 2 +- 16 files changed, 92 insertions(+), 6 deletions(-) diff --git a/Makefile.inc1 b/Makefile.inc1 index 213b32a97ed3..8fa55f5cea2e 100644 --- a/Makefile.inc1 +++ b/Makefile.inc1 @@ -718,6 +718,7 @@ BSARGS= DESTDIR= \ MK_HTML=no NO_LINT=yes MK_MAN=no MK_MAN_UTILS=yes \ -DNO_PIC MK_PROFILE=no -DNO_SHARED \ -DNO_CPU_CFLAGS MK_WERROR=no MK_CTF=no \ + MK_ASAN=no MK_UBSAN=no \ MK_CLANG_EXTRAS=no MK_CLANG_FORMAT=no MK_CLANG_FULL=no \ MK_LLDB=no MK_RETPOLINE=no MK_TESTS=no \ MK_INCLUDES=yes @@ -739,6 +740,7 @@ TMAKE= \ SSP_CFLAGS= \ -DNO_LINT \ -DNO_CPU_CFLAGS MK_WERROR=no MK_CTF=no \ + MK_ASAN=no MK_UBSAN=no \ MK_CLANG_EXTRAS=no MK_CLANG_FORMAT=no MK_CLANG_FULL=no \ MK_LLDB=no MK_RETPOLINE=no MK_TESTS=no @@ -2844,6 +2846,16 @@ _prereq_libs= lib/libcompiler_rt .if ${MK_SSP} != "no" _prereq_libs+= lib/libssp_nonshared .endif +.if ${MK_ASAN} != "no" +_prereq_libs+= lib/libclang_rt/asan +_prereq_libs+= lib/libclang_rt/asan-preinit +_prereq_libs+= lib/libclang_rt/asan_cxx +.endif +.if ${MK_UBSAN} != "no" +_prereq_libs+= lib/libclang_rt/ubsan_minimal +_prereq_libs+= lib/libclang_rt/ubsan_standalone +_prereq_libs+= lib/libclang_rt/ubsan_standalone_cxx +.endif # These dependencies are not automatically generated: # diff --git a/Makefile.libcompat b/Makefile.libcompat index c400cef6fbe3..c2be63c29e73 100644 --- a/Makefile.libcompat +++ b/Makefile.libcompat @@ -104,6 +104,7 @@ build${libcompat}: .PHONY OBJROOT='$${OBJTOP}/' \ MAKEOBJDIRPREFIX= \ DIRPRFX=${_dir}/ -DNO_LINT -DNO_CPU_CFLAGS \ + MK_ASAN=no MK_UBSAN=no \ MK_CTF=no MK_RETPOLINE=no MK_WERROR=no \ ${_t} .endfor diff --git a/lib/Makefile b/lib/Makefile index a06e77dee022..674368a19ffd 100644 --- a/lib/Makefile +++ b/lib/Makefile @@ -180,6 +180,8 @@ SUBDIR.${MK_STATS}+= libstats ${MACHINE_CPUARCH} == "arm" || ${MACHINE_CPUARCH} == "i386" || \ ${MACHINE_CPUARCH} == "powerpc") _libclang_rt= libclang_rt +.elif ${MK_ASAN} != "no" || ${MK_UBSAN} != "no" +.error "Requested build with sanitizers but cannot build runtime libraries!" .endif .if ${MK_CXX} != "no" diff --git a/lib/csu/Makefile.inc b/lib/csu/Makefile.inc index 5681ef65117a..b089e1a947b1 100644 --- a/lib/csu/Makefile.inc +++ b/lib/csu/Makefile.inc @@ -3,6 +3,9 @@ SSP_CFLAGS= NO_WMISSING_VARIABLE_DECLARATIONS= +# Can't instrument these files since that breaks non-sanitized programs. +MK_ASAN:= no +MK_UBSAN:= no .include diff --git a/lib/libclang_rt/Makefile.inc b/lib/libclang_rt/Makefile.inc index 946d3f4c77a7..19b8868b60e3 100644 --- a/lib/libclang_rt/Makefile.inc +++ b/lib/libclang_rt/Makefile.inc @@ -12,6 +12,8 @@ SHLIBDIR= ${LIBDIR} NO_PIC= MK_PROFILE= no +MK_ASAN:= no +MK_UBSAN:= no WARNS?= 0 diff --git a/lib/libgcc_eh/Makefile.inc b/lib/libgcc_eh/Makefile.inc index 4fe1eff406e1..9e386992e78c 100644 --- a/lib/libgcc_eh/Makefile.inc +++ b/lib/libgcc_eh/Makefile.inc @@ -20,9 +20,17 @@ SRCS_EXC+= libunwind.cpp SRCS+= ${SRCS_EXC} .for file in ${SRCS_EXC:M*.c} CFLAGS.${file}+= -fno-exceptions -funwind-tables +.if ${MK_ASAN} != "no" +# False-positives during stack unwinding +CFLAGS.${file}+= -fno-sanitize=address +.endif .endfor .for file in ${SRCS_EXC:M*.cpp} CXXFLAGS.${file}+= -fno-exceptions -funwind-tables +.if ${MK_ASAN} != "no" +# False-positives during stack unwinding +CXXFLAGS.${file}+= -fno-sanitize=address +.endif .endfor CFLAGS+= -I${UNWINDINCDIR} diff --git a/lib/libgcc_s/Makefile b/lib/libgcc_s/Makefile index 84477ee00818..aa097d7255de 100644 --- a/lib/libgcc_s/Makefile +++ b/lib/libgcc_s/Makefile @@ -4,6 +4,9 @@ PACKAGE= clibs SHLIB_NAME= libgcc_s.so.1 SHLIBDIR?= /lib +# Enabling UBSan triggers "undefined reference to vtable for __cxxabiv1::__function_type_info" +MK_UBSAN:= no + .include MK_SSP= no diff --git a/libexec/rtld-elf/Makefile b/libexec/rtld-elf/Makefile index 9ae998942a12..db1bf70ca59e 100644 --- a/libexec/rtld-elf/Makefile +++ b/libexec/rtld-elf/Makefile @@ -9,7 +9,10 @@ RTLD_ELF_DIR:= ${.PARSEDIR} .include PACKAGE= clibs MK_PIE= no # Always position independent using local rules +# Not compatible with sanitizer instrumentation or SSP. +MK_ASAN= no MK_SSP= no +MK_UBSAN= no CONFS= libmap.conf PROG?= ld-elf.so.1 diff --git a/share/mk/bsd.README b/share/mk/bsd.README index 9eb91c1bce3c..8004d742d649 100644 --- a/share/mk/bsd.README +++ b/share/mk/bsd.README @@ -118,7 +118,7 @@ The profiled libraries are no longer built in a different directory than the regular libraries. A new suffix, ".po", is used to denote a profiled object, and ".pico" denotes a position-independent relocatable object. ".nossppico" denotes a position-independent relocatable object without -stack smashing protection. +stack smashing protection and without sanitizer instrumentation. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= diff --git a/share/mk/bsd.compat.mk b/share/mk/bsd.compat.mk index e063415bcbfc..6787209dcd25 100644 --- a/share/mk/bsd.compat.mk +++ b/share/mk/bsd.compat.mk @@ -162,6 +162,7 @@ LIBCOMPATCFLAGS+= -B${LIBCOMPATTMP}/usr/lib${libcompat} LIBDIR_BASE:= /usr/lib${libcompat} _LIB_OBJTOP= ${LIBCOMPAT_OBJTOP} LIBDESTDIR:= ${LIBCOMPATTMP} +SYSROOT:= ${LIBCOMPATTMP} CFLAGS+= ${LIBCOMPATCFLAGS} LDFLAGS+= ${CFLAGS} ${LIBCOMPATLDFLAGS} MACHINE= ${LIBCOMPAT_MACHINE} diff --git a/share/mk/bsd.lib.mk b/share/mk/bsd.lib.mk index db54055b7ae0..5de74b6fa056 100644 --- a/share/mk/bsd.lib.mk +++ b/share/mk/bsd.lib.mk @@ -6,6 +6,8 @@ .include .include +____: + .if defined(LIB_CXX) || defined(SHLIB_CXX) _LD= ${CXX} .else @@ -106,6 +108,8 @@ CXXFLAGS+= -ftrivial-auto-var-init=pattern .endif .endif +.include "bsd.sanitizer.mk" + .if ${MK_DEBUG_FILES} != "no" && empty(DEBUG_FLAGS:M-g) && \ empty(DEBUG_FLAGS:M-gdwarf*) CFLAGS+= ${DEBUG_FILES_CFLAGS} @@ -147,7 +151,7 @@ PO_FLAG=-pg ${CTFCONVERT_CMD} .c.nossppico: - ${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS:C/^-fstack-protector.*$//} ${CFLAGS:C/^-fstack-protector.*$//} -c ${.IMPSRC} -o ${.TARGET} + ${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS:C/^-fstack-protector.*$//:C/^-fsanitize.*$//} ${CFLAGS:C/^-fstack-protector.*$//:C/^-fsanitize.*$//} -c ${.IMPSRC} -o ${.TARGET} ${CTFCONVERT_CMD} .c.pieo: @@ -161,7 +165,7 @@ PO_FLAG=-pg ${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET} .cc.nossppico .C.nossppico .cpp.nossppico .cxx.nossppico: - ${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS:C/^-fstack-protector.*$//} ${CXXFLAGS:C/^-fstack-protector.*$//} -c ${.IMPSRC} -o ${.TARGET} + ${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS:C/^-fstack-protector.*$//:C/^-fsanitize.*$//} ${CXXFLAGS:C/^-fstack-protector.*$//:C/^-fsanitize.*$//} -c ${.IMPSRC} -o ${.TARGET} .cc.pieo .C.pieo .cpp.pieo .cxx.pieo: ${CXX} ${PIEFLAG} ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET} diff --git a/share/mk/bsd.opts.mk b/share/mk/bsd.opts.mk index 33d843593427..6e81484a09ab 100644 --- a/share/mk/bsd.opts.mk +++ b/share/mk/bsd.opts.mk @@ -69,6 +69,7 @@ __DEFAULT_YES_OPTIONS = \ WERROR __DEFAULT_NO_OPTIONS = \ + ASAN \ BIND_NOW \ CCACHE_BUILD \ CTF \ @@ -77,7 +78,8 @@ __DEFAULT_NO_OPTIONS = \ INSTALL_AS_USER \ MANSPLITPKG \ RETPOLINE \ - STALE_STAGED + STALE_STAGED \ + UBSAN __DEFAULT_DEPENDENT_OPTIONS = \ MAKE_CHECK_USE_SANDBOX/TESTS \ diff --git a/share/mk/bsd.prog.mk b/share/mk/bsd.prog.mk index 89eddb24abb0..96d72daf2611 100644 --- a/share/mk/bsd.prog.mk +++ b/share/mk/bsd.prog.mk @@ -81,6 +81,8 @@ CXXFLAGS+= -ftrivial-auto-var-init=pattern .endif .endif +.include "bsd.sanitizer.mk" + .if ${MACHINE_CPUARCH} == "riscv" && ${LINKER_FEATURES:Mriscv-relaxations} == "" CFLAGS += -mno-relax .endif diff --git a/share/mk/bsd.progs.mk b/share/mk/bsd.progs.mk index 54266a335177..79397f595688 100644 --- a/share/mk/bsd.progs.mk +++ b/share/mk/bsd.progs.mk @@ -23,7 +23,7 @@ PROGS += ${PROGS_CXX} .if defined(PROG) # just one of many PROG_OVERRIDE_VARS += BINDIR BINGRP BINOWN BINMODE CSTD CXXSTD DPSRCS MAN \ - NO_SHARED MK_WERROR PROGNAME SRCS STRIP WARNS + NO_SHARED MK_WERROR PROGNAME SRCS STRIP WARNS MK_ASAN MK_UBSAN PROG_VARS += CFLAGS CXXFLAGS DEBUG_FLAGS DPADD INTERNALPROG LDADD LIBADD \ LINKS LDFLAGS MLINKS ${PROG_OVERRIDE_VARS} .for v in ${PROG_VARS:O:u} diff --git a/share/mk/bsd.sanitizer.mk b/share/mk/bsd.sanitizer.mk new file mode 100644 index 000000000000..56d010767906 --- /dev/null +++ b/share/mk/bsd.sanitizer.mk @@ -0,0 +1,43 @@ +.include + +.include "../../lib/libclang_rt/compiler-rt-vars.mk" +_use_sanitizers= no +# Add the necessary sanitizer flags if requested +.if ${MK_ASAN} == "yes" && ${NO_SHARED:Uno:tl} == "no" +SANITIZER_CFLAGS+= -fsanitize=address -fPIC +# TODO: remove this once all basic errors have been fixed: +# https://github.com/google/sanitizers/wiki/AddressSanitizer#faq +SANITIZER_CFLAGS+= -fsanitize-recover=address +SANITIZER_LDFLAGS+= -fsanitize=address +_use_sanitizers= yes +.endif # ${MK_ASAN} == "yes" + +.if ${MK_UBSAN} == "yes" && ${NO_SHARED:Uno:tl} == "no" +# Unlike the other sanitizers, UBSan could also work for static libraries. +# However, this currently results in linker errors (even with the +# -fsanitize-minimal-runtime flag), so only enable it for dynamically linked +# code for now. +SANITIZER_CFLAGS+= -fsanitize=undefined +SANITIZER_CFLAGS+= -fsanitize-recover=undefined +SANITIZER_LDFLAGS+= -fsanitize=undefined +_use_sanitizers= yes +.endif # ${MK_UBSAN} == "yes" + +.if !defined(BOOTSTRAPPING) && ${_use_sanitizers} != 0 && \ + ${COMPILER_TYPE} != "clang" +.error "Sanitizer instrumentation currently only supported with clang" +.endif + +# For libraries we only instrument the shared and PIE libraries by setting +# SHARED_CFLAGS instead of CFLAGS. We do this since static executables are not +# compatible with the santizers (interceptors do not work). +.if ${_use_sanitizers} != "no" +.if target(____) +SHARED_CFLAGS+= ${SANITIZER_CFLAGS} +SOLINKOPTS+= ${SANITIZER_LDFLAGS} +LDFLAGS:= ${LDFLAGS:N-Wl,-no-undefined:N-Wl,--no-undefined} +.else +CFLAGS+= ${SANITIZER_CFLAGS} +LDFLAGS+= ${SANITIZER_LDFLAGS} +.endif +.endif # ${_use_sanitizers} != "no" diff --git a/share/mk/sys.mk b/share/mk/sys.mk index 89ac2c549656..462002ce7d18 100644 --- a/share/mk/sys.mk +++ b/share/mk/sys.mk @@ -242,7 +242,7 @@ LFLAGS ?= # compiler driver flags (e.g. -mabi=*) that conflict with flags to LD. LD ?= ld LDFLAGS ?= -_LDFLAGS = ${LDFLAGS:S/-Wl,//g:N-mabi=*:N-fuse-ld=*:N--ld-path=*} +_LDFLAGS = ${LDFLAGS:S/-Wl,//g:N-mabi=*:N-fuse-ld=*:N--ld-path=*:N-fsanitize=*:N-fno-sanitize=*} MAKE ?= make