Date: Sun, 12 Feb 2023 14:44:24 -0500 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: David Chisnall <theraven@FreeBSD.org> Cc: freebsd-hackers <freebsd-hackers@freebsd.org> Subject: Re: CFT: snmalloc as libc malloc Message-ID: <20230212194424.ka7omg6ya3ain3ml@mutt-hbsd> In-Reply-To: <193D1E1C-73EB-42BB-8A6A-87B0E4AD9D7C@FreeBSD.org> References: <2f3dcda0-5135-290a-2dff-683b2e9fe271@FreeBSD.org> <20230210162350.gg5g7dihp3zef3ov@mutt-hbsd> <193D1E1C-73EB-42BB-8A6A-87B0E4AD9D7C@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] On Sat, Feb 11, 2023 at 05:10:02PM +0000, David Chisnall wrote: > On 10 Feb 2023, at 16:23, Shawn Webb <shawn.webb@hardenedbsd.org> wrote: > > > > So I took a little bit of a different approach, which should provide > > the same end result as your submodule approach. Note that I'm doing > > this in HardenedBSD 14-CURRENT (the hardened/current/master branch). > > > > 1. git cherry-pick -xs a5c83c69817d03943b8be982dd815c7e263d1a83 > > 2. git rm -f .gitmodules contrib/snmalloc > > 3. git commit > > 4. git subtree add -P contrib/snmalloc \ > > git@github.com:microsoft/snmalloc.git main > > > > I believe this should leave me with a tree that populates > > contrib/snmalloc and pulls in your non-contrib/ changes, leading me to > > end up in the same end state as your submodule approach. > > > > I am seeing some build errors. I've uploaded a WITHOUT_CLEAN=yes log > > to: > > > > https://hardenedbsd.org/~shawn/2023-02-10_snmalloc-01.log.txt > > > > Note that this is with llvm 15.0.7 that just landed in FreeBSD main. > > The error is a bit confusing, because nullptr_t has been in libc++ since C++11. Does HardenedBSD change anything in include orders? Can you add -v to the end of the compile command: > > > c++ -target x86_64-unknown-freebsd14.0 --sysroot=/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp -B/usr/obj/data/src/hardenedbsd/amd64.amd64/tmp/usr/bin -fomit-frame-pointer -O2 -pipe -fno-common -DHARDENEDBSD -DNO__SCCSID -DNO__RCSID -I/data/src/hardenedbsd/lib/libc/include -I/data/src/hardenedbsd/include -I/data/src/hardenedbsd/lib/libc/amd64 -DNLS -ftls-model=initial-exec -D__DBINTERFACE_PRIVATE -I/data/src/hardenedbsd/contrib/gdtoa -I/data/src/hardenedbsd/contrib/libc-vis -DINET6 -I/usr/obj/data/src/hardenedbsd/amd64.amd64/lib/libc -I/data/src/hardenedbsd/lib/libc/resolv -D_ACL_PRIVATE -DPOSIX_MISTAKE -I/data/src/hardenedbsd/lib/libmd -I/data/src/hardenedbsd/lib/libc/locale -DBROKEN_DES -DPORTMAP -DDES_BUILTIN -I/data/src/hardenedbsd/lib/libc/rpc -DWANT_HYPERV -DYP -DNS_CACHING -DSYMBOL_VERSIONING -g -gz=zlib -mretpoline -fPIC -flto -MD -MF.depend.malloc.o -MTmalloc.o -Wno-format-zero-length -fstack-protector-strong -Wsystem-headers -Werror -Wall -Wno-format-y2k -Wno-uninitialized -Wdate-time -Wno-empty-body -Wno-string-plus-int -Wno-unused-const-variable -Wno-error=unused-but-set-variable -Wno-error=array-parameter -Wno-error=deprecated-non-prototype -Wno-error=unused-but-set-parameter -Wno-tautological-compare -Wno-unused-value -Wno-parentheses-equality -Wno-unused-function -Wno-enum-conversion -Wno-unused-local-typedef -Wno-address-of-packed-member -Wno-switch -Wno-switch-enum -Wno-knr-promoted-parameter -Qunused-arguments -I/data/src/hardenedbsd/lib/libutil -I/data/src/hardenedbsd/lib/msun/amd64 -I/data/src/hardenedbsd/lib/msun/x86 --include-directory-after /data/src/hardenedbsd/lib/msun/src -DHARDENEDBSD -I/data/src/hardenedbsd/contrib/snmalloc/src/snmalloc -std=c++20 -mcx16 -fno-exceptions -fno-rtti -DSNMALLOC_USE_THREAD_CLEANUP -DSNMALLOC_BOOTSTRAP_ALLOCATOR -DSNMALLOC_JEMALLOC3_EXPERIMENTAL -DSNMALLOC_JEMALLOC_NONSTANDARD -DSNMALLOC_PLATFORM_HAS_GETENTROPY -DSNMALLOC_STATIC_LIBRARY_PREFIX=__ -ftls-model=initial-exec -DSNMALLOC_CHECK_CLIENT -DSNMALLOC_FAIL_FAST=false -DNDEBUG -g -gz=zlib -mretpoline -flto -Wno-c++11-extensions -c /data/src/hardenedbsd/lib/libc/stdlib/snmalloc/malloc.cc -o malloc.o > > > > That should dump the include paths. It’s possible that the libc++ headers are being included in the wrong order with respect to the C paths? Hey David, HardenedBSD doesn't have any changes to userland that would cause changes in include header paths (and the priorities of them.) I made sure to `pkg delete -g gcc\* binutils\*` before running another buildworld, ending up with the same error. I tried running your command above. Here's the result: http://ix.io/4nS9 I think Mark might be on the right track with regards to llvm 15. I'll try importing the snmalloc patch (via the same subtree method) to a pre-llvm15 import branch and report back. It might be a few days before I get you a result. BTW, I appreciate the work. One long-term goal I've had for HardenedBSD is to be able to switch between multiple heap implementations at buildworld-time. I think it would be nifty for users to have a choice of jemalloc, snmalloc, and GrapheneOS' hardened_malloc. Thanks, -- Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmPpQZEACgkQ/y5nonf4 4frmqQ//eWViZtBrwkq3zrzEBbBEyxpWpQvzyG68scSvtdJ6/+bVzGbk/xR3mzso DDiWwaL2ehEBdWwj1WDS4bPKF33Tm8jdOje8gf0qG+ajgcnhSpZFTqNbTmOxWdNO YIKJkCuhxP4AOMD8mhEgdzZseOlYcO7txlFefI8e6+B9nc4siM7OmO76CgGrzN1Q MP9YPDAhvEI5txsRHh/BXPxDU5hk/stHsdnwdIrZn5hq65qXU49+omMxx/T62JQ9 fpapv2rZOc0PmkyrMuAQlKo6EnSzANG2Cudd5Bx4MI8oZbpCwZKXGIuOmYwgN1U5 PjOYJ0J4IPyU7u3nwerKAB9RpG+HNFgt4KJlXvc/YRi7732GcCKamcpXUkYmZWXC 3l/1tQH8NtX/quXwH5tOUI2PWjxRHJoHfRv1Dtw3hzzB5FEVNxa8A4D4TtH7EsJU oW81yfuFW59wLmLlnXV2u77/uyaoafEW/gxUjWzc8+9c4o1ZQ7Fogj0nf7cT8g0P Q13N92pEEtwF863uJ4X1VQtXlvDBYAa1rMfYGMNMrqrhkO0QwhY5J2krmM7+tSzP 4mpT1fPwHYg64gRJpoOKDQrSs9PvybmfxXht1wXHg5j9exJS4CgeCBuBj5D5+9rp 39oIeJvAKwk/f0zdi6EvhNBTjwV3hYAqGalYvgix4csASIDiAyA= =blgy -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20230212194424.ka7omg6ya3ain3ml>