Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 16 Jan 2022 17:20:27 -0800
From:      Mark Millard <marklmi@yahoo.com>
To:        freebsd-current <freebsd-current@freebsd.org>
Cc:        Dimitry Andric <dim@FreeBSD.org>
Subject:   Re: WITH_ASAN= vs. vfork: ASAN can not deal with vfork in a reliable manor, so what to do to avoid vfork fairly generally, including for kyua?
Message-ID:  <DE55AE11-9506-4FCC-81F7-0F96A9B26102@yahoo.com>
In-Reply-To: <E0E46025-3D63-4906-A9F0-06D2225E1C65@yahoo.com>
References:  <E0E46025-3D63-4906-A9F0-06D2225E1C65@yahoo.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 2022-Jan-15, at 15:25, Mark Millard <marklmi@yahoo.com> wrote:

> ASAN is documented to not be able to deal reliably with vfork use
> (inaccurate results, result mixing interpretations of the old
> and new contexts, and so on). [There may be other routines
> sufficiently analogous to vfork to have the same sorts of issues.
> I'll write only of vfork explicitly.]
>=20
> It turns out that using SH_DISABLE_VFORK=3D with kyua runs
> helps avoid a bunch of junk failure ASAN reports that involve
> /bin/sh using vfork. So I've been recently using the likes of:
>=20
> env SH_DISABLE_VFORK=3D ASAN_OPTIONS=3Ddetect_container_overflow=3D0 \
> kyua test -k /usr/tests/Kyuafile
>=20
> in my kyua runs. But that need not cover all the vfork use in
> the kyua runs --or in general system operation.
>=20
>=20
>=20
> There is more that could be done. For example . . .
>=20
> contrib/llvm-project/compiler-rt/lib/asan/asan_interceptors.h
> currently has:
>=20
> #if SANITIZER_LINUX &&                                                =
\
>    (defined(__arm__) || defined(__aarch64__) || defined(__i386__) || \
>     defined(__x86_64__) || SANITIZER_RISCV64)
> # define ASAN_INTERCEPT_VFORK 1
> #else
> # define ASAN_INTERCEPT_VFORK 0
> #endif
>=20
> The "# define ASAN_INTERCEPT_VFORK 1" causes interception to
> use fork for the vfork implementation, instead of an actual
> vfork.
>=20
> It looks to me like asan_interceptors.h for FreeBSD should
> also be using:
>=20
> # define ASAN_INTERCEPT_VFORK 1
>=20
> but it currently is not.

Well, looking around there does not seem to be a FreeBSD-supporting
implementation to go with use of "# define ASAN_INTERCEPT_VFORK 1".
So there would be more to it than just causing the #define to happen.

Thus, there does not seem to be a quick way for me to see what would
result in the kyua run from more avoiding of vfork use (beyond what
use of SH_DISABLE_VFORK=3D did).

> There is even the possibility that a WITH_ASAN=3D buildworld could
> build a world that uses fork behavior for vfork and never does an
> actual vfork (by behavior). Basically restricting itself to things
> that fit with ASAN use.
>=20
>=20
>=20
> I tracked this issue down via a report in a kyua run that
> referenced:
>=20
> QUOTE
> ERROR: AddressSanitizer: stack-buffer-underflow on address =
0x7fffffffa580 at pc 0x000801e0f8ed bp 0x7fffffff9bf0 sp 0x7fffffff9be8
> WRITE of size 8 at 0x7fffffffa580 thread T0
> . . .
> Address 0x7fffffffa580 is located in stack of thread T0 at offset 0 in =
frame
>    #0 0x7fffffffa5ef  (<unknown module>)
>=20
>  This frame has 1 object(s):
>    [32, 288) 'buf' (line 180)
> HINT: this may be a false positive if your program uses some custom =
stack unwind mechanism, swapcontext or vfork
>      (longjmp and C++ exceptions *are* supported)
> END QUOTE
>=20
> The "'buf' (line 180)" turned out to be from the declaration in
> bin/sh/exec.c 's tryexec :
>=20
> static void
> tryexec(char *cmd, char **argv, char **envp)
> {
>        int e, in;
>        ssize_t n;
>        char buf[256];
>=20
>        execve(cmd, argv, envp);
>        e =3D errno;
>        if (e =3D=3D ENOEXEC) {
>                INTOFF;
>                in =3D open(cmd, O_RDONLY | O_NONBLOCK);
>                if (in !=3D -1) {
>                        n =3D pread(in, buf, sizeof buf, 0);
>                        close(in);
>                        if (n > 0 && isbinary(buf, n)) {
>                                errno =3D ENOEXEC;
>                                return;
>                        }
>                }
>                *argv =3D cmd;
>                *--argv =3D __DECONST(char *, _PATH_BSHELL);
>                execve(_PATH_BSHELL, argv, envp);
>        }
>        errno =3D e;
> }
>=20
> So I could finally see/validate that an example
> stack-buffer-* report was tied to vfork handling
> (analyzing other code related to the tryexec use).
>=20
> The report happens well after tryexec did its
> execve. The information about buf is just old,
> no longer accurate information, leading to a
> false report:
>=20
>=20
> =3D=3D87961=3D=3DERROR: AddressSanitizer: stack-buffer-underflow on =
address 0x7fffffffa580 at pc 0x000801e0f8ed bp 0x7fffffff9bf0 sp =
0x7fffffff9be8
> WRITE of size 8 at 0x7fffffffa580 thread T0
>    #0 0x801e0f8ec in bintime2timespec =
/usr/obj/BUILDs/main-amd64-nodbg-clang-alt/usr/main-src/amd64.amd64/tmp/us=
r/include/sys/time.h:285:14
>    #1 0x801e0f8ec in __vdso_clock_gettime =
/usr/main-src/lib/libc/sys/__vdso_gettimeofday.c:195:2
>    #2 0x801e0e1a0 in clock_gettime =
/usr/main-src/lib/libc/sys/clock_gettime.c:48:11
>    #3 0x10d54da in clock_gettime =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/saniti=
zer_common_interceptors.inc:2189:13
>    #4 0x11234f5 in __sanitizer::MonotonicNanoTime() =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/saniti=
zer_linux_libcdep.cpp:860:3
>    #5 0x10ba02c in =
__sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddressSp=
aceView> >::PopulateFreeArray(__sanitizer::AllocatorStats*, unsigned =
long, __sanitizer::SizeClassAllocator
> 64<__asan::AP64<__sanitizer::LocalAddressSpaceView> >::RegionInfo*, =
unsigned long) =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/saniti=
zer_allocator_primary64.h:790:45
>    #6 0x10b9c4b in =
__sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddressSp=
aceView> >::GetFromAllocator(__sanitizer::AllocatorStats*, unsigned =
long, unsigned int*, unsigned long) /u
> =
sr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/sanitize=
r_allocator_primary64.h:220:11
>    #7 0x10b9955 in =
__sanitizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocato=
r64<__asan::AP64<__sanitizer::LocalAddressSpaceView> > =
>::Refill(__sanitizer::SizeClassAllocator64LocalCac
> =
he<__sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddres=
sSpaceView> > >::PerClass*, =
__sanitizer::SizeClassAllocator64<__asan::AP64<__sanitizer::LocalAddressSp=
aceView> >*, unsigned lo
> ng) =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/saniti=
zer_allocator_local_cache.h:103:9
>    #8 0x10b9615 in =
__sanitizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocato=
r64<__asan::AP64<__sanitizer::LocalAddressSpaceView> > =
>::Allocate(__sanitizer::SizeClassAllocator64<__asa
> n::AP64<__sanitizer::LocalAddressSpaceView> >*, unsigned long) =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/sanitizer_common/saniti=
zer_allocator_local_cache.h:39:11
>    #9 0x10b9511 in =
__sanitizer::CombinedAllocator<__sanitizer::SizeClassAllocator64<__asan::A=
P64<__sanitizer::LocalAddressSpaceView> >, =
__sanitizer::LargeMmapAllocatorPtrArrayDynamic>::Allocate(__san
> =
itizer::SizeClassAllocator64LocalCache<__sanitizer::SizeClassAllocator64<_=
_asan::AP64<__sanitizer::LocalAddressSpaceView> > >*, unsigned long, =
unsigned long) /usr/main-src/contrib/llvm-project/compile
> r-rt/lib/sanitizer_common/sanitizer_allocator_combined.h:69:20
>    #10 0x10b6086 in __asan::Allocator::Allocate(unsigned long, =
unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, =
bool) /usr/main-src/contrib/llvm-project/compiler-rt/lib/asan/asan_a
> llocator.cpp:537:29
>    #11 0x10b4818 in __asan::asan_malloc(unsigned long, =
__sanitizer::BufferedStackTrace*) =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/asan/asan_allocator.cpp=
:980:34
>    #12 0x110be9b in malloc =
/usr/main-src/contrib/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.=
cpp:130:10
>    #13 0x117aca3 in ckmalloc /usr/main-src/bin/sh/memalloc.c:71:6
>    #14 0x115b1b1 in reprocess /usr/main-src/bin/sh/expand.c:912:9
>    #15 0x1155163 in evalvar /usr/main-src/bin/sh/expand.c:784:3
>    #16 0x1155163 in argstr /usr/main-src/bin/sh/expand.c:319:8
>    #17 0x1151178 in expandarg /usr/main-src/bin/sh/expand.c:241:2
>    #18 0x11427c8 in evalcommand /usr/main-src/bin/sh/eval.c:857:4
>    #19 0x114705b in evalbackcmd /usr/main-src/bin/sh/eval.c:681:4
>    #20 0x1151bfc in expbackq /usr/main-src/bin/sh/expand.c:476:2
>    #21 0x1151bfc in argstr /usr/main-src/bin/sh/expand.c:323:4
>    #22 0x1151178 in expandarg /usr/main-src/bin/sh/expand.c:241:2
>    #23 0x11427c8 in evalcommand /usr/main-src/bin/sh/eval.c:857:4
>    #24 0x113eeb7 in evaltree /usr/main-src/bin/sh/eval.c:289:4
>    #25 0x113eb88 in evalstring /usr/main-src/bin/sh/eval.c
>    #26 0x113e9f9 in evalcmd /usr/main-src/bin/sh/eval.c:139:17
>    #27 0x1145289 in evalcommand /usr/main-src/bin/sh/eval.c:1107:16
>    #28 0x113eeb7 in evaltree /usr/main-src/bin/sh/eval.c:289:4
>    #29 0x113f86b in evaltree /usr/main-src/bin/sh/eval.c:212:4
>    #30 0x1144d89 in evalcommand /usr/main-src/bin/sh/eval.c:1053:3
>    #31 0x113eeb7 in evaltree /usr/main-src/bin/sh/eval.c:289:4
>    #32 0x117a316 in cmdloop /usr/main-src/bin/sh/main.c:228:4
>    #33 0x1179788 in main /usr/main-src/bin/sh/main.c:175:3
>=20
> Address 0x7fffffffa580 is located in stack of thread T0 at offset 0 in =
frame
>    #0 0x7fffffffa5ef  (<unknown module>)
>=20
>  This frame has 1 object(s):
>    [32, 288) 'buf' (line 180)
> HINT: this may be a false positive if your program uses some custom =
stack unwind mechanism, swapcontext or vfork
>      (longjmp and C++ exceptions *are* supported)
> SUMMARY: AddressSanitizer: stack-buffer-underflow =
/usr/obj/BUILDs/main-amd64-nodbg-clang-alt/usr/main-src/amd64.amd64/tmp/us=
r/include/sys/time.h:285:14 in bintime2timespec
> Shadow bytes around the buggy address:
>  0x4ffffffff460: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff470: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff480: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff490: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff4a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> =3D>0x4ffffffff4b0:[f1]f1 f1 f1 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff4c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff4d0: 00 00 00 00 f3 f3 f3 f3 f3 f3 f3 f3 00 00 00 00
>  0x4ffffffff4e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>  0x4ffffffff4f0: f1 f1 f1 f1 00 f2 f2 f2 00 f3 f3 f3 00 00 00 00
>  0x4ffffffff500: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> Shadow byte legend (one shadow byte represents 8 application bytes):
>  Addressable:           00
>  Partially addressable: 01 02 03 04 05 06 07=20
>  Heap left redzone:       fa
>  Freed heap region:       fd
>  Stack left redzone:      f1
>  Stack mid redzone:       f2
>  Stack right redzone:     f3
>  Stack after return:      f5
>  Stack use after scope:   f8
>  Global redzone:          f9
>  Global init order:       f6
>  Poisoned by user:        f7
>  Container overflow:      fc
>  Array cookie:            ac
>  Intra object redzone:    bb
>  ASan internal:           fe
>  Left alloca redzone:     ca
>  Right alloca redzone:    cb
> =3D=3D87961=3D=3DABORTING




=3D=3D=3D
Mark Millard
marklmi at yahoo.com




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?DE55AE11-9506-4FCC-81F7-0F96A9B26102>