Date: Fri, 14 Jan 2022 05:17:52 -0800 From: Mark Millard <marklmi@yahoo.com> To: freebsd-current <freebsd-current@freebsd.org> Subject: Re: An explanation of some "Container overflow" ASAN reports: libc++ does sometimes temporarily overflow the used range of a container Message-ID: <439BD3A5-68D4-40A7-A4CD-77A05A847B18@yahoo.com> In-Reply-To: <22B11810-DB3F-4E81-AB40-ED207CD83EEE@yahoo.com> References: <22B11810-DB3F-4E81-AB40-ED207CD83EEE@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On 2022-Jan-14, at 04:44, Mark Millard <marklmi@yahoo.com> wrote:
>=20
> Looks like libc++ does the following sort of thing
> (from lldb list):
>=20
> . . .
> 1635=09
> 1636 template <class _Tp, class _Allocator>
> 1637 template <class _Up>
> 1638 void
> 1639 #ifndef _LIBCPP_CXX03_LANG
> (lldb)=20
> 1640 vector<_Tp, _Allocator>::__push_back_slow_path(_Up&& =
__x)
> 1641 #else
> 1642 vector<_Tp, _Allocator>::__push_back_slow_path(_Up& __x)
> 1643 #endif
> 1644 {
> 1645 allocator_type& __a =3D this->__alloc();
> 1646 __split_buffer<value_type, allocator_type&> =
__v(__recommend(size() + 1), size(), __a);
> 1647 // __v.push_back(_VSTD::forward<_Up>(__x));
> 1648 __alloc_traits::construct(__a, =
_VSTD::__to_address(__v.__end_), _VSTD::forward<_Up>(__x));
> 1649 __v.__end_++;
> (lldb)=20
> 1650 __swap_out_circular_buffer(__v);
> 1651 }
> . . . (the bt points to 1650) . . .
>=20
> 1648 constructs into __v at __v.__end_ and 1649 then corrects
> __v.__end_ to cause the constructed object to no longer be
> an example of "Container overflow" but now in the Container.
> (At least that is my interpretation.)
>=20
> The compiler's code generation may move the detailed
> place where __v.__end_++ happens relative to some other
> of the activity but the compiler has been told an order
> relative to the construction that would lead to writing
> memory in the capacity of the container __v but outside
> the size of the __v container at the time.
>=20
> For reference:
>=20
> 970 template <class _Tp, class _Allocator>
> 971 void
> 972 vector<_Tp, =
_Allocator>::__swap_out_circular_buffer(__split_buffer<value_type, =
allocator_type&>& __v)
> 973 {
> 974 =09
> 975 __annotate_delete();
> 976 =
_VSTD::__construct_backward_with_exception_guarantees(this->__alloc(), =
this->__begin_, this->__end_, __v.__begin_);
> 977 _VSTD::swap(this->__begin_, __v.__begin_);
> 978 _VSTD::swap(this->__end_, __v.__end_);
> 979 _VSTD::swap(this->__end_cap(), __v.__end_cap());
> (lldb)=20
> 980 __v.__first_ =3D __v.__begin_;
> 981 __annotate_new(size());
> 982 __invalidate_all_iterators();
> 983 }
> . . . (the bt for this points to 976) . . .
>=20
>=20
> This suggests to me that using some equivalent of:
>=20
> env ASAN_OPTIONS=3Ddetect_container_overflow=3D0
>=20
> my be required fairly generally when libc++ can
> be involved.
>=20
>=20
> Other notes . . .
>=20
> I used ld -v as an example for the above via:
>=20
> env ASAN_OPTIONS=3Ddetect_container_overflow=3D0 lldb ld
>=20
> and used:
>=20
> (lldb) env ASAN_OPTIONS=3D
> (lldb) run -v
>=20
> in order to have ld itself not have detect_container_overflow
> disabled.
>=20
> lldb suffers the libc++ Container overflow problems via its
> libc++ use and fails to operate without the:
>=20
> ASAN_OPTIONS=3Ddetect_container_overflow=3D0
>=20
Hmm. The code in:
772 template <class _Alloc, class _Ptr>
773 _LIBCPP_INLINE_VISIBILITY
774 void __construct_backward_with_exception_guarantees(_Alloc& __a, =
_Ptr __begin1, _Ptr __end1, _Ptr& __end2) {
775 static_assert(__is_cpp17_move_insertable<_Alloc>::value,
776 "The specified type does not meet the requirements of =
Cpp17MoveInsertable");
777 typedef allocator_traits<_Alloc> _Traits;
778 while (__end1 !=3D __begin1) {
779 _Traits::construct(__a, _VSTD::__to_address(__end2 - 1),
(lldb)=20
780 #ifdef _LIBCPP_NO_EXCEPTIONS
781 _VSTD::move(*--__end1)
782 #else
783 _VSTD::move_if_noexcept(*--__end1)
784 #endif
785 );
786 --__end2;
787 }
788 }
has the same sort of problem going in the other direction.
__end2 references the __v.__begin_ mentioned earlier for
the context at hand.
line 779 constructs just before where __v.__begin_ refers to
at the time and then 786 updates __v.__begin_ to make the
constructed object no longer be an example of "Container
overflow" but now in the Container.
This is likely the WRITE activity that is actually reported,
although I've not analyzed the machine code at all.
This also suggests to me that using some equivalent of:
env ASAN_OPTIONS=3Ddetect_container_overflow=3D0
my be required fairly generally when libc++ can
be involved.
Important . . .
I'll note that the construct-then-include order is tied
to exception safety. I'm not claiming that the libc++
code is wrong. It just that it and ASAN are currently
mismatched so ASAN by default reports libc++ as having
an addressing issue for which aborting the program is
the handling ASAN does by default.
=3D=3D=3D
Mark Millard
marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?439BD3A5-68D4-40A7-A4CD-77A05A847B18>