From owner-freebsd-stable@freebsd.org Thu Feb 21 09:36:54 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D96014D9176 for ; Thu, 21 Feb 2019 09:36:54 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9ED586AAC8 for ; Thu, 21 Feb 2019 09:36:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x1L9akU5044708 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 21 Feb 2019 11:36:49 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x1L9akU5044708 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x1L9akS3044707; Thu, 21 Feb 2019 11:36:46 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 21 Feb 2019 11:36:45 +0200 From: Konstantin Belousov To: Harry Schmalzbauer Cc: freebsd-stable Subject: Re: libcrypto.so.111 linked binaries SIGSEGV (in bhyve guest) Message-ID: <20190221093645.GS2420@kib.kiev.ua> References: <335630bc-a558-9e31-5e2d-aee6237e83b0@omnilan.de> <596d1486-e2af-43f7-6f3f-65881e91819d@omnilan.de> <20190221085432.GR2420@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.11.2 (2019-01-07) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Feb 2019 09:36:54 -0000 On Thu, Feb 21, 2019 at 10:03:29AM +0100, Harry Schmalzbauer wrote: > Am 21.02.2019 um 09:54 schrieb Konstantin Belousov: > > On Thu, Feb 21, 2019 at 09:24:43AM +0100, Harry Schmalzbauer wrote: > >> Am 20.02.2019 um 17:51 schrieb Harry Schmalzbauer: > >>> Hello, > >>> > >> … > >>> gdb shows: > >>> Core was generated by `/usr/sbin/auditdistd'. > >>> Program terminated with signal 11, Segmentation fault. > >>> Reading symbols from /lib/libutil.so.9...Reading symbols from > >>> /usr/lib/debug//lib/libutil.so.9.debug...done. > >>> done. > >>> Loaded symbols for /lib/libutil.so.9 > >>> Reading symbols from /libexec/ld-elf.so.1...Reading symbols from > >>> /usr/lib/debug//libexec/ld-elf.so.1.debug...done. > >>> done. > >>> Loaded symbols for /libexec/ld-elf.so.1 > >>> #0  memset (dest=0x80056f790, c=0, len=) > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624 > >>> 5624                    ((char *)dest)[i] = c; > >>> (gdb) bt > >>> #0  memset (dest=0x80056f790, c=0, len=) > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624 > >>> #1  0x0000000800235b07 in map_object (fd=3, path=0x800246140 > >>> "/lib/libcrypto.so.111", > >>>     sb=0x7fffffffd4a8) > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249 > >>> #2  0x0000000800230806 in load_object (name=0x201dba > >>> "libcrypto.so.111", fd_u=-1, > >>>     refobj=0x800248000, flags=) > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493 > >>> #3  0x0000000800229972 in _rtld (sp=, > >>> exit_proc=0x7fffffffea30, > >>>     objp=0x7fffffffea38) > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315 > >>> #4  0x0000000800228019 in .rtld_start () > >>>     at > >>> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39 > >>> #5  0x0000000000000000 in ?? () > >>> Current language:  auto; currently minimal > >>> > >>> Any help highly appreciated. > >>> > >>> This is with a live CD (amd64), compiled with stable/12 from today (so > >>> clang 7.01). > >>> The bhyve guest has 2GB hardwired and ran stable/11 beforehand, which > >>> compiled the live CD. > >>> bhyve host is 11.2.  But that shouldn't play a role, does it? > >> > >> I'm really interested what happens here. > >> I built stable/11 in that bhyve guest and updated that guest to > >> stable/11 from yesterday. > >> To my surpise llvm 7.01 was also merged to stable/11.  Thank you for > >> that great supprt! > >> No problems with any binary in the stable/11 bhyve guest. > >> > >> Then I built stable/12 in that re-built stable/11 guest. > >> As result, again all binaries linked to /lib/libcrypto.so.111 crash > >> (signal 11) with the stable/12 iso in the same bhyve guest. > >> > >> Here the example from ntpq: > >> Program terminated with signal 11, Segmentation fault. > >> Reading symbols from /lib/libedit.so.7...Reading symbols from > >> /usr/lib/debug//lib/libedit.so.7.debug...done. > >> done. > >> Loaded symbols for /lib/libedit.so.7 > >> Reading symbols from /lib/libm.so.5...Reading symbols from > >> /usr/lib/debug//lib/libm.so.5.debug...done. > >> done. > >> Loaded symbols for /lib/libm.so.5 > >> Reading symbols from /libexec/ld-elf.so.1...Reading symbols from > >> /usr/lib/debug//libexec/ld-elf.so.1.debug...done. > >> done. > >> #0  memset (dest=0x8005ef790, c=0, len=) at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624 > >> 5624                    ((char *)dest)[i] = c; > >> (gdb) bt > >> #0  memset (dest=0x8005ef790, c=0, len=) at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:5624 > >> #1  0x000000080025db07 in map_object (fd=3, path=0x80026e1a0 > >> "/lib/libcrypto.so.111", sb=0x7fffffffd4c8) at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/map_object.c:249 > >> #2  0x0000000800258806 in load_object (name=0x201b40 "libcrypto.so.111", > >> fd_u=-1, refobj=0x800270000, flags=) at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2493 > >> #3  0x0000000800251972 in _rtld (sp=, > >> exit_proc=0x7fffffffea50, objp=0x7fffffffea58) at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/rtld.c:2315 > >> #4  0x0000000800250019 in .rtld_start () at > >> /usr/local/share/deploy-tools/RELENG_12/src/libexec/rtld-elf/amd64/rtld_start.S:39 > >> #5  0x0000000000000000 in ?? () > >> > >> So please correct me if I'm comletely wrong, but the problem here seems > >> to be reproducably rtld-elf related. > >> Unfortunately I don't know anything about object files and linkers and > >> the related fundamental stuff. > > If you do not know about linkers, why do you claim that the problem > > is related to rtld ? > > > >> But maybe someone else has an idea what's going wrong here? > > > > The fault happens during zeroing of bss. Most likely it is due to some > > strangeness of the object being loaded. For diagnostic, show > > the output of "readelf -a libcrypto.so.111". > > Thanks for your help! > I just guess it's rtld related, since I obviously misinterpreted the > backtrace. Reverting topic change… > > ELF Header: > Magic: 7f 45 4c 46 02 01 01 09 00 00 00 00 00 00 00 00 > Class: ELF64 > Data: 2's complement, little endian > Version: 1 (current) > OS/ABI: FreeBSD > ABI Version: 0 > Type: DYN (Shared object file) > Machine: Advanced Micro Devices x86-64 > Version: 0x1 > Entry point address: 0x116000 > Start of program headers: 64 (bytes into file) > Start of section headers: 3090864 (bytes into file) > Flags: 0 > Size of this header: 64 (bytes) > Size of program headers: 56 (bytes) > Number of program headers: 8 > Size of section headers: 64 (bytes) > Number of section headers: 29 > Section header string table index: 28 > > Elf file type is DYN (Shared object file) > Entry point 0x116000 > There are 8 program headers, starting at offset 64 > > Program Headers: > Type Offset VirtAddr PhysAddr > FileSiz MemSiz Flg Align > PHDR 0x0000000000000040 0x0000000000000040 0x0000000000000040 > 0x00000000000001c0 0x00000000000001c0 R 0x8 > LOAD 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000115a7c 0x0000000000115a7c R 0x1000 > LOAD 0x0000000000116000 0x0000000000116000 0x0000000000116000 > 0x00000000001acb20 0x00000000001acb20 R E 0x1000 > LOAD 0x00000000002c3000 0x00000000002c3000 0x00000000002c3000 > 0x000000000002f790 0x00000000000325e0 RW 0x1000 > DYNAMIC 0x00000000002f1a80 0x00000000002f1a80 0x00000000002f1a80 > 0x0000000000000190 0x0000000000000190 RW 0x8 > GNU_RELRO 0x00000000002c9000 0x00000000002c9000 0x00000000002c9000 > 0x0000000000029790 0x0000000000029790 R 0x1 > GNU_EH_FRAME 0x00000000000d0050 0x00000000000d0050 0x00000000000d0050 > 0x000000000000bc74 0x000000000000bc74 R 0x4 > GNU_STACK 0x0000000000000000 0x0000000000000000 0x0000000000000000 > 0x0000000000000000 0x0000000000000000 RW 0 > > Section to Segment mapping: > Segment Sections... > 00 > 01 (null) (null) (null) (null) (null) (null) (null) (null) > (null) (null) (null) (null) (null) (null) (null) (null) (null) (null) > (null) (null) (null) (null) (null) (null) (null) (null) (null) (null) > 02 > 03 > 04 > 05 > 06 > 07 (null) (null) (null) (null) (null) (null) (null) (null) > (null) (null) (null) (null) (null) (null) (null) (null) (null) (null) > (null) (null) (null) (null) (null) (null) (null) (null) (null) (null) > There are 29 section headers, starting at offset 0x2f29b0: > > Section Headers: > [Nr] Name Type Address Offset > Size EntSize Flags Link Info Align > [ 0] (null) NULL 0000000000000000 00000000 > 0000000000000000 0000000000000000 0 0 0 > : > : > : > [28] (null) NULL 0000000000000000 00000000 > 0000000000000000 0000000000000000 0 0 0 > Key to Flags: > W (write), A (alloc), X (execute), M (merge), S (strings) > I (info), L (link order), G (group), x (unknown) > O (extra OS processing required) o (OS specific), p (processor specific) > > There is no dynamic section in this file. The object is clearly corrupted.