From owner-freebsd-current@freebsd.org Sun Mar 13 18:32:21 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8FADAACFF7B for ; Sun, 13 Mar 2016 18:32:21 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F034D48 for ; Sun, 13 Mar 2016 18:32:21 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version; bh=qNzsib3vNc8uw7HWYJ/Kwm/QVd4ZR0miQMMQjBEXNco=; b=BYQlUNr5lCb9dQmweUq1Vm4SZX g1hfwpTq2d3G6C+ONMGElkf0Z9rNJEBm1pgKz3TjCJlTYbh2+UVsDoGJRmzA7y9FjccH2hl+Kh4xS 0iWoVUuriD2JglNtdlBI/vbQEk2GXutS78+xITtJxR2jAlRJYA80rDVQTsLxSK5zjmic=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:47795 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.86_2 (FreeBSD)) (envelope-from ) id 1afAoO-000CI7-J6; Sun, 13 Mar 2016 13:32:20 -0500 Received: from cpe-70-113-41-107.austin.res.rr.com ([70.113.41.107]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Sun, 13 Mar 2016 13:32:20 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Sun, 13 Mar 2016 13:32:20 -0500 From: Larry Rosenman To: Konstantin Belousov Cc: Freebsd current Subject: Re: Crashes in libthr? In-Reply-To: <20160313181234.GI1741@kib.kiev.ua> References: <70c63304fceddc1a91e16d63152ff33a@thebighonker.lerctr.org> <20160313181234.GI1741@kib.kiev.ua> Message-ID: <276f6b11da9a005256d24fa5b37d7d5f@thebighonker.lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.1.4 X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 Mar 2016 18:32:21 -0000 On 2016-03-13 13:12, Konstantin Belousov wrote: > On Sun, Mar 13, 2016 at 11:16:20AM -0500, Larry Rosenman wrote: >> I updated one of my servers, and WHILE DOING THE INSTALLWORLD, I get >> segfaults. >> >> ANY multithreaded program crashes. >> >> I reverted libthr, and it's fine. >> >> borg.lerctr.org / # gdb -c zfs.core /sbin/zfs >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and >> you >> are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> Core was generated by `zfs'. >> Program terminated with signal 11, Segmentation fault. >> Reading symbols from /lib/libjail.so.1...Reading symbols from >> /usr/lib/debug//lib/libjail.so.1.debug...done. >> done. >> Loaded symbols for /lib/libjail.so.1 >> Reading symbols from /lib/libnvpair.so.2...Reading symbols from >> /usr/lib/debug//lib/libnvpair.so.2.debug...done. >> done. >> Loaded symbols for /lib/libnvpair.so.2 >> Reading symbols from /lib/libuutil.so.2...Reading symbols from >> /usr/lib/debug//lib/libuutil.so.2.debug...done. >> done. >> Loaded symbols for /lib/libuutil.so.2 >> Reading symbols from /lib/libzfs_core.so.2...Reading symbols from >> /usr/lib/debug//lib/libzfs_core.so.2.debug...done. >> done. >> Loaded symbols for /lib/libzfs_core.so.2 >> Reading symbols from /lib/libzfs.so.2...Reading symbols from >> /usr/lib/debug//lib/libzfs.so.2.debug...done. >> done. >> Loaded symbols for /lib/libzfs.so.2 >> Reading symbols from /lib/libc.so.7...Reading symbols from >> /usr/lib/debug//lib/libc.so.7.debug...done. >> done. >> Loaded symbols for /lib/libc.so.7 >> Reading symbols from /lib/libmd.so.6...Reading symbols from >> /usr/lib/debug//lib/libmd.so.6.debug...done. >> done. >> Loaded symbols for /lib/libmd.so.6 >> Reading symbols from /lib/libumem.so.2...Reading symbols from >> /usr/lib/debug//lib/libumem.so.2.debug...done. >> done. >> Loaded symbols for /lib/libumem.so.2 >> Reading symbols from /lib/libutil.so.9...Reading symbols from >> /usr/lib/debug//lib/libutil.so.9.debug...done. >> done. >> Loaded symbols for /lib/libutil.so.9 >> Reading symbols from /lib/libm.so.5...Reading symbols from >> /usr/lib/debug//lib/libm.so.5.debug...done. >> done. >> Loaded symbols for /lib/libm.so.5 >> Reading symbols from /lib/libavl.so.2...Reading symbols from >> /usr/lib/debug//lib/libavl.so.2.debug...done. >> done. >> Loaded symbols for /lib/libavl.so.2 >> Reading symbols from /lib/libbsdxml.so.4...Reading symbols from >> /usr/lib/debug//lib/libbsdxml.so.4.debug...done. >> done. >> Loaded symbols for /lib/libbsdxml.so.4 >> Reading symbols from /lib/libgeom.so.5...Reading symbols from >> /usr/lib/debug//lib/libgeom.so.5.debug...done. >> done. >> Loaded symbols for /lib/libgeom.so.5 >> Reading symbols from /lib/libz.so.6...Reading symbols from >> /usr/lib/debug//lib/libz.so.6.debug...done. >> done. >> Loaded symbols for /lib/libz.so.6 >> Reading symbols from /lib/libthr.so.3...done. >> Loaded symbols for /lib/libthr.so.3 > Why all libs have debug symbols, while your most interesting one, > libthr.so.3, does not ? > >> Reading symbols from /lib/libsbuf.so.6...Reading symbols from >> /usr/lib/debug//lib/libsbuf.so.6.debug...done. >> done. >> Loaded symbols for /lib/libsbuf.so.6 >> Reading symbols from /libexec/ld-elf.so.1...done. >> Loaded symbols for /libexec/ld-elf.so.1 >> #0 0x0000000802703f81 in __pthread_cxa_finalize () from >> /lib/libthr.so.3 >> [New LWP 100957] >> (gdb) bt >> #0 0x0000000802703f81 in __pthread_cxa_finalize () from >> /lib/libthr.so.3 >> #1 0x0000000802703e85 in __pthread_cxa_finalize () from >> /lib/libthr.so.3 >> #2 0x0000000802707052 in ?? () from /lib/libthr.so.3 >> #3 0x000000080063fc00 in ?? () >> #4 0x00007fffffffe638 in ?? () >> #5 0x00007fffffffe5b0 in ?? () >> #6 0x00000008026f8fd6 in atoi@plt () from /lib/libthr.so.3 >> #7 0x00007fffffffe5b0 in ?? () >> #8 0x000000080061adfd in r_debug_state () from /libexec/ld-elf.so.1 >> Previous frame inner to this frame (corrupt stack?) >> (gdb) >> >> old SVN: r296103 >> new SVN: r296796M >> (The M is a nd6 patch from markj@) >> >> this was a FULL buildworld/buildkernel. > > If you cd lib/libthr and do > make clean all install DEBUG_FLAGS=-g > on the broken world, does it fix the problem ? If not, do debugging > symbols from libthr appear accessible to gdb at least ? Try this to > get useful backtrace with source lines information. ar crashes linking the library...... -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 7011 W Parmer Ln, Apt 1115, Austin, TX 78729-6961