Date: Sat, 13 Jun 2020 19:14:38 +0300 From: Konstantin Belousov <kostikbel@gmail.com> To: Dewayne Geraghty <dewayne@heuristicsystems.com.au> Cc: freebsd-stable@freebsd.org, emaste@freebsd.org Subject: Re: Upgrading to 12.1S 362003 - a few issues Message-ID: <20200613161438.GT48478@kib.kiev.ua> In-Reply-To: <355bf1bc-82cf-a558-16df-1c7c2503c7a8@heuristicsystems.com.au> References: <978b8cb5-de88-4264-25ec-e7ecaaf80c75@heuristicsystems.com.au> <20200613013450.GS48478@kib.kiev.ua> <355bf1bc-82cf-a558-16df-1c7c2503c7a8@heuristicsystems.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jun 13, 2020 at 12:34:46PM +1000, Dewayne Geraghty wrote:
> Hi Konstantin,
> I did try ktrace/kdump but kdump complained of "data too short". Using
> your suggestion about LD_DEBUG nearly caused me to fall off my chair. I
> think this is most relevant:
>
> # setenv LD_DEBUG 1
> # ktrace -f /tmp/sq3.kt /usr/local/sbin/squid start
> /libexec/ld-elf.so.1 is initialized, base address = 0x8aded000
> RTLD dynamic = 0x8ae0f6c0
> RTLD pltgot = 0
> initializing thread locks
> _rtld_thread_init: done
> processing main program's program header
> note osrel 1201517
> note fctl0 0
> note crt_no_init
> AT_EXECPATH 0xffffdfe0 /usr/bin/ktrace
> obj_main path /usr/bin/ktrace
> Filling in DT_DEBUG entry
> Ignoring d_tag 1879048186 = 0x6ffffffa
> /usr/bin/ktrace valid_hash_sysv 1 valid_hash_gnu 0 dynsymcount 30
> lm_init("(null)")
> loading LD_PRELOAD libraries
> loading needed objects
> Searching for "libc.so.7"
> search_library_pathfds('libc.so.7', '(null)', fdp)
> lm_find("/usr/bin/ktrace", "/lib")
> lmp_find("/usr/bin/ktrace")
> lmp_find("$DEFAULT$")
> Trying "/lib/libc.so.7"
> Opened "/lib/libc.so.7", fd 3
> loading "/lib/libc.so.7"
> Ignoring d_tag 1879048186 = 0x6ffffffa
> /lib/libc.so.7 valid_hash_sysv 1 valid_hash_gnu 1 dynsymcount 3126
> 0x6d20f000 .. 0x6d3eafff: /lib/libc.so.7
> checking for required versions
> initializing initial thread local storage offsets
> relocating "/usr/bin/ktrace"
> reloc_jmpslot: *0x4b177c04 = 0x6d39bac0
> reloc_jmpslot: *0x4b177c08 = 0x6d3b6480
> ... [ Lots of these ]
> reloc_jmpslot: *0x9d4859fc = 0x9d46e560
> reloc_jmpslot: *0x9d485a00 = 0x9d47a428
> reloc_jmpslot: *0x9d485a04 = 0x9d47a448
> relocating "/usr/local/lib/heimdal/libasn1.so.8"
> relocating "/usr/local/lib/heimdal/libwind.so.0"
> relocating "/usr/local/lib/heimdal/libheimbase.so.1"
> relocating "/usr/local/lib/heimdal/libhx509.so.5"
> relocating "/usr/local/lib/heimdal/libhcrypto.so.4"
> relocating "/usr/local/lib/heimdal/libheimsqlite.so.0"
> relocating "/usr/local/lib/heimdal/libcom_err.so.1"
> relocating "/usr/local/lib/heimdal/libroken.so.18"
> relocating "/usr/local/lib/libintl.so.8"
> relocating "/usr/local/lib/heimdal/libheimntlm.so.0"
> doing copy relocations
> initializing initial thread local storage
> initializing key program variables
> "__progname": *0x73be38b8 <-- 0xffffdd44
> "environ": *0x73cbbc08 <-- 0xfffc69cc
> "__elf_aux_vector": *0x9d4a6a5c <-- 0xfffc6a4c
> resolving ifuncs
> reloc_jmpslot: *0x9d484d88 = 0x9d352cc0
> reloc_jmpslot: *0x9d484d8c = 0x9d352c70
> reloc_jmpslot: *0x9cb16e04 = 0x9d352c70
> reloc_jmpslot: *0x9cb16e0c = 0x9d352cc0
> calling init function for /lib/libc.so.7 at 0x9d47c440
> calling init function for /lib/libc.so.7 at 0x9d40fc20
> calling init function for /lib/libc.so.7 at 0x9d3f0f40
> calling init function for /lib/libthr.so.3 at 0x9cb15210
> calling init function for /lib/libthr.so.3 at 0x9cb10cb0
> _rtld_thread_init: done
> calling init function for /lib/libgcc_s.so.1 at 0x9bfad540
> calling init function for /lib/libgcc_s.so.1 at 0x9bfa3de0
> calling init function for /lib/libcxxrt.so.1 at 0x9b698480
> calling init function for /usr/lib/libc++.so.1 at 0x9ab43d90
> calling init function for /usr/lib/libc++.so.1 at 0x9aae3e40
> calling init function for /lib/libcrypt.so.5 at 0x965d3210
> calling init function for /usr/local/lib/libcrypto.so.11 at 0x96e9f420
> calling init function for /usr/local/lib/heimdal/libroken.so.18 at
> 0xa32b6f60
> calling init function for /usr/local/lib/libintl.so.8 at 0xa23cb104
> calling init function for /usr/local/lib/heimdal/libheimbase.so.1 at
> 0x9ed15de0
> calling init function for /usr/local/lib/heimdal/libcom_err.so.1 at
> 0x9fec3768
> calling init function for /usr/local/lib/heimdal/libasn1.so.8 at 0x9e23f054
> calling init function for /usr/local/lib/heimdal/libhcrypto.so.4 at
> 0x9fb5641c
> calling init function for /usr/local/lib/heimdal/libheimsqlite.so.0 at
> 0xa1b9603c
> calling init function for /usr/local/lib/heimdal/libwind.so.0 at 0x9f77a990
> calling init function for /usr/local/lib/heimdal/libhx509.so.5 at 0xa0c4e3b4
> calling init function for /usr/local/lib/heimdal/libkrb5.so.26 at 0x99527188
> calling init function for /usr/local/lib/heimdal/libheimntlm.so.0 at
> 0xa498c814
> calling init function for /usr/local/lib/heimdal/libgssapi.so.3 at
> 0x9a45bfd0
> calling init function for /usr/local/lib/libpcre.so.1 at 0x97550ca8
> calling init function for /usr/local/lib/libpcreposix.so.0 at 0x9728e518
> calling init function for /lib/libm.so.5 at 0x97139f10
> calling init function for /usr/local/lib/libssl.so.11 at 0x98579078
> calling init function for /usr/lib/libregex.so.1 at 0x95e8f0a0
> calling init function for /usr/lib/librt.so.1 at 0x953ca240
> loading filtees
> enforcing main obj relro
> transferring control to program entry point = 0x73757360
So the app code started executing.
> "atexit" in "squid" ==> 0x9d457ac0 in "libc.so.7"
> reloc_jmpslot: *0x73be32b4 = 0x9d457ac0
> "_ZSt13set_terminatePFvvE" in "squid" ==> 0x9b697a30 in "libcxxrt.so.1"
> reloc_jmpslot: *0x73be3794 = 0x9b697a30
> "strlen" in "squid" ==> 0x9d4750b0 in "libc.so.7"
> reloc_jmpslot: *0x73be3108 = 0x9d4750b0
> "malloc" in "squid" ==> 0x9d3e6620 in "libc.so.7"
> reloc_jmpslot: *0x73be3820 = 0x9d3e6620
> "memcpy" in "squid" ==> 0x9d47aa20 in "libc.so.7"
> reloc_jmpslot: *0x73be3500 = 0x9d47aa20
> "_Znwj" in "squid" ==> 0x9ab437c0 in "libc++.so.1"
> reloc_jmpslot: *0x73be3578 = 0x9ab437c0
> "free" in "squid" ==> 0x9d3e9ea0 in "libc.so.7"
> reloc_jmpslot: *0x73be3790 = 0x9d3e9ea0
> "_ZdlPv" in "squid" ==> 0x9ab438d0 in "libc++.so.1"
> reloc_jmpslot: *0x73be32b8 = 0x9ab438d0
> "getopt_long" in "squid" ==> 0x9d3515f0 in "libc.so.7"
> reloc_jmpslot: *0x73be3614 = 0x9d3515f0
> Segmentation fault
>
> I've placed the ktrace (sq3.kt.xz) and LD_DEBUG output (sq.deb.xz) at
> http://www.heuristicsystems.com/squid4K
No need, the problem occurs somewhere in user code. Try to build the
binary with debug symbols and look into the coredump with gdb.
>
> Pity about ldd, but I appreciate the complexity & the effort. I'm also
> comforted that it isn't an error, rather the evolution of ldd :)
ldd should be fixed after https://reviews.freebsd.org/D25257 is committed.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20200613161438.GT48478>