Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 9 Apr 2014 00:23:19 +0300
From:      Konstantin Belousov <kostikbel@gmail.com>
To:        Karl Pielorz <kpielorz_lst@tdx.co.uk>
Cc:        freebsd-hackers@freebsd.org
Subject:   Re: Stuck CLOSED sockets / sshd / zombies...
Message-ID:  <20140408212319.GC21331@kib.kiev.ua>
In-Reply-To: <92366925229B4C5B21B04D81@study64.tdx.co.uk>
References:  <3FE645E9723756F22EF901AE@Mail-PC.tdx.co.uk> <20140408164353.GB21331@kib.kiev.ua> <277FA3F7B4E7A98921F4D631@study64.tdx.co.uk> <201404081533.53990.jhb@freebsd.org> <92366925229B4C5B21B04D81@study64.tdx.co.uk>

index | next in thread | previous in thread | raw e-mail

[-- Attachment #1 --]
On Tue, Apr 08, 2014 at 10:06:36PM +0100, Karl Pielorz wrote:
> 
> 
> --On 8 April 2014 15:33:53 -0400 John Baldwin <jhb@freebsd.org> wrote:
> 
> > I think this patch only changes debug info, so I think you can just build
> > and install and re-attach gdb without having to restart sshd.  Also, this
> > patch should only require you to build and install libexec/rtld-elf.
> 
> Ok, patched, re-attached and re-ran bt (output is below). I've got gdb from 
> ports building on that machine but I probably won't be able to do any more 
> with it tonight.
> 
> -Karl
> 
> ---
> 
> [Switching to LWP 100218]
> 0x00000008038ea89c in __error () from /lib/libthr.so.3
> (gdb) bt
> #0  0x00000008038ea89c in __error () from /lib/libthr.so.3
> #1  0x00000008038e104f in __thr_rwlock_rdlock (rwlock=0x803af9480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #2  0x00000008038e821c in _thr_rtld_rlock_acquire (lock=0x803af9480) at 
> atomic.h:143
> #3  0x000000080064f9a2 in _rtld_get_stack_prot () from /libexec/ld-elf.so.1
> #4  0x00000008006498c9 in r_debug_state () from /libexec/ld-elf.so.1
> #5  0x00000008006470cd in .text () from /libexec/ld-elf.so.1
> #6  0x000000000041072c in grace_alarm_handler (sig=-17504) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:378
> #7  <signal handler called>
> #8  0x00000008038ea89c in __error () from /lib/libthr.so.3
> #9  0x00000008038e104f in __thr_rwlock_rdlock (rwlock=0x803af9480, 
> flags=<value optimized out>, tsp=<value optimized out>)
>     at /usr/src/lib/libthr/thread/thr_umtx.c:277
> #10 0x00000008038e821c in _thr_rtld_rlock_acquire (lock=0x803af9480) at 
> atomic.h:143
> #11 0x000000080064f9a2 in _rtld_get_stack_prot () from /libexec/ld-elf.so.1
> #12 0x00000008006498c9 in r_debug_state () from /libexec/ld-elf.so.1
> #13 0x00000008006470cd in .text () from /libexec/ld-elf.so.1
> #14 0x000000000042f9dd in sshpam_sigchld_handler (sig=<value optimized 
> out>) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:152
> #15 <signal handler called>
> #16 0x0000000800653aea in _rtld_atfork_post () from /libexec/ld-elf.so.1
> #17 0x000000080064a1eb in dlclose () from /libexec/ld-elf.so.1
> #18 0x0000000800edd121 in openpam_clear_chains () from /usr/lib/libpam.so.5
> #19 0x0000000800edd0bc in openpam_clear_chains () from /usr/lib/libpam.so.5
> #20 0x0000000800edd0bc in openpam_clear_chains () from /usr/lib/libpam.so.5
> #21 0x0000000800edd061 in openpam_clear_chains () from /usr/lib/libpam.so.5
> #22 0x0000000800ed99e7 in pam_end () from /usr/lib/libpam.so.5
> #23 0x000000000042e15d in sshpam_cleanup () at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/auth-pam.c:614
> #24 0x000000000041d58f in do_cleanup (authctxt=0x80401a600) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/session.c:2732
> #25 0x000000000041064f in ssh_cleanup_exit (i=255) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:2545
> #26 0x0000000000428f83 in mm_request_receive (sock=<value optimized out>, 
> m=<value optimized out>)
>     at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor_wrap.c:153
> #27 0x0000000000427e26 in monitor_read (pmonitor=0x804022220, 
> ent=0x6465a0, pent=0x7fffffffd0c0)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:593
> #28 0x0000000000427b49 in monitor_child_preauth (_authctxt=<value 
> optimized out>, pmonitor=0x804022220)
>     at /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/monitor.c:387
> #29 0x000000000040fd15 in main (ac=<value optimized out>, av=<value 
> optimized out>) at 
> /usr/src/secure/usr.sbin/sshd/../../../crypto/openssh/sshd.c:679

Ok, so the patch sort of worked, but your rtld does not have debugging
information for compiled .c files.  Please, in the patched tree, do
the following:

cd libexec/rtld-elf
make clean
make DEBUG_FLAGS=-g DEBUG=-DDEBUG
make install

and then re-do the backtracing.

[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (FreeBSD)

iQIcBAEBAgAGBQJTRGjGAAoJEJDCuSvBvK1BOuAQAJRg17vxmNhZalUk2fS0AEoG
ElU7P/gzTeS3Y3NQ/vDm7PHSkSJ4H+3MvisyR3s/1OPoO7NGxhZ8rIKBh7sfmTUh
soy+RlY3pawhlwVO6/+VMk2AUPUe75PIyDPyY4A4T+8aqDm9kxcDKtszwGbYW0vY
RpZEHcm77sn5CC64uZV9ppcuis/uKtZi/VWN69Uvt9UzWXA8MsxQYXXBUg3FCs/o
cIua0+3nrhmSkYz/euD8hyyUUoCNefErtC3NMptBHtQ22InKpFvLoZEL+d77UG9x
cDunUAm4LUaTJzzvzcwtxw/bTTplnYzOXqCPLFUouSw4HcNZhPkrJAPDjkMApHMn
27uM6ONvP9ZXitAGJ8o7O1OqFoK/8cURaq7+JmTmpgi2ZOvTDl+ga7vGVPUa/vDb
WbHzKg5v/JS3B6IhbNmhPI07VK0OsMQPDYVaDeEBvvEniegzj9kTa2W0vkNVIqHy
RyW53DiYPTk9PdW093IqVmW36xJEfCk8NaCse0D7p8GwdBnYRBlrgJGZwgJEgG0n
Sm0Ri27g1PLI/W5wW+E08OtMWjgHNtroDua2DAH0OTDItJW5NhQLIvpBFz7mD0Ij
ARFd3Ld67XQtH3RYAfMwNkEfgv8Hzufvzr87FQx5mdZHiAyOURJQcdPfdk5X7r+B
I9/MhdVcosf+cHOZu/um
=Hsia
-----END PGP SIGNATURE-----
help

Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140408212319.GC21331>