From owner-freebsd-hackers@FreeBSD.ORG  Thu Apr  3 15:59:15 2014
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 2633FD5A;
 Thu,  3 Apr 2014 15:59:15 +0000 (UTC)
Received: from mail.tdx.com (mail.tdx.com [62.13.128.18])
 by mx1.freebsd.org (Postfix) with ESMTP id A5D8EDB0;
 Thu,  3 Apr 2014 15:59:14 +0000 (UTC)
Received: from Mail-PC.tdx.co.uk (storm.tdx.co.uk [62.13.130.251])
 (authenticated bits=0)
 by mail.tdx.com (8.14.3/8.14.3/) with ESMTP id s33Fx7HP030102
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
 Thu, 3 Apr 2014 16:59:08 +0100 (BST)
Date: Thu, 03 Apr 2014 16:59:07 +0100
From: Karl Pielorz <kpielorz_lst@tdx.co.uk>
To: John Baldwin <jhb@freebsd.org>
Subject: Re: Stuck CLOSED sockets / sshd / zombies...
Message-ID: <6F730B3126CC5AE636D1E2A0@Mail-PC.tdx.co.uk>
In-Reply-To: <201404031103.41171.jhb@freebsd.org>
References: <3FE645E9723756F22EF901AE@Mail-PC.tdx.co.uk>
 <201404021405.56878.jhb@freebsd.org>
 <9FDC091D98AB2CF92DE4399F@Mail-PC.tdx.co.uk>
 <201404031103.41171.jhb@freebsd.org>
X-Mailer: Mulberry/4.0.8 (Win32)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Cc: freebsd-hackers@freebsd.org
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 03 Apr 2014 15:59:15 -0000



--On 03 April 2014 11:03 -0400 John Baldwin <jhb@freebsd.org> wrote:

> Hmm, that fd value doesn't make any sense now.  Do you have the backtrace
> for that process?  The fd may show up in the arguments to kern_readv().

Ok, bt shows:

"
#0  sched_switch (td=0xfffff800238bb920, newtd=<value optimized out>, 
flags=<value optimized out>) at ../../../kern/sched_ule.c:1938
#1  0xffffffff808be76e in mi_switch (flags=260, newtd=0x0) at 
../../../kern/kern_synch.c:494
#2  0xffffffff808f9002 in sleepq_catch_signals (wchan=0xfffff80002da4c24, 
pri=104) at ../../../kern/subr_sleepqueue.c:429
#3  0xffffffff808f8eaf in sleepq_wait_sig (wchan=0x0, pri=0) at 
../../../kern/subr_sleepqueue.c:634
#4  0xffffffff808be195 in _sleep (ident=<value optimized out>, lock=<value 
optimized out>, priority=360, wmesg=0xffffffff80efbd30 "sbwait",
    sbt=<value optimized out>, pr=0, flags=<value optimized out>) at 
../../../kern/kern_synch.c:254
#5  0xffffffff8092328c in sbwait (sb=<value optimized out>) at 
../../../kern/uipc_sockbuf.c:130
#6  0xffffffff80926b44 in soreceive_generic (so=0xfffff80002da4ae0, 
psa=0x0, uio=0xfffffe0000341ab0, mp0=0x0, controlp=0x0, flagsp=0x0)
    at ../../../kern/uipc_socket.c:1496
#7  0xffffffff8090346b in dofileread (td=0xfffff800238bb920, fd=8, 
fp=0xfffff80002cf86e0, auio=0xfffffe0000341ab0, offset=<value optimized 
out>, flags=0)
    at file.h:295


#8  0xffffffff809031a5 in kern_readv (td=0xfffff800238bb920, fd=8, 
auio=0xfffffe0000341ab0) at ../../../kern/sys_generic.c:256


#9  0xffffffff80903133 in sys_read (td=<value optimized out>, uap=<value 
optimized out>) at ../../../kern/sys_generic.c:171
#10 0xffffffff80c96cd7 in amd64_syscall (td=0xfffff800238bb920, traced=0) 
at subr_syscall.c:134
#11 0xffffffff80c7d3fb in Xfast_syscall () at 
../../../amd64/amd64/exception.S:391
#12 0x000000080320d9ea in ?? ()
"

So, fd=8? - fstat seems to show that as:

"
USER   CMD     PID  FD MOUNT  INUM MODE         SZ|DV R/W
root   sshd   4346   8* local stream fffff80002e55c30 <-> fffff80002e552d0
...
root   sshd   4344   4* local stream fffff80002e552d0 <-> fffff80002e55c30
"

Netstat shows those as:

"
Active UNIX domain sockets
Address  Type   Recv-Q Send-Q  Inode Conn     Refs  Nextref Addr
fffff80002e55c30 stream      0    0    0 fffff80002e552d0        0        0
fffff80002e552d0 stream      0    0    0 fffff80002e55c30        0        0
"

Let me know what you want running next, maybe something more sensible, or 
useful than what I just ran above :-)

Regards,

-Karl