Date: Wed, 2 Oct 2002 23:51:41 -0400 (EDT) From: Robert Watson <rwatson@FreeBSD.org> To: current@FreeBSD.org Subject: NFS hang on rmdir(2) with 5.0-current client, server Message-ID: <Pine.NEB.3.96L.1021002234608.55209A-100000@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
Running into an odd (and apparently recent) problem involving rmdir(2) and
NFS. I have a diskless box started using pxeboot: NFS /, MFS /var, MFS
/tmp, recent 5.0-CURRENT. Attempt to rmdir /usr/local (on NFS) results in
NFS hanging. It appears to send out the RPC, but no response comes back
from the (also recent) FreeBSD 5.0 NFS server. Here's the quicky on the
client:
crash1# which rmdir
/bin/rmdir
crash1# rmdir /usr/local
load: 0.04 cmd: rmdir 466 [sbwait] 0.00u 0.00s 0% 76k
~^B
Stopped at siointr1+0xf4: movl $0,brk_state1.0
db> trace 466
mi_switch(c0baa8d8,1f4,c02eb8f0,c0baa840,3d4) at mi_switch+0x238
msleep(c1996eec,0,58,c04bdba0,1f4) at msleep+0x474
sbwait(c1996ed4,c0518e80,c0bac630,c0bac630,f4240) at sbwait+0x51
soreceive(c1996e88,0,c86a1aec,c86a1b38,0) at soreceive+0x306
nfs_receive(c1bcb800,c86a1b3c,c86a1b38,c199d000,c0e34844) at
nfs_receive+0x4c0
nfs_reply(c1bcb800,c18d7850,c0bb3400,c1bcb800,44) at nfs_reply+0x5e
nfs_request(c19d94a0,c0bb3d00,d,c0baa840,c1cdb280) at nfs_request+0x2dc
nfs_rmdir(c86a1c04,c04e8600,c19d94a0,c1bc8de0,c86a1c98) at nfs_rmdir+0x11a
VOP_RMDIR(c19d94a0,c1bc8de0,c86a1c98,c86a1c98,0) at VOP_RMDIR+0x33
kern_rmdir(c0baa840,bfbffd3e,0,c86a1d40,c044ff4e) at kern_rmdir+0x1be
rmdir(c0baa840,c86a1d10,c04dedc9,418,1) at rmdir+0x22
syscall(2f,2f,2f,0,0) at syscall+0x2be
Xint0x80_syscall() at Xint0x80_syscall+0x1d
--- syscall (137, FreeBSD ELF32, rmdir), eip = 0x804886f, esp =
0xbfbffbfc, ebp = 0xbfbffc20 ---
db>
I can't do much debugging on the server just yet as it's actually the
development box and I don't have remote serial console access / debugger
access / etc on it. It strikes me that either the client is sending an
invalid request that the server blocks, or the server is getting unhappy
about the request. The server NFS processes seem to be idle, so at the
very least the server doesn't care:
0 354 1 85 106 0 1104 776 select Is ?? 0:00.12 nfsd:
master
0 356 354 0 4 0 1012 628 nfsd I ?? 1:16.66 nfsd:
server
0 357 354 0 4 0 1012 628 nfsd I ?? 0:00.18 nfsd:
server
0 358 354 0 4 0 1012 628 nfsd I ?? 0:00.03 nfsd:
server
0 359 354 0 4 0 1012 628 nfsd I ?? 0:00.02 nfsd:
server
I've attached the tcpdump output from the server interface below, if
anyone is interested. Has anyone else experienced this problem?
I believe / is mounted using NFSv3, and quite possibly "soft":
file:/home on /home (nfs)
file:/projects on /projects (nfs)
It looks like the client is basically hung waiting for an RPC response.
I'd be glad to provide more debugging information if someone can point me
in the right direction.
Robert N M Watson FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org Network Associates Laboratories
23:46:09.588003 192.168.50.10.1260868882 > 192.168.50.1.2049: 152 lookup
[|nfs]
4500 00b4 12b2 0000 4011 822b c0a8 320a
c0a8 3201 03ff 0801 00a0 22d0 4b27 5512
0000 0000 0000 0002 0001 86a3 0000 0002
0000 0004 0000 0001 0000 0044 0000 0000
0000 0000 0000 0000 0000 0000 0000 000c
0000
23:46:09.588168 192.168.50.1.2049 > 192.168.50.10.1260868882: reply ok 128
lookup [|nfs]
4500 009c 7e3b 0000 4011 16ba c0a8 3201
c0a8 320a 0801 03ff 0088 15d3 4b27 5512
0000 0001 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 4f29 ba3b db0f a103
0c00 0000 3356 0a00 6e81 461d 0000 0000
0000
23:46:09.588675 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir
[|nfs]
4500 00b4 12b3 0000 4011 822a c0a8 320a
c0a8 3201 03ff 0801 00a0 22c4 4b27 5513
0000 0000 0000 0002 0001 86a3 0000 0002
0000 000f 0000 0001 0000 0044 0000 0000
0000 0000 0000 0000 0000 0000 0000 000c
0000
23:46:10.592213 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir
[|nfs]
4500 00b4 12b4 0000 4011 8229 c0a8 320a
c0a8 3201 03ff 0801 00a0 22c4 4b27 5513
0000 0000 0000 0002 0001 86a3 0000 0002
0000 000f 0000 0001 0000 0044 0000 0000
0000 0000 0000 0000 0000 0000 0000 000c
0000
23:46:12.602134 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir
[|nfs]
4500 00b4 12b5 0000 4011 8228 c0a8 320a
c0a8 3201 03ff 0801 00a0 22c4 4b27 5513
0000 0000 0000 0002 0001 86a3 0000 0002
0000 000f 0000 0001 0000 0044 0000 0000
0000 0000 0000 0000 0000 0000 0000 000c
0000
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1021002234608.55209A-100000>