Date: Wed, 2 Oct 2002 23:51:41 -0400 (EDT) From: Robert Watson <rwatson@FreeBSD.org> To: current@FreeBSD.org Subject: NFS hang on rmdir(2) with 5.0-current client, server Message-ID: <Pine.NEB.3.96L.1021002234608.55209A-100000@fledge.watson.org>
next in thread | raw e-mail | index | archive | help
Running into an odd (and apparently recent) problem involving rmdir(2) and NFS. I have a diskless box started using pxeboot: NFS /, MFS /var, MFS /tmp, recent 5.0-CURRENT. Attempt to rmdir /usr/local (on NFS) results in NFS hanging. It appears to send out the RPC, but no response comes back from the (also recent) FreeBSD 5.0 NFS server. Here's the quicky on the client: crash1# which rmdir /bin/rmdir crash1# rmdir /usr/local load: 0.04 cmd: rmdir 466 [sbwait] 0.00u 0.00s 0% 76k ~^B Stopped at siointr1+0xf4: movl $0,brk_state1.0 db> trace 466 mi_switch(c0baa8d8,1f4,c02eb8f0,c0baa840,3d4) at mi_switch+0x238 msleep(c1996eec,0,58,c04bdba0,1f4) at msleep+0x474 sbwait(c1996ed4,c0518e80,c0bac630,c0bac630,f4240) at sbwait+0x51 soreceive(c1996e88,0,c86a1aec,c86a1b38,0) at soreceive+0x306 nfs_receive(c1bcb800,c86a1b3c,c86a1b38,c199d000,c0e34844) at nfs_receive+0x4c0 nfs_reply(c1bcb800,c18d7850,c0bb3400,c1bcb800,44) at nfs_reply+0x5e nfs_request(c19d94a0,c0bb3d00,d,c0baa840,c1cdb280) at nfs_request+0x2dc nfs_rmdir(c86a1c04,c04e8600,c19d94a0,c1bc8de0,c86a1c98) at nfs_rmdir+0x11a VOP_RMDIR(c19d94a0,c1bc8de0,c86a1c98,c86a1c98,0) at VOP_RMDIR+0x33 kern_rmdir(c0baa840,bfbffd3e,0,c86a1d40,c044ff4e) at kern_rmdir+0x1be rmdir(c0baa840,c86a1d10,c04dedc9,418,1) at rmdir+0x22 syscall(2f,2f,2f,0,0) at syscall+0x2be Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall (137, FreeBSD ELF32, rmdir), eip = 0x804886f, esp = 0xbfbffbfc, ebp = 0xbfbffc20 --- db> I can't do much debugging on the server just yet as it's actually the development box and I don't have remote serial console access / debugger access / etc on it. It strikes me that either the client is sending an invalid request that the server blocks, or the server is getting unhappy about the request. The server NFS processes seem to be idle, so at the very least the server doesn't care: 0 354 1 85 106 0 1104 776 select Is ?? 0:00.12 nfsd: master 0 356 354 0 4 0 1012 628 nfsd I ?? 1:16.66 nfsd: server 0 357 354 0 4 0 1012 628 nfsd I ?? 0:00.18 nfsd: server 0 358 354 0 4 0 1012 628 nfsd I ?? 0:00.03 nfsd: server 0 359 354 0 4 0 1012 628 nfsd I ?? 0:00.02 nfsd: server I've attached the tcpdump output from the server interface below, if anyone is interested. Has anyone else experienced this problem? I believe / is mounted using NFSv3, and quite possibly "soft": file:/home on /home (nfs) file:/projects on /projects (nfs) It looks like the client is basically hung waiting for an RPC response. I'd be glad to provide more debugging information if someone can point me in the right direction. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories 23:46:09.588003 192.168.50.10.1260868882 > 192.168.50.1.2049: 152 lookup [|nfs] 4500 00b4 12b2 0000 4011 822b c0a8 320a c0a8 3201 03ff 0801 00a0 22d0 4b27 5512 0000 0000 0000 0002 0001 86a3 0000 0002 0000 0004 0000 0001 0000 0044 0000 0000 0000 0000 0000 0000 0000 0000 0000 000c 0000 23:46:09.588168 192.168.50.1.2049 > 192.168.50.10.1260868882: reply ok 128 lookup [|nfs] 4500 009c 7e3b 0000 4011 16ba c0a8 3201 c0a8 320a 0801 03ff 0088 15d3 4b27 5512 0000 0001 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 4f29 ba3b db0f a103 0c00 0000 3356 0a00 6e81 461d 0000 0000 0000 23:46:09.588675 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir [|nfs] 4500 00b4 12b3 0000 4011 822a c0a8 320a c0a8 3201 03ff 0801 00a0 22c4 4b27 5513 0000 0000 0000 0002 0001 86a3 0000 0002 0000 000f 0000 0001 0000 0044 0000 0000 0000 0000 0000 0000 0000 0000 0000 000c 0000 23:46:10.592213 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir [|nfs] 4500 00b4 12b4 0000 4011 8229 c0a8 320a c0a8 3201 03ff 0801 00a0 22c4 4b27 5513 0000 0000 0000 0002 0001 86a3 0000 0002 0000 000f 0000 0001 0000 0044 0000 0000 0000 0000 0000 0000 0000 0000 0000 000c 0000 23:46:12.602134 192.168.50.10.1260868883 > 192.168.50.1.2049: 152 rmdir [|nfs] 4500 00b4 12b5 0000 4011 8228 c0a8 320a c0a8 3201 03ff 0801 00a0 22c4 4b27 5513 0000 0000 0000 0002 0001 86a3 0000 0002 0000 000f 0000 0001 0000 0044 0000 0000 0000 0000 0000 0000 0000 0000 0000 000c 0000 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.NEB.3.96L.1021002234608.55209A-100000>