Date: Tue, 7 Dec 2010 16:59:19 +0100 (CET) From: Oliver Fromme <olli@lurza.secnetix.de> To: freebsd-fs@FreeBSD.ORG Subject: Re: NFS hangs (7.3) Message-ID: <201012071559.oB7FxJqf048700@lurza.secnetix.de> In-Reply-To: <201011171705.oAHH5age003849@lurza.secnetix.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Oliver Fromme wrote: > I've got a problem on a server farm. Every now and then, > some NFS mounts hang. This happens after a few days or > after a few weeks. All processes trying to access files > from the hanging mount go to state "D" and freeze. The > only way to resolve the problem is to reboot the server. > [...] > The machine is quite busy. The hangs seem to always occur > in the night when lots of cron jobs are running. The machine > has 221 NFS mounts and 26 nullfs mounts, and it has 26 jails, > if that matters. All NFS shares are mounted from a virtual > filer running on a NetApp filer. The mounts use the default > settings, so they should be v3 TCP (this is the default, > right?). The only extra option we use is -L in order to > "fake" locking locally. Shortly after I posted the above, I found out that stable/7 does *not* use TCP by default. tcpdump showed that NFS was using UDP. So I changed the mamangement scripts to force TCP when mounting NFS shares. So far, there were no further hangs. It's still possible that it might occur in the future (sometimes it took a few weeks to produce a hang), but it seems as if the problem is really fixed now. In case it happens again, I will follow Rick's and Kostik's advice (procstat -ka, ps axHl etc.). Thanks! Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "... there are two ways of constructing a software design: One way is to make it so simple that there are _obviously_ no deficiencies and the other way is to make it so complicated that there are no _obvious_ deficiencies." -- C.A.R. Hoare, ACM Turing Award Lecture, 1980
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201012071559.oB7FxJqf048700>