Date: Sat, 20 Jun 2009 12:47:53 +1000 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Kostik Belousov <kostikbel@gmail.com> Cc: Jilles Tjoelker <jilles@stack.nl>, freebsd-arch@FreeBSD.org Subject: Re: deadlocks with intr NFS mounts and ^Z (or: PCATCH and sleepable locks) Message-ID: <20090620121543.F29239@delplex.bde.org> In-Reply-To: <20090619194654.GC2884@deviant.kiev.zoral.com.ua> References: <20090619162328.GA79975@stack.nl> <20090619194654.GC2884@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 19 Jun 2009, Kostik Belousov wrote: > On Fri, Jun 19, 2009 at 06:23:28PM +0200, Jilles Tjoelker wrote: >> I have been having trouble with deadlocks with NFS mounts for a while, >> and I have found at least one way it can deadlock. It seems an issue >> with the sleep/lock system. >> >> NFS sleeps while holding a lockmgr lock, waiting for a reply from the >> server. When the mount is set intr, this is an interruptible sleep, so >> that interrupting signals can abort the sleep. However, this also means >> that SIGSTOP etc will suspend the thread without waking it up first, so >> it will be suspended with a lock held. >> >> If it holds the wrong locks, it is possible that the shell will not be >> able to run, and the process cannot be continued in the normal manner. >> >> Due to some other things I do not understand, it is then possible that >> the process cannot be continued at all (SIGCONT seems ignored), but in >> simple cases SIGCONT works, and things go back to normal. >> ... >> Also, making SIGSTOP and the like interrupt/restart syscalls is not >> acceptable unless you find some way to do it such that userland won't >> notice. For example, a read of 10 megabytes from a regular file with >> that much available must not return less then 10 megabytes. > > See > http://lists.freebsd.org/pipermail/freebsd-smp/2009-January/001611.html Have any fixes been applied? I now remember seeing problems like the first set above on FreeBSD cluster machines (I don't encounter "intr" nfs mounts anywhere else; mount(8) still doesn't show the "intr" option so I assume that the "intr" specified in fstab is in use on the FreeBSD machines): normal resume after ^Z on a parallel build not working, sometimes hanging the whole file system but other times recoverable after re-logging in and sending suitable SIGCONTs manually These problems seemed to go away, but right now the following problem like the second set above occurs consistently (I first noticed this last week): Script started on Sat Jun 20 02:32:51 2009 pts/0:bde@ref8-i386:~/sys7/i386/compile> sh zm ^Z [1]+ Stopped sh zm pts/0:bde@ref8-i386:~/sys7/i386/compile> % sh zm *** Stopped -- signal 18 *** Stopped -- signal 18 *** Stopped -- signal 18 *** Signal 1 *** Signal 1 *** Signal 1 `all' not remade because of errors. linking kernel ^C pts/0:bde@ref8-i386:~/sys7/i386/compile> exit Script done on Sat Jun 20 02:34:41 2009 The shell script zm builds 6 kernels in parallel using make -k -j8 for each. Signal 18 is SIGTSTP. Receiving this is normal, but the shell shouldn't print any meesages about it. Signal 1 is SIGHUP. This shouldn't occur. On another run, ISTR getting messages about i/o errors or unrestartable processes. Anyway, the messages about signals are associated with failing jobs in the build. ref7-i386 now behaves normally -- ^Z and resume just work; no messages are printed and the build completes successfully after resuming. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20090620121543.F29239>