Date: Thu, 27 Dec 2007 15:15:21 +0200 From: Kostik Belousov <kostikbel@gmail.com> To: Raja Sivaramakrishnan <srajag00@yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: namei lookup vnode locking Message-ID: <20071227131521.GO57756@deviant.kiev.zoral.com.ua> In-Reply-To: <701759.46468.qm@web90511.mail.mud.yahoo.com> References: <701759.46468.qm@web90511.mail.mud.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--5tiY7shzwSGI9p6W Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 26, 2007 at 04:46:48PM -0800, Raja Sivaramakrishnan wrote: > Hello, > I encountered an issue with FreeBSD 6.1 and would > appreciate some feedback on this. The problem happens First, there were the significant locking fixes for the devfs after 6.1, in particular, check that 6.2, or, even better 6.3-latest RC shows the errant behaviour. It may be all fixed already. For proper reporting of the deadlock, see kernel debug chapter of the developer handbook, in particular, deadlock section. I have a doubt regarding you analysis as far as I was able to understand it. ttydrain() ioctl does not hold the dev vnode lock while calling the driver. Anyway, do what I recommended above. > when a perl script running on some client system does=20 > a telnet into the FreeBSD box, exits from the login=20 > shell and immediately exits the perl script too. After > the script exits, there is a deadlock on the FreeBSD > box that prevents new processes (such as ps, top etc.) > from starting. Upon investigation, this seems to be > caused due to the following sequence of events on the > FreeBSD system. >=20 > 1) login process exits. exit call in the kernel closes > all file descriptors. One of these is the fd for > /dev/ttyp0, used for the telnet session. login locks > the vnode for /dev/ttyp0 and waits for 5 minutes in > order for the tty to drain (ttywait() call). >=20 > 2) The tty is supposed to be drained by telnetd.=20 > However, telnetd sees the network connection go=20 > down when the perl script exits. As a result, it > jumps to cleanup code, where it tries to do chmod > on /dev/ttyp0. chmod syscall attempts to lock > /dev/ttyp0, but fails as the lock is held by login, > which puts telnetd process to sleep. However,=20 > telnetd holds the lock on the vnode for /dev. > It appears that the lock was acquired when doing the > namei lookup for /dev/ttyp0. The current state is > that there is output in the tty that has to be > read by telnetd, but it can't because it is sleeping > for the /dev/ttyp0 lock. telnetd is holding the > /dev vnode lock while sleeping. >=20 > 3) As a result, any process that needs the /dev > vnode lock is put to sleep for 5 minutes (ttywait > waits for a default of 5 minutes). Even if a > process wants to open an unrelated device file, > /dev/foo, it is not able to do so because the /dev > lock is held by telnetd. >=20 > Few questions: >=20 > 1) Does namei lookup need to acquire an exclusive > lock on intermediate vnodes when looking up a pathname > i.e. if telnetd is trying to lookup /dev/ttyp0, does > it need to get an exclusive lock on /dev? Can it > be a shared lock that will allow at least other > readers > to make progress? >=20 > 2) Besides relaxing the locking above, any other > thoughts on how to fix this? Reducing the tty timeout > from the close routine is another option, but that > only limits the duration of the deadlock. >=20 > Thanks, >=20 > Raja >=20 >=20 > ___________________________________________________________________= _________________ > Never miss a thing. Make Yahoo your home page.=20 > http://www.yahoo.com/r/hs > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --5tiY7shzwSGI9p6W Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFHc6VpC3+MBN1Mb4gRAhrBAJsFhdoxgeovIz5GlGNnUq+FrEXJUwCgpzpX 5pLJcQy/LtDqoVIk1Ce/9Jo= =NQA7 -----END PGP SIGNATURE----- --5tiY7shzwSGI9p6W--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20071227131521.GO57756>