Date: Wed, 15 Sep 2010 15:09:49 -0400 From: John Baldwin <jhb@freebsd.org> To: freebsd-hackers@freebsd.org Cc: cronfy <cronfy@gmail.com> Subject: Re: is vfs.lookup_shared unsafe in 7.3? Message-ID: <201009151509.49728.jhb@freebsd.org> In-Reply-To: <AANLkTinEje-%2B1P1n33YMKAaciaYHQH%2BdpwgX6UY1dOux@mail.gmail.com> References: <AANLkTinEje-%2B1P1n33YMKAaciaYHQH%2BdpwgX6UY1dOux@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday, September 13, 2010 4:57:15 pm cronfy wrote: > Hello, > > Trying to overtake high server load (sudden peaks of 15%us/85%sy, LA > > 40, very slow lstat() at these moments, looks like some kind of lock > contention) I enabled vfs.lookup_shared=1 on two servers today. One is > FreeBSD-7.3 kernel csup'ed and built Sep 9 2010 and other is > FreeBSD-7.3 csup'ed and built Jul 16 2010. > > The server with more fresh kernel is running nice and does not show > high load anymore. But on the second server it did not help. More, > after a few hours of work with vfs.lookup_shared=1 I noticed processes > stucked in "ufs" state. I tried to kill them with no luck. Disabling > vfs.lookup_shared freezed the whole system. > > So, is vfs.lookup_shared=1 unsafe in 7.3? Did it become more stable > between 16 Jul and 9 Sep (is it the reason why first system is still > running?), or should I expect that it will freeze in a near time too? > > Thanks in advance! No, 7.3 has a bug that can cause these hangs that is probably made worse by vfs.lookup_shared=1, but can occur even if it is disabled. You want these fixes applied (in order, one of them reverts part of another): Author: jhb Date: Fri Jul 16 20:23:24 2010 New Revision: 210173 URL: http://svn.freebsd.org/changeset/base/210173 Log: When the MNTK_EXTENDED_SHARED mount option was added, some filesystems were changed to defer the setting of VN_LOCK_ASHARE() (which clears LK_NOSHARE in the vnode lock's flags) until after they had determined if the vnode was a FIFO. This occurs after the vnode has been inserted into a VFS hash or some similar table, so it is possible for another thread to find this vnode via vget() on an i-node number and block on the vnode lock. If the lockmgr interlock (vnode interlock for vnode locks) is not held when clearing the LK_NOSHARE flag, then the lk_flags field can be clobbered. As a result the thread blocked on the vnode lock may never get woken up. Fix this by holding the vnode interlock while modifying the lock flags in this case. The softupdates code also toggles LK_NOSHARE in one function to close a race with snapshots. Fix this code to grab the interlock while fiddling with lk_flags. Modified: stable/7/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c stable/7/sys/fs/cd9660/cd9660_vfsops.c stable/7/sys/fs/udf/udf_vfsops.c stable/7/sys/ufs/ffs/ffs_softdep.c stable/7/sys/ufs/ffs/ffs_vfsops.c Author: jhb Date: Fri Aug 20 20:33:13 2010 New Revision: 211532 URL: http://svn.freebsd.org/changeset/base/211532 Log: MFC: Use VN_LOCK_AREC() and VN_LOCK_ASHARE() rather than manipulating lockmgr lock flags directly. Modified: stable/7/sys/fs/nwfs/nwfs_node.c stable/7/sys/fs/pseudofs/pseudofs_vncache.c stable/7/sys/fs/smbfs/smbfs_node.c stable/7/sys/gnu/fs/xfs/FreeBSD/xfs_freebsd_iget.c stable/7/sys/kern/vfs_lookup.c Author: jhb Date: Fri Aug 20 20:58:57 2010 New Revision: 211533 URL: http://svn.freebsd.org/changeset/base/211533 Log: Revert 210173 as it did not properly fix the bug. It assumed that the VI_LOCK() for a given vnode was used as the internal interlock for that vnode's v_lock lockmgr lock. This is not the case. Instead, add dedicated routines to toggle the LK_NOSHARE and LK_CANRECURSE flags. These routines lock the lockmgr lock's internal interlock to synchronize the updates to the flags member with other threads attempting to acquire the lock. The VN_LOCK_A*() macros now invoke these routines, and the softupdates code uses these routines to temporarly enable recursion on buffer locks. Reviewed by: kib Modified: stable/7/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c stable/7/sys/fs/cd9660/cd9660_vfsops.c stable/7/sys/fs/udf/udf_vfsops.c stable/7/sys/kern/kern_lock.c stable/7/sys/sys/lockmgr.h stable/7/sys/sys/vnode.h stable/7/sys/ufs/ffs/ffs_softdep.c stable/7/sys/ufs/ffs/ffs_vfsops.c -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201009151509.49728.jhb>