From owner-svn-src-all@FreeBSD.ORG Sun Sep 22 20:34:32 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 2E693135; Sun, 22 Sep 2013 20:34:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A1FB3251E; Sun, 22 Sep 2013 20:34:31 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r8MKYQ1A088879; Sun, 22 Sep 2013 23:34:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r8MKYQ1A088879 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r8MKYQSX088878; Sun, 22 Sep 2013 23:34:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 22 Sep 2013 23:34:26 +0300 From: Konstantin Belousov To: Matthew Fleming Subject: Re: svn commit: r255797 - head/sys/kern Message-ID: <20130922203426.GM41229@kib.kiev.ua> References: <201309221923.r8MJNm3u021657@svn.freebsd.org> <20130922201916.GL41229@kib.kiev.ua> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dgZ3gzdqzaCbtzwD" Content-Disposition: inline In-Reply-To: <20130922201916.GL41229@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: "svn-src-head@freebsd.org" , Attilio Rao , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Sep 2013 20:34:32 -0000 --dgZ3gzdqzaCbtzwD Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 22, 2013 at 11:19:16PM +0300, Konstantin Belousov wrote: > On Sun, Sep 22, 2013 at 01:14:21PM -0700, Matthew Fleming wrote: > > On Sun, Sep 22, 2013 at 12:23 PM, Konstantin Belousov = wrote: > >=20 > > > Author: kib > > > Date: Sun Sep 22 19:23:48 2013 > > > New Revision: 255797 > > > URL: http://svnweb.freebsd.org/changeset/base/255797 > > > > > > Log: > > > Increase the chance of the buffer write from the bufdaemon helper > > > context to succeed. If the locked vnode which owns the buffer to be > > > written is shared locked, try the non-blocking upgrade of the lock = to > > > exclusive. > > > > > > PR: kern/178997 > > > Reported and tested by: Klaus Weber < > > > fbsd-bugs-2013-1@unix-admin.de> > > > Sponsored by: The FreeBSD Foundation > > > MFC after: 1 week > > > Approved by: re (marius) > > > > > > Modified: > > > head/sys/kern/vfs_bio.c > > > > > > Modified: head/sys/kern/vfs_bio.c > > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > > --- head/sys/kern/vfs_bio.c Sun Sep 22 19:15:24 2013 (r255= 796) > > > +++ head/sys/kern/vfs_bio.c Sun Sep 22 19:23:48 2013 (r255= 797) > > > @@ -2624,6 +2624,8 @@ flushbufqueues(struct vnode *lvp, int ta > > > int hasdeps; > > > int flushed; > > > int queue; > > > + int error; > > > + bool unlock; > > > > > > flushed =3D 0; > > > queue =3D QUEUE_DIRTY; > > > @@ -2699,7 +2701,16 @@ flushbufqueues(struct vnode *lvp, int ta > > > BUF_UNLOCK(bp); > > > continue; > > > } > > > - if (vn_lock(vp, LK_EXCLUSIVE | LK_NOWAIT | LK_CANRECU= RSE) > > > =3D=3D 0) { > > > + if (lvp =3D=3D NULL) { > > > + unlock =3D true; > > > + error =3D vn_lock(vp, LK_EXCLUSIVE | LK_NOWAI= T); > > > + } else { > > > + ASSERT_VOP_LOCKED(vp, "getbuf"); > > > + unlock =3D false; > > > + error =3D VOP_ISLOCKED(vp) =3D=3D LK_EXCLUSIV= E ? 0 : > > > + vn_lock(vp, LK_UPGRADE | LK_NOWAIT); > > > > >=20 > > I don't think this is quite right. > >=20 > > When the lock is held shared, and VOP_LOCK is implemented by lockmgr(9), > > (i.e. all in-tree filesystems?), LK_UPGRADE may drop the lock, and not > > reacquire it. This would happen when the vnode is locked shared, the > > upgrade fails (2 shared owners), then lockmgr(9) will try to lock EX, w= hich > > will also fail (still one shared owner). The caller's lock is no longer > > held. > >=20 > > Doesn't that scenario (LK_UPGRADE failing) cause problems both for the > > caller (unexpected unlock) and for flushbufqueues(), which expects the > > vnode lock to be held since lvp is non-NULL? >=20 > Does it ? If the lock is dropped, the code is indeed in trouble. > Please note that LK_NOWAIT is specified for upgrade, and I believe > that this causes lockmgr to return with EBUSY without dropping > the lock. Yes, you are right, I reverted the patch. Thank you for noting this. I am bitten by unreasonable behaviour of non-blocking upgrade once more. It has a history. Some time ago I proposed the following patch, which was turned down. That time, I was able to work-around the case. For the bufdaemon helper, I do not see any way to avoid this, except of sometimes locking the reader vnode exclusive in anticipation of the too high dirty buffer mark. diff --git a/sys/kern/kern_lock.c b/sys/kern/kern_lock.c index 74a5b19..2f2dbf6 100644 --- a/sys/kern/kern_lock.c +++ b/sys/kern/kern_lock.c @@ -694,6 +692,7 @@ __lockmgr_args(struct lock *lk, u_int flags, struct loc= k_object *ilk, } break; case LK_UPGRADE: + case LK_TRYUPGRADE: _lockmgr_assert(lk, KA_SLOCKED, file, line); v =3D lk->lk_lock; x =3D v & LK_ALL_WAITERS; @@ -714,6 +713,17 @@ __lockmgr_args(struct lock *lk, u_int flags, struct lo= ck_object *ilk, } =20 /* + * In LK_TRYUPGRADE mode, do not drop the lock, + * returning EBUSY instead. + */ + if (op =3D=3D LK_TRYUPGRADE) { + LOCK_LOG2(lk, "%s: %p failed the nowait upgrade", + __func__, lk); + error =3D EBUSY; + break; + } + + /* * We have been unable to succeed in upgrading, so just * give up the shared lock. */ diff --git a/sys/sys/lockmgr.h b/sys/sys/lockmgr.h index f525a06..7c51830 100644 --- a/sys/sys/lockmgr.h +++ b/sys/sys/lockmgr.h @@ -169,6 +168,7 @@ _lockmgr_args_rw(struct lock *lk, u_int flags, struct r= wlock *ilk, #define LK_RELEASE 0x100000 #define LK_SHARED 0x200000 #define LK_UPGRADE 0x400000 +#define LK_TRYUPGRADE 0x800000 =20 #define LK_TOTAL_MASK (LK_INIT_MASK | LK_EATTR_MASK | LK_TYPE_MASK) =20 --dgZ3gzdqzaCbtzwD Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.21 (FreeBSD) iQIcBAEBAgAGBQJSP1RRAAoJEJDCuSvBvK1BrvAP/0W+SSKTKV3lg+bEjwRWEF4W aV9nEKqDKbmdjeH+wiVU+EQ1I2uyFWj8z4+JhheoRTlvtB0QOfruvuAy7xXAcVun F8yUA8ssdEzNka9BSbbv9cD4oGJ+/kZJks+l4ncOCRnATQfDMErk9GrWQGSB/TIG 6edDD3NQc6ixxo1sB0kS6ciVzMd9QWP6SQnLripjJC6OaSKsFnM0YiVwaKPt+hZ5 cbk8JTdULvftFvY/3PjaNhWVSHUvqoNa9b0Hf5CkaeysScmE5QV+LMRSXRHI3iVO zz8tazcJV5JKl9VaLn7cjM9hEkez4NHRrQ2AayHOt5z/k6m6QMxJCY7Rsxcy0TQc qVnBixODgTbrfyyeSiAg1mbAMvs9cnnxdyf/vvFUAV2Tpw5BDb8e1LW/fO/ooxgj vyTkzY6aWrzf5YMiXIhU3SInlKeWJYEOdUpRrSqWY2fjWpQrq3sAb849g2Zve4K2 jlbxY9gRwluisZDiVkbPO4nDF+6o6w3Y/CfsB0mEPK2VspfJbdFYF+WCgpNVARnt 4FnToefcmv/0NdDVNt/2ufvskiCafKPHN8d6529/PItN60b/Vl5R/i6HldXBfWLP 6l+AoYJT6zNjjnFC/q3thDAS5JhaSQKskgFpQYBXe4lMs82DS/Sm/J59qs3lv5Ig FedWIS7IKQp/1iRsM1yl =y/NC -----END PGP SIGNATURE----- --dgZ3gzdqzaCbtzwD--