From owner-freebsd-virtualization@FreeBSD.ORG Sun Jul 6 17:25:24 2014 Return-Path: Delivered-To: virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E8E6EDF3; Sun, 6 Jul 2014 17:25:23 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A4FF423AD; Sun, 6 Jul 2014 17:25:23 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s66HPCoU084581 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 6 Jul 2014 17:25:17 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s66HPCu8084580; Sun, 6 Jul 2014 17:25:12 GMT (envelope-from swills) Date: Sun, 6 Jul 2014 17:25:12 +0000 From: Steve Wills To: Ryan Stone Subject: Re: tmpfs panic Message-ID: <20140706172511.GA84461@mouf.net> References: <20140706135333.GA80856@mouf.net> <20140706154621.GA81830@mouf.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="OXfL5xGRrasGEqWY" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Sun, 06 Jul 2014 17:25:18 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.98.1 at mouf.net X-Virus-Status: Clean Cc: virtualization@freebsd.org, FreeBSD Current X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jul 2014 17:25:24 -0000 --OXfL5xGRrasGEqWY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Jul 06, 2014 at 12:28:07PM -0400, Ryan Stone wrote: > On Sun, Jul 6, 2014 at 11:46 AM, Steve Wills wrote: > > I should have noted this system is running in bhyve. Also I'm told this= panic > > may be related to the fact that the system is running in bhyve. > > > > Looking at it a little more closely: > > > > (kgdb) list *__mtx_lock_sleep+0xb1 > > 0xffffffff809638d1 is in __mtx_lock_sleep (/usr/src/sys/kern/kern_mutex= =2Ec:431). > > 426 * owner stops running or the state of the lock= changes. > > 427 */ > > 428 v =3D m->mtx_lock; > > 429 if (v !=3D MTX_UNOWNED) { > > 430 owner =3D (struct thread *)(v & ~MTX_FL= AGMASK); > > 431 if (TD_IS_RUNNING(owner)) { > > 432 if (LOCK_LOG_TEST(&m->lock_obje= ct, 0)) > > 433 CTR3(KTR_LOCK, > > 434 "%s: spinning on %p= held by %p", > > 435 __func__, m, owner); > > (kgdb) > > > > I'm told that MTX_CONTESTED was set on the unlocked mtx and that MTX_CO= NTENDED > > is spuriously left behind, and to ask how lock prefix is handled in bhy= ve. Any > > of that make sense to anyone? >=20 > The mutex has both MTX_CONTESTED and MTX_UNOWNED set on it? That is a > special sentinel value that is set on a mutex when it is destroyed > (see MTX_DESTROYED in sys/mutex.h). If that is the case it looks like > you've stumbled upon some kind of use-after-free in tmpfs. I doubt > that bhyve is responsible (other than perhaps changing the timing > around making the panic more likely to happen). Given the first thing seen was: Freed UMA keg (TMPFS node) was not empty (16 items). Lost 1 pages of memor= y. this sounds reasonable to me. What can I do to help find and elliminate the source of the error? Steve --OXfL5xGRrasGEqWY Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJTuYZ3AAoJEPXPYrMgexuh06IIAJXfbe7rm1wQYoEz0ZpJU3jL QJvtyCllWlP/0w2zuGl7HSr0iGIdLqURKUmY7deA0tt+F5gP6IoSsXm5esAOEnFU 5ZTyOoHlkbSC4rMnjyRN/2isfdjmRruOG/NR37wvIXVPtH/8eJPK02utaBHJg3/6 8/PcGDxi3GOUaf0Kf3TURwSmLnG1ddhODBcCjMaC1GW5+OYSxsFN0fj5gAvkqzXY 6yFBZhRrvD9slOVkV1s3SaMDpjhLA6kd6Cipb2DT0HtgGTE5UF0RRc9OJUiTiKkb GSQEYfOymUdPUETJol1L7aZEk+lczflJAbC7Sn/HsirZgnL3Il72DQprZL6Jqsc= =+qu/ -----END PGP SIGNATURE----- --OXfL5xGRrasGEqWY--