From owner-freebsd-stable@FreeBSD.ORG Thu May 16 09:42:33 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 3999E5C0 for ; Thu, 16 May 2013 09:42:33 +0000 (UTC) (envelope-from db@nipsi.de) Received: from fop.bsdsystems.de (mx.bsdsystems.de [88.198.57.43]) by mx1.freebsd.org (Postfix) with ESMTP id 63CA1A80 for ; Thu, 16 May 2013 09:42:31 +0000 (UTC) Received: from [172.16.6.25] (p579A6028.dip0.t-ipconnect.de [87.154.96.40]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by fop.bsdsystems.de (Postfix) with ESMTP id ECC4C53E11; Thu, 16 May 2013 11:42:23 +0200 (CEST) Subject: Re: still mbuf leak in 9.0 / 9.1? Mime-Version: 1.0 (Apple Message framework v1085) From: dennis berger X-Priority: 3 In-Reply-To: Date: Thu, 16 May 2013 11:42:23 +0200 Message-Id: <4F319A22-E611-4EE6-A970-98315B15C12F@nipsi.de> References: <004BC6EA-D8E6-473E-851C-9CDA7578510A@nipsi.de> <20130515211436.GA42790@icarus.home.lan> <696B5622-A95D-4187-A027-07ECC9B5AD1F@nipsi.de> To: Steven Hartland X-Mailer: Apple Mail (2.1085) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: Jeremy Chadwick , FreeBSD stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2013 09:42:33 -0000 This is indeed a ZFS+NFS system and I can see that istgt and nfs are = stuck in some ZIO state. Maybe it's this.=20 Thank's for pointing out.=20 Is it this ZFS+NFS deadlock? --- a/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c=20 +++ b/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c=20 @@ -3720,8 +3720,16 @@ arc_lowmem(void *arg __unused, int howto = __unused)=20 mutex_enter(&arc_reclaim_thr_lock);=20 needfree =3D 1;=20 cv_signal(&arc_reclaim_thr_cv);=20 - while (needfree)=20 - msleep(&needfree, &arc_reclaim_thr_lock, 0, "zfs:lowmem", 0);=20= +=20 + /*=20 + * It is unsafe to block here in arbitrary threads, because we = can come=20 + * here from ARC itself and may hold ARC locks and thus risk a = deadlock=20 + * with ARC reclaim thread.=20 + */=20 + if (curproc =3D=3D pageproc) {=20 + while (needfree)=20 + msleep(&needfree, &arc_reclaim_thr_lock, 0, "zfs:lowmem", 0);=20= + }=20 mutex_exit(&arc_reclaim_thr_lock);=20 mutex_exit(&arc_lowmem_lock);=20 } I'll try to crash our testsystem. I'll assume that stressing NFS backed = with ZFS a lot might trigger this bug? -dennis Am 16.05.2013 um 00:03 schrieb Steven Hartland: > ----- Original Message ----- From: "dennis berger" >> FreeBSD 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4 = 09:23:10 UTC 2012 >>=20 >>> 3. Regarding this: >>>>> A clean shutdown isn't possible though. It hangs after vnode >>>>> cleaning, normally you would see detaching of usb devices here, or >>>>> other devices maybe? >>> Please don't conflate this with your above issue. This is almost >>> certainly unrelated. Please start a new thread about that if = desired. >>=20 >> Maybe this is a misunderstanding normally this system will shutdown = cleanly, of course. >> This hang only appears after the network problem above. >=20 > If this is a ZFS system, its a known issue which is fixed in current, > stable-9, stable-8 and the upcoming 8.4 release. >=20 > If not and you have USB devices see if the following sysctl helps: > hw.usb.no_shutdown_wait=3D1 >=20 > Regards > Steve >=20 > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > This e.mail is private and confidential between Multiplay (UK) Ltd. = and the person or entity to whom it is addressed. In the event of = misdirection, the recipient is prohibited from using, copying, printing = or otherwise disseminating it or any information contained in it.=20 > In the event of misdirection, illegible or incomplete transmission = please telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. >=20 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to = "freebsd-stable-unsubscribe@freebsd.org"