From owner-freebsd-stable@FreeBSD.ORG Wed Oct 31 21:23:55 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9660FE12; Wed, 31 Oct 2012 21:23:55 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (host-122-100-2-194.octopus.com.au [122.100.2.194]) by mx1.freebsd.org (Postfix) with ESMTP id 092A28FC16; Wed, 31 Oct 2012 21:23:53 +0000 (UTC) Received: from server.rulingia.com (c220-239-241-202.belrs5.nsw.optusnet.com.au [220.239.241.202]) by vps.rulingia.com (8.14.5/8.14.5) with ESMTP id q9VLNqO2037647 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Thu, 1 Nov 2012 08:23:52 +1100 (EST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.14.5/8.14.5) with ESMTP id q9VLNkcA040212 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 1 Nov 2012 08:23:46 +1100 (EST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.14.5/8.14.5/Submit) id q9VLNk2i040211; Thu, 1 Nov 2012 08:23:46 +1100 (EST) (envelope-from peter) Date: Thu, 1 Nov 2012 08:23:46 +1100 From: Peter Jeremy To: Steven Hartland Subject: Re: ZFS corruption due to lack of space? Message-ID: <20121031212346.GL3309@server.rulingia.com> References: <27087376D1C14132A3CC1B4016912F6D@multiplay.co.uk> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="IMjqdzrDRly81ofr" Content-Disposition: inline In-Reply-To: <27087376D1C14132A3CC1B4016912F6D@multiplay.co.uk> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2012 21:23:55 -0000 --IMjqdzrDRly81ofr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2012-Oct-31 17:25:09 -0000, Steven Hartland wro= te: >Been running some tests on new hardware here to verify all >is good. One of the tests was to fill the zfs array which >seems like its totally corrupted the tank. I've accidently "filled" a pool, and had multiple processes try to write to the full pool, without either emptying the free space reserve (so I could still delete the offending files) or corrupting the pool. Had you tried to read/write the raw disks before you tried the ZFS testing? Do you have compression and/or dedupe enabled on the pool? >1. Given the information it seems like the multiple writes filling >the disk may have caused metadata corruption? I don't recall seeing this reported before. >2. Is there anyway to stop the scrub? Other than freeing up some space, I don't think so. If this is a test pool that you don't need, you could try destroying it and re-creating it - that may be quicker and easier than recovering the existing pool. >3. Surely low space should never prevent stopping a scrub? As Artem noted, ZFS is a copy-on-write filesystem. It is supposed to reserve some free space to allow metadata updates (stop scrubs, delete files, etc) even when it is "full" but I have seen reports of this not working correctly in the past. A truncate-in-place may work. You could also try asking on zfs-discuss@opensolaris.org=20 --=20 Peter Jeremy --IMjqdzrDRly81ofr Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.19 (FreeBSD) iEYEARECAAYFAlCRluIACgkQ/opHv/APuIf5XwCePbniJH+FqKmFdUYvRlHobjbE U74AoIBMqgc6dVkhg9Znx5K9IVh4Spa2 =1SD/ -----END PGP SIGNATURE----- --IMjqdzrDRly81ofr--