Date: Fri, 03 Jan 2014 11:21:24 -0500 From: Allan Jude <freebsd@allanjude.com> To: freebsd-current@freebsd.org Subject: Re: ZFS command can block the whole ZFS subsystem! Message-ID: <52C6E384.1070905@allanjude.com> In-Reply-To: <20140103171457.0fbf0cd4@telesto> References: <20140103130021.30569db4@thor.walstatt.dyndns.org> <FC618C2B94D9425EAE5C11FEF2042F49@multiplay.co.uk> <20140103171457.0fbf0cd4@telesto>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --M9wQJR7DWN4JiS9ktGuN2hxKo0WlnEXGm Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable On 2014-01-03 11:14, O. Hartmann wrote: > On Fri, 3 Jan 2014 14:38:03 -0000 > "Steven Hartland" <killing@multiplay.co.uk> wrote: > >> ----- Original Message -----=20 >> From: "O. Hartmann" <ohartman@zedat.fu-berlin.de> >>> For some security reasons, I dumped via "dd" a large file onto a 3TB >>> disk. The systems is 11.0-CURRENT #1 r259667: Fri Dec 20 22:43:56 >>> CET 2013 amd64. Filesystem in question is a single ZFS pool. >>> >>> Issuing the command >>> >>> "rm dumpfile.txt" >>> >>> and then hitting Ctrl-Z to bring the rm command into background via >>> fg" (I use FreeBSD's csh in that console) locks up the entire >>> command and even worse - it seems to wind up the pool in question >>> for being exported! >> I cant think of any reason why backgrounding a shell would export a >> pool. > I sent the job "rm" into background and I didn't say that implies an > export of the pool! > > I said that the pool can not be exported once the bg-command has been > issued.=20 > >>> I expect to get the command into the background as every other UNIX >>> command does when sending Ctrl-Z in the console. Obviously, ZFS >>> related stuff in FreeBSD doesn't comply.=20 >>> >>> The file has been removed from the pool but the console is still >>> stuck with "^Z fg" (as I typed this in). Process list tells me: >>> >>> top >>> 17790 root 1 20 0 8228K 1788K STOP 10 0:05 >>> 0.00% rm >>> >>> for the particular "rm" command issued. >> Thats not backgrounded yet otherwise it wouldnt be in the state STOP. > As I said - the job never backgrounded, locked up the terminal and > makes the whole pool inresponsive. > > >>> Now, having the file deleted, I'd like to export the pool for >>> further maintainance >> Are you sure the delete is complete? Also don't forget ZFS has TRIM by= >> default, so depending on support of the underlying devices you could >> be seeing deletes occuring. > Quite sure it didn't! It takes hours (~ 8 now) and the drive is still > working, although I tried to stop.=20 >> You can check that gstat -d > command report 100% acticity on the drive. I exported the pool in > question in single user mode and now try to import it back while in > miltiuser mode. > > Shortly after issuing the command > > zpool import POOL00 > > the terminal is stuck again, the drive is working at 100% for two > hours now and it seems the great ZFS is deleting every block per pedes.= > Is this supposed to last days or a week? > > =20 >>> but that doesn't work with >>> >>> zpool export -f poolname >>> >>> This command is now also stuck blocking the terminal and the pool >>> from further actions. >> If the delete hasnt completed and is stuck in the kernel this is >> to be expected. > At this moment I will not imagine myself what will happen if I have to > delete several deka terabytes. If the weird behaviour of the current > system can be extrapolated, then this is a no-go. > >>> This is painful. Last time I faced the problem, I had to reboot >>> prior to take any action regarding any pool in the system, since >>> one single ZFS command could obviously block the whole subsystem (I >>> tried to export and import). >>> >>> What is up here? >> Regards >> Steve >> >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> This e.mail is private and confidential between Multiplay (UK) Ltd. >> and the person or entity to whom it is addressed. In the event of >> misdirection, the recipient is prohibited from using, copying, >> printing or otherwise disseminating it or any information contained >> in it.=20 >> >> In the event of misdirection, illegible or incomplete transmission >> please telephone +44 845 868 1337 or return the E.mail to >> postmaster@multiplay.co.uk. > Regards, > Oliver Deleting large amounts of data with 'rm' is slow. When destroying a dataset, ZFS grew a feature flag, async_destroy that lets this happen in the background, and avoids a lot of these issues. An async_delete might be something to consider some day. --=20 Allan Jude --M9wQJR7DWN4JiS9ktGuN2hxKo0WlnEXGm Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.16 (MingW32) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSxuOKAAoJEJrBFpNRJZKfWwQP/iEQRDU0BcugqHFJfk5El70V urwT2CR2RNoau5HOYdsepZaGvGU/6/eAi8NnUq7Fn3tsMaOZ9JCMflMw9QnS4oq5 vpk+mqCe3A2gmDRe7f4UKrhe6eJJxsbFHHNzDutcMY4O9coI/y7ENekqu5Emy8lZ Sbf/+WlnKL3KmITYRSKAotTa5qDigYsnIaCP3nTtIQGQKihb8ExQUoEqUISNMHA9 yHJ6LMP7mqSFhcXttYjdf2pHc2nXEKmCHvznrbFBKAxrVt8opOkGtVGYGUBIyWzW +T7D6f/OFztenHsKJIoKF2q75JxPoWw6DTJWIJvAhCrJ292BDSvxSBceLC4QtT+N GOXd7B3nMVwCBFiDoiofWHDQnGyeuFTpgvIL8QrAH7+wP64KwVc0bp//rlZfaNmP UHbpeB5dHzON1GJAn5sM14Qz+HEbsV0w0w+X8KVDdMTrRBJGGIJkBYfM8MvBDwUX V0CIemt7bXn2Hoteh51U4bxPg1uI1mOGG9URXv5XCe1O2QQps1+7NEaBdiSz9FXL O2luyteM2YL2W7ysz/E0+lq9N45osJC3cugyMrsQdKwD4vHRxJXEuoxvzJMxYm8E iDSDOqbx0KnKlhT18gKK/A82YWF3/jF49k6EjSl+hO8C0Rzu88UMS5lbqssf3sRD 9zvbSkaz81Zr/EDT7GIA =wIYK -----END PGP SIGNATURE----- --M9wQJR7DWN4JiS9ktGuN2hxKo0WlnEXGm--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52C6E384.1070905>