Date: Fri, 15 Feb 2019 11:34:23 +0100 From: Ole <ole@free.de> To: freebsd-questions@freebsd.org Subject: ZFS deadlock on parallel ZFS operations FreeBSD 11.2 and 12.0 Message-ID: <20190215113423.01edabe9.ole@free.de>
next in thread | raw e-mail | index | archive | help
--Sig_/PFqTLl197+HIroS.1xohi5P Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Hi, I observed that FreeBSD Systems with ZFS will run into a deadlock if there are many parallel zfs send/receive/snapshot processes. I observed this on bare metal and virtual machines with FreeBSD 11.2 and 12.0. With RAM from 20 to 64G. If the system is also on ZFS the whole system crashes. With only jails on ZFS they freeze, but the Host system stays stable. But you can't kill -9 the zfs processes. Only a poweroff stops the machine. On a FreeBSD 12.0 VM (bhyve), 30G RAM, 5 CPUs, about 30 zfs operations, mostly send and receive will crash the system. There is no heavy load on the machine: # top | head -8 last pid: 91503; load averages: 0.34, 0.31, 0.29 up 0+22:50:47 11:24:00 536 processes: 1 running, 529 sleeping, 6 zombie CPU: 0.9% user, 0.0% nice, 1.5% system, 0.2% interrupt, 97.4% idle Mem: 165M Active, 872M Inact, 19G Wired, 264M Buf, 9309M Free ARC: 11G Total, 2450M MFU, 7031M MRU, 216M Anon, 174M Header, 1029M Other 8423M Compressed, 15G Uncompressed, 1.88:1 Ratio Swap: 1024M Total, 1024M Free I wonder if this is a BUG or normal behaviour. I could live with a limited amount of parallel ZFS operation, but I don't want the whole system to crash.=20 Reducing the vfs.zfs.arc_max wont help. Any Idea to handle with this? regards Ole --Sig_/PFqTLl197+HIroS.1xohi5P Content-Type: application/pgp-signature Content-Description: Digitale Signatur von OpenPGP -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE60BGd7KVfL83NXCUJZaRRqjklFAFAlxmla8ACgkQJZaRRqjk lFDO/w//fZkynrf21xaOHips2oijOsnPydvKz1WO+YPeNiHHYhfcCyYKLj8+tQyI /YMCo6vuqpPlX91tsFNPedjgc5VmspuCkLuUqL2iomR3sc3eUZNDsdMqr+PDeiN4 IyDoguZmpUAffO28fF/MPcp4PPZ7Usyp0jLhPStbhAYmGsu0IGfAnLF7IlI0fXoN 7uk3z2KOm+xVBVAm5vBtDgCo85OUYW6Pq+/wLWq90aSBFBeiQkgQjwvdbZ0pi797 9dEt+7IegIHODHl116s+psEjVwxCzxneN/Uski8d0qTolnoYiizEN3KtA6FMNDd2 cpqRlZWfAxbjVRVa6OGT44NhtwEMQlubPIO409u81d8M8hFfRV7wRPkkXwqSlOu7 9s45jT+mLSLChFvFwB34ki+dYLaQpFhyH4KUy/QhWmXNVl6BwtcILVqiel3JgQRg yxlutVjkTeFoyjXLFjWVbddy45Irm+Q9qG6vDdU4OvT8z8EkXtMNKVqfCVXh0zZD pjImy6YrXkz69cQ8naXxtfAwLr22feYbCqBXA897vRWBTx6KaLnmC19KaiIbyaHJ ptr67ikSa2LMAFkhDWME2STLmAz2AGlmCUlDqq5ReaWj+L0FtUKwaOKe9mO3nR/o Klev2Y7nv4nSnzNoLixdW2IudCb3N82tGja6qKVBpsk6AKSlgfE= =dHWu -----END PGP SIGNATURE----- --Sig_/PFqTLl197+HIroS.1xohi5P--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190215113423.01edabe9.ole>