Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 29 Apr 2019 05:18:48 +0000
From:      bugzilla-noreply@freebsd.org
To:        bugs@FreeBSD.org
Subject:   [Bug 237636] bhyve guest ZFS filesystem freezes in zcw->zcw_cv state
Message-ID:  <bug-237636-227@https.bugs.freebsd.org/bugzilla/>

next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237636

            Bug ID: 237636
           Summary: bhyve guest ZFS filesystem freezes in zcw->zcw_cv
                    state
           Product: Base System
           Version: 12.0-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: freebsd-bugs@morgandavis.net

I experience random freezes during file I/O that locks up the entire virtual
machine (except networking). Environment is FreeBSD 12.0-RELEASE bhyve guest
running with ZFS on a FreeBSD 12.0-RELEASE host also on ZFS.

During some specific disk operations, such as a sqlite database vacuum or g=
it
'gc' operation, the VM will freeze up and cease performing disk operations.=
=20
The VM can still respond to pings and accept network connections, but any d=
isk
operation will just cause complete blocking.  The only remedy is to kill -9=
 the
bhyve host process and restart the guest.

Trying Control-T during one of these episodes always shows the kernel stuck=
 in
zcw->zcw_cv.  For example, this "git gc" operation:

Enumerating objects: 12282, done.
Counting objects: 100% (12282/12282), done.
Delta compression using up to 4 threads
Compressing objects: 100% (12259/12259), done.
<--- system completely stops responding here, control-T entered -->
load: 0.47  cmd: git 90093 [zcw->zcw_cv] 31.31r 9.77u 0.09s 0% 123612k

This does not always happen and is not reproducible at will. However, the
freezes are occurring mostly with sqlite and occasionally with git.

Host system has 16GB RAM and guest has 5GB allocated.  The host does nothing
but run this one guest bhyve instance.

Guest's /boot/loader.conf disables ZFS prefetching:

vfs.zfs.prefetch_disable=3D"1"

And after much searching for help on the freezing, I found this article whi=
ch
suggested reducing ZFS arc max.
https://www.reddit.com/r/freebsd/comments/1u358s/freebsd_random_freeze_wher=
e_to_start/

So I added this to loader.conf, too:

vfs.zfs.arc_max=3D"1G"

This occurs on FreeBSD 11 and now 12 as well.  I don't recall having this i=
ssue
under FreeBSD 10.

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-237636-227>