Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 02 Feb 2024 07:54:33 +0000
From:      bugzilla-noreply@freebsd.org
To:        fs@FreeBSD.org
Subject:   [Bug 273772] zpool nested in ZVOL becomes inaccessible when zfsd(8) is running on the host
Message-ID:  <bug-273772-3630-9zxIKFhOUs@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-273772-3630@https.bugs.freebsd.org/bugzilla/>
References:  <bug-273772-3630@https.bugs.freebsd.org/bugzilla/>

next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D273772

--- Comment #11 from Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> ---
Now both: bhyve host and guest are running FreeBSD stable/14. ZFS pools on =
host
and guest were upgraded. The report is still valid and the issue is persist=
ent.
 Maybe the transition to mirror (the guest is nested on a ZVOL of pool with=
 a
top-level-vdev that had previously been removed) has something to do with it
(see bug 273663). The guest is using host's ZVOL zroot/ZVOL/virtbsd as vtbd=
0.




This is how ZFS looks on the host:=20

# zpool status zroot
  pool: zroot
 state: ONLINE
  scan: scrub repaired 0B in 00:11:06 with 0 errors on Fri Jan 26 03:21:55 =
2024
    scan warning: skipped blocks that are only referenced by the checkpoint.
remove: Removal of vdev 1 copied 15.1G in 0h0m, completed on Fri May 15
20:04:34 2020
        119K memory used for removed device mappings
checkpoint: created Mon Dec  4 20:10:12 2023, consumes 49.6G
config:

        NAME          STATE     READ WRITE CKSUM
        zroot         ONLINE       0     0     0
          mirror-0    ONLINE       0     0     0
            da0p2     ONLINE       0     0     0
            da1p2     ONLINE       0     0     0

errors: No known data errors

# zpool get all zroot
NAME   PROPERTY                       VALUE                          SOURCE
zroot  size                           476G                           -
zroot  capacity                       63%                            -
zroot  altroot                        -                              default
zroot  health                         ONLINE                         -
zroot  guid                           6762271100094851702            -
zroot  version                        -                              default
zroot  bootfs                         zroot/ROOT/default14           local
zroot  delegation                     on                             default
zroot  autoreplace                    off                            default
zroot  cachefile                      -                              default
zroot  failmode                       wait                           default
zroot  listsnapshots                  off                            default
zroot  autoexpand                     off                            default
zroot  dedupratio                     1.00x                          -
zroot  free                           176G                           -
zroot  allocated                      300G                           -
zroot  readonly                       off                            -
zroot  ashift                         0                              default
zroot  comment                        -                              default
zroot  expandsize                     -                              -
zroot  freeing                        0                              -
zroot  fragmentation                  58%                            -
zroot  leaked                         0                              -
zroot  multihost                      off                            default
zroot  checkpoint                     49.6G                          -
zroot  load_guid                      2254167222130845727            -
zroot  autotrim                       on                             local
zroot  compatibility                  off                            default
zroot  bcloneused                     0                              -
zroot  bclonesaved                    0                              -
zroot  bcloneratio                    1.00x                          -
zroot  feature@async_destroy          enabled                        local
zroot  feature@empty_bpobj            active                         local
zroot  feature@lz4_compress           active                         local
zroot  feature@multi_vdev_crash_dump  enabled                        local
zroot  feature@spacemap_histogram     active                         local
zroot  feature@enabled_txg            active                         local
zroot  feature@hole_birth             active                         local
zroot  feature@extensible_dataset     active                         local
zroot  feature@embedded_data          active                         local
zroot  feature@bookmarks              enabled                        local
zroot  feature@filesystem_limits      enabled                        local
zroot  feature@large_blocks           enabled                        local
zroot  feature@large_dnode            enabled                        local
zroot  feature@sha512                 enabled                        local
zroot  feature@skein                  enabled                        local
zroot  feature@edonr                  enabled                        local
zroot  feature@userobj_accounting     active                         local
zroot  feature@encryption             enabled                        local
zroot  feature@project_quota          active                         local
zroot  feature@device_removal         active                         local
zroot  feature@obsolete_counts        active                         local
zroot  feature@zpool_checkpoint       active                         local
zroot  feature@spacemap_v2            active                         local
zroot  feature@allocation_classes     enabled                        local
zroot  feature@resilver_defer         enabled                        local
zroot  feature@bookmark_v2            enabled                        local
zroot  feature@redaction_bookmarks    enabled                        local
zroot  feature@redacted_datasets      enabled                        local
zroot  feature@bookmark_written       enabled                        local
zroot  feature@log_spacemap           active                         local
zroot  feature@livelist               enabled                        local
zroot  feature@device_rebuild         enabled                        local
zroot  feature@zstd_compress          enabled                        local
zroot  feature@draid                  enabled                        local
zroot  feature@zilsaxattr             active                         local
zroot  feature@head_errlog            active                         local
zroot  feature@blake3                 enabled                        local
zroot  feature@block_cloning          enabled                        local
zroot  feature@vdev_zaps_v2           active                         local

# zfs get all zroot/ZVOL/virtbsd
NAME                PROPERTY              VALUE                     SOURCE
zroot/ZVOL/virtbsd  type                  volume                    -
zroot/ZVOL/virtbsd  creation              Wed Mar 10  8:59 2021     -
zroot/ZVOL/virtbsd  used                  109G                      -
zroot/ZVOL/virtbsd  available             144G                      -
zroot/ZVOL/virtbsd  referenced            11.3G                     -
zroot/ZVOL/virtbsd  compressratio         1.13x                     -
zroot/ZVOL/virtbsd  reservation           none                      default
zroot/ZVOL/virtbsd  volsize               50G                       local
zroot/ZVOL/virtbsd  volblocksize          8K                        -
zroot/ZVOL/virtbsd  checksum              on                        default
zroot/ZVOL/virtbsd  compression           lz4                       inherit=
ed
from zroot
zroot/ZVOL/virtbsd  readonly              off                       default
zroot/ZVOL/virtbsd  createtxg             5153440                   -
zroot/ZVOL/virtbsd  copies                1                         default
zroot/ZVOL/virtbsd  refreservation        51.6G                     local
zroot/ZVOL/virtbsd  guid                  1428736673261547338       -
zroot/ZVOL/virtbsd  primarycache          all                       default
zroot/ZVOL/virtbsd  secondarycache        all                       default
zroot/ZVOL/virtbsd  usedbysnapshots       45.8G                     -
zroot/ZVOL/virtbsd  usedbydataset         11.3G                     -
zroot/ZVOL/virtbsd  usedbychildren        0B                        -
zroot/ZVOL/virtbsd  usedbyrefreservation  51.5G                     -
zroot/ZVOL/virtbsd  logbias               latency                   default
zroot/ZVOL/virtbsd  objsetid              66109                     -
zroot/ZVOL/virtbsd  dedup                 off                       default
zroot/ZVOL/virtbsd  mlslabel              none                      default
zroot/ZVOL/virtbsd  sync                  standard                  default
zroot/ZVOL/virtbsd  refcompressratio      1.21x                     -
zroot/ZVOL/virtbsd  written               85.1M                     -
zroot/ZVOL/virtbsd  logicalused           64.4G                     -
zroot/ZVOL/virtbsd  logicalreferenced     13.6G                     -
zroot/ZVOL/virtbsd  volmode               default                   default
zroot/ZVOL/virtbsd  snapshot_limit        none                      default
zroot/ZVOL/virtbsd  snapshot_count        none                      default
zroot/ZVOL/virtbsd  snapdev               hidden                    default
zroot/ZVOL/virtbsd  context               none                      default
zroot/ZVOL/virtbsd  fscontext             none                      default
zroot/ZVOL/virtbsd  defcontext            none                      default
zroot/ZVOL/virtbsd  rootcontext           none                      default
zroot/ZVOL/virtbsd  redundant_metadata    all                       default
zroot/ZVOL/virtbsd  encryption            off                       default
zroot/ZVOL/virtbsd  keylocation           none                      default
zroot/ZVOL/virtbsd  keyformat             none                      default
zroot/ZVOL/virtbsd  pbkdf2iters           0                         default
zroot/ZVOL/virtbsd  snapshots_changed     Fri Feb  2  4:20:00 2024  -






This is how ZFS looks inside the guest:=20

[virtbsd] ~# gpart show
=3D>       40  104857520  vtbd0  GPT  (50G)
         40       1024      1  freebsd-boot  (512K)
       1064   94371840      2  freebsd-zfs  (45G)
   94372904   10484656      3  freebsd-swap  (5.0G)

[virtbsd] ~# zpool list
NAME      SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEA=
LTH=20
ALTROOT
vmzroot  44.5G  8.98G  35.5G        -         -    32%    20%  1.00x    ONL=
INE=20
-
[virtbsd] ~# zpool status
  pool: vmzroot
 state: ONLINE
  scan: scrub repaired 0B in 00:00:16 with 0 errors on Sat Jan 27 03:01:33 =
2024
config:

        NAME          STATE     READ WRITE CKSUM
        vmzroot       ONLINE       0     0     0
          gpt/vmzfs0  ONLINE       0     0     0

errors: No known data errors

[virtbsd] ~# zpool get all vmzroot
NAME     PROPERTY                       VALUE                          SOUR=
CE
vmzroot  size                           44.5G                          -
vmzroot  capacity                       20%                            -
vmzroot  altroot                        -                              defa=
ult
vmzroot  health                         ONLINE                         -
vmzroot  guid                           14496911069470365237           -
vmzroot  version                        -                              defa=
ult
vmzroot  bootfs                         vmzroot/ROOT/default           local
vmzroot  delegation                     on                             defa=
ult
vmzroot  autoreplace                    off                            defa=
ult
vmzroot  cachefile                      -                              defa=
ult
vmzroot  failmode                       wait                           defa=
ult
vmzroot  listsnapshots                  off                            defa=
ult
vmzroot  autoexpand                     off                            defa=
ult
vmzroot  dedupratio                     1.00x                          -
vmzroot  free                           35.5G                          -
vmzroot  allocated                      8.98G                          -
vmzroot  readonly                       off                            -
vmzroot  ashift                         0                              defa=
ult
vmzroot  comment                        -                              defa=
ult
vmzroot  expandsize                     -                              -
vmzroot  freeing                        0                              -
vmzroot  fragmentation                  32%                            -
vmzroot  leaked                         0                              -
vmzroot  multihost                      off                            defa=
ult
vmzroot  checkpoint                     -                              -
vmzroot  load_guid                      9652613418712187666            -
vmzroot  autotrim                       on                             local
vmzroot  compatibility                  off                            defa=
ult
vmzroot  bcloneused                     0                              -
vmzroot  bclonesaved                    0                              -
vmzroot  bcloneratio                    1.00x                          -
vmzroot  feature@async_destroy          enabled                        local
vmzroot  feature@empty_bpobj            active                         local
vmzroot  feature@lz4_compress           active                         local
vmzroot  feature@multi_vdev_crash_dump  enabled                        local
vmzroot  feature@spacemap_histogram     active                         local
vmzroot  feature@enabled_txg            active                         local
vmzroot  feature@hole_birth             active                         local
vmzroot  feature@extensible_dataset     active                         local
vmzroot  feature@embedded_data          active                         local
vmzroot  feature@bookmarks              enabled                        local
vmzroot  feature@filesystem_limits      enabled                        local
vmzroot  feature@large_blocks           enabled                        local
vmzroot  feature@large_dnode            enabled                        local
vmzroot  feature@sha512                 enabled                        local
vmzroot  feature@skein                  enabled                        local
vmzroot  feature@edonr                  enabled                        local
vmzroot  feature@userobj_accounting     active                         local
vmzroot  feature@encryption             enabled                        local
vmzroot  feature@project_quota          active                         local
vmzroot  feature@device_removal         enabled                        local
vmzroot  feature@obsolete_counts        enabled                        local
vmzroot  feature@zpool_checkpoint       enabled                        local
vmzroot  feature@spacemap_v2            active                         local
vmzroot  feature@allocation_classes     enabled                        local
vmzroot  feature@resilver_defer         enabled                        local
vmzroot  feature@bookmark_v2            enabled                        local
vmzroot  feature@redaction_bookmarks    enabled                        local
vmzroot  feature@redacted_datasets      enabled                        local
vmzroot  feature@bookmark_written       enabled                        local
vmzroot  feature@log_spacemap           active                         local
vmzroot  feature@livelist               enabled                        local
vmzroot  feature@device_rebuild         enabled                        local
vmzroot  feature@zstd_compress          enabled                        local
vmzroot  feature@draid                  enabled                        local
vmzroot  feature@zilsaxattr             active                         local
vmzroot  feature@head_errlog            active                         local
vmzroot  feature@blake3                 enabled                        local
vmzroot  feature@block_cloning          enabled                        local
vmzroot  feature@vdev_zaps_v2           active                         local

--=20
You are receiving this mail because:
You are on the CC list for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-273772-3630-9zxIKFhOUs>