Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 1 Apr 2013 16:18:23 +0200
From:      Fabian Keil <freebsd-listen@fabiankeil.de>
To:        FreeBSD Current <freebsd-current@freebsd.org>
Subject:   panic: solaris assert: sa.sa_magic == 0x2F505A (0x4d5ea364 == 0x2f505a), file: /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vfsops.c, line: 625
Message-ID:  <20130401161813.6e42e2a1@fabiankeil.de>

next in thread | raw e-mail | index | archive | help
--Sig_/rndEUROgXS3aLZUHjnb9sGB
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

I got the following panic on 10.0-CURRENT from two days ago
while receiving an incremental snapshot to a certain pool:

(kgdb) where
#0  doadump (textdump=3D0) at pcpu.h:229
#1  0xffffffff8031a3ce in db_dump (dummy=3D<value optimized out>, dummy2=3D=
0, dummy3=3D0, dummy4=3D0x0) at /usr/src/sys/ddb/db_command.c:543
#2  0xffffffff80319eca in db_command (last_cmdp=3D<value optimized out>, cm=
d_table=3D<value optimized out>, dopager=3D1) at /usr/src/sys/ddb/db_comman=
d.c:449
#3  0xffffffff80319c82 in db_command_loop () at /usr/src/sys/ddb/db_command=
.c:502
#4  0xffffffff8031c5d0 in db_trap (type=3D<value optimized out>, code=3D0) =
at /usr/src/sys/ddb/db_main.c:231
#5  0xffffffff805d0da3 in kdb_trap (type=3D3, code=3D0, tf=3D<value optimiz=
ed out>) at /usr/src/sys/kern/subr_kdb.c:654
#6  0xffffffff8087fdc3 in trap (frame=3D0xffffff80dc9d6520) at /usr/src/sys=
/amd64/amd64/trap.c:579
#7  0xffffffff80869cb2 in calltrap () at exception.S:228
#8  0xffffffff805d058e in kdb_enter (why=3D0xffffffff80a47e7a "panic", msg=
=3D<value optimized out>) at cpufunc.h:63
#9  0xffffffff80599216 in panic (fmt=3D<value optimized out>) at /usr/src/s=
ys/kern/kern_shutdown.c:747
#10 0xffffffff8130323f in assfail3 (a=3D<value optimized out>, lv=3D<value =
optimized out>, op=3D<value optimized out>, rv=3D<value optimized out>, f=
=3D<value optimized out>, l=3D<value optimized out>)
    at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/=
opensolaris_cmn_err.c:89
#11 0xffffffff8117924e in zfs_space_delta_cb (bonustype=3D<value optimized =
out>, data=3D0xffffff8015eeb8c0, userp=3D0xfffffe004261c640, groupp=3D0xfff=
ffe004261c648)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/f=
s/zfs/zfs_vfsops.c:625
#12 0xffffffff8110003b in dmu_objset_userquota_get_ids (dn=3D0xfffffe004261=
c358, before=3D0, tx=3D<value optimized out>) at /usr/src/sys/modules/zfs/.=
./../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249
#13 0xffffffff811071b6 in dnode_sync (dn=3D0xfffffe004261c358, tx=3D0xfffff=
e00186e1300) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts=
/common/fs/zfs/dnode_sync.c:554
#14 0xffffffff810ff98b in dmu_objset_sync_dnodes (list=3D0xfffffe00691a5250=
, newlist=3D<value optimized out>, tx=3D<value optimized out>)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/f=
s/zfs/dmu_objset.c:910
#15 0xffffffff810ff825 in dmu_objset_sync (os=3D0xfffffe00691a5000, pio=3D<=
value optimized out>, tx=3D0xfffffe00186e1300)
    at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/f=
s/zfs/dmu_objset.c:1027
#16 0xffffffff8110cb0d in dsl_dataset_sync (ds=3D0xfffffe001f3d0c00, zio=3D=
0x780, tx=3D0xfffffe00186e1300) at /usr/src/sys/modules/zfs/../../cddl/cont=
rib/opensolaris/uts/common/fs/zfs/dsl_dataset.c:1411
#17 0xffffffff8111399a in dsl_pool_sync (dp=3D0xfffffe0069ec4000, txg=3D<va=
lue optimized out>) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensola=
ris/uts/common/fs/zfs/dsl_pool.c:409
#18 0xffffffff8112f0ee in spa_sync (spa=3D0xfffffe0050f00000, txg=3D3292) a=
t /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs=
/spa.c:6328
#19 0xffffffff81137c45 in txg_sync_thread (arg=3D0xfffffe0069ec4000) at /us=
r/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/txg.=
c:493
#20 0xffffffff80569c1a in fork_exit (callout=3D0xffffffff811378d0 <txg_sync=
_thread>, arg=3D0xfffffe0069ec4000, frame=3D0xffffff80dc9d6c00) at /usr/src=
/sys/kern/kern_fork.c:991
#21 0xffffffff8086a1ee in fork_trampoline () at exception.S:602
#22 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal
(kgdb) f 12
#12 0xffffffff8110003b in dmu_objset_userquota_get_ids (dn=3D0xfffffe004261=
c358, before=3D0, tx=3D<value optimized out>) at /usr/src/sys/modules/zfs/.=
./../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c:1249
1249		error =3D used_cbs[os->os_phys->os_type](dn->dn_bonustype, data,
(kgdb) p *dn
$1 =3D {dn_struct_rwlock =3D {lock_object =3D {lo_name =3D 0xffffffff811da0=
a9 "dn->dn_struct_rwlock", lo_flags =3D 40960000, lo_data =3D 0, lo_witness=
 =3D 0x0}, sx_lock =3D 1}, dn_link =3D {list_next =3D 0xfffffe0042629020,=20
    list_prev =3D 0xfffffe00691a5360}, dn_objset =3D 0xfffffe00691a5000, dn=
_object =3D 55652, dn_dbuf =3D 0xfffffe00427ad0e0, dn_handle =3D 0xfffffe00=
69f70128, dn_phys =3D 0xffffff8015eeb800,=20
  dn_type =3D DMU_OT_PLAIN_FILE_CONTENTS, dn_bonuslen =3D 192, dn_bonustype=
 =3D 44 ',', dn_nblkptr =3D 1 '\001', dn_checksum =3D 0 '\0', dn_compress =
=3D 0 '\0', dn_nlevels =3D 1 '\001', dn_indblkshift =3D 14 '\016',=20
  dn_datablkshift =3D 0 '\0', dn_moved =3D 0 '\0', dn_datablkszsec =3D 10, =
dn_datablksz =3D 5120, dn_maxblkid =3D 0, dn_next_nblkptr =3D "\000\000\000=
", dn_next_nlevels =3D "\000\000\000",=20
  dn_next_indblkshift =3D "\000\000\000", dn_next_bonustype =3D ",\000\000"=
, dn_rm_spillblk =3D "\000\000\000", dn_next_bonuslen =3D {192, 0, 0, 0}, d=
n_next_blksz =3D {0, 0, 0, 0}, dn_dbufs_count =3D 0, dn_dirty_link =3D {{
      list_next =3D 0xfffffe00691a51f0, list_prev =3D 0xfffffe0042628ab0}, =
{list_next =3D 0x0, list_prev =3D 0x0}, {list_next =3D 0x0, list_prev =3D 0=
x0}, {list_next =3D 0x0, list_prev =3D 0x0}}, dn_mtx =3D {lock_object =3D {
      lo_name =3D 0xffffffff811da0bf "dn->dn_mtx", lo_flags =3D 40960000, l=
o_data =3D 0, lo_witness =3D 0x0}, sx_lock =3D 1}, dn_dirty_records =3D {{l=
ist_size =3D 208, list_offset =3D 0, list_head =3D {
        list_next =3D 0xfffffe004261c470, list_prev =3D 0xfffffe004261c470}=
}, {list_size =3D 208, list_offset =3D 0, list_head =3D {list_next =3D 0xff=
fffe004261c490, list_prev =3D 0xfffffe004261c490}}, {list_size =3D 208,=20
      list_offset =3D 0, list_head =3D {list_next =3D 0xfffffe004261c4b0, l=
ist_prev =3D 0xfffffe004261c4b0}}, {list_size =3D 208, list_offset =3D 0, l=
ist_head =3D {list_next =3D 0xfffffe004261c4d0,=20
        list_prev =3D 0xfffffe004261c4d0}}}, dn_ranges =3D {{avl_root =3D 0=
x0, avl_compar =3D 0xffffffff81106ec0 <free_range_compar>, avl_offset =3D 0=
, avl_numnodes =3D 0, avl_size =3D 40}, {avl_root =3D 0x0,=20
      avl_compar =3D 0xffffffff81106ec0 <free_range_compar>, avl_offset =3D=
 0, avl_numnodes =3D 0, avl_size =3D 40}, {avl_root =3D 0x0, avl_compar =3D=
 0xffffffff81106ec0 <free_range_compar>, avl_offset =3D 0,=20
      avl_numnodes =3D 0, avl_size =3D 40}, {avl_root =3D 0x0, avl_compar =
=3D 0xffffffff81106ec0 <free_range_compar>, avl_offset =3D 0, avl_numnodes =
=3D 0, avl_size =3D 40}}, dn_allocated_txg =3D 3292, dn_free_txg =3D 0,=20
  dn_assigned_txg =3D 0, dn_notxholds =3D {cv_description =3D 0xffffffff811=
da0dd "dn->dn_notxholds", cv_waiters =3D 0}, dn_dirtyctx =3D DN_UNDIRTIED, =
dn_dirtyctx_firstset =3D 0x0, dn_tx_holds =3D {rc_count =3D 0},=20
  dn_holds =3D {rc_count =3D 3}, dn_dbufs_mtx =3D {lock_object =3D {lo_name=
 =3D 0xffffffff811da0cb "dn->dn_dbufs_mtx", lo_flags =3D 40960000, lo_data =
=3D 0, lo_witness =3D 0x0}, sx_lock =3D 1}, dn_dbufs =3D {list_size =3D 224=
,=20
    list_offset =3D 176, list_head =3D {list_next =3D 0xfffffe004261c5f8, l=
ist_prev =3D 0xfffffe004261c5f8}}, dn_bonus =3D 0x0, dn_have_spill =3D 0, d=
n_zio =3D 0xfffffe00695af000, dn_oldused =3D 2560, dn_oldflags =3D 3,=20
  dn_olduid =3D 1001, dn_oldgid =3D 1001, dn_newuid =3D 0, dn_newgid =3D 0,=
 dn_id_flags =3D 5, dn_zfetch =3D {zf_rwlock =3D {lock_object =3D {lo_name =
=3D 0xffffffff811dd156 "zf->zf_rwlock", lo_flags =3D 40960000, lo_data =3D =
0,=20
        lo_witness =3D 0x0}, sx_lock =3D 1}, zf_stream =3D {list_size =3D 1=
12, list_offset =3D 88, list_head =3D {list_next =3D 0xfffffe004261c688, li=
st_prev =3D 0xfffffe004261c688}}, zf_dnode =3D 0xfffffe004261c358,=20
    zf_stream_cnt =3D 0, zf_alloc_fail =3D 0}}

The incremental was created with:
zfs send -i @2013-03-28_14:21 tank/home/fk@2013-04-01_12:31
piped through mbuffer and received with:
zfs receive -v -u -F rockbox/backup/r500/tank/home/fk

Reading the incremental directly from a file triggers the
panic as as well, but sometimes it takes more than one attempt.

The offending sa_magic in the panic message is always the same.

The receiving pool appears to be okay:

fk@r500 ~ $zpool status rockbox
  pool: rockbox
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
	still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(7) for details.
  scan: scrub repaired 0 in 0h3m with 0 errors on Mon Apr  1 13:57:35 2013
config:

	NAME                 STATE     READ WRITE CKSUM
	rockbox              ONLINE       0     0     0
	  label/rockbox.eli  ONLINE       0     0     0

errors: No known data errors

The feature that isn't yet enabled is lz4 but after upgrading
a copy of the pool the panic was still reproducible. On the
receiving side gzip compression is enabled.

Fabian

--Sig_/rndEUROgXS3aLZUHjnb9sGB
Content-Type: application/pgp-signature; name=signature.asc
Content-Disposition: attachment; filename=signature.asc

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (FreeBSD)

iEYEARECAAYFAlFZly4ACgkQBYqIVf93VJ1R4QCgoIwD4kojtAN8nO/m1yxVpbFS
4JMAn2INHipf/UWOAvwSkaWXlBc4teHB
=ay0r
-----END PGP SIGNATURE-----

--Sig_/rndEUROgXS3aLZUHjnb9sGB--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130401161813.6e42e2a1>