Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 12 Apr 2018 12:06:40 +0000
From:      bugzilla-noreply@freebsd.org
To:        bugs@FreeBSD.org
Subject:   [Bug 227465] [zfs][iscsi]repeatable panics on iSCSI target/zvol creation
Message-ID:  <bug-227465-227@https.bugs.freebsd.org/bugzilla/>

next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227465

            Bug ID: 227465
           Summary: [zfs][iscsi]repeatable panics on iSCSI target/zvol
                    creation
           Product: Base System
           Version: 11.1-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: emz@norma.perm.ru

I'm getting repeatable panics while massively creating zvols and iSCSI targ=
ets
pointing to these zvols. At the time of the panicking volmode was default, =
and
vfs.zfs.vol.mode was 1, thus the effective volmode was geom.

I hope that switching to volmode=3Ddev will help me with getting rid of thi=
s, but
I'm still reporting it just in case someone would want to have a look.

Backtrace from 11.1-RELEASE-p6 r329259:
=3D=3D=3DCut=3D=3D=3D
# kgdb /boot/kernel.old/kernel vmcore.0=20
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain condition=
s.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
stack pointer           =3D 0x28:0xfffffe10485e9300
frame pointer           =3D 0x28:0xfffffe10485e9300
code segment            =3D base 0x0, limit 0xfffff, type 0x1b
                        =3D DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        =3D interrupt enabled, resume, IOPL =3D 0
current process         =3D 761 (ctld)
trap number             =3D 12
panic: page fault
cpuid =3D 7
KDB: stack backtrace:
#0 0xffffffff80aadac7 at kdb_backtrace+0x67
#1 0xffffffff80a6bba6 at vpanic+0x186
#2 0xffffffff80a6ba13 at panic+0x43
#3 0xffffffff80edf832 at trap_fatal+0x322
#4 0xffffffff80edf889 at trap_pfault+0x49
#5 0xffffffff80edf0c6 at trap+0x286
#6 0xffffffff80ec36d1 at calltrap+0x8
#7 0xffffffff822492f6 at zvol_geom_start+0xf6
#8 0xffffffff809bc477 at g_io_request+0x297
#9 0xffffffff809bc195 at g_io_getattr+0x75
#10 0xffffffff809b80bf at g_dev_ioctl+0x33f
#11 0xffffffff8263afe6 at ctl_be_block_open_dev+0x626
#12 0xffffffff82639ddd at ctl_be_block_ioctl+0x92d
#13 0xffffffff82632bdc at ctl_ioctl+0x48c
#14 0xffffffff8093ae38 at devfs_ioctl_f+0x128
#15 0xffffffff80ac9415 at kern_ioctl+0x255
#16 0xffffffff80ac914f at sys_ioctl+0x16f
#17 0xffffffff80ee0394 at amd64_syscall+0x6c4
Uptime: 6d20h47m1s
Dumping 25781 out of 65386 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%.=
.91%

Reading symbols from /boot/kernel.old/zfs.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/zfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/zfs.ko
Reading symbols from /boot/kernel.old/opensolaris.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/opensolaris.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/opensolaris.ko
Reading symbols from /boot/kernel.old/geom_mirror.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/geom_mirror.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/geom_mirror.ko
Reading symbols from /boot/kernel.old/pf.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/pf.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/pf.ko
Reading symbols from /boot/kernel.old/mlx5.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/mlx5.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/mlx5.ko
Reading symbols from /boot/kernel.old/linuxkpi.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/linuxkpi.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/linuxkpi.ko
Reading symbols from /boot/kernel.old/mlx5en.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/mlx5en.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/mlx5en.ko
Reading symbols from /boot/kernel.old/ichwd.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/ichwd.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/ichwd.ko
Reading symbols from /boot/kernel.old/ctl.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/ctl.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/ctl.ko
Reading symbols from /boot/kernel.old/cfiscsi.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/cfiscsi.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/cfiscsi.ko
Reading symbols from /boot/kernel.old/iscsi.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/iscsi.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/iscsi.ko
Reading symbols from /boot/kernel.old/ng_socket.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/ng_socket.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/ng_socket.ko
Reading symbols from /boot/kernel.old/netgraph.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/netgraph.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/netgraph.ko
Reading symbols from /boot/kernel.old/ng_netflow.ko...Reading symbols from
/usr/lib/debug//boot/kernel.old/ng_netflow.ko.debug...done.
done.
Loaded symbols for /boot/kernel.old/ng_netflow.ko
#0  doadump (textdump=3D<value optimized out>) at pcpu.h:222
222     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump (textdump=3D<value optimized out>) at pcpu.h:222
#1  0xffffffff80a6b721 in kern_reboot (howto=3D260) at
/usr/src/sys/kern/kern_shutdown.c:366
#2  0xffffffff80a6bbe0 in vpanic (fmt=3D<value optimized out>, ap=3D<value
optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:759
#3  0xffffffff80a6ba13 in panic (fmt=3D<value optimized out>) at
/usr/src/sys/kern/kern_shutdown.c:690
#4  0xffffffff80edf832 in trap_fatal (frame=3D0xfffffe10485e9240, eva=3D8) =
at
/usr/src/sys/amd64/amd64/trap.c:801
#5  0xffffffff80edf889 in trap_pfault (frame=3D0xfffffe10485e9240, usermode=
=3D0) at
pcpu.h:222
#6  0xffffffff80edf0c6 in trap (frame=3D0xfffffe10485e9240) at
/usr/src/sys/amd64/amd64/trap.c:421
#7  0xffffffff80ec36d1 in calltrap () at
/usr/src/sys/amd64/amd64/exception.S:236
#8  0xffffffff821ab494 in dmu_objset_spa (os=3D0x0)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c=
:86
#9  0xffffffff822492f6 in zvol_geom_start (bp=3D0xfffff80b1e45a000)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:2752
#10 0xffffffff809bc477 in g_io_request (bp=3D0xfffff80b1e45a000,
cp=3D0xfffff801d9a4c080)
    at /usr/src/sys/geom/geom_io.c:626
#11 0xffffffff809bc195 in g_io_getattr (attr=3D0xfffffe10485e94e8
"GEOM::candelete", cp=3D<value optimized out>,=20
    len=3D0xfffffe10485e9528, ptr=3D0xfffffe10485e9530) at
/usr/src/sys/geom/geom_io.c:303
#12 0xffffffff809b80bf in g_dev_ioctl (dev=3D<value optimized out>, cmd=3D<=
value
optimized out>,=20
    data=3D0xfffffe10485e94e8 "GEOM::candelete", fflag=3D<value optimized o=
ut>,
td=3D0xfffff80144281000) at atomic.h:188
#13 0xffffffff8263afe6 in ctl_be_block_open_dev (be_lun=3D0xfffff8097076b40=
0,
req=3D<value optimized out>)
    at /usr/src/sys/cam/ctl/ctl_backend_block.c:2072
#14 0xffffffff82639ddd in ctl_be_block_ioctl (dev=3D<value optimized out>,
cmd=3D<value optimized out>,=20
    addr=3D0xfffff8084df9e000 "block", flag=3D<value optimized out>, td=3D<=
value
optimized out>)
    at /usr/src/sys/cam/ctl/ctl_backend_block.c:2610
#15 0xffffffff82632bdc in ctl_ioctl (dev=3D0xfffff80106088200, cmd=3D<value
optimized out>,=20
    addr=3D0xfffff8084df9e000 "block", flag=3D3, td=3D0xfffff80144281000) at
/usr/src/sys/cam/ctl/ctl.c:3012
#16 0xffffffff8093ae38 in devfs_ioctl_f (fp=3D0xfffff8004a9f66e0, com=3D324=
3303201,
data=3D0xfffff8084df9e000,=20
    cred=3D0xfffff80019719000, td=3D0xfffff80144281000) at
/usr/src/sys/fs/devfs/devfs_vnops.c:791
#17 0xffffffff80ac9415 in kern_ioctl (td=3D<value optimized out>, fd=3D4,
com=3D<value optimized out>,=20
    data=3D<value optimized out>) at file.h:323
#18 0xffffffff80ac914f in sys_ioctl (td=3D<value optimized out>,
uap=3D0xfffffe10485e9a30)
    at /usr/src/sys/kern/sys_generic.c:745
#19 0xffffffff80ee0394 in amd64_syscall (td=3D0xfffff80144281000, traced=3D=
0) at
subr_syscall.c:135
#20 0xffffffff80ec39bb in Xfast_syscall () at
/usr/src/sys/amd64/amd64/exception.S:396
#21 0x000000080163f56a in ?? ()
Previous frame inner to this frame (corrupt stack?)
Current language:  auto; currently minimal
(kgdb)
=3D=3D=3DCut=3D=3D=3D

Panic from 11.1-STABLE r332062:
=3D=3D=3DCut=3D=3D=3D
# kgdb /boot/kernel/kernel vmcore.0=20=20=20=20=20
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain condition=
s.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:
trap number             =3D 12
panic: page fault
cpuid =3D 21
KDB: stack backtrace:
#0 0xffffffff80b3a367 at kdb_backtrace+0x67
#1 0xffffffff80af3f77 at vpanic+0x177
#2 0xffffffff80af3df3 at panic+0x43
#3 0xffffffff80f71d42 at trap_fatal+0x352
#4 0xffffffff80f71d99 at trap_pfault+0x49
#5 0xffffffff80f71617 at trap+0x2c7
#6 0xffffffff80f51cdc at calltrap+0x8
#7 0xffffffff822fd526 at zvol_geom_start+0xf6
#8 0xffffffff80a463b5 at g_io_schedule_down+0x1a5
#9 0xffffffff80a46cad at g_down_procbody+0x6d
#10 0xffffffff80ab7973 at fork_exit+0x83
#11 0xffffffff80f529fe at fork_trampoline+0xe
Uptime: 6d4h36m7s
Dumping 13242 out of 65385 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%.=
.91%

Reading symbols from /boot/kernel/zfs.ko...Reading symbols from
/usr/lib/debug//boot/kernel/zfs.ko.debug...done.
done.
Loaded symbols for /boot/kernel/zfs.ko
Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from
/usr/lib/debug//boot/kernel/opensolaris.ko.debug...done.
done.
Loaded symbols for /boot/kernel/opensolaris.ko
Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from
/usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done.
done.
Loaded symbols for /boot/kernel/geom_mirror.ko
Reading symbols from /boot/kernel/pf.ko...Reading symbols from
/usr/lib/debug//boot/kernel/pf.ko.debug...done.
done.
Loaded symbols for /boot/kernel/pf.ko
Reading symbols from /boot/kernel/mlx5.ko...Reading symbols from
/usr/lib/debug//boot/kernel/mlx5.ko.debug...done.
done.
Loaded symbols for /boot/kernel/mlx5.ko
Reading symbols from /boot/kernel/linuxkpi.ko...Reading symbols from
/usr/lib/debug//boot/kernel/linuxkpi.ko.debug...done.
done.
Loaded symbols for /boot/kernel/linuxkpi.ko
Reading symbols from /boot/kernel/mlx5en.ko...Reading symbols from
/usr/lib/debug//boot/kernel/mlx5en.ko.debug...done.
done.
Loaded symbols for /boot/kernel/mlx5en.ko
Reading symbols from /boot/kernel/ichwd.ko...Reading symbols from
/usr/lib/debug//boot/kernel/ichwd.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ichwd.ko
Reading symbols from /boot/kernel/ctl.ko...Reading symbols from
/usr/lib/debug//boot/kernel/ctl.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ctl.ko
Reading symbols from /boot/kernel/cfiscsi.ko...Reading symbols from
/usr/lib/debug//boot/kernel/cfiscsi.ko.debug...done.
done.
Loaded symbols for /boot/kernel/cfiscsi.ko
Reading symbols from /boot/kernel/iscsi.ko...Reading symbols from
/usr/lib/debug//boot/kernel/iscsi.ko.debug...done.
done.
Loaded symbols for /boot/kernel/iscsi.ko
Reading symbols from /boot/kernel/ng_socket.ko...Reading symbols from
/usr/lib/debug//boot/kernel/ng_socket.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_socket.ko
Reading symbols from /boot/kernel/netgraph.ko...Reading symbols from
/usr/lib/debug//boot/kernel/netgraph.ko.debug...done.
done.
Loaded symbols for /boot/kernel/netgraph.ko
Reading symbols from /boot/kernel/ng_netflow.ko...Reading symbols from
/usr/lib/debug//boot/kernel/ng_netflow.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ng_netflow.ko
Reading symbols from /boot/kernel/dtraceall.ko...Reading symbols from
/usr/lib/debug//boot/kernel/dtraceall.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtraceall.ko
Reading symbols from /boot/kernel/dtrace.ko...Reading symbols from
/usr/lib/debug//boot/kernel/dtrace.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtrace.ko
Reading symbols from /boot/kernel/dtmalloc.ko...Reading symbols from
/usr/lib/debug//boot/kernel/dtmalloc.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtmalloc.ko
Reading symbols from /boot/kernel/dtnfscl.ko...Reading symbols from
/usr/lib/debug//boot/kernel/dtnfscl.ko.debug...done.
done.
Loaded symbols for /boot/kernel/dtnfscl.ko
Reading symbols from /boot/kernel/fbt.ko...Reading symbols from
/usr/lib/debug//boot/kernel/fbt.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fbt.ko
Reading symbols from /boot/kernel/fasttrap.ko...Reading symbols from
/usr/lib/debug//boot/kernel/fasttrap.ko.debug...done.
done.
Loaded symbols for /boot/kernel/fasttrap.ko
Reading symbols from /boot/kernel/sdt.ko...Reading symbols from
/usr/lib/debug//boot/kernel/sdt.ko.debug...done.
done.
Loaded symbols for /boot/kernel/sdt.ko
Reading symbols from /boot/kernel/systrace.ko...Reading symbols from
/usr/lib/debug//boot/kernel/systrace.ko.debug...done.
done.
Loaded symbols for /boot/kernel/systrace.ko
Reading symbols from /boot/kernel/systrace_freebsd32.ko...Reading symbols f=
rom
/usr/lib/debug//boot/kernel/systrace_freebsd32.ko.debug...done.
done.
Loaded symbols for /boot/kernel/systrace_freebsd32.ko
Reading symbols from /boot/kernel/profile.ko...Reading symbols from
/usr/lib/debug//boot/kernel/profile.ko.debug...done.
done.
Loaded symbols for /boot/kernel/profile.ko
Reading symbols from /boot/kernel/ksyms.ko...Reading symbols from
/usr/lib/debug//boot/kernel/ksyms.ko.debug...done.
done.
Loaded symbols for /boot/kernel/ksyms.ko
#0  doadump (textdump=3D<value optimized out>) at pcpu.h:229
229     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump (textdump=3D<value optimized out>) at pcpu.h:229
#1  0xffffffff80af3b8b in kern_reboot (howto=3D260) at
/usr/src/sys/kern/kern_shutdown.c:383
#2  0xffffffff80af3fb1 in vpanic (fmt=3D<value optimized out>, ap=3D<value
optimized out>)
    at /usr/src/sys/kern/kern_shutdown.c:776
#3  0xffffffff80af3df3 in panic (fmt=3D<value optimized out>) at
/usr/src/sys/kern/kern_shutdown.c:707
#4  0xffffffff80f71d42 in trap_fatal (frame=3D0xfffffe10470bd910, eva=3D8) =
at
/usr/src/sys/amd64/amd64/trap.c:817
#5  0xffffffff80f71d99 in trap_pfault (frame=3D0xfffffe10470bd910, usermode=
=3D0) at
pcpu.h:229
#6  0xffffffff80f71617 in trap (frame=3D0xfffffe10470bd910) at
/usr/src/sys/amd64/amd64/trap.c:410
#7  0xffffffff80f51cdc in calltrap () at
/usr/src/sys/amd64/amd64/exception.S:232
#8  0xffffffff82256ed4 in dmu_objset_spa (os=3D0x0)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c=
:94
#9  0xffffffff822fd526 in zvol_geom_start (bp=3D0xfffff80ac4bcd468)
    at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zvol.c:2769
#10 0xffffffff80a463b5 in g_io_schedule_down (tp=3D<value optimized out>) at
/usr/src/sys/geom/geom_io.c:881
#11 0xffffffff80a46cad in g_down_procbody (arg=3D<value optimized out>) at
/usr/src/sys/geom/geom_kern.c:109
#12 0xffffffff80ab7973 in fork_exit (callout=3D0xffffffff80a46c40
<g_down_procbody>, arg=3D0x0, frame=3D0xfffffe10470bdac0)
    at /usr/src/sys/kern/kern_fork.c:1053
#13 0xffffffff80f529fe in fork_trampoline () at
/usr/src/sys/amd64/amd64/exception.S:843
#14 0x0000000000000000 in ?? ()
Current language:  auto; currently minimal
(kgdb)
=3D=3D=3DCut=3D=3D=3D

--=20
You are receiving this mail because:
You are the assignee for the bug.=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-227465-227>