Date: Mon, 27 Oct 2025 12:19:38 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 290570] [zfs] panic: buffer modified while frozen! Message-ID: <bug-290570-227@https.bugs.freebsd.org/bugzilla/>
index | next in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290570 Bug ID: 290570 Summary: [zfs] panic: buffer modified while frozen! Product: Base System Version: 14.3-STABLE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: vova@fbsd.ru running stable/14-087fc6ae0a8c, amd64 with debug configuration (std.debug): got panic on snapshot removal, while resilver was in progress, pool state was online: (kgdb) bt #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80b0d420 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:523 #3 0xffffffff80b0d939 in vpanic (fmt=0xffffffff828755e0 "buffer modified while frozen!", ap=ap@entry=0xfffffe01590f5ce0) at /usr/src/sys/kern/kern_shutdown.c:967 #4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:891 #5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446 #6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980, zio@entry=<error reading variable: value is not available>) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593 #7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #9 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #10 0xffffffff80b75392 in taskqueue_run_locked (queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 #11 0xffffffff80b76283 in taskqueue_thread_loop (arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 #12 0xffffffff80ac0022 in fork_exit (callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40) at /usr/src/sys/kern/kern_fork.c:1153 #13 <signal handler called> #14 0x6bec65f367ac65f7 in ?? () Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb from core.txt: __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 57 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (offsetof(struct pcpu, (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 td = <optimized out> #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 error = 0 coredump = <optimized out> #2 0xffffffff80b0d420 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:523 once = 0 __pc = 0x0 #3 0xffffffff80b0d939 in vpanic ( fmt=0xffffffff828755e0 "buffer modified while frozen!", ap=ap@entry=0xfffffe01590f5ce0) at /usr/src/sys/kern/kern_shutdown.c:967 buf = "buffer modified while frozen!", '\000' <repeats 226 times> __pc = 0x0 __pc = 0x0 __pc = 0x0 other_cpus = {__bits = {4093, 0 <repeats 15 times>}} td = 0xfffff8000e2e2000 bootopt = <unavailable> newpanic = <optimized out> #4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:891 ap = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0xfffffe01590f5d10, reg_save_area = 0xfffffe01590f5cb0}} #5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446 zc = {zc_word = {9183713730946906330, 4141878062941643272, 14528101124437143189, 16728594463488619406}} hdr = 0xfffff804979abbc0 #6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980, zio@entry=<error reading variable: value is not available>) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593 hash_lock = 0x0 exists = <optimized out> callback = 0xfffff8026374a140 buf = 0xfffff80603cb39a0 hdr = 0xfffff804979abbc0 #7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 psize = 8192 zl = 0x0 pio = <optimized out> pio_next = <optimized out> next_to_execute = <optimized out> #8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 stage = ZIO_STAGE_DONE pipeline = <optimized out> #9 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 cookie = 0 #10 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 et = {et_link = {tqe_next = 0xffffffff812183c3, tqe_prev = 0x0}, et_td = 0x0, et_section = {bucket = 2166642066}, et_old_priority = 255 '\377'} tb = {tb_running = 0xfffffe038b8c1bb8, tb_seq = 6978482, tb_canceling = false, tb_link = {le_next = 0xfffffe0159104e78, le_prev = 0xfffffe01590fae88}} in_net_epoch = false pending = 1 task = <optimized out> #11 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 tqp = <optimized out> tq = 0xfffff8000e2f2e00 #12 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40) at /usr/src/sys/kern/kern_fork.c:1153 __pc = 0x0 __pc = 0x0 td = 0xfffff8000e2e2000 p = 0xffffffff81b80fa0 <proc0> dtd = <optimized out> #13 <signal handler called> No locals. #14 0x6bec65f367ac65f7 in ?? () No symbol table info available. Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb (kgdb) (kgdb) Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100266 (CPU 0) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xffffffff81cef340 <nmi0_stack+3888>) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 _sx_xunlock_hard (sx=sx@entry=0xfffff804976ca298, x=<optimized out>, x@entry=18446735277854429184, file=<optimized out>, line=<optimized out>) at /usr/src/sys/kern/kern_sx.c:937 #5 0xffffffff80b197a9 in _sx_xunlock (sx=sx@entry=0xfffff804976ca298, file=<optimized out>, line=line@entry=1447) at /usr/src/sys/kern/kern_sx.c:412 #6 0xffffffff825e343d in arc_cksum_verify (buf=buf@entry=0xfffff804343bf620) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1447 #7 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344be0ac0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593 #8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be0ac0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #10 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #11 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 #12 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 #13 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590ebf40) at /usr/src/sys/kern/kern_fork.c:1153 #14 <signal handler called> #15 0xc4719ce7c8319ce3 in ?? () Backtrace stopped: Cannot access memory at address 0x6b4607ab6f0607af Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100268 (CPU 1) #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 #1 doadump (textdump=textdump@entry=1) at /usr/src/sys/kern/kern_shutdown.c:405 #2 0xffffffff80b0d420 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:523 #3 0xffffffff80b0d939 in vpanic ( fmt=0xffffffff828755e0 "buffer modified while frozen!", ap=ap@entry=0xfffffe01590f5ce0) at /usr/src/sys/kern/kern_shutdown.c:967 #4 0xffffffff80b0d6c3 in panic (fmt=<unavailable>) at /usr/src/sys/kern/kern_shutdown.c:891 #5 0xffffffff825e34a1 in arc_cksum_verify (buf=0xfffff80603cb39a0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:1446 #6 0xffffffff825f14d8 in arc_write_done (zio=0xfffffe0344bd9980, zio@entry=<error reading variable: value is not available>) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6593 #7 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bd9980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #8 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #9 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #10 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 #11 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 #12 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590f5f40) at /usr/src/sys/kern/kern_fork.c:1153 #13 <signal handler called> #14 0x6bec65f367ac65f7 in ?? () Backtrace stopped: Cannot access memory at address 0xccdbfebfc09bfebb Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100258 (CPU 2) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a4cbf30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 _sx_xunlock_hard (sx=sx@entry=0xfffffe0371bde9b0, x=<optimized out>, x@entry=18446735277854445568, file=<optimized out>, line=<optimized out>) at /usr/src/sys/kern/kern_sx.c:937 #5 0xffffffff80b197a9 in _sx_xunlock (sx=0xfffffe0371bde9b0, file=<optimized out>, line=line@entry=742) at /usr/src/sys/kern/kern_sx.c:412 #6 0xffffffff827b7f51 in zio_add_child_first (pio=<optimized out>, cio=cio@entry=0xfffffe0336bac000) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:742 #7 0xffffffff827b8464 in zio_create (pio=pio@entry=0xfffffe0371bde600, spa=0xfffffe010f784000, txg=51255565, bp=0x0, data=0x0, lsize=lsize@entry=4096, psize=4096, done=0x0, private=0x0, type=ZIO_TYPE_WRITE, priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1314944, vd=0xfffffe01595c7000, offset=1105181822976, zb=0xfffffe0371bde600, stage=ZIO_STAGE_READY, pipeline=(ZIO_STAGE_ENCRYPT | ZIO_STAGE_CHECKSUM_GENERATE | ZIO_STAGE_BRT_FREE | ZIO_STAGE_DDT_READ_START | ZIO_STAGE_DDT_WRITE | ZIO_STAGE_GANG_ISSUE | ZIO_STAGE_DVA_FREE | ZIO_STAGE_DVA_CLAIM | ZIO_STAGE_VDEV_IO_START | unknown: 0x28000000)) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:973 #8 0xffffffff827b9cfe in zio_vdev_child_io (pio=0xfffffe0371bde600, bp=<optimized out>, vd=0xfffffe01595c7000, offset=1105181822976, data=<optimized out>, size=4096, type=2, priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1314944, done=0x0, private=0x0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:1589 #9 0xffffffff82704fca in vdev_raidz_io_start_write (zio=0xfffffe0371bde600, rr=0xfffff80057111000, ashift=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649 #10 vdev_raidz_io_start (zio=0xfffffe0371bde600) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748 #11 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe0371bde600) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #12 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #13 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #14 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d06000) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687 #15 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d06000) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917 #16 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #17 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #18 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f1100) at /usr/src/sys/kern/subr_taskqueue.c:518 #19 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830 #20 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004baff30, frame=0xfffffe01590c3f40) at /usr/src/sys/kern/kern_fork.c:1153 #21 <signal handler called> #22 0x94c89f3998889f3d in ?? () Backtrace stopped: Cannot access memory at address 0x33ff04753fbf0471 Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100254 (CPU 3) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a4daf30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 0xffffffff80ebeb13 in trash_ctor (mem=mem@entry=0xfffff80580056900, size=size@entry=384, arg=arg@entry=0x0, flags=flags@entry=257) at /usr/src/sys/vm/uma_dbg.c:76 #5 0xffffffff80eb71d6 in item_ctor (zone=0xfffffe00e2400000, uz_flags=1090584576, size=384, udata=0x0, flags=257, item=0xfffff80580056900) at /usr/src/sys/vm/uma_core.c:3471 #6 0xffffffff80a402ad in uma_zalloc (zone=0xdeadc0dedeadc0de, zone@entry=0xfffff8000e2b2300, flags=257) at /usr/src/sys/vm/uma.h:367 #7 g_clone_bio (bp=bp@entry=0xfffff8000e2b2300) at /usr/src/sys/geom/geom_io.c:197 #8 0xffffffff80a429a9 in g_slice_start (bp=0xfffff8000e2b2300) at /usr/src/sys/geom/geom_slice.c:276 #9 0xffffffff80a40832 in g_io_request (bp=bp@entry=0xfffff8000e2b2300, cp=cp@entry=0xfffff80004239e00) at /usr/src/sys/geom/geom_io.c:587 #10 0xffffffff825aa0ef in vdev_geom_io_start (zio=0xfffffe02fd667600) at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/vdev_geom.c:1255 #11 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe02fd667600) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #12 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #13 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #14 0xffffffff82704fd6 in vdev_raidz_io_start_write (zio=0xfffffe02efd66140, rr=0xfffff807adf52000, ashift=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649 #15 vdev_raidz_io_start (zio=0xfffffe02efd66140) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748 #16 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe02efd66140) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #17 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #18 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #19 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d05ac0) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687 #20 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d05ac0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917 #21 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #22 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #23 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f1100) at /usr/src/sys/kern/subr_taskqueue.c:518 #24 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830 #25 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004baff30, frame=0xfffffe010bdf9f40) at /usr/src/sys/kern/kern_fork.c:1153 #26 <signal handler called> #27 0xe699f066ead9f062 in ?? () Backtrace stopped: Cannot access memory at address 0x41ae6b2a4dee6b2e Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100267 (CPU 4) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a4e9f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 trash_dtor (mem=mem@entry=0xfffff8042f2b5000, size=size@entry=4096, arg=arg@entry=0xfffff80006f65240) at /usr/src/sys/vm/uma_dbg.c:101 #5 0xffffffff80eb65a0 in item_dtor (zone=0xfffffe0109828000, item=0xfffff8042f2b5000, size=4096, udata=0xfffff80006f65240, skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3513 #6 uma_zfree_arg (zone=0xfffffe0109828000, item=0xfffff8042f2b5000, udata=0xfffff80006f65240) at /usr/src/sys/vm/uma_core.c:4517 #7 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff80006f65240, buf=0x1000) at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209 #8 0xffffffff827b7982 in zio_data_buf_free (buf=0x1000, size=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:421 #9 0xffffffff825e02c6 in abd_free_linear (abd=0xfffff800755d7400) at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:244 #10 abd_free (abd=0xfffff800755d7400) at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:306 #11 0xffffffff8270059f in vdev_raidz_row_free (rr=0xfffff801ca082800) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:145 #12 vdev_raidz_map_free (rm=0xfffff80177b75700) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:160 #13 0xffffffff827c2db6 in zio_vdev_io_assess (zio=0xfffffe0344c33e40) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4158 #14 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #15 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #16 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 #17 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 #18 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe01590faf40) at /usr/src/sys/kern/kern_fork.c:1153 #19 <signal handler called> #20 0xa15df4f6ad1df4f2 in ?? () Backtrace stopped: Cannot access memory at address 0x66a6fba0a2a6fbe Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100259 (CPU 5) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a4f8f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 cache_bucket_pop (cache=0xfffffe00e2400400, bucket=0xfffffe00e2400410) at /usr/src/sys/vm/uma_core.c:916 #5 cache_alloc_item (zone=0xfffffe00e2400000, cache=0xfffffe00e2400400, bucket=0xfffffe00e2400410, udata=0x0, flags=258) at /usr/src/sys/vm/uma_core.c:3642 #6 uma_zalloc_arg (zone=0xfffffe00e2400000, udata=udata@entry=0x0, flags=flags@entry=258) at /usr/src/sys/vm/uma_core.c:3740 #7 0xffffffff80a40268 in uma_zalloc (zone=0x0, flags=258) at /usr/src/sys/vm/uma.h:367 #8 0xffffffff825a9f90 in vdev_geom_io_start (zio=0xfffffe035d281600) at /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/vdev_geom.c:1199 #9 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe035d281600) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #10 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #11 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #12 0xffffffff82704fd6 in vdev_raidz_io_start_write (zio=0xfffffe0388482e40, rr=0xfffff8079b57c000, ashift=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649 #13 vdev_raidz_io_start (zio=0xfffffe0388482e40) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748 #14 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe0388482e40) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #15 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #16 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #17 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d064c0) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687 #18 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d064c0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917 #19 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #20 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #21 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f1100) at /usr/src/sys/kern/subr_taskqueue.c:518 #22 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830 #23 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004baff30, frame=0xfffffe01590d2f40) at /usr/src/sys/kern/kern_fork.c:1153 #24 <signal handler called> #25 0xf1e4f728fda4f72c in ?? () Backtrace stopped: Cannot access memory at address 0x56d36c645a936c60 Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100256 (CPU 6) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a507f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 0xffffffff826d0e63 in spa_config_held (spa=spa@entry=0xfffffe010f784000, locks=locks@entry=22, rw=rw@entry=RW_READER) at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:585 #5 0xffffffff827b8170 in zio_create (pio=pio@entry=0xfffffe035d2e0980, spa=0xfffffe010f784000, txg=51255565, bp=0x0, data=0xfffff801e799d278, lsize=lsize@entry=4096, psize=4096, done=0xffffffff82701610 <vdev_raidz_child_done>, private=0xfffff801e799d260, type=ZIO_TYPE_WRITE, priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1048704, vd=0xfffffe01595cb000, offset=1105181818880, zb=0xfffffe035d2e0980, stage=ZIO_STAGE_READY, pipeline=(ZIO_STAGE_VDEV_IO_START | ZIO_STAGE_VDEV_IO_DONE | ZIO_STAGE_VDEV_IO_ASSESS | ZIO_STAGE_DONE)) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:903 #6 0xffffffff827b9cfe in zio_vdev_child_io (pio=0xfffffe035d2e0980, bp=<optimized out>, vd=0xfffffe01595cb000, offset=1105181818880, data=<optimized out>, size=4096, type=2, priority=ZIO_PRIORITY_ASYNC_WRITE, flags=1048704, done=0xffffffff82701610 <vdev_raidz_child_done>, private=0xfffff801e799d260) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:1589 #7 0xffffffff82704fca in vdev_raidz_io_start_write (zio=0xfffffe035d2e0980, rr=0xfffff801e799d000, ashift=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1649 #8 vdev_raidz_io_start (zio=0xfffffe035d2e0980) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:1748 #9 0xffffffff827c27ce in zio_vdev_io_start (zio=0xfffffe035d2e0980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4042 #10 0xffffffff827ba208 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #11 zio_nowait (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2437 #12 0xffffffff826fd684 in vdev_mirror_io_start (zio=0xfffffe0344d06980) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_mirror.c:687 #13 0xffffffff827c28d3 in zio_vdev_io_start (zio=0xfffffe0344d06980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:3917 #14 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #15 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #16 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f1100) at /usr/src/sys/kern/subr_taskqueue.c:518 #17 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004baff30) at /usr/src/sys/kern/subr_taskqueue.c:830 #18 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004baff30, frame=0xfffffe01590b9f40) at /usr/src/sys/kern/kern_fork.c:1153 #19 <signal handler called> #20 0x1a7a3fee163a3fea in ?? () Backtrace stopped: Cannot access memory at address 0xbd4da4a2b10da4a6 Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100269 (CPU 7) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a516f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 trash_dtor (mem=mem@entry=0xfffff8028709ddc8, size=size@entry=392, arg=arg@entry=0xfffff80006f66200) at /usr/src/sys/vm/uma_dbg.c:101 #5 0xffffffff80eb65a0 in item_dtor (zone=0xfffffe010984b000, item=0xfffff8028709ddc8, size=392, udata=0xfffff80006f66200, skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3513 #6 uma_zfree_arg (zone=0xfffffe010984b000, item=item@entry=0xfffff8028709ddc8, udata=0xfffff80006f66200) at /usr/src/sys/vm/uma_core.c:4517 #7 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff80006f66200, buf=0x188, buf@entry=0xfffff8028709ddc8) at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209 #8 0xffffffff8260f28a in dbuf_destroy (db=0xfffff8028709ddc8) at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:3116 #9 0xffffffff8261374e in dbuf_rele_and_unlock (db=0xfffff8028709df00, tag=<optimized out>, evicting=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:3999 #10 0xffffffff8261b196 in dbuf_write_done (zio=0xfffffe0344bcaac0, buf=<optimized out>, vdb=0xfffff8028709ddc8) at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:4858 #11 0xffffffff825f196f in arc_write_done (zio=0xfffffe0344bcaac0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6639 #12 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bcaac0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #13 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #14 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #15 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2e00) at /usr/src/sys/kern/subr_taskqueue.c:518 #16 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafef0) at /usr/src/sys/kern/subr_taskqueue.c:830 #17 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafef0, frame=0xfffffe0159104f40) at /usr/src/sys/kern/kern_fork.c:1153 #18 <signal handler called> #19 0xa9ff18e6a5bf18e2 in ?? () Backtrace stopped: Cannot access memory at address 0xec883aa028883ae Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100271 (CPU 8) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a525f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 arc_write_done (zio=zio@entry=0xfffffe0344bf2980) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6564 #5 0xffffffff827c45cf in zio_done (zio=0xfffffe0344bf2980) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #6 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #7 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #8 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2d00) at /usr/src/sys/kern/subr_taskqueue.c:518 #9 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830 #10 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafee0, frame=0xfffffe015910ef40) at /usr/src/sys/kern/kern_fork.c:1153 #11 <signal handler called> #12 0xbe5f9160b21f9164 in ?? () Backtrace stopped: Cannot access memory at address 0x19684a3c15280a28 Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100273 (CPU 9) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a534f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 slab_item_index (slab=0xfffff80483ec7be8, keg=0xfffff80006f38a80, item=0xfffff804810a0000) at /usr/src/sys/vm/uma_int.h:435 #5 uma_dbg_free (zone=zone@entry=0xfffffe0109828000, slab=0xfffff80483ec7be8, slab@entry=0xfffff80006f65240, item=item@entry=0xfffff804810a0000) at /usr/src/sys/vm/uma_core.c:5816 #6 0xffffffff80eb6566 in item_dtor (zone=0xfffffe0109828000, item=0xfffff804810a0000, size=4096, udata=0xfffff80006f65240, skip=SKIP_NONE) at /usr/src/sys/vm/uma_core.c:3501 #7 uma_zfree_arg (zone=0xfffffe0109828000, item=0xfffff804810a0000, udata=0xfffff80006f65240) at /usr/src/sys/vm/uma_core.c:4517 #8 0xffffffff8259ead1 in kmem_cache_free (cache=0xfffff804810a0000, buf=0xfffff80006f38a80) at /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_kmem.c:209 #9 0xffffffff827b7982 in zio_data_buf_free (buf=0xfffff80006f38a80, size=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:421 #10 0xffffffff825e02c6 in abd_free_linear (abd=0xfffff8061b8a3e00) at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:244 #11 abd_free (abd=0xfffff8061b8a3e00) at /usr/src/sys/contrib/openzfs/module/zfs/abd.c:306 #12 0xffffffff8270059f in vdev_raidz_row_free (rr=0xfffff803e72eb000) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:145 #13 vdev_raidz_map_free (rm=0xfffff804f1092200) at /usr/src/sys/contrib/openzfs/module/zfs/vdev_raidz.c:160 #14 0xffffffff827c2db6 in zio_vdev_io_assess (zio=0xfffffe0383e9a7c0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4158 #15 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #16 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #17 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2d00) at /usr/src/sys/kern/subr_taskqueue.c:518 #18 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830 #19 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafee0, frame=0xfffffe0159118f40) at /usr/src/sys/kern/kern_fork.c:1153 #20 <signal handler called> #21 0xc4527342c8127346 in ?? () Backtrace stopped: Cannot access memory at address 0x6365e80e6f25e80a Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100272 (CPU 10) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a543f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 0xffffffff826d48b4 in bp_get_dsize_sync (spa=0xfffffe010f784000, bp=bp@entry=0xfffffe0327b7da00) at /usr/src/sys/contrib/openzfs/module/zfs/spa_misc.c:2140 #5 0xffffffff82652161 in dsl_dataset_block_born ( ds=ds@entry=0xfffff80037ed2000, bp=bp@entry=0xfffffe0327b7da00, tx=tx@entry=0xfffff8043315e200) at /usr/src/sys/contrib/openzfs/module/zfs/dsl_dataset.c:140 #6 0xffffffff8261ad22 in dbuf_write_done (zio=0xfffffe0344be5ac0, buf=<optimized out>, vdb=0xfffff8021e32dc40) at /usr/src/sys/contrib/openzfs/module/zfs/dbuf.c:4810 #7 0xffffffff825f196f in arc_write_done (zio=0xfffffe0344be5ac0) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6639 #8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be5ac0) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #10 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #11 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2d00) at /usr/src/sys/kern/subr_taskqueue.c:518 #12 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830 #13 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafee0, frame=0xfffffe0159109f40) at /usr/src/sys/kern/kern_fork.c:1153 #14 <signal handler called> #15 0xa17e1b53ad3e1b57 in ?? () Backtrace stopped: Cannot access memory at address 0x1649801f0a09801b Tracing command "kernel", '\000' <repeats 13 times> pid 0 tid 100270 (CPU 11) #0 cpustop_handler () at /usr/src/sys/x86/x86/mp_x86.c:1530 #1 0xffffffff81009388 in ipi_nmi_handler () at /usr/src/sys/x86/x86/mp_x86.c:1487 #2 0xffffffff810406a6 in trap (frame=0xfffffe003a552f30) at /usr/src/sys/amd64/amd64/trap.c:248 #3 <signal handler called> #4 zfs_refcount_remove_many (rc=rc@entry=0xfffff8048def2098, number=number@entry=1, holder=0xfffff8048def2000) at /usr/src/sys/contrib/openzfs/module/zfs/refcount.c:176 #5 0xffffffff826ab722 in zfs_refcount_remove (rc=rc@entry=0xfffff8048def2098, holder=0xfffff8048def2000) at /usr/src/sys/contrib/openzfs/module/zfs/refcount.c:212 #6 0xffffffff825e7200 in remove_reference (hdr=hdr@entry=0xfffff8048def2000, tag=tag@entry=0xfffff8048def2000) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:2314 #7 0xffffffff825f182c in arc_write_done (zio=0xfffffe0344be4c80) at /usr/src/sys/contrib/openzfs/module/zfs/arc.c:6629 #8 0xffffffff827c45cf in zio_done (zio=0xfffffe0344be4c80) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:4988 #9 0xffffffff827baea8 in __zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2350 #10 zio_execute (zio=<optimized out>) at /usr/src/sys/contrib/openzfs/module/zfs/zio.c:2261 #11 0xffffffff80b75392 in taskqueue_run_locked ( queue=queue@entry=0xfffff8000e2f2d00) at /usr/src/sys/kern/subr_taskqueue.c:518 #12 0xffffffff80b76283 in taskqueue_thread_loop ( arg=arg@entry=0xfffff80004bafee0) at /usr/src/sys/kern/subr_taskqueue.c:830 #13 0xffffffff80ac0022 in fork_exit ( callout=0xffffffff80b761b0 <taskqueue_thread_loop>, arg=0xfffff80004bafee0, frame=0xfffffe01590fff40) at /usr/src/sys/kern/kern_fork.c:1153 #14 <signal handler called> #15 0xfc4cec75700cec71 in ?? () Backtrace stopped: Cannot access memory at address 0xdb7b7739d73b773d (kgdb) -- You are receiving this mail because: You are the assignee for the bug.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-290570-227>