From owner-freebsd-stable@freebsd.org Tue Dec 15 21:39:02 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 046834C33DC for ; Tue, 15 Dec 2020 21:39:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from pyroxene2a.sentex.ca (pyroxene19.sentex.ca [IPv6:2607:f3e0:0:3::19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "pyroxene.sentex.ca", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CwWq93V5Tz4vh9 for ; Tue, 15 Dec 2020 21:39:01 +0000 (UTC) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:20e5:8bab:f6d6:8c81] ([IPv6:2607:f3e0:0:4:20e5:8bab:f6d6:8c81]) by pyroxene2a.sentex.ca (8.15.2/8.15.2) with ESMTPS id 0BFLd1ns096184 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Tue, 15 Dec 2020 16:39:01 -0500 (EST) (envelope-from mike@sentex.net) To: FreeBSD-STABLE Mailing List From: mike tancsa Subject: zfs panic RELENG_12 Message-ID: <878824fe-dde2-b551-4685-e8bd27371275@sentex.net> Date: Tue, 15 Dec 2020 16:39:01 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: en-US X-Rspamd-Queue-Id: 4CwWq93V5Tz4vh9 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:3::19 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-2.00 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[mike]; FROM_HAS_DN(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f3e0:0:3::19:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; HFILTER_HELO_IP_A(1.00)[pyroxene2a.sentex.ca]; RCPT_COUNT_ONE(0.00)[1]; SPAMHAUS_ZRD(0.00)[2607:f3e0:0:3::19:from:127.0.2.255]; DMARC_NA(0.00)[sentex.net]; TO_DN_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; HFILTER_HELO_NORES_A_OR_MX(0.30)[pyroxene2a.sentex.ca]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Dec 2020 21:39:02 -0000 Was doing a backup via zfs send | zfs recv when the box panic'd.=C2=A0 It= s a not so old RELENG_12 box from last week. Any ideas if this is a hardware issue or a bug ? Its r368493 from last Wednesday. I dont see an ECC errors logged, so dont think its hardware. Reading symbols from /boot/kernel/kernel... Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 01 fault virtual address=C2=A0=C2=A0 =3D 0x0 fault code=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 =3D supervisor read data, page not present instruction pointer=C2=A0=C2=A0=C2=A0=C2=A0 =3D 0x20:0xffffffff823a554b stack pointer=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =3D 0x28:0xfffffe0343231000 frame pointer=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =3D 0x28:0xfffffe03432310c0 code segment=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 =3D base 0x0, limit 0xfffff, type 0x1b =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D DPL 0= , pres 1, long 1, def32 0, gran 1 processor eflags=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D interrupt = enabled, resume, IOPL =3D 0 current process=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D 87427= (zfs) trap number=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 =3D 12 panic: page fault cpuid =3D 1 time =3D 1608065221 KDB: stack backtrace: #0 0xffffffff80a3fa05 at kdb_backtrace+0x65 #1 0xffffffff809f3beb at vpanic+0x17b #2 0xffffffff809f3a63 at panic+0x43 #3 0xffffffff80e400d1 at trap_fatal+0x391 #4 0xffffffff80e4012f at trap_pfault+0x4f #5 0xffffffff80e3f776 at trap+0x286 #6 0xffffffff80e19568 at calltrap+0x8 #7 0xffffffff82393a5e at dmu_object_info+0x1e #8 0xffffffff823983a5 at dmu_recv_stream+0x7b5 #9 0xffffffff8244b706 at zfs_ioc_recv+0xac6 #10 0xffffffff8244dd3d at zfsdev_ioctl+0x62d #11 0xffffffff808a35e0 at devfs_ioctl+0xb0 #12 0xffffffff80f3becb at VOP_IOCTL_APV+0x7b #13 0xffffffff80ad1b0a at vn_ioctl+0x16a #14 0xffffffff808a3bce at devfs_ioctl_f+0x1e #15 0xffffffff80a5d807 at kern_ioctl+0x2b7 #16 0xffffffff80a5d4aa at sys_ioctl+0xfa #17 0xffffffff80e40c87 at amd64_syscall+0x387 Uptime: 3d14h59m52s Dumping 17213 out of 65366 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55 55=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" (offsetof(struct pcpu, (kgdb) #0=C2=A0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:5= 5 #1=C2=A0 doadump (textdump=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/kern_shutdown.c:371 #2=C2=A0 0xffffffff809f3805 in kern_reboot (howto=3D260) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/kern_shutdown.c:451 #3=C2=A0 0xffffffff809f3c43 in vpanic (fmt=3D, ap=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/kern_shutdown.c:880 #4=C2=A0 0xffffffff809f3a63 in panic (fmt=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/kern_shutdown.c:807 #5=C2=A0 0xffffffff80e400d1 in trap_fatal (frame=3D0xfffffe0343230f40, ev= a=3D0) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/amd64/amd64/trap.c:921 #6=C2=A0 0xffffffff80e4012f in trap_pfault (frame=3D0xfffffe0343230f40, =C2=A0=C2=A0=C2=A0 usermode=3D, signo=3D, u= code=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/amd64/amd64/trap.c:739 #7=C2=A0 0xffffffff80e3f776 in trap (frame=3D0xfffffe0343230f40) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/amd64/amd64/trap.c:405 #8=C2=A0 #9=C2=A0 0xffffffff823a554b in dnode_hold_impl (os=3D0xfffff805e1d2b800, =C2=A0=C2=A0=C2=A0 object=3D, flag=3D, slot= s=3D, =C2=A0=C2=A0=C2=A0 tag=3D, dnp=3D0xfffffe03432310d8) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs= /zfs/dnode.c:1370 #10 0xffffffff82393a5e in dmu_object_info (os=3D0xfffff80777890070, =C2=A0=C2=A0=C2=A0 object=3D18446744071600721588, doi=3D0xfffffe03432312e= 0) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs= /zfs/dmu.c:2615 #11 0xffffffff823983a5 in receive_read_record (ra=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c:2821 #12 dmu_recv_stream (drc=3D0xfffffe0343231430, fp=3D, =C2=A0=C2=A0=C2=A0 voffp=3D, cleanup_fd=3D8, action_handle= p=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_send.c:3203 #13 0xffffffff8244b706 in zfs_ioc_recv (zc=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:4745 #14 0xffffffff8244dd3d in zfsdev_ioctl (dev=3D, =C2=A0=C2=A0=C2=A0 zcmd=3D, arg=3D, flag=3D= , =C2=A0=C2=A0=C2=A0 td=3D) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c:6956 #15 0xffffffff808a35e0 in devfs_ioctl (ap=3D0xfffffe0343231778) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/fs/devfs/devfs_vnops.c:797 #16 0xffffffff80f3becb in VOP_IOCTL_APV ( =C2=A0=C2=A0=C2=A0 vop=3D0xffffffff816a2fe0 , a=3D0xfffffe= 0343231778) =C2=A0=C2=A0=C2=A0 at vnode_if.c:1067 #17 0xffffffff80ad1b0a in vn_ioctl (fp=3D0xfffff8001802b5a0, =C2=A0=C2=A0=C2=A0 com=3D, data=3D0xfffffe0343231910, =C2=A0=C2=A0=C2=A0 active_cred=3D0xfffff80032214300, td=3D0x20700000) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/vfs_vnops.c:1508 #18 0xffffffff808a3bce in devfs_ioctl_f (fp=3D0xfffff80777890070, =C2=A0=C2=A0=C2=A0 com=3D18446744071600721588, data=3D0xffffffff824e34ed = <.L.str+1>, cred=3D0x0, =C2=A0=C2=A0=C2=A0 td=3D0xfffff80298850000) at /usr/src/sys/fs/devfs/devf= s_vnops.c:755 #19 0xffffffff80a5d807 in fo_ioctl (fp=3D0xfffff8001802b5a0, com=3D322282= 1403, =C2=A0=C2=A0=C2=A0 data=3D0xffffffff824e34ed <.L.str+1>, active_cred=3D0x= 0, =C2=A0=C2=A0=C2=A0 td=3D0xfffff80298850000) at /usr/src/sys/sys/file.h:33= 7 #20 kern_ioctl (td=3D0x20700000, fd=3D, com=3D3222821403, =C2=A0=C2=A0=C2=A0 data=3D0xffffffff824e34ed <.L.str+1> "zrl->zr_mtx") =C2=A0=C2=A0=C2=A0 at /usr/src/sys/kern/sys_generic.c:805 #21 0xffffffff80a5d4aa in sys_ioctl (td=3D0xfffff80298850000, =C2=A0=C2=A0=C2=A0 uap=3D0xfffff802988503c0) at /usr/src/sys/kern/sys_gen= eric.c:713 #22 0xffffffff80e40c87 in syscallenter (td=3D0xfffff80298850000) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:= 144 #23 amd64_syscall (td=3D0xfffff80298850000, traced=3D0) =C2=A0=C2=A0=C2=A0 at /usr/src/sys/amd64/amd64/trap.c:1163 #24 #25 0x00000008004a8cba in ?? () Backtrace stopped: Cannot access memory at address 0x7fffffffa988 (kgdb)