From owner-freebsd-fs@freebsd.org Sat Mar 16 12:21:16 2019 Return-Path: <owner-freebsd-fs@freebsd.org> Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06FEE152943F for <freebsd-fs@mailman.ysv.freebsd.org>; Sat, 16 Mar 2019 12:21:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 9068B8662C for <freebsd-fs@freebsd.org>; Sat, 16 Mar 2019 12:21:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 4A5E3152943D; Sat, 16 Mar 2019 12:21:15 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22F63152943C for <fs@mailman.ysv.freebsd.org>; Sat, 16 Mar 2019 12:21:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AB0C486627 for <fs@FreeBSD.org>; Sat, 16 Mar 2019 12:21:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id DD7902AA0 for <fs@FreeBSD.org>; Sat, 16 Mar 2019 12:21:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2GCLDn4022566 for <fs@FreeBSD.org>; Sat, 16 Mar 2019 12:21:13 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2GCLDfi022557 for fs@FreeBSD.org; Sat, 16 Mar 2019 12:21:13 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 235783] Repeated ZFS-related kernel panic Date: Sat, 16 Mar 2019 12:21:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.2-RELEASE X-Bugzilla-Keywords: panic X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: jurij.kovacic@ocpea.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: <bug-235783-3630-yZcSKiASgV@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-235783-3630@https.bugs.freebsd.org/bugzilla/> References: <bug-235783-3630@https.bugs.freebsd.org/bugzilla/> Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/> List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 16 Mar 2019 12:21:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D235783 --- Comment #1 from Jurij Kovacic <jurij.kovacic@ocpea.com> --- Hello, After approximately 1 month, the server (FreeBSD 11.2-RELEASE-p9 #0 r344062) has crashed again.=20 It seems to me, the reason for the crash is most likely the same as the last time: Fatal trap 9: general protection fault while in kernel mode cpuid =3D 2; apic id =3D 02 instruction pointer =3D 0x20:0xffffffff82299013 stack pointer =3D 0x28:0xfffffe0352893ad0 frame pointer =3D 0x28:0xfffffe0352893b10 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 9 (dbuf_evict_thread) trap number =3D 9 panic: general protection fault cpuid =3D 2 KDB: stack backtrace: #0 0xffffffff80b3d5a7 at kdb_backtrace+0x67 #1 0xffffffff80af6b47 at vpanic+0x177 #2 0xffffffff80af69c3 at panic+0x43 #3 0xffffffff80f77fdf at trap_fatal+0x35f #4 0xffffffff80f7759e at trap+0x5e #5 0xffffffff80f580bc at calltrap+0x8 #6 0xffffffff8229c049 at dbuf_evict_one+0xe9 #7 0xffffffff82297a15 at dbuf_evict_thread+0x1a5 #8 0xffffffff80aba0c3 at fork_exit+0x83 #9 0xffffffff80f58fee at fork_trampoline+0xe Uptime: 31d1h57m16s Uptime: 31d1h57m16s Dumping 2771 out of 12248 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..= 91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done. done. Loaded symbols for /boot/kernel/ums.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /usr/lib/debug//boot/kernel/pflog.ko.debug...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /usr/lib/debug//boot/kernel/pf.ko.debug...done. done. Loaded symbols for /boot/kernel/pf.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/blank_saver.ko...Reading symbols from /usr/lib/debug//boot/kernel/blank_saver.ko.debug...done. done. Loaded symbols for /boot/kernel/blank_saver.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/smbfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/smbfs.ko.debug...done. done. Loaded symbols for /boot/kernel/smbfs.ko Reading symbols from /boot/kernel/libiconv.ko...Reading symbols from /usr/lib/debug//boot/kernel/libiconv.ko.debug...done. done. Loaded symbols for /boot/kernel/libiconv.ko Reading symbols from /boot/kernel/libmchain.ko...Reading symbols from /usr/lib/debug//boot/kernel/libmchain.ko.debug...done. done. Loaded symbols for /boot/kernel/libmchain.ko Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko #0 doadump (textdump=3D<value optimized out>) at pcpu.h:229 229 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=3D<value optimized out>) at pcpu.h:229 #1 0xffffffff80af675b in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:383 #2 0xffffffff80af6b81 in vpanic (fmt=3D<value optimized out>, ap=3D<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:776 #3 0xffffffff80af69c3 in panic (fmt=3D<value optimized out>) at /usr/src/sys/kern/kern_shutdown.c:707 #4 0xffffffff80f77fdf in trap_fatal (frame=3D0xfffffe0352893a10, eva=3D0) at /usr/src/sys/amd64/amd64/trap.c:875 #5 0xffffffff80f7759e in trap (frame=3D0xfffffe0352893a10) at pcpu.h:229 #6 0xffffffff80f580bc in calltrap () at /usr/src/sys/amd64/amd64/exception.S:231 #7 0xffffffff82299013 in dbuf_destroy (db=3D0xfffff8028de9cae0) at atomic.= h:79 #8 0xffffffff8229c049 in dbuf_evict_one () at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:487 #9 0xffffffff82297a15 in dbuf_evict_thread (unused=3D<value optimized out>) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:525 #10 0xffffffff80aba0c3 in fork_exit ( callout=3D0xffffffff82297870 <dbuf_evict_thread>, arg=3D0x0, frame=3D0xfffffe0352893c00) at /usr/src/sys/kern/kern_fork.c:1054 #11 0xffffffff80f58fee in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:959 #12 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) The "crashinfo" txt file is available at thie URL: http://www.ocpea.com/dump/core-5.txt original core dump files are also available if needed. Can someone, please, take a look at this? I will be more than happy to help with debugging, supplying additional info etc if needed. Kind regards, Jurij --=20 You are receiving this mail because: You are the assignee for the bug.=