From owner-freebsd-current@freebsd.org Tue Jan 29 14:43:19 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2993414BF9CC for ; Tue, 29 Jan 2019 14:43:19 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (unknown [IPv6:2001:470:1f0f:3ad::53:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F282389501 for ; Tue, 29 Jan 2019 14:43:17 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding: Content-Type:MIME-Version:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=F9t06iGNnU2r9FsiXiNXS7Antd6eD4h+MEU0SlaZnpg=; b=rRgKfleNWye4W0TlIvxRuzLoOK ET/pa3EDVLghZeCqRRIVDSniBqEO2hCL2QQqDj0kJBTFnLksJuX/WmFL1SJvGw3irzvdE5d+N5TR9 Tgj2C9e668NKv25TcMk2woQyX90l9mZRirrjZxknZzN39g7g3VoLVzL0qgLDHXK1Kv5U=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:23416 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1goUbc-0006EO-HG for freebsd-current@freebsd.org; Tue, 29 Jan 2019 08:43:16 -0600 Received: from 2600:1700:210:b18f:2985:5695:7960:1704 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 29 Jan 2019 08:43:16 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 29 Jan 2019 08:43:16 -0600 From: Larry Rosenman To: Freebsd current Subject: Use after Free panic: ZFS? Message-ID: <59aaaf6f2b821e9f96f5441274f19957@lerctr.org> X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.3.8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jan 2019 14:43:19 -0000 I've seen a couple of these... ⌂70% [ler@borg.lerctr.org:/var/crash] $ uname -aKU FreeBSD borg.lerctr.org 13.0-CURRENT FreeBSD 13.0-CURRENT r343437 LER-MINIMAL amd64 1300009 1300009 ⌂66% [ler@borg.lerctr.org:/var/crash] $ Ideas? vmcore/symbols available. borg.lerctr.org dumped core - see /var/crash/vmcore.7 Tue Jan 29 04:00:46 CST 2019 FreeBSD borg.lerctr.org 13.0-CURRENT FreeBSD 13.0-CURRENT r343437 LER-MINIMAL amd64 panic: Memory modified after free 0xfffff807019ca980(32) val=0 @ 0xfffff807019ca980 GNU gdb (GDB) 8.2 [GDB v8.2 for FreeBSD] Copyright (C) 2018 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd13.0". Type "show configuration" for configuration details. For bug reporting instructions, please see: . Find the GDB manual and other documentation resources online at: . For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /boot/kernel/kernel...Reading symbols from /usr/lib/debug//boot/kernel/kernel.debug...done. done. Unread portion of the kernel message buffer: panic: Memory modified after free 0xfffff807019ca980(32) val=0 @ 0xfffff807019ca980 cpuid = 5 time = 1548755136 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f750c880 vpanic() at vpanic+0x1b4/frame 0xfffffe00f750c8e0 panic() at panic+0x43/frame 0xfffffe00f750c940 trash_ctor() at trash_ctor+0x4c/frame 0xfffffe00f750c950 uma_zalloc_arg() at uma_zalloc_arg+0x9df/frame 0xfffffe00f750c9e0 uma_zfree_arg() at uma_zfree_arg+0x46a/frame 0xfffffe00f750ca40 arc_buf_destroy_impl() at arc_buf_destroy_impl+0x133/frame 0xfffffe00f750ca80 arc_buf_destroy() at arc_buf_destroy+0x17a/frame 0xfffffe00f750cab0 dbuf_destroy() at dbuf_destroy+0x87/frame 0xfffffe00f750cb10 dbuf_evict_one() at dbuf_evict_one+0x187/frame 0xfffffe00f750cb40 dbuf_evict_thread() at dbuf_evict_thread+0x185/frame 0xfffffe00f750cbb0 fork_exit() at fork_exit+0x84/frame 0xfffffe00f750cbf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f750cbf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Uptime: 3d16h49m14s Dumping 22587 out of 131028 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% __curthread () at ./machine/pcpu.h:230 230 __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD)); (kgdb) #0 __curthread () at ./machine/pcpu.h:230 #1 doadump (textdump=) at /usr/src/sys/kern/kern_shutdown.c:371 #2 0xffffffff80491760 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:451 #3 0xffffffff80491bc0 in vpanic (fmt=, ap=0xfffffe00f750c920) at /usr/src/sys/kern/kern_shutdown.c:877 #4 0xffffffff80491913 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:804 #5 0xffffffff8071255c in trash_ctor (mem=, size=, arg=, flags=) at /usr/src/sys/vm/uma_dbg.c:82 #6 0xffffffff8070cf4f in uma_zalloc_arg (zone=0xfffff8203ffdc000, udata=0x108, flags=1) at /usr/src/sys/vm/uma_core.c:2418 #7 0xffffffff8070d69a in bucket_alloc (zone=, udata=, flags=) at /usr/src/sys/vm/uma_core.c:433 #8 uma_zfree_arg (zone=0xfffff801059a0000, item=, udata=0xfffff81042431940) at /usr/src/sys/vm/uma_core.c:3153 #9 0xffffffff812f8c13 in arc_free_data_buf (hdr=, buf=0xfffffe025fe1e000, size=8192, tag=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:5248 #10 arc_buf_destroy_impl (buf=0xfffff8190202ef00) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3270 #11 0xffffffff812f859a in arc_buf_destroy (buf=0xfffff8190202ef00, tag=0xfffff80aea618840) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:3687 #12 0xffffffff8130d3d7 in dbuf_destroy (db=0xfffff80aea618840) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:2328 #13 0xffffffff81313bb7 in dbuf_evict_one () at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:717 #14 0xffffffff8130b1d5 in dbuf_evict_thread (unused=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:757 #15 0xffffffff80458a94 in fork_exit ( callout=0xffffffff8130b050 , arg=0x0, frame=0xfffffe00f750cc00) at /usr/src/sys/kern/kern_fork.c:1055 #16 (kgdb) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106