From owner-freebsd-current@freebsd.org Wed Jan 30 12:55:06 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4DC414C007F for ; Wed, 30 Jan 2019 12:55:06 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lj1-f175.google.com (mail-lj1-f175.google.com [209.85.208.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 337736A710 for ; Wed, 30 Jan 2019 12:55:06 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lj1-f175.google.com with SMTP id n18-v6so20596917lji.7 for ; Wed, 30 Jan 2019 04:55:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=fjNTVHDmn/mw3F24bCYlxMO2o8zQYMTxT7sdgX4b2Ss=; b=QwQHwHIonju0/CYs4r17bQDfxg0eshSg+q8MukbGUHZbqnx/1bQw+3q+fXf3QiSyra WOw7iZvsfbMDcSwasBTbJJskcVTVv5qbfAPWnRk1hJePqRijZ8hpEBLKeN9T2jGXEhnN QpEqlRHtXBud/tr18FMfE1EuwDnmIDLH54NhKs7I5B6z806ZGCvw/TJZVE3rYU0wP3KZ OcQwtW48rqA7aECG7i/UkEhqy2RINhPZFSBEoF2SBIHL2CZdmTnsvPfawZwhOGzRxc4W nEGK2LmXr4J7HCfPIn5nucgaoO/+jHp02qxk43iShaSGmKyUzI2JM+VFIPq8IyHHYUVd A8eg== X-Gm-Message-State: AJcUukcZxm2alYRB4oiX0KfytVrlOfvPrB84zTD8njbHC1ovkdvtmGkJ Gd4U+ltblGBaqwGtZyzwm6G9bNzG X-Google-Smtp-Source: ALg8bN7ualXnmjFSFb7+tD2MDA0g11wYdr/lho1nmGJjU8J4/mFmlTSDSqRfGkTA+S+I5oeKV/pPZQ== X-Received: by 2002:a2e:b1ca:: with SMTP id e10-v6mr24502191lja.16.1548852898634; Wed, 30 Jan 2019 04:54:58 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id a127sm274992lfe.73.2019.01.30.04.54.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Jan 2019 04:54:57 -0800 (PST) Subject: Re: Use after Free panic: ZFS? To: Larry Rosenman , Freebsd current References: <59aaaf6f2b821e9f96f5441274f19957@lerctr.org> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= xsFNBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABzR5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz7CwZQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryM7BTQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAcLBfAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <1ef4a133-10db-0cde-54a2-801e0db18f9c@FreeBSD.org> Date: Wed, 30 Jan 2019 14:54:56 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <59aaaf6f2b821e9f96f5441274f19957@lerctr.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 337736A710 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.99)[-0.995,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jan 2019 12:55:07 -0000 On 29/01/2019 16:43, Larry Rosenman wrote: > panic: Memory modified after free 0xfffff807019ca980(32) val=0 @ 0xfffff807019ca980 > > cpuid = 5 > time = 1548755136 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00f750c880 > vpanic() at vpanic+0x1b4/frame 0xfffffe00f750c8e0 > panic() at panic+0x43/frame 0xfffffe00f750c940 > trash_ctor() at trash_ctor+0x4c/frame 0xfffffe00f750c950 > uma_zalloc_arg() at uma_zalloc_arg+0x9df/frame 0xfffffe00f750c9e0 > uma_zfree_arg() at uma_zfree_arg+0x46a/frame 0xfffffe00f750ca40 > arc_buf_destroy_impl() at arc_buf_destroy_impl+0x133/frame 0xfffffe00f750ca80 > arc_buf_destroy() at arc_buf_destroy+0x17a/frame 0xfffffe00f750cab0 > dbuf_destroy() at dbuf_destroy+0x87/frame 0xfffffe00f750cb10 > dbuf_evict_one() at dbuf_evict_one+0x187/frame 0xfffffe00f750cb40 > dbuf_evict_thread() at dbuf_evict_thread+0x185/frame 0xfffffe00f750cbb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe00f750cbf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00f750cbf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > Uptime: 3d16h49m14s > Dumping 22587 out of 131028 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > __curthread () at ./machine/pcpu.h:230 > 230             __asm("movq %%gs:%P1,%0" : "=r" (td) : "n" (OFFSETOF_CURTHREAD)); > (kgdb) #0  __curthread () at ./machine/pcpu.h:230 > #1  doadump (textdump=) >     at /usr/src/sys/kern/kern_shutdown.c:371 > #2  0xffffffff80491760 in kern_reboot (howto=260) >     at /usr/src/sys/kern/kern_shutdown.c:451 > #3  0xffffffff80491bc0 in vpanic (fmt=, ap=0xfffffe00f750c920) >     at /usr/src/sys/kern/kern_shutdown.c:877 > #4  0xffffffff80491913 in panic (fmt=) >     at /usr/src/sys/kern/kern_shutdown.c:804 > #5  0xffffffff8071255c in trash_ctor (mem=, size=, >     arg=, flags=) >     at /usr/src/sys/vm/uma_dbg.c:82 > #6  0xffffffff8070cf4f in uma_zalloc_arg (zone=0xfffff8203ffdc000, >     udata=0x108, flags=1) at /usr/src/sys/vm/uma_core.c:2418 > #7  0xffffffff8070d69a in bucket_alloc (zone=, >     udata=, flags=) >     at /usr/src/sys/vm/uma_core.c:433 > #8  uma_zfree_arg (zone=0xfffff801059a0000, item=, >     udata=0xfffff81042431940) at /usr/src/sys/vm/uma_core.c:3153 The problem is with an item in an (internal) UMA bucket zone. So, this is probably not ZFS specific. -- Andriy Gapon