From owner-freebsd-fs@freebsd.org Mon Aug 27 16:27:44 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A7C9108F882 for ; Mon, 27 Aug 2018 16:27:44 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:5001:17bd::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1FD0B781E1; Mon, 27 Aug 2018 16:27:44 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [2001:470:6845:1:52d4:a912:70e6:df0e] (helo=yuuta.localdomain) by mail1.yamagi.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91 (FreeBSD)) (envelope-from ) id 1fuKMZ-000FN1-5x; Mon, 27 Aug 2018 18:27:42 +0200 Date: Mon, 27 Aug 2018 18:27:23 +0200 From: Yamagi Burmeister To: lev@FreeBSD.org Cc: freebsd-fs@freebsd.org, MacCrest@gmail.com Subject: Re: ZFS (ARC) performance regression in r321610 Message-Id: <20180827182723.cd8ec6bac6448793b25ef91c@yamagi.org> In-Reply-To: <3c6f8c96-6ac9-7257-c8c0-8be2063a7c19@FreeBSD.org> References: <20180827154727.80f92fff9bbc931b37928d43@yamagi.org> <3c6f8c96-6ac9-7257-c8c0-8be2063a7c19@FreeBSD.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; x86_64-unknown-linux-gnu) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Mon__27_Aug_2018_18_27_23_+0200_C52TiSw_Mx8WrCMb" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2018 16:27:44 -0000 --Signature=_Mon__27_Aug_2018_18_27_23_+0200_C52TiSw_Mx8WrCMb Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, 27 Aug 2018 17:19:10 +0300 Lev Serebryakov wrote: > On 27.08.2018 16:47, Yamagi Burmeister wrote: >=20 > > With this program I was able to bisect the source and identify commit > > r321610 (MFV r318946: 8021 ARC buf data scatter-ization) as the culprit: > > https://svnweb.freebsd.org/base?view=3Drevision&revision=3D321610 > You could try to set "zfs_abd_scatter_enabled =3D 0" via kgdb and repeat > tests to be sure, that this code is a problem. With my testcase this looks very, very good. I'll give it a try in production and report back. That'll take at least 12 hours to be sure that it really works. --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Mon__27_Aug_2018_18_27_23_+0200_C52TiSw_Mx8WrCMb Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAluEJmsACgkQ6xRy5x1Q JRXXzg/+K9zUlMg95NoJKxqVk0u08YH7X8Kcw7fvNnLEQNrDvX9BeLjp3I9en0Ar oR6brCV2dId6VshF/ytWOXf7dmNHLlgrWK2DWy1GOk8uQKja/Wg9srWOWXT6PzKO fzdF7y/NQxoDlnd1SJs1LVe3tNZMxJTsB5M1ZMEkTdpnRRuwjgeVDHCC0cNLnwCM En+/KljkYvXTlDGeeZm3dNRCijZUmoSpeJwTzDT4gqRlMwa9kpmqogpqriS2adCd ZhOxsYLCcc9cBfgBD4zDeCdLbJpRYFvuEsZV6+Jvyi75a+gDw69CFLX3gJiTohcA Ij7bHTN3RH4CXrkhF2nBqIN8/PM1HtGLGUmovWH03yf7FeHOVi6Yqt6ytNHxCq8O oCv8QmSwnz1QJEZ5FtQgQ0uLV0jRlBMD29l2rWNu9zh3UsKpWKbleXqJJDAamdsq qzkMrDTZZQFm9yAO4EGnEFPe5t9HrbLBDuDpGuM10WxKN3ffLeoRgaUWAmfWObQT tMiIZqP216b8jOBxH9yPDywnJdPiFZQ6QCgGQ1Vv01dkmuTIWukRcu28yVNQKeew yJg0PWuja8r9W4ffROas2OfKoOLl93Vk5/IIKOWqeMa0TX50hpnn4HP+2QN3jBAB Iwwi3b78ekmYQB0N/XC9B/qxZlflVRc370rHFbNrfb/yEDHZEjc= =jzs8 -----END PGP SIGNATURE----- --Signature=_Mon__27_Aug_2018_18_27_23_+0200_C52TiSw_Mx8WrCMb--