From owner-freebsd-fs@freebsd.org Fri Nov 4 14:46:48 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6CF2C2FD12 for ; Fri, 4 Nov 2016 14:46:48 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A63CEDE0; Fri, 4 Nov 2016 14:46:47 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA00925; Fri, 04 Nov 2016 16:46:45 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1c2flV-000Of6-5t; Fri, 04 Nov 2016 16:46:45 +0200 Subject: Re: ZFS L2ARC checksum errors after compression To: lev@FreeBSD.org, freebsd-fs References: <921575537.20161029143626@serebryakov.spb.ru> <3dae7691-fcd1-b3b9-445c-b81d6f0cdc52@FreeBSD.org> From: Andriy Gapon Message-ID: <3bd7cb79-ec5a-3b7c-0642-24a7b1f001e6@FreeBSD.org> Date: Fri, 4 Nov 2016 16:20:20 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Nov 2016 14:46:48 -0000 On 03/11/2016 21:43, Lev Serebryakov wrote: > On 29.10.2016 16:32, Andriy Gapon wrote: > > Looks like L2ARC is unusable now if your have compression enabled on > FSes. It shows 2x compression (ALLOC = 2xSIZE), and tons of checksum > errors. I simply don't have compressible enough data on my FSes! It is > mostly media files! Looks like this data is bogus. > >> I think that a recent upstream change, compressed ARC support, reintroduced an a >> old problem that was fixed a while ago. > Lev, because of the confusing variable names I made a mistake in the patch that I offered you. Could you please try a new slight different patch? Also, I think that there could be another problem in addition to the one that I see. But I am quite busy at the moment, no time to investigate. Maybe on the weekend or some time next week. Thank you for reporting and testing. Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c =================================================================== --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision 308225) +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy) @@ -7028,7 +7028,22 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, continue; } - if ((write_asize + HDR_GET_LSIZE(hdr)) > target_sz) { + /* + * We rely on the L1 portion of the header below, so + * it's invalid for this header to have been evicted out + * of the ghost cache, prior to being written out. The + * ARC_FLAG_L2_WRITING bit ensures this won't happen. + */ + ASSERT(HDR_HAS_L1HDR(hdr)); + + ASSERT3U(HDR_GET_PSIZE(hdr), >, 0); + ASSERT3P(hdr->b_l1hdr.b_pdata, !=, NULL); + ASSERT3U(arc_hdr_size(hdr), >, 0); + uint64_t size = arc_hdr_size(hdr); + uint64_t asize = vdev_psize_to_asize(dev->l2ad_vdev, + size); + + if ((write_psize + asize) > target_sz) { full = B_TRUE; mutex_exit(hash_lock); ARCSTAT_BUMP(arcstat_l2_write_full); @@ -7063,21 +7078,6 @@ l2arc_write_buffers(spa_t *spa, l2arc_dev_t *dev, list_insert_head(&dev->l2ad_buflist, hdr); mutex_exit(&dev->l2ad_mtx); - /* - * We rely on the L1 portion of the header below, so - * it's invalid for this header to have been evicted out - * of the ghost cache, prior to being written out. The - * ARC_FLAG_L2_WRITING bit ensures this won't happen. - */ - ASSERT(HDR_HAS_L1HDR(hdr)); - - ASSERT3U(HDR_GET_PSIZE(hdr), >, 0); - ASSERT3P(hdr->b_l1hdr.b_pdata, !=, NULL); - ASSERT3U(arc_hdr_size(hdr), >, 0); - uint64_t size = arc_hdr_size(hdr); - uint64_t asize = vdev_psize_to_asize(dev->l2ad_vdev, - size); - (void) refcount_add_many(&dev->l2ad_alloc, size, hdr); /* -- Andriy Gapon