From owner-freebsd-fs@freebsd.org  Tue Dec  6 17:14:24 2016
Return-Path: <owner-freebsd-fs@freebsd.org>
Delivered-To: freebsd-fs@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A93EC693A3
 for <freebsd-fs@mailman.ysv.freebsd.org>; Tue,  6 Dec 2016 17:14:24 +0000 (UTC)
 (envelope-from avg@FreeBSD.org)
Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140])
 by mx1.freebsd.org (Postfix) with ESMTP id BE919173A
 for <freebsd-fs@FreeBSD.org>; Tue,  6 Dec 2016 17:14:23 +0000 (UTC)
 (envelope-from avg@FreeBSD.org)
Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua
 [212.40.38.100])
 by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA23360;
 Tue, 06 Dec 2016 19:14:21 +0200 (EET) (envelope-from avg@FreeBSD.org)
Received: from localhost ([127.0.0.1])
 by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD))
 id 1cEJJt-000Gp6-CM; Tue, 06 Dec 2016 19:14:21 +0200
Subject: Re: 11-STABLE vs 11.0-RELENG test
To: Alex Tutubalin <lexa@lexa.ru>, freebsd-fs@FreeBSD.org
References: <477264f9-9263-6df8-4566-2911e84f2ad8@lexa.ru>
 <a10b10ed-9200-26f3-5ca4-976a13922dee@lexa.ru>
From: Andriy Gapon <avg@FreeBSD.org>
Message-ID: <dd8f644b-8555-f8ad-a18a-58e2a28cc209@FreeBSD.org>
Date: Tue, 6 Dec 2016 19:12:58 +0200
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101
 Thunderbird/45.5.1
MIME-Version: 1.0
In-Reply-To: <a10b10ed-9200-26f3-5ca4-976a13922dee@lexa.ru>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Dec 2016 17:14:24 -0000

On 06/12/2016 18:17, Alex Tutubalin wrote:
> Followup, same box as in 1st message, but with different HBA and 5 HDD:
> 
> Summary:
>  1) read speed depends on 'on-disk' data, not by code used while read.
>  2) FreeBSD 11.0-releng (svn up today) creates 'fast' big files (420+ Mb/s read
> speed)
>  3)  FreeBSD 11-STABLE creates 'slow' big files. (~200  Mb/s)
>  4) zfs send slow-dataset | zfs recv   (under 11.0-releng) creates
> 'intermediate' files (320Mb/s)
>  5) file copy slow-file ... (under 11.0-releng) creates fast copy (400+Mb/s)
> 
> So, ZFS write code in 11-STABLE looks broken.
> 
> Here detailed log of my tests:
> 
> Box: Intel i5-2400, 16Gb of RAM, Adaptec 5805, 5x4Tb drives (HGST NAS drives) in
> JBOD, merged in RAIDZ2, enough free space:
> $ zpool list
> NAME      SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH ALTROOT
> zdata    18,1T  5,11T  13,0T         -     9%    28%  1.00x  ONLINE -

That's an interesting observation.

Could you please try starting with a freshly created pool and writing some data?
I wonder if 'zpool list -v' will show any significant differences between
different FreeBSD versions?
Another thing to check would be 'zdb -mm' but it could be very large.  However,
a difference should be sufficiently small as most of the space would remain unused.

And, just in case, please compare 'zpool get all' and 'zfs get all' between
those two versions.

-- 
Andriy Gapon