From owner-freebsd-fs@FreeBSD.ORG  Sun Jan 26 03:01:59 2014
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 6BA1481F
 for <freebsd-fs@freebsd.org>; Sun, 26 Jan 2014 03:01:59 +0000 (UTC)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id 250B9109D
 for <freebsd-fs@freebsd.org>; Sun, 26 Jan 2014 03:01:58 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s0Q31s1K045572;
 Sat, 25 Jan 2014 22:01:54 -0500 (EST)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s0Q31rXu045569;
 Sat, 25 Jan 2014 22:01:53 -0500 (EST) (envelope-from wollman)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <21220.31393.533799.991163@hergotha.csail.mit.edu>
Date: Sat, 25 Jan 2014 22:01:53 -0500
From: Garrett Wollman <wollman@bimajority.org>
To: Matthew Ahrens <mahrens@delphix.com>
Subject: Re: ZFS read performance anomaly / performance regression
In-Reply-To: <CAJjvXiHnBq3bg-ZGgyzFw6WBNau0SmoLUfxWLtFCnBwb13fqaQ@mail.gmail.com>
References: <21219.21690.165105.561362@hergotha.csail.mit.edu>
 <CAJjvXiHnBq3bg-ZGgyzFw6WBNau0SmoLUfxWLtFCnBwb13fqaQ@mail.gmail.com>
X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (hergotha.csail.mit.edu [127.0.0.1]); Sat, 25 Jan 2014 22:01:54 -0500 (EST)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: freebsd-fs <freebsd-fs@freebsd.org>
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs/>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 26 Jan 2014 03:01:59 -0000

<<On Sat, 25 Jan 2014 11:43:28 -0800, Matthew Ahrens <mahrens@delphix.com> said:

> How are you reading the files?  Could you be reading in less than
> block-size chunks (default 128K)?

Nope.  dd bs=1024k.

> This sounds similar to issue discussed in the thread with subject "ZFS slow
> reads for unallocated blocks", which was comparing sparse vs cached reads.
>  Since your storage is so fast, prefetch will probably read and cache the
> blocks by the time you issue the read() syscall, so you may be hitting a
> similar issue.

How would that have changed between 9.1 and 9.2?

-GAWollman