From owner-freebsd-fs@FreeBSD.ORG  Sat Sep 17 18:14:55 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 02546106566C
	for <freebsd-fs@freebsd.org>; Sat, 17 Sep 2011 18:14:55 +0000 (UTC)
	(envelope-from lytboris@gmail.com)
Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com
	[209.85.161.54])
	by mx1.freebsd.org (Postfix) with ESMTP id 8F5A28FC0A
	for <freebsd-fs@freebsd.org>; Sat, 17 Sep 2011 18:14:53 +0000 (UTC)
Received: by fxg9 with SMTP id 9so3624875fxg.13
	for <freebsd-fs@freebsd.org>; Sat, 17 Sep 2011 11:14:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type;
	bh=D7pX2ZciewCoDqC/df7VmKv4UcMsVm9AX9JR2B9Coxo=;
	b=nfonVJa/D1rSDBnq5pHeJCo4vwYd1auDk3vWi07QZrf/BzIE3+CeMUdtqs2JKZhf79
	0aSt+qmSVpdNM904EMpcDyX5KtFlRpRomq0sUYgyskYFjrh4X8U+aa4AWIzWmk9U5rA3
	Lkqy0gg+q+Wlk8l6Lv3aRhCMPmsaRWKEuCyCk=
MIME-Version: 1.0
Received: by 10.223.17.3 with SMTP id q3mr1593897faa.71.1316283292674; Sat, 17
	Sep 2011 11:14:52 -0700 (PDT)
Received: by 10.223.106.15 with HTTP; Sat, 17 Sep 2011 11:14:52 -0700 (PDT)
In-Reply-To: <6B437FA4-B422-4BE7-BDF5-F90717F3865B@bitgravity.com>
References: <CAEJYa-Si+4Tj5sj8fuxWfqjZgMX1cB8y=JWqJqe_F+R4Er9g_A@mail.gmail.com>
	<6B437FA4-B422-4BE7-BDF5-F90717F3865B@bitgravity.com>
Date: Sat, 17 Sep 2011 22:14:52 +0400
Message-ID: <CAEJYa-TKjQtoR_39yDJVW8vxPDyLOMvBWsyvUVQ5gRzji+mSiQ@mail.gmail.com>
From: Lytochkin Boris <lytboris@gmail.com>
To: "David P. Discher" <dpd@bitgravity.com>
Content-Type: text/plain; charset=ISO-8859-1
Cc: freebsd-fs@freebsd.org
Subject: Re: [ZFS] starving reads while idle disks
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 17 Sep 2011 18:14:55 -0000

Hi.

> Do you see the same read-stravation when writing the tar to a file ? (possibly outside the zpool).
Yep.
> I have anecdotal suspicion that /dev/null has some performance hit of blocking or locking.
No-no. Every program that tries to read from ZFS faces this issue actually.

 I found something more interesting. Lets presume I have 10 big dirs in .
 Issuing tar|dd command on separate dirs (so spawning 10 "threads")
 simultaneously will result in 10x faster reads cumulatively (I saw
 15Mb/s in zpool iostat), and disk load may be as high as 70%.
 So I think there is some read throttling thing that limits read speed
 per read(). But still no clue where to find this bottleneck.

-- 
Wbr,
Boris.