From owner-freebsd-fs@FreeBSD.ORG  Fri Oct 29 14:42:05 2010
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 6EF76106566B;
	Fri, 29 Oct 2010 14:42:05 +0000 (UTC) (envelope-from avg@icyb.net.ua)
Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140])
	by mx1.freebsd.org (Postfix) with ESMTP id 871C98FC08;
	Fri, 29 Oct 2010 14:42:04 +0000 (UTC)
Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua
	[212.40.38.101])
	by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA07090;
	Fri, 29 Oct 2010 17:41:59 +0300 (EEST)
	(envelope-from avg@icyb.net.ua)
Message-ID: <4CCADD37.7000306@icyb.net.ua>
Date: Fri, 29 Oct 2010 17:41:59 +0300
From: Andriy Gapon <avg@icyb.net.ua>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US;
	rv:1.9.2.11) Gecko/20101021 Lightning/1.0b2 Thunderbird/3.1.5
MIME-Version: 1.0
To: Artemiev Igor <ai@kliksys.ru>
References: <3D1C350B94A44E5D95BAA1596D1EBF13@vosz.local>
	<20101029090417.GA17537@two.kliksys.ru>
	<4CCABFC2.3040701@icyb.net.ua>
In-Reply-To: <4CCABFC2.3040701@icyb.net.ua>
X-Enigmail-Version: 1.1.2
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org
Subject: Re: 8.1-STABLE: zfs and sendfile: problem still exists
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Oct 2010 14:42:05 -0000

on 29/10/2010 15:36 Andriy Gapon said the following:
> on 29/10/2010 12:04 Artemiev Igor said the following:
>> Yep, this problem exists. You may workaround it via bumping up
>> net.inet.tcp.sendspace up to 128k.  zfs sendfile is very ineffective. I have
>> made a small investigation via DTrace, it reads MAXBSIZE chunks, but map in vm
>> only one page (4K).  I.e. if you have a file with size 512K, sendfile make
>> calls freebsd_zfs_read 128 times.
> 
> What svn revision of FreeBSD source tree did you test?
> 

Ah, I think I see what's going on.
Either sendfile should (have an option to) use VOP_GETPAGES to request data or ZFS
mappedread should use vm_grab_page instead of vm_lookup_page for UIO_NOCOPY case.
Currently ZFS would read a whole FS block into ARC, but populate only one page
with data and for the rest it would just wastefully do uiomove(UIO_NOCOPY) from
ARC data.
So, e.g. zpool iostat would show that there are only few actual reads from a pool.
 The rest of the time must be spent churning over the data already in ARC and
doing page-per-VOP_READ copies from it.

-- 
Andriy Gapon