From owner-freebsd-stable@FreeBSD.ORG  Fri Oct 29 09:47:38 2010
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id E2D55106564A;
	Fri, 29 Oct 2010 09:47:38 +0000 (UTC) (envelope-from ai@kliksys.ru)
Received: from gate.kliksys.ru (gate.kliksys.ru [78.110.241.113])
	by mx1.freebsd.org (Postfix) with ESMTP id 984148FC0A;
	Fri, 29 Oct 2010 09:47:38 +0000 (UTC)
Received: from [192.168.0.204] (helo=two.kliksys.ru)
	by gate.kliksys.ru with esmtp (Exim 4.71 (FreeBSD))
	(envelope-from <ai@kliksys.ru>)
	id 1PBksW-0008qe-FF; Fri, 29 Oct 2010 13:04:04 +0400
Date: Fri, 29 Oct 2010 13:04:17 +0400
From: Artemiev Igor <ai@kliksys.ru>
To: freebsd-stable@freebsd.org, freebsd-fs@freebsd.org
Message-ID: <20101029090417.GA17537@two.kliksys.ru>
References: <3D1C350B94A44E5D95BAA1596D1EBF13@vosz.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3D1C350B94A44E5D95BAA1596D1EBF13@vosz.local>
User-Agent: Mutt/1.5.20 (2009-06-14)
X-Spam_score: 0.0
Cc: 
Subject: Re: 8.1-STABLE: zfs and sendfile: problem still exists
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>, 
	<mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
	<mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 29 Oct 2010 09:47:39 -0000

On Thu, Oct 28, 2010 at 09:57:22AM +0400, Alexander Zagrebin wrote:
> Hi!
> 
> I've noticed that ZFS on 8.1-STABLE still has problems with sendfile.
> When accessing a file at first time the transfer speed is too low, but
> on following attempts the transfer speed is normal.
...
> I've tried ftpd and nginx with "sendfile on". The behavior is the same.
> After disabling using sendfile in nginx ("sendfile off") the problem has
> gone.

Yep, this problem exists. You may workaround it via bumping up
net.inet.tcp.sendspace up to 128k.  zfs sendfile is very ineffective. I have
made a small investigation via DTrace, it reads MAXBSIZE chunks, but map in vm
only one page (4K).  I.e. if you have a file with size 512K, sendfile make
calls freebsd_zfs_read 128 times.