From owner-freebsd-fs@FreeBSD.ORG  Mon Sep 12 18:58:25 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id B1DEE106574B
	for <freebsd-fs@freebsd.org>; Mon, 12 Sep 2011 18:58:25 +0000 (UTC)
	(envelope-from bfriesen@simple.dallas.tx.us)
Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74])
	by mx1.freebsd.org (Postfix) with ESMTP id 75B888FC16
	for <freebsd-fs@freebsd.org>; Mon, 12 Sep 2011 18:58:24 +0000 (UTC)
Received: from freddy.simplesystems.org (freddy.simplesystems.org
	[65.66.246.65])
	by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id
	p8CIwNEa029634; Mon, 12 Sep 2011 13:58:23 -0500 (CDT)
Date: Mon, 12 Sep 2011 13:58:23 -0500 (CDT)
From: Bob Friesenhahn <bfriesen@simple.dallas.tx.us>
X-X-Sender: bfriesen@freddy.simplesystems.org
To: "K. Macy" <kmacy@freebsd.org>
In-Reply-To: <CAHM0Q_Piojt=Ob+sniMpaO9v1j_mBxCrKXFi6yxWktn2tnnSVQ@mail.gmail.com>
Message-ID: <alpine.GSO.2.01.1109121356080.12948@freddy.simplesystems.org>
References: <freemail.20110912131045.97846.1@xmldata03.freemail.hu>
	<F959B6B4287E4B7A9C55B34E7A2EDAC9@multiplay.co.uk>
	<CAHM0Q_Piojt=Ob+sniMpaO9v1j_mBxCrKXFi6yxWktn2tnnSVQ@mail.gmail.com>
User-Agent: Alpine 2.01 (GSO 1266 2009-07-14)
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2
	(blade.simplesystems.org [65.66.246.90]);
	Mon, 12 Sep 2011 13:58:23 -0500 (CDT)
Cc: freebsd-fs@freebsd.org
Subject: Re: ZFS-lighttpd2-sendfile, too high IO
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 12 Sep 2011 18:58:25 -0000

On Mon, 12 Sep 2011, K. Macy wrote:

> 2011/9/12 Steven Hartland <killing@multiplay.co.uk>:
>> sendfile doesn't work as you might expect on zfs, its not zero copy due to
>> the use of none buffer pool memory.
>>
>> You do get a benefit but it requires double the amount of memory to get it
>> so we've disabled sendfile
>> under nginx for zfs based hosts for this very reason.
>
> In my performance testing, sending from mmaped I/O on ZFS was
> dramatically faster (> 2x) than sendfile.

The problem is that since the zfs ARC and the backing memory used for 
the mapping are not the same, twice as much memory is used.  This 
would not be good for streaming if the machine is short on RAM.

Bob
-- 
Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/