From owner-freebsd-fs@FreeBSD.ORG Mon Sep 12 18:38:52 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8B94106566C for ; Mon, 12 Sep 2011 18:38:52 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-vw0-f44.google.com (mail-vw0-f44.google.com [209.85.212.44]) by mx1.freebsd.org (Postfix) with ESMTP id 9CC108FC08 for ; Mon, 12 Sep 2011 18:38:52 +0000 (UTC) Received: by vws12 with SMTP id 12so3883430vws.17 for ; Mon, 12 Sep 2011 11:38:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=FVjdH7lC12Q3Dq7MDwoxG1yJQhVdETO8/+D6Xn6I7Yk=; b=Niemeo5nQUiSGU5Hr+zaSmlqkzo8497osmwQbOVB8EbS3CCb1NjYQUKztTazckrGEJ efNqeQzfzNU3RQcfY/6L+YjvQ/HmMH5fLVjLlzdo0QAEIBmnejoRRfLYkSzg9gYa6W7S lofBXmNPmcq3ppNorcW22CvP9H6eUBiPEKKS8= MIME-Version: 1.0 Received: by 10.52.90.229 with SMTP id bz5mr2920823vdb.77.1315851135169; Mon, 12 Sep 2011 11:12:15 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.52.113.225 with HTTP; Mon, 12 Sep 2011 11:12:15 -0700 (PDT) In-Reply-To: References: Date: Mon, 12 Sep 2011 20:12:15 +0200 X-Google-Sender-Auth: 9D2wAP5hf6OUB8L9LbYZ8slQYwU Message-ID: From: "K. Macy" To: Steven Hartland Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: ZFS-lighttpd2-sendfile, too high IO X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Sep 2011 18:38:53 -0000 2011/9/12 Steven Hartland : > sendfile doesn't work as you might expect on zfs, its not zero copy due t= o > the use of none buffer pool memory. > > You do get a benefit but it requires double the amount of memory to get i= t > so we've disabled sendfile > under nginx for zfs based hosts for this very reason. In my performance testing, sending from mmaped I/O on ZFS was dramatically faster (> 2x) than sendfile. Cheers > =A0 Regards > =A0 Steve > ----- Original Message ----- From: "Laszlo KAROLYI" > To: > Sent: Monday, September 12, 2011 12:10 PM > Subject: ZFS-lighttpd2-sendfile, too high IO > > >> Hello, Recently I installed a FreeBSD with the newest 8.2-STABLE and zfs >> version. I use lighttpd2 and zfs on it, and munin to monitor the outgoin= g >> bandwidth. Zpool version 28, zfs version 5, with the latest kernel. We h= ave >> a big mp3 archive (half hour, 256kbit/s mp3-s) which lighty serves. This >> means full random IO. When I see that the server serves 15mbit/sec, i se= e >> constantly 8-10Mbyte/sec reads on the zfs raidz1 array, which is too muc= h. >> As I could see from truss logs, lighty uses sendfile and writev.My setti= ngs: >> vfs.zfs.l2c_only_size: 15106737664 > > ... > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > This e.mail is private and confidential between Multiplay (UK) Ltd. and t= he > person or entity to whom it is addressed. In the event of misdirection, t= he > recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >