From owner-freebsd-stable@FreeBSD.ORG Mon Nov 2 09:52:58 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D8151065694 for ; Mon, 2 Nov 2009 09:52:58 +0000 (UTC) (envelope-from freebsd-stable@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id CCF378FC24 for ; Mon, 2 Nov 2009 09:52:57 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.50) id 1N4tak-0000Hs-LO for freebsd-stable@freebsd.org; Mon, 02 Nov 2009 10:52:50 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Nov 2009 10:52:50 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Mon, 02 Nov 2009 10:52:50 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Ivan Voras Date: Mon, 02 Nov 2009 10:52:31 +0100 Lines: 18 Message-ID: References: <772532900-1257123963-cardhu_decombobulator_blackberry.rim.net-1402739480-@bda715.bisx.prod.on.blackberry> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Thunderbird 2.0.0.23 (X11/20090928) In-Reply-To: <772532900-1257123963-cardhu_decombobulator_blackberry.rim.net-1402739480-@bda715.bisx.prod.on.blackberry> Sender: news Subject: Re: Performance issues with 8.0 ZFS and sendfile/lighttpd X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Nov 2009 09:52:58 -0000 gnukix@alltel.blackberry.com wrote: > I can send in more documentation later but I am seeing severe zfs performance issues with lighttpd. Same machine using UFS will push 1gbit or more but same content and traffic load can not hit 200mbit. Ufs does around 3 megabytes/sec IO at 800mbit network but zfs pushes the disks into the ground with 50+ megabytes/sec dusk i/o. No compression no atime no checksums on zfs and still same IO levels. Ufs with soft updates and atime on. Orders of magnitude more disk IO... Like zfs isn't using cache or isn't coalescing disk reads or both. > > Has anyone else seen this or have any recommendations? Lighttpd config remains exactly the same as well FYI. Only difference is ufs vs zfs. AFAIK, ZFS is incompatible (currently) with some advanced VM operations (like mmap, and I think sendfile relies on the same mechanism as mmap), so that could be a cause of the slowdown. Though I'm surprised you can only get 200 MBit/s - that's 25 MB/s and I think that even with multiple memcpy-ing data around the kernel you should be able to get hundreds of MB/s on newer hardware (which normally really can achieve tens of gigabytes/s of sustained memory access). What else can you observe from your system? Do you have exceedingly high sys times and load numbers? I'm also interested in what does 10 seconds of running 'vmstat 1' looks like on your system. Is it a bare machine or a virtual machine?