From owner-freebsd-fs@FreeBSD.ORG Wed Sep 15 07:42:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBD811065670 for ; Wed, 15 Sep 2010 07:42:28 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1AA668FC0A for ; Wed, 15 Sep 2010 07:42:27 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id KAA00433; Wed, 15 Sep 2010 10:42:25 +0300 (EEST) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1OvmdN-00003U-6q; Wed, 15 Sep 2010 10:42:25 +0300 Message-ID: <4C9078E0.2050402@freebsd.org> Date: Wed, 15 Sep 2010 10:42:24 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100912 Lightning/1.0b2 Thunderbird/3.1.3 MIME-Version: 1.0 To: Steven Hartland References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua> <4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua> <4C874F00.3050605@freebsd.org> <4C8D087B.5040404@freebsd.org> <03537796FAB54E02959E2D64FC83004F@multiplay.co.uk> <4C8D280F.3040803@freebsd.org> <3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk> <4C8E4212.30000@freebsd.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 07:42:28 -0000 on 14/09/2010 20:30 Steven Hartland said the following: > > ----- Original Message ----- From: "Andriy Gapon" >> I'd really prefer to see description of your sources as svn revision rXXXXX plus >> http link to a diff of your actual sources to that revision. >> That would greatly help to see what you actually have, and what you don't have. > > The zfs files don't seem to have any svn revision information in them. Is there > something > else that would id the revision of > sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c > or the svn revision of the version of stable in use? I don't know. I haven't used cvsup since the switch to svn, but generally in CVS each file has its own independent revision, so it's impossible to tell that the whole tree is at revision R. Maybe you could get yourself another source tree by doing svn checkout and applying the patches there. Then you could update it with svn update. For you branch URI is svn://svn.freebsd.org/base/stable/8 You would be able to get a diff of your local changes with svn diff command. >> Well, I would love to see the mentioned above graphs for this real test load. >> Going below c_min likely means that you don't have all the latest stable/8 ZFS >> code, but i am not sure. > > It defintely is :( > If its relavent the source was downloaded via cvsup from the uk mirror. > >>> If someone could suggest a set of tests that would help I'll be happy to run >>> them but >>> from what's been said thus far is seems that the use of sendfile is forcing >>> memory >>> use >>> other than that coming from arc which is what's expected? >>> >>> Would running the same test with sendfile disabled in nginx help? >> >> The more test data the better, we could have some base for comparison and >> separation of general ARC issues from sendfile-specific issues. > > Going to run the following tests:- > 1. run a live test with "sendfile off" in the nginx config > 2. run a live test with "sendfile on" in the nginx config. > > During these tests I'm going to monitor the following every minute:- > time, kstat.zfs.misc.arcstats.size, vm.stats.vm.v_pdwakeups, > vm.stats.vm.v_cache_count, vm.stats.vm.v_inactive_count, > vm.stats.vm.v_active_count, vm.stats.vm.v_wire_count, > vm.stats.vm.v_free_count > > Anything else that should be monitored? > > Before each test the machine will be rebooted to try to ensure as direct a > comparison > as possible. > > Anything else that I should add / change before running said tests or should > monitor? This sounds sufficiently good. If you could arrange to draw the graphs of the data it would be terrific :) -- Andriy Gapon