From owner-freebsd-fs@FreeBSD.ORG Tue Sep 14 17:40:59 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7EDB106564A; Tue, 14 Sep 2010 17:40:59 +0000 (UTC) (envelope-from prvs=1873ccaa52=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 106EE8FC26; Tue, 14 Sep 2010 17:40:58 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Tue, 14 Sep 2010 18:30:06 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Tue, 14 Sep 2010 18:30:06 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 by mail1.multiplay.co.uk (MDaemon PRO v10.0.4) with ESMTP id md50011232724.msg; Tue, 14 Sep 2010 18:30:06 +0100 X-Authenticated-Sender: Killing@multiplay.co.uk X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1873ccaa52=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Andriy Gapon" References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua> <4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua> <4C874F00.3050605@freebsd.org> <4C8D087B.5040404@freebsd.org> <03537796FAB54E02959E2D64FC83004F@multiplay.co.uk> <4C8D280F.3040803@freebsd.org> <3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk> <4C8E4212.30000@freebsd.org> Date: Tue, 14 Sep 2010 18:30:07 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5931 Cc: freebsd-fs@freebsd.org, Pawel Jakub Dawidek , jhell Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Sep 2010 17:40:59 -0000 ----- Original Message ----- From: "Andriy Gapon" > I'd really prefer to see description of your sources as svn revision rXXXXX plus > http link to a diff of your actual sources to that revision. > That would greatly help to see what you actually have, and what you don't have. The zfs files don't seem to have any svn revision information in them. Is there something else that would id the revision of sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c or the svn revision of the version of stable in use? > Well, I would love to see the mentioned above graphs for this real test load. > Going below c_min likely means that you don't have all the latest stable/8 ZFS > code, but i am not sure. It defintely is :( If its relavent the source was downloaded via cvsup from the uk mirror. >> If someone could suggest a set of tests that would help I'll be happy to run them but >> from what's been said thus far is seems that the use of sendfile is forcing memory >> use >> other than that coming from arc which is what's expected? >> >> Would running the same test with sendfile disabled in nginx help? > > The more test data the better, we could have some base for comparison and > separation of general ARC issues from sendfile-specific issues. Going to run the following tests:- 1. run a live test with "sendfile off" in the nginx config 2. run a live test with "sendfile on" in the nginx config. During these tests I'm going to monitor the following every minute:- time, kstat.zfs.misc.arcstats.size, vm.stats.vm.v_pdwakeups, vm.stats.vm.v_cache_count, vm.stats.vm.v_inactive_count, vm.stats.vm.v_active_count, vm.stats.vm.v_wire_count, vm.stats.vm.v_free_count Anything else that should be monitored? Before each test the machine will be rebooted to try to ensure as direct a comparison as possible. Anything else that I should add / change before running said tests or should monitor? Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.