From owner-freebsd-fs@FreeBSD.ORG Wed Sep 15 10:32:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9069E1065695; Wed, 15 Sep 2010 10:32:25 +0000 (UTC) (envelope-from prvs=1874f602db=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id D10878FC19; Wed, 15 Sep 2010 10:32:24 +0000 (UTC) X-MDAV-Processed: mail1.multiplay.co.uk, Wed, 15 Sep 2010 11:32:19 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Wed, 15 Sep 2010 11:32:19 +0100 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mail1.multiplay.co.uk X-Spam-Level: X-Spam-Status: No, score=-5.0 required=6.0 tests=USER_IN_WHITELIST shortcircuit=ham autolearn=disabled version=3.2.5 Received: from r2d2 by mail1.multiplay.co.uk (MDaemon PRO v10.0.4) with ESMTP id md50011248038.msg; Wed, 15 Sep 2010 11:32:18 +0100 X-Authenticated-Sender: Killing@multiplay.co.uk X-MDRemoteIP: 188.220.16.49 X-Return-Path: prvs=1874f602db=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Andriy Gapon" References: <5DB6E7C798E44D33A05673F4B773405E@multiplay.co.uk><4C85E91E.1010602@icyb.net.ua><4C873914.40404@freebsd.org><20100908084855.GF2465@deviant.kiev.zoral.com.ua><4C874F00.3050605@freebsd.org><4C8D087B.5040404@freebsd.org><03537796FAB54E02959E2D64FC83004F@multiplay.co.uk><4C8D280F.3040803@freebsd.org><3FBF66BF11AA4CBBA6124CA435A4A31B@multiplay.co.uk><4C8E4212.30000@freebsd.org> Date: Wed, 15 Sep 2010 11:32:18 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.5931 Cc: freebsd-fs@freebsd.org, jhell , Pawel Jakub Dawidek Subject: Re: zfs very poor performance compared to ufs due to lack of cache? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2010 10:32:25 -0000 Ok the results are in, the conclusions I can see from the data, others may see more, are:- === Common === * arc size on boot is ~180M with a target size of 6.5G === sendfile on === * arc size increases on demand but peaks the min value * The difference between min and max arc is taken up by inactive pages * vm page daemon wakeups sit at a constant level once the machine has filled memory and never fully empties swap. === sendfile off === * arc size increases on demand all the way up the the max value * vm cache count stays at almost zero all the time === conclusion === The interaction of zfs and sendfile is causing large amounts of memory to end up in the inactive pool and only the use of a hard min arc limit is ensuring that zfs forces the vm to release said memory so that it can be used by zfs arc. The source data, xls's and exported graphs can be found here:- http://www.multiplaygameservers.com/dropzone/zfs-sendfile-results.zip Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.