From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 10:23:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BACD1065674; Wed, 24 Feb 2010 10:23:53 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 7540B8FC19; Wed, 24 Feb 2010 10:23:52 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id o1OANll9017628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 24 Feb 2010 11:23:50 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.3/8.14.3) with ESMTP id o1OANeVK080424 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 24 Feb 2010 11:23:40 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id o1OANZMf025426; Wed, 24 Feb 2010 11:23:40 +0100 (CET) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id o1OANHA7025425; Wed, 24 Feb 2010 11:23:17 +0100 (CET) (envelope-from ticso) Date: Wed, 24 Feb 2010 11:23:17 +0100 From: Bernd Walter To: Alexander Leidinger Message-ID: <20100224102317.GR13767@cicely7.cicely.de> References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> <20100223230844.GP13767@cicely7.cicely.de> <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on spamd.cicely.de Cc: freebsd-fs@freebsd.org, Bernd Walter , ticso@cicely.de, Ivan Voras Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 10:23:53 -0000 On Wed, Feb 24, 2010 at 10:24:33AM +0100, Alexander Leidinger wrote: > > Quoting Bernd Walter (from Wed, 24 Feb 2010 > 00:08:44 +0100): > > >Yes - your values seem to fit with my assumed values, but I'm talking > >about L2ARC cache devices here. > >Those are written linear with small bandwith and read random. > >If they are too slow ZFS just drops data and cache fill slower. > > The cache is filled asynchron. And there is also no purge from ARC to > L2ARC, so no increased latency if something needs to be thrown out of > the ARC. The write speed does not really matter for L2ARC devices. > > >Random read access for USB sticks is great compared to HDD - although > >USB has a high latency overhead. > > It depends, the corresponding numbers I present in > http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/ > are mostly taken with gstat and depending on the workload I see only > 0.4ms per read on the L2ARC. This matches with my experience, which is already impressive fast for USB as such. But under load the latency only increses to a few ms, while HDD easily climb up to a few hundred ms. Even half rotational delay without seek time for a 7200 HDD is about 4ms - 10 times slower than USB stick. With USB sticks there is only a high latency expection from time to time when there was also write load, but this rarely happens for me, especially after adding multiple devices. What I want to say is that Performance wise SSD are the better choice, but depending on the workload the higher capacity you get with USB sticks can be more important. I'm currently running a mix of 1x 4G CF, 4x 4G USB, 1x 32G SATA SSD. Speed with SSD is best, read latency with CF isn't much different from USB, but price is higher. I've ordered 4x 16G cheap sticks, since I'm very pleased with the speed, but want a higher hit rate. The only drawback with USB sticks is that my BIOS sees them and is horrible slow to boot. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.