From owner-freebsd-fs@FreeBSD.ORG Wed Feb 24 09:40:47 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 81273106568D for ; Wed, 24 Feb 2010 09:40:47 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 27CB98FC0C for ; Wed, 24 Feb 2010 09:40:46 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2D1AD.dip.t-dialin.net [217.226.209.173]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 25FEB84469B; Wed, 24 Feb 2010 10:24:37 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E29051B19AB; Wed, 24 Feb 2010 10:24:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1267003474; bh=ccEb+qYxbZ8/QgDUnaYG9Mz9sWEMvAx5fqBRgrchWwI=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=DxlwHCryZg0yFsWCFT/s72ETwcqBJeU3/Y24SHQ8Ijn3DS70zEVFOHPPrtFMnbIjF 2KmIoBMIig0icGCxSLNcSdddpmOwekf/b7Cssnyb6bQhsX7aUYGZ2PLOlZ9fTjNYO/ HFY8+IkAOS7sUz/OPfJIpTc54aV34Uv+6chkMibFlDpsi2J/vQPqtajJ8y3iih/ER2 cPNawWtlYfcuer5OBk1NOdgQHpNBjmlg+cW8s6KzmvvOaqNYtyEJAqtIIQw5glKofo e/QHRxIScTR5c+F1gRlA0OBUSVTGu6lsgxT+n90sor3sU4nJ5woYZD7omlv4ailp2x kkTKj8bDioppw== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o1O9OXx1014000; Wed, 24 Feb 2010 10:24:33 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 24 Feb 2010 10:24:33 +0100 Message-ID: <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> Date: Wed, 24 Feb 2010 10:24:33 +0100 From: Alexander Leidinger To: ticso@cicely.de, Bernd Walter References: <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> <20100223230844.GP13767@cicely7.cicely.de> In-Reply-To: <20100223230844.GP13767@cicely7.cicely.de> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 25FEB84469B.2CD4B X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.763, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, J_CHICKENPOX_33 0.60, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1267608278.62427@A6xXwX/8fCrwF6Ye+OgG6A X-EBL-Spam-Status: No Cc: freebsd-fs@freebsd.org, Ivan Voras , ticso@cicely.de Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Feb 2010 09:40:47 -0000 Quoting Bernd Walter (from Wed, 24 Feb 2010 00:08:44 +0100): > On Tue, Feb 23, 2010 at 10:21:41PM +0100, Ivan Voras wrote: >> On 23 February 2010 20:34, Bernd Walter wrote: >> > On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: >> >> http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf >> >> >> >> It's curious how ZIL on SSD doesn't help them with NFS when they >> >> increase the load. >> > >> > My assumption is because they already write linear on SSD and get a more >> > or less fixed write rate, while parallel write rate with disks can >> > increase because of reordering. See below for whta the SLOG is used for. I do not think the SLOG is used at all in this case (I haven't looked at the report, but NFS is not really used in situations for which the SLOG was designed for). >> > I'm personally impressed by my own tests on how much our current >> > USB stack can speed up random reads even with cheap USB flash sticks >> > used as cache devices. >> >> This is surprising to hear - I've just run some randomio >> (http://www.arctic.org/~dean/randomio/) tests on two little used USB >> flash sticks and got around 110 IOPS sequential writing (~~ 7 MB/s) >> and a bit less than 30 IOPS random writes of 4 KB buffers (amounting >> to ~~ 1 MB/s). > > Yes - your values seem to fit with my assumed values, but I'm talking > about L2ARC cache devices here. > Those are written linear with small bandwith and read random. > If they are too slow ZFS just drops data and cache fill slower. The cache is filled asynchron. And there is also no purge from ARC to L2ARC, so no increased latency if something needs to be thrown out of the ARC. The write speed does not really matter for L2ARC devices. > Random read access for USB sticks is great compared to HDD - although > USB has a high latency overhead. It depends, the corresponding numbers I present in http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/ are mostly taken with gstat and depending on the workload I see only 0.4ms per read on the L2ARC. >> (the test command was "randomio file 16 1 1 4096 10"). >> >> The ZIL should be written practically linearly - the sequential write >> rate is relevant here - and it is actually significantly slower than >> what mechanical HDDs can achieve. > > A small test with ZIL on USB sticks was horrible because it couldn't > take up with write speed. The write latency for USB sticks is just too big (at least for the USB stick I use it is about 10 times slower than the write latency of my disks). Forget about it. FYI: what Sun is selling as their state of the art Storage appliances now comes with special SSD firmware. The SSDs for the L2ARC are optimized for read operations, while the SSDs for the ZIL are optimized for write performance. If you use the respective other one, the performance will not be as good as expected. >> Is your result with ZIL perhaps simply because you moved it to another >> device and so freed the main device? > > I assume for ZIL you really need a device with fast write speed, but > I have not much experience with ZIL devices. That is correct. And AFAIK the SLOG will only be used for direct I/O (O_FSYNC and the like), not for the normal write operations. So this should only speed up DBs with a good amount of changes (those DBs which take ACID sort of serious), and maybe busy mailservers... and similar workloads. Bye, Alexander. -- "Virtual" means never knowing where your next byte is coming from. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137