Date: Wed, 24 Feb 2010 10:24:33 +0100 From: Alexander Leidinger <Alexander@Leidinger.net> To: ticso@cicely.de, Bernd Walter <ticso@cicely7.cicely.de> Cc: freebsd-fs@freebsd.org, Ivan Voras <ivoras@freebsd.org>, ticso@cicely.de Subject: Re: Some ZFS+NFS benchmarks (OpenSolaris) Message-ID: <20100224102433.18641xjz8tdvmwdc@webmail.leidinger.net> In-Reply-To: <20100223230844.GP13767@cicely7.cicely.de> References: <hm19h4$8ah$1@dough.gmane.org> <20100223193458.GO13767@cicely7.cicely.de> <9bbcef731002231321t352ce3e6y5fdafbf75b7fac54@mail.gmail.com> <20100223230844.GP13767@cicely7.cicely.de>
next in thread | previous in thread | raw e-mail | index | archive | help
Quoting Bernd Walter <ticso@cicely7.cicely.de> (from Wed, 24 Feb 2010 00:08:44 +0100): > On Tue, Feb 23, 2010 at 10:21:41PM +0100, Ivan Voras wrote: >> On 23 February 2010 20:34, Bernd Walter <ticso@cicely7.cicely.de> wrote: >> > On Tue, Feb 23, 2010 at 08:15:48PM +0100, Ivan Voras wrote: >> >> http://staff.science.uva.nl/~delaat/sne-2009-2010/p02/report.pdf >> >> >> >> It's curious how ZIL on SSD doesn't help them with NFS when they >> >> increase the load. >> > >> > My assumption is because they already write linear on SSD and get a more >> > or less fixed write rate, while parallel write rate with disks can >> > increase because of reordering. See below for whta the SLOG is used for. I do not think the SLOG is used at all in this case (I haven't looked at the report, but NFS is not really used in situations for which the SLOG was designed for). >> > I'm personally impressed by my own tests on how much our current >> > USB stack can speed up random reads even with cheap USB flash sticks >> > used as cache devices. >> >> This is surprising to hear - I've just run some randomio >> (http://www.arctic.org/~dean/randomio/) tests on two little used USB >> flash sticks and got around 110 IOPS sequential writing (~~ 7 MB/s) >> and a bit less than 30 IOPS random writes of 4 KB buffers (amounting >> to ~~ 1 MB/s). > > Yes - your values seem to fit with my assumed values, but I'm talking > about L2ARC cache devices here. > Those are written linear with small bandwith and read random. > If they are too slow ZFS just drops data and cache fill slower. The cache is filled asynchron. And there is also no purge from ARC to L2ARC, so no increased latency if something needs to be thrown out of the ARC. The write speed does not really matter for L2ARC devices. > Random read access for USB sticks is great compared to HDD - although > USB has a high latency overhead. It depends, the corresponding numbers I present in http://www.leidinger.net/blog/2010/02/10/making-zfs-faster/ are mostly taken with gstat and depending on the workload I see only 0.4ms per read on the L2ARC. >> (the test command was "randomio file 16 1 1 4096 10"). >> >> The ZIL should be written practically linearly - the sequential write >> rate is relevant here - and it is actually significantly slower than >> what mechanical HDDs can achieve. > > A small test with ZIL on USB sticks was horrible because it couldn't > take up with write speed. The write latency for USB sticks is just too big (at least for the USB stick I use it is about 10 times slower than the write latency of my disks). Forget about it. FYI: what Sun is selling as their state of the art Storage appliances now comes with special SSD firmware. The SSDs for the L2ARC are optimized for read operations, while the SSDs for the ZIL are optimized for write performance. If you use the respective other one, the performance will not be as good as expected. >> Is your result with ZIL perhaps simply because you moved it to another >> device and so freed the main device? > > I assume for ZIL you really need a device with fast write speed, but > I have not much experience with ZIL devices. That is correct. And AFAIK the SLOG will only be used for direct I/O (O_FSYNC and the like), not for the normal write operations. So this should only speed up DBs with a good amount of changes (those DBs which take ACID sort of serious), and maybe busy mailservers... and similar workloads. Bye, Alexander. -- "Virtual" means never knowing where your next byte is coming from. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20100224102433.18641xjz8tdvmwdc>