Date: Mon, 20 May 2013 18:28:51 +0300 From: Daniel Kalchev <daniel@digsys.bg> To: freebsd-fs@freebsd.org Subject: Re: L2ARC and ZIL on same SSD? Message-ID: <519A4133.3090502@digsys.bg> In-Reply-To: <CAHDrHSv-%2BPjLiPcZrE7CJx85Qyo%2B%2BYv%2Bc_1myNH_mZ1nS5Xwog@mail.gmail.com> References: <CAHDrHSv-%2BPjLiPcZrE7CJx85Qyo%2B%2BYv%2Bc_1myNH_mZ1nS5Xwog@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 20.05.13 17:53, Josh Beard wrote: > Hello, > > I have a file server running 9.1-RELEASE with 24 GB of RAM and a 6 TB zpool > spread across 12 500GB disks in two raidz1 and 2 spares. This server > houses hundreds of home directories for end-users and stays pretty busy > with both read and writes. > > I'm planning to add two SSDs soon (128 GB each) to help performance and was > planning to mirror them. I was curious if having the L2ARC and ZIL on the > same disks (mirrored) is okay. With some searching, I haven't seen any > "official" word on this. I've seen some folks say not to and others > recommend it. Adding a second set of SSDs isn't desirable, as I'll start > cutting into my available storage capacity (16 bay system). > > Any advice or experience with this? > Indeed, there is no official position on how SSDs should be used with ZFS, because SSDs are so much different from each other. Here is what you should consider, however: There is no need to mirror the L2ARC. On failure, it is simply ignored. Therefore, for L2ARC in general you put as large SSD as you find, that has good read rate. You might want to have this on a separate controller, which provides the highest bandwidth. There is suggestion to mirror the ZIL, in order to avoid pool damage, but in post-v28 pools, the ZIL can be removed and replaced on the fly. Since the ZIL is only ever read after crash, it all boils down what your recovery scenario is. If you can tolerate the loss of most recent data, you really do not need to mirror ZIL. In any case, it is a bad idea to mirror the SSDs before giving them to ZFS. Let ZFS manage it's redundancy, by giving it either the raw devices or raw slices/partittions. You also need to start thinking of SSDs in ZFS as "expense". You are wearing it during use and at some point you will have to replace the SSD. That is, you trade SSD wear for performance. L2ARC and ZIL have quite different wearing characteristics. In theory, ZIL would work best if it is either an high endurance SLC flash drive, or if you use only small portion of it for writes. For example, use only 8GB out of an 30GB (or larger) SSD for ZIL. L2ARC, if sufficiently large would wear much less, because it will be mostly used for reads. But, if your L2ARC is smaller than required -- it will be overwriten many many times and thus wear faster. In both cases you can monitor the "Media Wearout Indicator" of SMART to see how much life you have left. So considering especially the "expense" concept, you may want to have one higher performance but small SSD for ZIL and another lower performance but cheap SSD for L2ARC. If your SSD has sufficient read/write performance, you might use slices of the same SSD for ZIL and L2ARC. But consider consumer grade "performance" SSD wear faster. One good things is that as SSDs wear, larger and cheaper SSDs will be available :) I don't know how much this helps -- you need to put it into perspective for your workload and other constraints. Daniel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?519A4133.3090502>