Date: Thu, 24 Sep 2015 16:57:36 +0300 From: Dmitrijs <war@dim.lv> To: Paul Kraus <paul@kraus-haus.org>, FreeBSD Questions <freebsd-questions@freebsd.org> Subject: Re: zfs performance degradation Message-ID: <56040150.90403@dim.lv> In-Reply-To: <782C9CEF-BE07-4E05-83ED-133B7DA96780@kraus-haus.org> References: <56019211.2050307@dim.lv> <37A37E9D-9D65-4553-BBA2-C5B032163499@kraus-haus.org> <56038054.5060906@dim.lv> <782C9CEF-BE07-4E05-83ED-133B7DA96780@kraus-haus.org>
next in thread | previous in thread | raw e-mail | index | archive | help
2015.09.24. 16:17, Paul Kraus wrote: > So if I plan to work with data a lot, get decent performance and still be sure I'm on the safe side with mirror-raid1, should I choose another filesystem? Especially, if i do not really need snapshots, clones, etc. > What is your definition of “decent” performance ? What does your _real_ workload look like ? > > Did you have performance issues doing real work which caused you to try to find the cause -or- were you benchmarking before trying to use the system for real work ? As we use our file server mostly to store users' backup and hdd images for old and new computers. File sizes mostly go 10-100Gb+. In the daytime the usually have 1 "writer" and several "readers". And datetime performance is very important to us. In the nighttime several "writers" and no "readers". Most file operations are sequential. Working environment mostly ms windows 7. We used to have software mirror raid on one windows 7 computer, shared to our gigabit network. Now we decided to have dedicate box for file storage. In my terms "decent" where performance is limited mostly by gigabit network, not the file operations. So if zfs raid mirror can give me total 100Mbs+ output at least on 1-2 readers - it's ok. When I have total 70-80Mbs and less in perspective - I worry that I'm doing something wrong. And as you see, I'm just beginning to dig into freebsd file systems etc, so my question and answers may be inadequate. >> Or is it not possible at all, and I should put something like raid0 for work and tolerate slow backup on raid1 at nights? > There are many places in ZFS where you can run into performance bottlenecks. Remember, ZFS was designed for data integrity (end to end checksums), data reliability (lots of ways to get redundancy), and scalability. Performance was secondary from the very beginning. There are lots of other filesystems with much better performance, there are few (if any) with more protection for your data. Do not get me wrong, the performance of ZFS _can_ be very good, but you need to understand your workload and layout the zpool to accommodate that workload. Thank you for clarification. I made some research and found out that zfs is really very good system for the points you mentioned. That's why we choose freebsd and zfs. And that's exactly where I need help - to understand how big is price for zfs reliability and options. If we need to buy better CPU, add more RAM, more disks to get better performance - it must be considered, that's one thing. And if there are tweaks, that can be adjusted via settings - that's another thing. > For example, one of my critical workloads is NFS with sync writes. My zpool layout is many vdevs of 3-way mirrors with a separate ZIL device (SLOG). I have not been able to go production with this server yet because I am waiting on backordered SSDs for the SLOG. The original SSDs I used just did not have the small block write performance I needed. > > Another example is one of my _backup_ servers, which has a 6 drive RAIDz2 zpool layout. In this case I am not terribly concerned about performance as I am limited by the 1 Gbps network connection. > > Also note that in general, the _best_ performance you can expect of any zpool layout is equivalent to _1_ drives worth of I/O per _vdev_. So my 6 drive RAIDz2 has performance equivalent to _one_ of the drives that make up that vdev. Which is fine for _my_ workload. The rule of thumb for performance that I received over on the OpenZFS mailing list a while back was to assume you can get 100 MB/sec and 100 random I/Ops from a consumer SATA hard disk drive. I have seen nothing, even using “enterprise” grade HDDs, to convince me that is a bad rule of thumb. If your workload is strictly sequential you _may_ get more. > > So a zpool made up of one single vdev, no matter how many drives, will average the performance of one of those drives. It does not really matter if it is a 2-way mirror vdev, a 3-way mirror vdev, a RAIDz2 vdev, a RAIDz3 vdev, etc. This is more true for write operations that read (mirrors can achieve higher performance by reading from multiple copies at once). Thanks! Now I understand. Although it is strange, that you did not mention how RAM and\or CPU matters. Or do they? I start observing that my 4core Celeron J1900 is throttling writes. Still haven't found at least approximate specification\recommendation as simple as "if you need zfs mirror 2 drives, take at least core i3 or e3 processor, 10 drives - go for e5 xeon, etc". I did not notice cpu impact on windows machine, still i've got " load averages: 1.60, 1.43, 1.24 " on write on zfs. best regards, Dmitriy
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?56040150.90403>