Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 12 Jun 2013 19:54:36 +0200
From:      Adam Nowacki <nowakpl@platinum.linux.pl>
To:        freebsd-fs@freebsd.org
Subject:   Re: An order of magnitude higher IOPS needed with ZFS than UFS
Message-ID:  <51B8B5DC.2010703@platinum.linux.pl>
In-Reply-To: <20130612114937.GA13688@icarus.home.lan>
References:  <51B79023.5020109@fsn.hu> <op.wykdduw834t2sn@markf.office.supranet.net> <20130612114937.GA13688@icarus.home.lan>

next in thread | previous in thread | raw e-mail | index | archive | help
On 2013-06-12 13:49, Jeremy Chadwick wrote:
> On Wed, Jun 12, 2013 at 06:40:32AM -0500, Mark Felder wrote:
>> On Tue, 11 Jun 2013 16:01:23 -0500, Attila Nagy <bra@fsn.hu> wrote:
>>
>>> BTW, the file systems are 77-78% full according to df (so ZFS
>>> holds more, because UFS is -m 8).
>>
>> ZFS write performance can begin to drop pretty badly when you get
>> around 80% full. I've not seen any benchmarks showing an improvement
>> with a very fast and large ZIL or tons of memory, but I'd expect
>> that would help significantly. Just note that you're right at the
>> edge where performance gets impacted.
>
> Mark, do you have any references for this?  I'd love to learn/read more
> about this engineering/design aspect (I won't say flaw, I'll just say
> aspect) to ZFS, as it's the first I've heard of it.
>
> The reason I ask: (respectfully, not judgementally) I'm worried you
> might be referring to something that has to do with SSDs and not ZFS,
> specifically SSD wear-levelling performing better with lots of free
> space (i.e. a small FTL map; TRIM helps with this immensely) -- where
> the performance hit tends to begin around the 70-80% mark.  (I can talk
> more about that if asked, but want to make sure the two things aren't
> being mistaken for one another)
>

So I went hunting for some evidence and created this:
http://tepeserwery.pl/nowak/fillingzfs.png

Columns are groups of sectors, new row is created every time a FLUSH 
command is sent to a disk. Percentage is the amount of filled space in 
the pool. Red means a write happened there, Pool is 1GB with writes of 
50MB between black lines.

It looks like past 80% there simply isn't enough continuous disk space 
and writes are becoming more and more random. For some unknown to me 
reason there is also a lot more flushing which certainly doesn't help 
for performance. There is also this odd hole left untouched by any 
write, reserved space of some sort?




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51B8B5DC.2010703>