Date: Sat, 21 Jan 2017 01:03:51 -0800 From: Jordan Hubbard <jordanhubbard@icloud.com> To: Alex Povolotsky <tarkhil@webmail.sub.ru> Cc: freebsd-fs@freebsd.org Subject: Re: Poor ZFS performance Message-ID: <AA732462-2738-40DD-A689-B6C36B333CB3@icloud.com> In-Reply-To: <595d8117-e2f2-fa4f-a45e-3a9fb93d0687@webmail.sub.ru> References: <595d8117-e2f2-fa4f-a45e-3a9fb93d0687@webmail.sub.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Jan 21, 2017, at 12:49 AM, Alex Povolotsky <tarkhil@webmail.sub.ru = <mailto:tarkhil@webmail.sub.ru>> wrote: >=20 > I'm writing lots of (tens of millions) relatively small files, hashing = them out in three-level directory, 100 entries per level. You=E2=80=99re never going to get good performance doing that. The = constraints placed on ZFS=E2=80=99 design for directories (UNIX API, = POSIX compliance, etc) and small file representation formats means = it=E2=80=99ll never be a =E2=80=9Cdatabase=E2=80=9D - the filesystem's = design, to say nothing of UNIX=E2=80=99s directory iteration / lookup = APIs, just aren't optimized for millions of small files because it was = never the intention that any UNIX filesystem be a low-cost KVS or = database analog. Things will get quickly pathological from a = performance perspective and fixing the pathologies would require such a = significant redesign of a number of different pieces of the puzzle here = that it=E2=80=99s never likely to happen. Your application would be far better served by using an actual database. = I=E2=80=99m not just suggesting this as a hypothetical, either. I=E2=80=99= ve dealt with several folks who went down this path, storing millions = aornd even billions of small files into ZFS, and the results have never = been pretty, nor have there been any easy options or =E2=80=9Csimple = tunables=E2=80=9D that were going to make those scenarios significantly = prettier. The advice was the same: This needs to be a database, and = there are plenty of those to choose from with all kinds of performance / = consistency / redundancy characteristics to pick and choose between. - Jordan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AA732462-2738-40DD-A689-B6C36B333CB3>