Date: Thu, 5 Apr 2018 18:11:25 +0100 From: Stilez Stilezy <stilezy@gmail.com> To: Andriy Gapon <avg@freebsd.org> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS dedup write pathway - possible inefficiency or..? Message-ID: <CAFwhr75nEEhFB=jUJibZXYDWo=sSeGK=i-x7wZL4pB0q00_wJw@mail.gmail.com> In-Reply-To: <a9a355a7-234e-2adf-bf6b-0cc9ffabf9d6@FreeBSD.org> References: <CAFwhr76S-hqJya5P91HsJh94SnYR-scn%2BcmBEUFCDpQT_gD_OA@mail.gmail.com> <14c857cc-463f-a56e-bcf6-c0702da6d3bc@FreeBSD.org> <CAFwhr76%2B4G_ZVK1kHYvVegs1Oocdz3aPeqF3t%2BQtdsVmwtCmjg@mail.gmail.com> <a9a355a7-234e-2adf-bf6b-0cc9ffabf9d6@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks Andriy, That does suck, but it's how bugs are. I'm using 10G and dedup, so even with good hardware the test case is definitely piling on the incoming data and workload during writes, probably as fast as the OS can take it, or a little more. Without dedup it flies, though. Hard to disentangle what's dedup overhead and what's dedup write bug, though, or to get an idea which is responsible for how much of the problem. As I don't know Illumos/OpenZFS's track record with bugs, is this likely to get attention/resolved "at some time this year" or is it a "who knows, bugs take forever, could be still here in 5 years time"? Is it helpful if I nudge and offer a "causing a problem" note on their bug tracker? Perhaps it's worth it in case it gives extra data? What do you reckon? Last, if I have high data rates but want to minimise the dirty data issue, wcan you suggest broadly, how to customise the dirty data/caching sysctls/loaders, to try and mitigate the impact (get best possible handling without losing 99% of throughput or sky-high latency?). I understand from your reply that there's no recipes in debugging, but any suggestions at all which way to try for at least some mitigation, or which values might be worth experimenting with, to reduce the effect of the problem pathway? Stilez On 4 April 2018 at 18:04, Andriy Gapon <avg@freebsd.org> wrote: > On 02/04/2018 21:30, Stilez Stilezy wrote: > > Thanks on that tip, Andriy, > > > > I don't have the knowledge to tell if that's the first of the 2 issues > I'm > > seeing. It could be. What exactly is that bug describing and what > behaviour > > would it create? > > Sub-optimal performance for ZFS write throughput (and latency) when dedup > is > enabled and you are trying to write as fast as you can. > > > Assuming it's the same - > > > > 1) Are there any known workarounds or sysctls that help to reduce the > issue? > > I don't know of any. > > > 2) Are there any easy diagnostic tests I can easily do, to confirm if > this is > > the same behaviour as that bug? Nobody else uses the test system, so I > can > > change any sysctls to sane or extreme values for testing, or create > large txg's > > and spread out reads and writes to see what's happening and what > coincides with > > what. > > We used DTrace to observe internal ZFS behavior. > I do not have any simple recipes. > > > On 2 April 2018 at 18:47, Andriy Gapon <avg@freebsd.org > > <mailto:avg@freebsd.org>> wrote: > > > > On 02/04/2018 16:36, Stilez Stilezy wrote: > > > The first issue is specific to the > > > dedup write pathway. I've tested locally to a point where it > seems it's > > > not due to inadequate hardware and it's very consistent and > specific, even > > > on idle conditions/minimal load. I'm wondering whether there's a > code > > > bottleneck specifically affecting just the dedup write pathway. > > > > I think that this might be https://www.illumos.org/issues/8353 > > <https://www.illumos.org/issues/8353>; > > > > -- > > Andriy Gapon > > > > > > > -- > Andriy Gapon >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFwhr75nEEhFB=jUJibZXYDWo=sSeGK=i-x7wZL4pB0q00_wJw>