Date: Mon, 3 Oct 2016 13:12:41 -0400 From: Evan Susarret <evansus@gmail.com> To: ohartman@zedat.fu-berlin.de Cc: freebsd-current@freebsd.org Subject: Re: ZFS - Abyssal slow on copying Message-ID: <93D024AE-408C-4C1D-882D-9C526F2BB276@gmail.com>
next in thread | raw e-mail | index | archive | help
In case you are still trying to get this resolved: Since you have enabled deduplication in the past, unfortunately the deduplic= ation tables must be loaded into memory for any and all writes. ZFS must det= ermine if all writes match a previous block, to 1) increment the dedup count= for matches, or 2) decrement the dedup count for a no-longer needed block. This often means that the table must be loaded from disk repeatedly, especia= lly if the table is large enough that it is purged from memory often. It is e= ven possible to get your pool into a state where it cannot be imported due t= o the amount of memory required. =46rom the disk activity and slowness you described, it sounds like the dedu= p table (DDT) is being loaded, dumped, and loaded again repeatedly. I don't know of the top of my head if you can use the 'zdb' command to exami= ne the current DDT table, but there is a way to simulate the effect of enabl= ing dedup on a given dataset using zdb in order to estimate the size of the D= DT table that would result. The only resolution, as Allan stated, is to zfs send and zfs recv the datase= t into a new copy that does not have dedup enabled, or recreate it from scra= tch. Depending if you enabled dedup on single dataset, multiple datasets, or= the whole pool, it may be easiest to recreate the whole pool. This is a sid= e-effect of dedup that some have referred to as 'toxic', and some googling w= ill show that it is a common issue encountered when experimenting with dedup= (on all platforms). Some guides will recommend only enabling dedup if you know that your data wi= ll have many duplicated blocks, and also that the tables will easily fit int= o memory. There are also guides that can help to estimate the amount of ram r= equired to hold a DDT based on the blocksize and number of blocks in the dat= aset. This can be complicated by many factors, so it is usually best to avoi= d turning dedup on unless you are sure that it will help rather than hinder u= sage. Hope this helps! Evan=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?93D024AE-408C-4C1D-882D-9C526F2BB276>