Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 16 Jan 2014 13:58:17 +0200
From:      Volodymyr Kostyrko <c.kworr@gmail.com>
To:        Ulysse 31 <ulysse31@gmail.com>, Freddie Cash <fjwcash@gmail.com>
Cc:        FreeBSD Filesystems <freebsd-fs@freebsd.org>
Subject:   Re: zpool import taking weeks ...
Message-ID:  <52D7C959.9080305@gmail.com>
In-Reply-To: <CAFSDvD25ceAZT_WBj9yFfh13KrVeqVtcNHWB85B3rNzk1b_Ang@mail.gmail.com>
References:  <CAFSDvD2hLNN_CupeVKu02bLLUS1EFuTdZZw5Z9xoiNFZ0p4_fg@mail.gmail.com> <CAOjFWZ5dgw0nZzWFhWOCS-HSseBC=uavc=aQM%2Byht-R5a4Xfow@mail.gmail.com> <CAB7-odmXb%2B0KvEh6vDPstt89-O=ccL81MAtX2hRLWk%2BRcVZzUA@mail.gmail.com> <CAOjFWZ6r2LYWAw_eB4f9AEoy_w313rRy44t9uBdNMK2MJTmQwg@mail.gmail.com> <CAFSDvD25ceAZT_WBj9yFfh13KrVeqVtcNHWB85B3rNzk1b_Ang@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
15.01.2014 10:29, Ulysse 31 wrote:
> Thanks for the answer.
>
> I was beginning to ... well ... have a little start of panic ^^'
> The import still running but no real activity on disks : sometimes,
> maybe something like each hour, there is some really brief led
> activity on the hdd, but nothing more.
> one thing that would be great is to have an overall percentage or just
> counting of what is done or what is left ... for now i don't even know
> if it's gonna end on the next 5 mins or on the next 5 weeks :s
> So now since i have no real activity on the actual zpool import (the
> last import started a week ago, it do not have go out of swap since i
> added lot of swap (260G HDD swap)), should i just let it run, or make
> a reboot and start over ?
> Also once imported, and since i don't care about the data actually on
> the incriminated dataset (the one that crashed with the
> recv/rollback), I was planning to simply delete that dataset and make
> a new full sync, what do you think about it ?
> Thanks again for your help.

You odds are grim. I had the same problem with crashed ZFS when dropping 
some snapshots (deduplicated ones). The situation looks like:

1. Boot. Import start.
2. Memory depletes.
3. When memory is over the process is stuck or machine crashes. In your 
case no crash means you are running a lil bit newer code.

All of this comes in something like 15 minutes. My machine was at 4G of 
memory back then. I added 4G more and process successfully completed in 
near 20 minutes eating ~6G of mem. The culprit was deleting deduplicated 
snapshot worth 14G of disk space.

I personally think that previous versions of ZFS tend to interpret all 
snapshot operations as atomic ones meaning all changes to fs should 
happen simultaneously thus pushing to memory:

- all old dedup data that still can be accessed,
- all new dedup data that should be active after the change,
- all metadata of filesystem in question.

I'm not a pro in this and haven't retested this situation after the 
recent changes with snapshots (like background deleting). But this was a 
final grave reason for me to stop thinking about dedup. Or at least in 
ZFS as I still like the way HAMMER does it.

-- 
Sphinx of black quartz, judge my vow.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?52D7C959.9080305>