Date: Sat, 17 Jun 2017 22:52:41 +0200 From: =?UTF-8?Q?=c5=81ukasz_W=c4=85sikowski?= <lukasz@wasikowski.net> To: Stephen McKay <mckay@FreeBSD.org> Cc: freebsd-fs@freebsd.org Subject: Re: Problem with zpool remove of log device Message-ID: <31ae8d40-9c28-a14c-2b7c-b62a6125df04@wasikowski.net> In-Reply-To: <caa9ae$1dr696j@ipmail07.adl2.internode.on.net> References: <9188a169-cd81-f64d-6b9e-0e3c6b4af1bb@wasikowski.net> <0410af$1dldvp4@ipmail04.adl6.internode.on.net> <4df1ea6d-148e-f3ab-d204-9277c513a468@wasikowski.net> <0fc687$sij78n@ipmail05.adl6.internode.on.net> <cd18dc67-cbaf-1178-1515-abcaa7fb4ccc@wasikowski.net> <caa9ae$1dr696j@ipmail07.adl2.internode.on.net>
next in thread | previous in thread | raw e-mail | index | archive | help
W dniu 2017-06-15 o 13:54, Stephen McKay pisze: > On Friday, 9th June 2017, lukasz@wasikowski.net wrote: > >> W dniu 2017-06-09 o 05:18, Stephen McKay pisze: >> >>> while : >>> do >>> date > foo >>> fsync foo >>> done >>> >>> With this running, my system does 600 writes per second to the log >>> according to zpool iostat. That drops to zero once I kill the script. >> >> Zero, so no writes to log are performed during execution of this script. > > OK. I believe this means your log is in a "pending removal" state and > has not been finally removed because ZFS thinks there's still data > stored on it. I'm happy for true ZFS experts to confirm or deny this > theory. Anybody? > >> I applied this patch to 11.1-PRERELASE, nothing changed. Still zpool >> remove exits with errcode 0, but log device is still attached to pool. > > Thanks for trying this out, but I managed to leave out essential > information. > > In my rush to do things before going away (see above), I didn't read my > notes on this event. The patch has the safety feature of requiring the > log to be offline. This means you will have to break the mirror (by > detaching one disk from it), then offline the remaining disk, and finally, > trigger the hack by attempting to remove the remaining disk. > > When I was stuck in this situation, I had already reduced my log to > a single disk before discovering the accounting error problem, so I > don't know if you can activate the patch code without first breaking > the mirror. I don't think you can offline a mirror. I've not tried. > > I've now had time to review my notes (sorry I didn't do that first up). > My pool had a data mirror-0 (gpt/data0 and gpt/data1) and a log mirror-1 > (gpt/log0 and gpt/log1). The sequence I did, minus most of the failed > attempts at random stuff, status checks, and so forth, was: > > # zpool remove pool mirror-1 #Did nothing but should have worked. > # zpool detach pool gpt/log1 #Broke the log mirror. > # zpool remove pool gpt/log0 #Did nothing but should have worked. > # zpool offline pool gpt/log0 #Just fiddling to change the state. > # zpool remove pool gpt/log0 #Still nothing. > ... Discovered plausible hack. Built and booted hacked kernel. ... > # zpool remove pool gpt/log0 #Glorious success! > > So, my log was already down to one offline disk before I got hacky. > That's why I forgot to mention it. > > You could break your mirror or you could modify the hack to remove the > VDEV_STATE_OFFLINE check. If you have already saved all your important > data then either would be fine as an experiment. My tests was done after zpool remove pool mirror-1, zpool detach pool gpt/log1. I don't remember if I offlined gpt/log0. It's possible that I haven't done it. Bulit and booted hacked kernel, zpool remove pool gpt/log0 didn't work. Unfortunately, lease of this box ended 14.06, I zeroed drives on 13.06 :( Again, thank you for your help! -- best regards, Lukasz Wasikowski
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?31ae8d40-9c28-a14c-2b7c-b62a6125df04>