From owner-freebsd-fs@freebsd.org Sat Jun 17 22:32:29 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B55BEC0AC6D for ; Sat, 17 Jun 2017 22:32:29 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) Received: from EUR03-DB5-obe.outbound.protection.outlook.com (mail-oln040092071057.outbound.protection.outlook.com [40.92.71.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41B2A79F98 for ; Sat, 17 Jun 2017 22:32:28 +0000 (UTC) (envelope-from kisscoolandthegangbang@hotmail.fr) Received: from DB5EUR03FT014.eop-EUR03.prod.protection.outlook.com (10.152.20.56) by DB5EUR03HT028.eop-EUR03.prod.protection.outlook.com (10.152.20.242) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1157.12; Sat, 17 Jun 2017 22:32:26 +0000 Received: from AM4PR05MB1714.eurprd05.prod.outlook.com (10.152.20.57) by DB5EUR03FT014.mail.protection.outlook.com (10.152.20.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1157.12 via Frontend Transport; Sat, 17 Jun 2017 22:32:26 +0000 Received: from AM4PR05MB1714.eurprd05.prod.outlook.com ([fe80::1926:1bbf:ff94:fe53]) by AM4PR05MB1714.eurprd05.prod.outlook.com ([fe80::1926:1bbf:ff94:fe53%15]) with mapi id 15.01.1157.022; Sat, 17 Jun 2017 22:32:26 +0000 From: kc atgb To: "freebsd-fs@freebsd.org" Subject: Re: Problem with zpool remove of log device Thread-Topic: Problem with zpool remove of log device Thread-Index: AQHS48l6WE5shmvvSUeFr52IoccjV6Ipq40A Date: Sat, 17 Jun 2017 22:32:26 +0000 Message-ID: References: <9188a169-cd81-f64d-6b9e-0e3c6b4af1bb@wasikowski.net> <0410af$1dldvp4@ipmail04.adl6.internode.on.net> In-Reply-To: Accept-Language: fr-FR, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=hotmail.fr; x-incomingtopheadermarker: OriginalChecksum:E1081F7F99241A16B64A7CFB1402AA9DA7E761A49B0A3CAC138AD389EA4F0E30; UpperCasedChecksum:220293D3CD204853E32C3430EC831864A08EA088C53A8C834F504C414746A798; SizeAsReceived:7415; Count:45 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [HT0buQjcUtkQrCTyLHgpqY4Q/3+777+I] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; DB5EUR03HT028; 7:cjdWjA7YAKf5f3wP7TZwxORy0Sn8MKlucaxm1XDfmoKibhDpTXjknIwc8I0DZGlwVID32Ensl171C7XCcPqC2LVvWgmok1MYDxYLX7GC0IGn+wgRTQVytNWPVbSratvdvYktd01PCx45QNJatNytyN+U1webBKufku8SDxBWY6k35K2xn8MnhTZ89UQtUH3+XXFRSvpadHyfcw0LBJhpxwCfvuCVUlnzy5j6rzriq+WdDl3xHiU+JQ2VcZHGswxjUYKQ5Fu47tVMYmRqciA2G5c8rQX5+UbGJOYQhQ28xTEcRcoJMn6ESW36fc6HFGFsvShBynIH33HmO1P94fXb71QA2otQMgabxsfMP3f4vjWfWlDbW92WGcREA4wKj8TMeLtx+4DRxP+CREtpYfPBriNbGa0Ws0HlAyRJVq7IwKfF4fgZAUMR2fjI6vLpxwgVK3C2cs1Hn4z2BfkQy2BXRvQKCjCB1DXMbzb29bLkqXtTdH6iUpC4JvgvWE2gRBSM095NmIej/fWb23BumprXmbZ0a6ucMeiUWFE7qVX+2qMlG0c0/v6l5jnzpZRft+83o5itKBPTrIZ1nVo3TW1ZwEcoOm/HYh2OdzVGtpuTvdkYy3b+T2VTBm6rN+krdQRhAEuZbbR2xremnz2b3a44de5oc7NribVbiajFV4EG63QuTfUiw+r48XbeRUMlJbK6ciBHdhfCaaf8YIZhYJr3D6mheQiIXhy8W73G3wH2U28vrO5Qfkcl3pbjIhCZHpVmwVll0xzfLDuh2elEaiaa1A== x-incomingheadercount: 45 x-eopattributedmessage: 0 x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:DB5EUR03HT028; H:AM4PR05MB1714.eurprd05.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-traffictypediagnostic: DB5EUR03HT028: x-ms-office365-filtering-correlation-id: be82fb74-2291-4dd3-529a-08d4b5d0ba28 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322274)(1603101448)(1601125374)(1701031045); SRVR:DB5EUR03HT028; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031); SRVR:DB5EUR03HT028; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:DB5EUR03HT028; x-forefront-prvs: 034119E4F6 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-ID: <765B861DD714984F885760EE371ED6E4@eurprd05.prod.outlook.com> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: outlook.com X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jun 2017 22:32:26.0550 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB5EUR03HT028 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Jun 2017 22:32:29 -0000 Le lun. 12 juin 2017 22:16:02 CEST kc atgb a =E9crit: >=20 >=20 > Le mer. 07 juin 2017 08:21:09 CEST > Stephen McKay a =E9crit: >=20 > > On Friday, 26th May 2017, lukasz@wasikowski.net wrote: > >=20 > > >I cant remove log device from pool - operation ends ok, but log device > > >is still in the pool (bug?). > > > > > ># uname -a > > >FreeBSD xxx.yyy.com 11.0-STABLE FreeBSD 11.0-STABLE #0 r316543: Thu Ap= r > > >6 08:22:43 CEST 2017 root@xxx.yyy.com:/usr/obj/usr/src/sys/YYY am= d64 > > > > > ># zpool status tank > > >[..snip..] > > > > > > NAME STATE READ WRITE CKSUM > > > tank ONLINE 0 0 0 > > > mirror-0 ONLINE 0 0 0 > > > ada2p3 ONLINE 0 0 0 > > > ada3p3 ONLINE 0 0 0 > > > logs > > > mirror-1 ONLINE 0 0 0 > > > gpt/tankssdzil0 ONLINE 0 0 0 block size: 5= 12B configured, 4096B native > > > gpt/tankssdzil1 ONLINE 0 0 0 block size: 5= 12B configured, 4096B native > >=20 > > >When I try to remove log device operation ends without errors: > > > > > ># zpool remove tank mirror-1; echo $? > > >0 > > > > > >But the log device is still there: > > >[..snip..] > > >I'd like to remove it - how should I proceed? > >=20 > > Does your system still write to the log? Use "zfs iostat -v 1" to > > check. I think it is probably no longer be in use and only the final > > disconnection failed. > >=20 > > What does "zpool list -v" tell you? If you have a non-zero ALLOC > > column for your log mirror and the log is no longer being used then > > you may have hit an accounting bug in zfs that the zfsonlinux people > > ran into a while ago. > >=20 > > I had this problem when I tried to remove a log mirror from a pool > > I have been using for years. I solved it by tweaking the zfsonlinux > > hack a bit and slotting it into 9.3. > >=20 > > If you apply this hack be sure to have a full backup first! When I > > used it, I did my backup and a scrub then booted the hacked kernel, > > issued the zfs remove command (which succeeded), reverted the kernel, > > then scrubbed again. All went well. > >=20 > > Good luck! > >=20 > > Here's the patch against 9.3 (should be close even for 11.0): > >=20 > > Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision 3098= 60) > > +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (working copy) > > @@ -5446,6 +5446,18 @@ > > ASSERT(vd =3D=3D vd->vdev_top); > > =20 > > /* > > + * slog stuck hack - barnes333@gmail.com > > + * https://github.com/zfsonlinux/zfs/issues/1422 > > + */ > > + if (vd->vdev_islog && vd->vdev_removing > > + && vd->vdev_state =3D=3D VDEV_STATE_OFFLINE > > + && vd->vdev_stat.vs_alloc > 0) { > > + printf("ZFS: slog stuck hack - clearing vs_alloc: %llu\n", > > + (unsigned long long)vd->vdev_stat.vs_alloc); > > + vd->vdev_stat.vs_alloc =3D 0; > > + } > > + > > + /* > > * Only remove any devices which are empty. > > */ > > if (vd->vdev_stat.vs_alloc !=3D 0) > >=20 > > Cheers, > >=20 > > Stephen. > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >=20 >=20 > I have this case once again. The first time it was one month ago.=20 > I had to backup ma datas and destroy and recreate the pool to remove the = "faulted" log device.=20 >=20 > I'll try your patch. I hope I'll be more lucky than OP. I have to backup = first again.=20 >=20 > In my opinion, maybe this problem is related to a certain type of data or= activity. I have my pool for few years now and added a log only some month= s ago.=20 > It is a little bit strange that it happened to me twice in so little laps= of time and others are not affected.=20 >=20 > K. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 I have succesfully applied the patch and build the kernel. The removal of l= og=20 device has worked too. It was in offline state, then I had to remove the dr= ive so it was marked as unavailable before removal. =20 My FreeBSD version : FreeBSD my.host.name 9.3-STABLE FreeBSD 9.3-STABLE #0 r315141: Sun Mar 12=20 16:00:24 CET 2017 root@my.host.name:/usr/obj/usr/src/sys/GENERIC amd64 I'm still curious about why is it happening. Any idea ?=20 K.