From owner-freebsd-geom@FreeBSD.ORG Wed May 5 22:05:58 2010 Return-Path: Delivered-To: freebsd-geom@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 3B936106570E for ; Wed, 5 May 2010 22:05:58 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id BFE4B8FC21 for ; Wed, 5 May 2010 22:05:57 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 61D0219E023 for ; Thu, 6 May 2010 00:05:56 +0200 (CEST) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id CAEF319E030 for ; Thu, 6 May 2010 00:05:53 +0200 (CEST) Message-ID: <4BE1EBC1.5040307@quip.cz> Date: Thu, 06 May 2010 00:05:53 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.9) Gecko/20100317 SeaMonkey/2.0.4 MIME-Version: 1.0 To: freebsd-geom@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: How can I disable automatic gjournal activation on the given device after hot-swap? X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2010 22:05:58 -0000 Hi, I have server with gjournal on top of the gmirror. Today I pulled out one failed drive and replace it with another used drive from spare server. The problem is, that this drive was in another gmirror with gjournal, so right after the detection of ad4 on the ATA channel, the gjournal detected meta data on it and prevents me from adding this drive in to gmirror gm0. Steps done: root@roxy ~/# gmirror status Name Status Components mirror/gm0 DEGRADED ad6 root@roxy ~/# gmirror forget -v gm0 Done. root@roxy ~/# gmirror insert -v gm0 ad4 gmirror: Cannot access provider ad4. root@roxy ~/# ls -al /dev/ad4* crw-r----- 1 root operator 0, 84 Dec 7 02:22 /dev/ad4 crw-r----- 1 root operator 0, 85 Dec 7 02:22 /dev/ad4s1 crw-r----- 1 root operator 0, 129 Dec 7 02:22 /dev/ad4s1a crw-r----- 1 root operator 0, 130 Dec 7 02:22 /dev/ad4s1b crw-r----- 1 root operator 0, 131 Dec 7 02:22 /dev/ad4s1c crw-r----- 1 root operator 0, 132 Dec 7 02:22 /dev/ad4s1d crw-r----- 1 root operator 0, 133 Dec 7 02:22 /dev/ad4s1e crw-r----- 1 root operator 0, 134 Dec 7 02:22 /dev/ad4s1f crw-r----- 1 root operator 0, 86 Dec 7 02:22 /dev/ad4s2 crw-r----- 1 root operator 0, 135 Dec 7 02:22 /dev/ad4s2c crw-r----- 1 root operator 0, 136 Dec 7 02:22 /dev/ad4s2d crw-r----- 1 root operator 0, 137 Dec 7 02:22 /dev/ad4s2e crw-r----- 1 root operator 0, 138 Dec 7 02:22 /dev/ad4s2f crw-r----- 1 root operator 0, 145 Dec 7 02:22 /dev/ad4s2f.journal root@roxy ~/# dd if=/dev/zero of=/dev/ad4s2 bs=512 count=100 dd: /dev/ad4s2: Operation not permitted root@roxy ~/# sysctl kern.geom.debugflags=16 kern.geom.debugflags: 0 -> 16 root@roxy ~/# dd if=/dev/zero of=/dev/ad4s2 bs=512 count=100 dd: /dev/ad4s2: Operation not permitted root@roxy ~/# gjournal list Geom name: gjournal 1576372395 ID: 1576372395 Providers: 1. Name: mirror/gm0s2d.journal Mediasize: 397253253632 (370G) Sectorsize: 512 Mode: r1w1e1 Consumers: 1. Name: mirror/gm0s2d Mediasize: 398326995968 (371G) Sectorsize: 512 Mode: r1w1e1 Jend: 398326995456 Jstart: 397253253632 Role: Data,Journal Geom name: gjournal 2419990975 ID: 2419990975 Providers: 1. Name: ad4s2f.journal Mediasize: 461697294848 (430G) Sectorsize: 512 Mode: r0w0e0 Consumers: 1. Name: ad4s2e Mediasize: 2147483648 (2.0G) Sectorsize: 512 Mode: r1w1e1 Jend: 2147483136 Jstart: 0 Role: Journal 2. Name: ad4s2f Mediasize: 461697295360 (430G) Sectorsize: 512 Mode: r1w1e1 Role: Data root@roxy ~/# gjournal stop -v ad4s2f.journal Done. root@roxy ~/# lsq /dev/ad4* crw-r----- 1 root operator 0, 84 Dec 7 02:22 /dev/ad4 crw-r----- 1 root operator 0, 85 Dec 7 02:22 /dev/ad4s1 crw-r----- 1 root operator 0, 129 Dec 7 02:22 /dev/ad4s1a crw-r----- 1 root operator 0, 130 Dec 7 02:22 /dev/ad4s1b crw-r----- 1 root operator 0, 131 Dec 7 02:22 /dev/ad4s1c crw-r----- 1 root operator 0, 132 Dec 7 02:22 /dev/ad4s1d crw-r----- 1 root operator 0, 133 Dec 7 02:22 /dev/ad4s1e crw-r----- 1 root operator 0, 134 Dec 7 02:22 /dev/ad4s1f crw-r----- 1 root operator 0, 86 Dec 7 02:22 /dev/ad4s2 crw-r----- 1 root operator 0, 135 Dec 7 02:22 /dev/ad4s2c crw-r----- 1 root operator 0, 136 Dec 7 02:22 /dev/ad4s2d crw-r----- 1 root operator 0, 137 Dec 7 02:22 /dev/ad4s2e crw-r----- 1 root operator 0, 138 Dec 7 02:22 /dev/ad4s2f root@roxy ~/# gjournal clear -v /dev/ad4s2f Cannot clear metadata on /dev/ad4s2f: Operation not permitted. gjournal: Not fully done. root@roxy ~/# gjournal clear -v /dev/ad4s2e Metadata cleared on /dev/ad4s2e. Done. root@roxy ~/# dd if=/dev/zero of=/dev/ad4s2 bs=512 count=100 dd: /dev/ad4s2: Operation not permitted So the drive ad4 is still not writable by any command. (dd, bsdlabel, gmirror...) How can I fix it without unloading the gjournal kernel module which is needed by running system? (I can't remove the drive, I have just ssh access to the server at this time) Miroslav Lachman