From owner-freebsd-geom@FreeBSD.ORG Tue Jul 6 19:49:00 2004 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D44916A4CE; Tue, 6 Jul 2004 19:49:00 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 570EB43D39; Tue, 6 Jul 2004 19:48:59 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost.freebsd.dk [127.0.0.1]) by critter.freebsd.dk (8.12.11/8.12.11) with ESMTP id i66JmvZC020838; Tue, 6 Jul 2004 21:48:57 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Pawel Jakub Dawidek From: "Poul-Henning Kamp" In-Reply-To: Your message of "Tue, 06 Jul 2004 20:37:27 +0200." <20040706183727.GN12007@darkness.comp.waw.pl> Date: Tue, 06 Jul 2004 21:48:57 +0200 Message-ID: <20837.1089143337@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: geom@FreeBSD.org cc: Scott Long Subject: Re: Design and implementation of GEOM_MIRROR:) X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jul 2004 19:49:00 -0000 In message <20040706183727.GN12007@darkness.comp.waw.pl>, Pawel Jakub Dawidek w rites: > >--GiGP7OV2v0ywwcSZ >Content-Type: text/plain; charset=iso-8859-2 >Content-Disposition: inline >Content-Transfer-Encoding: quoted-printable > >On Tue, Jul 06, 2004 at 10:46:10AM -0600, Scott Long wrote: >+> This sounds reasonable. Don't forget that you might want to deal with >+> both global spares and spares that are dedicated to a certain array. > >Ok, I implement GEOM_SPARE class already. It works as follow: OK, I made a cup of tea and thought about what I have learned about hot spares over the years: What you propose here would have had me cursing your name in many late night and morning hours over the years. It is not the way we will do it. Lets get some thoughts down about how we want hot sparing to work before we rush in with an implementation. Here is my analysis: First, there are two kinds of hot sparing: dedicated and pooled. Dedicated hot sparing is when an array has an extra disk member which is dedicated as a hot spare and nobody else can use it. This is what you use to have when you have very strict rules for which bits are the most important etc. Any policy can be implemented with dedicated hot spares, but at a cost in disk-space. It is straightforward to implement and I think that it would be fair to require all of our array classes to support it. Pooled hot spares on the other hand are tricky to get right: 1. I want us to have a prioritization of the arrays, so that the administrator gets to control what gets hot spared in a major loosage. This also means that we should not assign hot spares right away, but (at least) have a (configurable) waiting period during which higher-priority arrays get to pull rank. Ideally higher ranks should be able to reposses already assigned hot spares, as long as this doesn't make the previous owner go critical. 2. I want the arrays to have a say in which hot spare they get. Selecting the wrong hot spare (automagically) can be much worse than running without a hot spare. Putting two strips on the same physical disk is a catastrophy for instance. 3. We need some way to partition hot spares. If I dedicate an entire 72G disk as hot spare and an array need only 10G of hot spare, the remaining 62G should be available for other hot sparing uses. Considering the complexity of this, I am pretty sure that pooled spares do not belong in the kernel. It would be much better handled by a userland process where the user could configure (using a scripting language ?) how hot sparing decisions are to be made. To do it in userland, would require standardization of the g_ctl() messages to the arrays and a way for the arrays to signal to the userland process that they are in distress. The former is merely a matter of discipline, the latter we can use the XML output for. Poul-Henning -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.