From owner-freebsd-geom@FreeBSD.ORG Fri Jul 14 15:04:01 2006 Return-Path: X-Original-To: freebsd-geom@freebsd.org Delivered-To: freebsd-geom@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A852416A4DD for ; Fri, 14 Jul 2006 15:04:01 +0000 (UTC) (envelope-from anderson@centtech.com) Received: from mh1.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 31F1643D49 for ; Fri, 14 Jul 2006 15:04:00 +0000 (GMT) (envelope-from anderson@centtech.com) Received: from [10.177.171.220] (neutrino.centtech.com [10.177.171.220]) by mh1.centtech.com (8.13.1/8.13.1) with ESMTP id k6EF3xKt039817; Fri, 14 Jul 2006 10:03:59 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <44B7B266.2020701@centtech.com> Date: Fri, 14 Jul 2006 10:04:06 -0500 From: Eric Anderson User-Agent: Thunderbird 1.5.0.4 (X11/20060612) MIME-Version: 1.0 To: "R. B. Riddick" References: <20060714121530.46028.qmail@web30312.mail.mud.yahoo.com> In-Reply-To: <20060714121530.46028.qmail@web30312.mail.mud.yahoo.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV 0.87.1/1599/Fri Jul 14 00:35:31 2006 on mh1.centtech.com X-Virus-Status: Clean Cc: Deomid Ryabkov , freebsd-geom@freebsd.org Subject: Re: gmirror State: STALE X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Jul 2006 15:04:01 -0000 On 07/14/06 07:15, R. B. Riddick wrote: > --- Deomid Ryabkov wrote: >> Pawel Jakub Dawidek wrote: >>> BTW. Are you sure you don't want to use autosynchronization? >>> >> [...] >> I think, that ultimately should be handled in the base system: >> do not let mirror sync and fsck collide, ever. >> > I experienced the same problem... Especially when there are two mirrors on the > same two disks... Somehow the head positioning times increase the overall > processing time much more than linear... > > I think it would be a good idea to change the rebuild-strategy (maybe something > that is slower; e. g. one rebuild-request per 20 other requests or when there > are no pending other requests) in case of high load (e. g. fsck), although that > would increase the risk of data loss (another strategy could be, that other > requests are delayed until 20 rebuild-requests have been completed). > > There is another problem in my setting: The rebuild of the other mirror devices > that use the same disk(s) should be delayed... Maybe it's because the stale mirror disks are also trying to do reads for the mirror provider? Maybe during mirror synch the out-of-date geom should not participate in any reads? (Maybe that is how it is already - I didn't look at the code) Maybe also mirror rebuilds should happen 'behind' the real writes - so when a block is written to the mirror, the primary disk gets the data written in real-time and does not block waiting for all the other mirrored geoms to finish writing also. That way, if you have a slightly slower disk in a mirror, the slower disk doesn't penalize the faster one. The cost of this is of course there would be times during heavy IO that the mirror isn't *exactly* in sync. Sounds like a good sysctl tunable? Maybe a maximum delay time setting is the right choice, similar to what gjournal has. Maybe the same could be achieved by setting: kern.geom.mirror.syncs_per_sec: 1000 kern.geom.mirror.reqs_per_sync: 5 to something lower, like: kern.geom.mirror.syncs_per_sec: 100 kern.geom.mirror.reqs_per_sync: 2 Or higher - I'm not sure how they would impact performance while an fsck is running. Sounds like using your geom_cache on top of the mirror would help this. :) Eric -- ------------------------------------------------------------------------ Eric Anderson Sr. Systems Administrator Centaur Technology Anything that works is better than anything that doesn't. ------------------------------------------------------------------------