From owner-freebsd-geom@FreeBSD.ORG Wed Oct 1 20:31:52 2008 Return-Path: Delivered-To: freebsd-geom@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D73DC1065693 for ; Wed, 1 Oct 2008 20:31:52 +0000 (UTC) (envelope-from crahman@gmail.com) Received: from yw-out-2324.google.com (yw-out-2324.google.com [74.125.46.31]) by mx1.freebsd.org (Postfix) with ESMTP id 94C9F8FC16 for ; Wed, 1 Oct 2008 20:31:52 +0000 (UTC) (envelope-from crahman@gmail.com) Received: by yw-out-2324.google.com with SMTP id 9so137343ywe.13 for ; Wed, 01 Oct 2008 13:31:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type:content-transfer-encoding :content-disposition; bh=UTUEieBXWOm4REWnVmXS/XQ7FRC72I9eX6K0gGb3P4w=; b=eG6ykDQcwPFhUuRJQWKkPmxj2tEbro2LUqZtgRyUyUO2Ss35DTy5ja1waZQ3VAmDJl RH+88PzNVg4ftLku1ug4hThzJ9E1A+l33Y6A3K3kTWfSzXZPBXMkvFHOCL6UHLIoXUGF C8AVQuIxKEoH2Fq9ZicFEDlvR7wkTJHDwO/Sg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type :content-transfer-encoding:content-disposition; b=rMuepeKQW0kRIsSS2gLIXwW7qldCzrmXJ32dcpg6sFRjU6AqO43AXvkHk7jpfiBLvL mAzC4Pw/B/c1ZZ0oQt67T/TFMFtfiR1aMh1NqNDyI9YR0Mqzzd41m9bV4npFaCFSuwyi n2BHTDCxrFvrMRt7b54CzdnWjn0iioIT+rbEs= Received: by 10.65.212.18 with SMTP id o18mr14897583qbq.53.1222893110894; Wed, 01 Oct 2008 13:31:50 -0700 (PDT) Received: by 10.65.176.2 with HTTP; Wed, 1 Oct 2008 13:31:50 -0700 (PDT) Message-ID: <9e77bdb50810011331y7216eac3yf85907f96f5e8370@mail.gmail.com> Date: Wed, 1 Oct 2008 14:31:50 -0600 From: "Cyrus Rahman" To: freebsd-geom@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: Experiences with Gpart X-BeenThere: freebsd-geom@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: GEOM-specific discussions and implementations List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Oct 2008 20:31:52 -0000 I recently had occasion to set up a system that needed to contain numerous partitions, and I thought it would be a good occasion to try gpart to divide the disk. By and large it went very well, with the exception of bug in 'gpart bootcode' which made it necessary to manually dd in the bootstrap. I did discover a few oddities that I will mention here. My intent was to mirror the disks with gmirror. It didn't make sense to individually mirror each gpart partition and have some 30 simultaneous restores going should the mirror ever break. My first idea was to mirror the entire disks (ad4 and ad6) with gmirror and put gpart over the mirror. This sort of works, except that gpart itself if it finds itself at the beginning of the disk even when it doesn't properly taste itself on the last block of the disk. As a result, the same gpart partitions get attached to ad4 and ad6, as well as to the mirror. The gpart geoms attached directly to the disks complain about the last sector being incorrect (the backup table) and request that you fix it (which would not likely go well). But in addition to that, any further geoms contained within the mirror get attached to the outer gpart geoms on ad4 and ad6 and not the mirror. They don't work properly, because the mirror is already attached to ad4 and ad6, so you end up with a non-functional setup. In experimenting, I also discovered that if you directly nest gpart tables, things work fine until you reboot. Then the internal partitions vanish because gpart doesn't attach to them. Fortunately, if you put the inner gpart inside a gmirror inside the outer gpart, things do work fine. This I did for two reasons: First, to avoid the aforementioned trouble with an outermost gmirror containing a gpart geom, and second, so that I could better define how the disk was to be mirrored. I was able to put a number of smaller filesystems inside a single mirrored gpart partition, and then most of the really big ones inside a second mirrored gpart partition. This second partition used the -F option to gmirror so that it would not need to be re-synchonized in a crash. The filesystems within it are gjournal'd to make this possible. Booting was yet another issue, since the loader objects to nested gpart partitions. I handled this by simply making a third partition on the outermost gpt that contained the root. Since this is likely confusing, here's a diagram that will help if you use a fixed font: ad4 ad6 ------ ------ gpart 1 (ad4p1) gpart 1 (ad6p1) freebsd-boot freebsd-boot gpart 2 (ad4p2) gpart 2 (ad6p2) gmirror (gm0 = ad4p2 + ad6p2) root (gm0.p1) gpart 3 (ad4p3) gpart3 (ad6p3) gmirror (gm1 = ad4p3 + ad6p3) gpart 1 (gm1p1) gpart 2 (gm1p2 ... gpart n (gm1pn) gpart 3 (ad4p4) gpart 3 (ad6p4) gmirror (gm2 = ad4p4 + ad6p4) (mirrored with -F) gpart 1 (gm2p1) gjournal (gm2p1.journal) gpart 2 (gm2p2) gjournal (gm2p2.journal) ... Despite the intent of gpt's being to make such nesting unnecessary, as a means of defining the structure of gmirrors, which take up the entire extent of whatever encloses them, the nesting was very helpful.