From owner-freebsd-fs@FreeBSD.ORG Mon Nov 16 13:35:37 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0D23F106568F for ; Mon, 16 Nov 2009 13:35:37 +0000 (UTC) (envelope-from aaron@goflexitllc.com) Received: from mail.goflexitllc.com (mail.goflexitllc.com [70.38.81.12]) by mx1.freebsd.org (Postfix) with ESMTP id AB9318FC0A for ; Mon, 16 Nov 2009 13:35:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=goflexitllc.com; h=message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type; s=zeta; bh=y/mqTuBiO8eqtrd6zUlI2TkWtk M=; b=Ru+YOWQBF1Jh5T2xXh4vnT49waAkgTsqSFWETDpXu20B4rqiB6IAot5/T9 JlD0GfQqCAi6zmIagxCiRzz8uqB/kMTovevPvVG5Ey1s5RCv4SAh74aW7c4beYd4 ORqMBE DomainKey-Signature: a=rsa-sha1; c=nofws; d=goflexitllc.com; h= message-id:date:from:mime-version:to:cc:subject:references :in-reply-to:content-type; q=dns; s=zeta; b=PquLaIWI3gjeIj7g7Ik0 brPun/8PzVKIFTOVVgco+Kj2dWjU47wR+OtuOE2Hf+cuY9+vna3Fpb3eQN8s4xl0 K8IyvZ53vEZ8emaFGzSrfRACExdu9UsQ2RE56T4djJ9G Received: (qmail 96496 invoked by uid 89); 16 Nov 2009 13:35:33 -0000 Received: (simscan 1.4.1 ppid 96487 pid 96493 t 0.1964s) (scanners: regex: 1.4.1 attach: 1.4.1 clamav: 0.95.2/m:51/d:10027); 16 Nov 0109 13:35:33 -0000 Received: from temp4.wavelinx.net (HELO ?172.16.1.128?) (aaron@goflexitllc.com@69.27.151.4) by mail.goflexitllc.com with ESMTPA; 16 Nov 2009 13:35:33 -0000 Message-ID: <4B015520.7080109@goflexitllc.com> Date: Mon, 16 Nov 2009 07:35:28 -0600 From: Aaron Hurt User-Agent: Thunderbird 2.0.0.23 (X11/20091001) MIME-Version: 1.0 To: Miroslav Lachman <000.fbsd@quip.cz> References: <1A8F306A-8749-471B-94EA-FC8435A30C34@yellowspace.net> <4AFF15AE.4070902@quip.cz> In-Reply-To: <4AFF15AE.4070902@quip.cz> Content-Type: multipart/mixed; boundary="------------010909020406010003050503" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: gmirroring slices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Nov 2009 13:35:37 -0000 This is a multi-part message in MIME format. --------------010909020406010003050503 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Miroslav Lachman wrote: > Lorenzo Perone wrote: >> >> Hello, >> >> I was wondering if anyone could give me an advice on how viable and >> reliable it is, to use gmirror on a slice of an MBR-style partitioned >> disk, and use the second slice(s) within a zpool. >> >> I remember a discussion here on where metadata is kept (always at the >> end of the disk as opposed to the end of the given consumer?), so I >> wasn't sure about how much of a good idea this might be. > > I think metadata is stored at the end of the provider (slice in this > case), but I am not a GEOM expert. > >> The reason I'd >> like to have it like this is, that I had mixed bad experiences in the >> effort of using ZFS as a boot and root volume, so I'd rather keep a >> traditional slice for booting/rooting, and a zpool for the production >> jails on that machine. >> >> The example would be >> >> provider: mirror/gm0 >> consumers: ad6s1 and ad8s1 >> >> zpool mirror made out of >> ad6s2 and ad8s2 > > I am running following setup for year without any configuration problems > > # gmirror status > Name Status Components > mirror/gms1 COMPLETE ad4s1 > ad6s1 > > # zpool status > pool: tank > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 0 > mirror ONLINE 0 0 0 > ad4s2 ONLINE 0 0 0 > ad6s2 ONLINE 0 0 0 > > The first slice is 20GB partitioned as usual: > # mount -t ufs > /dev/mirror/gms1a on / (ufs, local) > /dev/mirror/gms1e on /usr (ufs, local, soft-updates) > /dev/mirror/gms1d on /var (ufs, local, nosuid, soft-updates) > /dev/mirror/gms1f on /tmp (ufs, local, noexec, nosuid, soft-updates) > > The rest (450GB) is used in ZFS mirrored zpool for jails (each jail > has its own filesystem) > >> while experimenting, I got into the problem that gmirror label -v -b >> round-robin gm0 ad6s1 got a permission denied (even with sysctl >> kern.geom.debugflags=16/17). Any hints on what can cause this (I might >> have screwed up something with fdisk/bsdlabel, but after doublechecking >> I wonder what it could be..) > > I did it in non-standard way - converting already installed system on > one disk to mirrored. So when I was in system running off ad6 I > created two slices on ad4, setup gmirror gms1 from first slice of ad4, > create partitions, newfs, mount it and transfer files from running > system by dump & restore, edit fstab. Then I rebooted system from > gms1, destroy content of ad6, create slices on ad6 and insert first > slice in to gms1. > After this I had ad4s2 and ad6s2 ready for zpool. > All was done remotely through ssh. > > Miroslav Lachman > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > !DSPAM:2,4aff15c1775218542073880! > An example with gpart ... this is how I know have all of my production dedicated servers setup and running 8.0-RCx ... net1# gpart show => 34 312581741 ad6 GPT (149G) 34 128 1 freebsd-boot (64K) 162 8388608 2 freebsd-swap (4.0G) 8388770 10485760 3 freebsd-ufs (5.0G) 18874530 293707245 4 freebsd-zfs (140G) => 34 312581741 ad16 GPT (149G) 34 128 1 freebsd-boot (64K) 162 8388608 2 freebsd-swap (4.0G) 8388770 10485760 3 freebsd-ufs (5.0G) 18874530 293707245 4 freebsd-zfs (140G) net1# gmirror status Name Status Components mirror/boot COMPLETE ad6p1 ad16p1 mirror/swap COMPLETE ad6p2 ad16p2 mirror/root COMPLETE ad6p3 ad16p3 net1# zpool status pool: pool0 state: ONLINE scrub: scrub completed after 0h5m with 0 errors on Wed Oct 14 12:45:40 2009 config: NAME STATE READ WRITE CKSUM pool0 ONLINE 0 0 0 mirror ONLINE 0 0 0 ad6p4 ONLINE 0 0 0 ad16p4 ONLINE 0 0 0 errors: No known data errors net1# mount -t ufs /dev/mirror/root on / (ufs, local, soft-updates) net1# mount -t zfs pool0 on /pool0 (zfs, local) pool0/tmp on /tmp (zfs, local, nosuid) pool0/usr on /usr (zfs, local) pool0/usr/home on /usr/home (zfs, local) pool0/usr/hosting on /usr/hosting (zfs, local, noexec, nosuid) pool0/usr/ports on /usr/ports (zfs, local, nosuid) pool0/usr/ports/distfiles on /usr/ports/distfiles (zfs, local, noexec, nosuid) pool0/usr/ports/packages on /usr/ports/packages (zfs, local, noexec, nosuid) pool0/usr/src on /usr/src (zfs, local, noexec, nosuid) pool0/var on /var (zfs, local) pool0/var/crash on /var/crash (zfs, local, noexec, nosuid) pool0/var/db on /var/db (zfs, local, noexec, nosuid) pool0/var/db/pkg on /var/db/pkg (zfs, local, nosuid) pool0/var/empty on /var/empty (zfs, local, noexec, nosuid, read-only) pool0/var/log on /var/log (zfs, local, noexec, nosuid) pool0/var/mail on /var/mail (zfs, local, noexec, nosuid) pool0/var/qmail on /var/qmail (zfs, local) pool0/var/run on /var/run (zfs, local, noexec, nosuid) pool0/var/tmp on /var/tmp (zfs, local, nosuid) It runs great and I haven't experienced any issues related to sharing disks between ufs and zfs using gpt partitioning. -- Aaron Hurt Managing Partner Flex I.T., LLC 611 Commerce Street Suite 3117 Nashville, TN 37203 Phone: 615.438.7101 E-mail: aaron@goflexitllc.com --------------010909020406010003050503--