Skip site navigation (1)Skip section navigation (2)
Date:      Tue, 4 Nov 2008 12:55:56 -0500
From:      "Gabriel Lavoie" <glavoie@gmail.com>
To:        "Volodymyr Kostyrko" <c.kworr@gmail.com>
Cc:        freebsd-questions@freebsd.org
Subject:   Re: gjournal: journaled slices vs. journaled partitions
Message-ID:  <d05df8620811040955m654f769bj8683bff66d7a6b42@mail.gmail.com>
In-Reply-To: <gepqhr$56u$1@ger.gmane.org>
References:  <48FD6665.5000102@telus.net> <48FD6803.7080802@shopzeus.com> <48FE6C64.7060606@telus.net> <gepqhr$56u$1@ger.gmane.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Hello,
     I built a similar setup last weekend on a new home server with two
500GB drives. I didn't want to only put gmirror and have full drives rebuild
on power failure/reset on the system. I was told that putting bsdlabels on a
gjournal provider wasn't a good idea but I have yet to have an answer about
why... I went with this setup anyway and I made some reset tests to see what
happens on reboot and everything always went fine.

When building this setup I got one big problem. If the root filesystem (/)
was on a gjournal provider, an unclean shutdown when data was being written
on the disk rendered the system completely unbootable. I got this message:

GEOM_MIRROR: Device mirror/gm launched (2/2)
GEOM_JOURNAL: Journal 3672855181: mirror/gma contains data.
GEOM_JOURNAL: Journal 3672855181: mirror/gma contains journal.
GEOM_JOURNAL: Journal 3868799910: mirror/gmd contains data.
GEOM_JOURNAL: Journal 3868799910: mirror/gmd contains journal.
GEOM_JOURNAL: Journal mirror/gmd consistent.
Trying to mount root from ufs:/dev/mirror/gm.journal

Manual root filesystem specification:
    <fstype>:<device>  Mount <device> using filesystem <fstype>
                                           eg. ufs:da0s1a
    ?                             List valid disk boot devices
    <empty line>          Abort manual input


mountroot> ?

List of GEOM managed disk devices:
     mirror/gmd.journal mirror/gmd mirror/gmc mirror/gma mirror/gm ad10s1c
ad10s1b ad8s1c ad8s1b ad10s2 ad10s1 ad8s1 ad10 ad8 acd0


As you can see, in the proposed list of disk devices devices to boot on,
"mirror/gm.journala" is absent. As I and Ivan Voras, that I contacted about
this problem, found, the GEOM_JOURNAL thread that is supposed to mark the
journal consistent takes too much time to do it with the root filesystem's
provider and the kernel try to mount a device that doesn't yet exist. A bug
report has been opened about this problem. For my final setup I decided to
put the root filesystem on a separate mirrorred slice of 1GB. Since this
slice isn't often written on, not many rebuilds should occur in case of
power failure. And I made my "power failure" test by hitting the reset
button while writing data on this filesystem and the rebuild on 1GB doesn't
takes too much time (at most 20-30 seconds).

Now I have the question. Why the "load" algorith wasn't recommended? Is it
fixed in 7.0-RELEASE-p5?

Here is my complete setup that seems to boot correctly every times I made my
reset tests while writing data on each filesystems. The 2GB gjournal
provider is directly on the mirror provider for all mirrored filesystems
exept the root one and I made my bsd labels on the gjournal provider,
instead of creating a journal for every filesystem.


[root@headless ~]# cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump
Pass#
/dev/ad10s1b            none            swap    sw              0       0
/dev/ad8s1b             none            swap    sw              0       0
/dev/mirror/root        /               ufs     rw              1       1
/dev/ufs/usr            /usr            ufs     rw,async        2       2
/dev/ufs/var            /var            ufs     rw,async        2       2
/dev/ufs/tmp            /tmp            ufs     rw,async        2       2
/dev/ufs/home           /home           ufs     rw,async        2       2
/dev/ufs/data           /mnt/data       ufs     rw,async        2       2
/dev/acd0               /cdrom          cd9660  ro,noauto       0       0


[root@headless ~]# mount
/dev/mirror/root on / (ufs, local, soft-updates)
devfs on /dev (devfs, local)
/dev/ufs/usr on /usr (ufs, asynchronous, local, gjournal)
/dev/ufs/var on /var (ufs, asynchronous, local, gjournal)
/dev/ufs/tmp on /tmp (ufs, asynchronous, local, gjournal)
/dev/ufs/home on /home (ufs, asynchronous, local, acls, gjournal)
/dev/ufs/data on /mnt/data (ufs, asynchronous, local, acls, gjournal)


[root@headless ~]# glabel status
    Name  Status  Components
 ufs/usr     N/A  mirror/data.journald
 ufs/var     N/A  mirror/data.journale
 ufs/tmp     N/A  mirror/data.journalf
ufs/home     N/A  mirror/data.journalg
ufs/data     N/A  mirror/data.journalh


[root@headless ~]# gjournal list
Geom name: gjournal 372943514
ID: 372943514
Providers:
1. Name: mirror/data.journal
   Mediasize: 495810966528 (462G)
   Sectorsize: 512
   Mode: r5w5e11
Consumers:
1. Name: mirror/data
   Mediasize: 497958450688 (464G)
   Sectorsize: 512
   Mode: r1w1e1
   Jend: 497958450176
   Jstart: 495810966528
   Role: Data,Journal


[root@headless ~]# gmirror list
Geom name: data
State: COMPLETE
Components: 2
Balance: split
Slice: 4096
Flags: NOFAILSYNC
GenID: 0
SyncID: 1
ID: 990032118
Providers:
1. Name: mirror/data
   Mediasize: 497958450688 (464G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: ad8s2
   Mediasize: 497958451200 (464G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   GenID: 0
   SyncID: 1
   ID: 235591066
2. Name: ad10s2
   Mediasize: 497958451200 (464G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   GenID: 0
   SyncID: 1
   ID: 2007880058

Geom name: root
State: COMPLETE
Components: 2
Balance: split
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 4098555256
Providers:
1. Name: mirror/root
   Mediasize: 1073022976 (1.0G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: ad8s1a
   Mediasize: 1073023488 (1.0G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   GenID: 0
   SyncID: 1
   ID: 3394521634
2. Name: ad10s1a
   Mediasize: 1073023488 (1.0G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: HARDCODED
   GenID: 0
   SyncID: 1
   ID: 3774466459


Gabriel


2008/11/4 Volodymyr Kostyrko <c.kworr@gmail.com>

> Carl wrote:
>
>> Volodymyr Kostyrko wrote:
>>
>>  I have some setups were gjournal was put on device rather the on
>>> partition, i.e.:
>>>
>>> [umgah] ~> gmirror status
>>>          Name    Status  Components
>>> mirror/umgah0  COMPLETE  ad0
>>>                          ad1
>>> [umgah] ~> gjournal status
>>>                  Name  Status  Components
>>> mirror/umgah0.journal     N/A  mirror/umgah0
>>> [umgah] ~> glabel status
>>>             Name  Status  Components
>>>   ufs/umgah0root     N/A  mirror/umgah0.journala
>>> label/umgah0swap     N/A  mirror/umgah0.journalb
>>>    ufs/umgah0usr     N/A  mirror/umgah0.journald
>>>    ufs/umgah0var     N/A  mirror/umgah0.journale
>>>
>>
>> Does the above suggest that you've ended up with individual journal
>> providers for each partition anyway? If so, where are they and have you
>> really achieved anything functionally different? Are they at the end of
>> their individually associated partitions or all together somewhere else? Has
>> the ill-advised journaled small partition issue been successfully overcome
>> through what you've done?
>>
>
> First, there is only one journal - for /dev/mirror/umgah0 and it is named
> /dev/mirror/umgah0.journal. Anything else is just a bsdlabel partitions,
> there are four of 'em.
>
>
>>  [umgah] ~> mount
>>> /dev/ufs/umgah0root on / (ufs, asynchronous, local, noatime, gjournal)
>>> devfs on /dev (devfs, local)
>>> /dev/md0 on /tmp (ufs, asynchronous, local)
>>> /dev/ufs/umgah0var on /var (ufs, asynchronous, local, noatime, gjournal)
>>> /dev/ufs/umgah0usr on /usr (ufs, asynchronous, local, noatime, gjournal)
>>> devfs on /var/named/dev (devfs, local)
>>>
>>> And yes, mirror autosynchronization is turned off, gjournal takes care of
>>> that too.
>>>
>>> It's not stated in manual, but gjournal is typically transparent for any
>>> type of access, just in case of UFS file system is marked as journaled so
>>> any metadata writes can be distinguished from data writes. Without that
>>> gjournal does literally nothing.
>>>
>>
>> And what does this mean for your swap partition?
>>
>
> Just nothing, it's just swap. It can't be journaled.
>
>  Laszlo Nagy wrote earlier:
>>
>>> Another tricky question: why would you journal a SWAP partition?
>>>
>>
>> Volodymyr, does your assertion that gjournal does nothing when a file
>> system is not UFS mean that there is no penalty with regard to your swap
>> partition despite the existence of "mirror/umgah0.journalb"?
>>
>
> I haven't seen any perfomance decrease in this configuration. And according
> to manual and articles about gjournal it should work this way.
>
>  Any chance you'd like to share your command sequence for constructing your
>> gmirror'd and gjournal'd filesystem, Volodymyr? :-)
>>
>
> If we have two disks (ad0, ad1) it should look like this:
>
> > gmirror label -b load -n umgah0 ad1
>
> We are getting all drive gmirrored without synchronization (we don't need
> it - journal would take care of any discrepancies) and with load balance
> (load was fixed not so long ago in stable and should be fine to go with).
>
> > gjournal label mirror/umgah0
>
> We are creating a journal on top of our gmirror. It eats 1G from the end of
> the disks and gives us the rest to use.
>
> > bsdlabel -wB mirror/umgah0.journal
>
> We are writing the standard bsdlabel to the disk and making it bootable.
> After that we will get one partition 'a'.
>
> <spam>
> Yes, no fdisk. I don't think this old piece of rough junk is ever needed on
> machine running FreeBSD solely. It just takes space, it requires
> compatibility to forgotten-and-abandoned standards and gives nothing more.
> You have your server dual-booting Windows or Linux? This is the only case
> you need fdisk for.
> </spam>
>
> > bsdlabel -e mirror/umgah0.journal
>
> Now we are splitting our journal to some partitions. I did it this way:
>
> # /dev/mirror/umgah0.journal:
> 8 partitions:
> #        size   offset    fstype   [fsize bsize bps/cpg]
>  a:   524288       16    4.2BSD
>  b: 16777216   *      swap
>  c: 779325614        0    unused        0     0         # "raw" part, don't
> edit
>  d: 33554432 *    4.2BSD
>  e: * *    4.2BSD
>
> After that we can format this filesystems:
>
> > newfs -J -L umgah0root /dev/mirror/umgah0.journala
> > newfs -J -L umgah0var /dev/mirror/umgah0.journald
> > newfs -J -L umgah0usr /dev/mirror/umgah0.journale
>
> And label the swap:
>
> > glabel label umgah0swap /dev/mirror/umgah0.journalb
>
> You can skip all this glabel thing, I just prefer to have slim fstab, as
> slim as possible.
>
> <fstab>
> /dev/label/umgah0swap none swap sw 0 0
>
> md /tmp mfs rw,-s1024m,-S,-oasync 0 0
>
> /dev/ufs/umgah0root / ufs rw,async,noatime 0 1
> /dev/ufs/umgah0var /var ufs rw,async,noatime 0 2
> /dev/ufs/umgah0usr /usr ufs rw,async,noatime 0 2
> </fstab>
>
> There's a lot more here to describe from moving system to newly created
> partitions to inserting and rebuilding our first disk to gmirror. All this
> issues are described in handbook or other articles found on the net.
>
>
> --
> Sphinx of black quartz judge my vow.
>
> _______________________________________________
> freebsd-questions@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-questions
> To unsubscribe, send any mail to "
> freebsd-questions-unsubscribe@freebsd.org"
>



-- 
Gabriel Lavoie
glavoie@gmail.com



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d05df8620811040955m654f769bj8683bff66d7a6b42>