Date: Fri, 21 Dec 2012 17:28:46 +0000 From: Arthur Chance <freebsd@qeng-ho.org> To: Paul Kraus <paul@kraus-haus.org> Cc: yudi.tux@gmail.com, freebsd-questions@freebsd.org Subject: Re: ZFS info WAS: new backup server file system options Message-ID: <50D49C4E.6060007@qeng-ho.org> In-Reply-To: <282CDB05-5607-4315-8F37-3EEC289E83F5@kraus-haus.org> References: <CACo--muRK_pqrBhL2LLcnByTrVKfXQfFGZDn-NqpQndm3Th=RA@mail.gmail.com> <282CDB05-5607-4315-8F37-3EEC289E83F5@kraus-haus.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/21/12 14:06, Paul Kraus wrote: > On Dec 21, 2012, at 7:49 AM, yudi v wrote: > >> I am building a new freebsd fileserver to use for backups, will be using 2 >> disk raid mirroring in a HP microserver n40l. >> I have gone through some of the documentation and would like to know what >> file systems to choose. >> >> According to the docs, ufs is suggested for the system partitions but >> someone on the freebsd irc channel suggested using zfs for the rootfs as >> well >> >> Are there any disadvantages of using zfs for the whole system rather than >> going with ufs for the system files and zfs for the user data? > > First a disclaimer, I have been working with Solaris since 1995 and managed > lots of data under ZFS, I have only been working with FreeBSD for about the past > 6 months. > > UFS is clearly very stable and solid, but to get redundancy you need to use > a separate "volume manager". Slight correction here - you don't need a volume manager (as I understand the term), you'd use the GEOM subsystem, specifically gmirror in this case. See "man gmirror" for details > ZFS is a completely different way of thinking about managing storage (not > just a filesystem). I prefer ZFS for a number of reasons: > > 1) End to end data integrity through checksums. With the advent of 1 TB plus > drives, the uncorrectable error rate (typically 10^-14 or 10^-15) means that > over the life of any drive you *are* now likely to run into uncorrectable errors. > This means that traditional volume managers (which rely on the drive reporting an > bad reads and writes) cannot detect these errors and bad data will be returned to > the application. > > 2) Simplicity of management. Since the volume management and filesystem layers > have been combined, you don't have to manage each separately. > > 3) Flexibility of storage. Once you build a zpool, the filesystems that reside > on it share the storage of the entire zpool. This means you don't have to decide > how much space to commit to a given filesystem at creation. It also means that all > the filesystems residing in that one zpool share the performance of all the drives > in that zpool. > > 4) Specific to booting off of a ZFS, if you move drives around (as I tend to do in > at least one of my lab systems) the bootloader can still find the root filesystem > under ZFS as it refers to it by zfs device name, not physical drive device name. > Yes, you can tell the bootloader where to find root if you move it, but zfs does > that automatically. > > 5) Zero performance penalty snapshots. The only cost to snapshots is the space > necessary to hold the data. I have managed systems with over 100,000 snapshots. > > I am running two production, one lab, and a bunch of VBox VMs all with ZFS. > The only issue I have seen is one I have also seen under Solaris with ZFS. Certain > kinds of hardware layer faults will cause the zfs management tools (the zpool and > zfs commands) to hang waiting on a blocking I/O that will never return. The data > continuos to be available, you just can't manage the zfs infrastructure until the > device issues are cleared. For example, if you remove a USB drive that hosts a > mounted ZFS, then any attempt to manage that ZFS device will hang (zpool export > -f <zpool name> hangs until a reboot). > > Previously I had been running (at home) a fileserver under OpenSolaris using > ZFS and it saved my data when I had multiple drive failures. At a certain client > we had a 45 TB configuration built on top of 120 750GB drives. We had multiple > redundancy and could survive a complete failure of 2 of the 5 disk enclosures (yes, > we tested this in pre-production). > > There are a number of good writeups on how setup a FreeBSD system to boot off > of ZFS, I like this one the best > http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/9.0-RELEASE , but I do the zpool/zfs > configuration slightly differently (based on some hard learned lessons on Solaris). > I am writing up my configuration (and why I do it this way), but it is not ready yet. > > Make sure you look at all the information here: http://wiki.freebsd.org/ZFS , > keeping in mind that lots of it was written before FreeBSD 9. I would NOT use ZFS, > especially for booting, prior to release 9 of FreeBSD. Some of the reason for this > is the bugs that were fixed in zpool version 28 (included in release 9). I would agree with all that. My current system uses UFS filesystems for the base install, and ZFS with a raidz zpool for everything else, but that's only because I started using ZFS in REL 8.0 when it was just out of experimental status, and I didn't want to risk having an unbootable system. (That last paragraph suggests I was wise in that decision.) My next machine I'm specing out now will be pure ZFS so I get the boot environment stuff.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?50D49C4E.6060007>