Date: Fri, 09 Feb 2007 07:38:07 -0600 From: Eric Anderson <anderson@freebsd.org> To: Ivan Voras <ivoras@fer.hr> Cc: freebsd-fs@freebsd.org Subject: Re: comments on newfs raw disk ? Safe ? (7 terabyte array) Message-ID: <45CC793F.7090003@freebsd.org> In-Reply-To: <eqhrv8$92j$1@sea.gmane.org> References: <646424.65334.qm@web58613.mail.re3.yahoo.com> <eqhrv8$92j$1@sea.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 02/09/07 07:12, Ivan Voras wrote: > Arone Silimantia wrote: > >> dd if=/dev/zero of=/dev/da1 bs=1k count=1 >> newfs -m 0 /dev/da1 >> mount /dev/da1 /mnt >> >> And that's that. But it seems too good to be true! Can someone please >> comment on this scheme and if there are some hidden dangers or lack of >> functionality that I will regret in the future ? > > No dangers at the system level - you can create your file system on any > storage-like device, use it and mount it any way you want. Raw disks are > a perfectly valid target. As a side benefit, one might also get a performance *increase* because not having a slice/partition in the way might make the file system blocks line up better with the stripe size. Scott Long has posted about this in the past, either here, or on -performance, or somewhere. I can't find the thread right off, but it's out there (he wrote a very good description of what happens, why, etc). FWIW, I almost always do it this way. >> Will it fsck just like any other UFS2 partition I run ? Can I run >> quotas and snapshots and everything else on it, just like normal ? > > Yes. > >> Other than the fact that I can't boot this, is there _any downside >> whatsoever_ to newfs'ing raw disk like this ? > > Only "collateral" problems because of the partition size: a regular > (non-softupdates) fsck will take a LONG time to finish and eat a LOT of > memory while it's doing its stuff. You'll need a lot of swap space (1GB > per TB? someone had empirical numbers on this, I'm sure) if you think > you'll need to fsck it entirely. Creating snapshots will also take a > long time on it, and you probably want to search the lists for > recommendations about creating snapshots in a second level directory in > order not to block the root directory. Related to this is > background-fsck which works by creating snapshots, so you'll probably > want to disable it. I have 5 10Tb file systems (and some 2Tb ones, but who cares about those tiny things? :)), and I can tell you that an empty huge file system is pretty easily fsck-able, but a full one will kill you. It greatly depends on how many files (inodes) you have used on the file system. If you have a massive amount of small files, you'll be eating up a ton of memory. My 'rule of thumb' for my data (which averages to about 16k/file) is 1G of memory for each 1Tb of disk space used. So, on a 10Tb file system, if I ever want the fsck to complete, I need an AMD64 box with *at least* 10G of memory, plus a lot of time. A *lot* of time. By 'a lot', I mean anywhere from a day, to several days. > In any case, try every feature you think you'll need before deploying it. > > Also, write about your experience on this list :) I second that. It's important to share anything we can, so we see what others are doing, what the needs are, etc. I also recommend looking at gjournal, now in -CURRENT. I'm not sure if it's still considered beta, but care should be taken when using it of course. However, I use it for many tens of TB of data, and I'm a happy gjournal fan (thanks Pawel!). Eric
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?45CC793F.7090003>