Date: Mon, 12 Feb 2007 16:52:44 +0100 (CET) From: Oliver Fromme <olli@lurza.secnetix.de> To: freebsd-fs@FreeBSD.ORG, aronesimi@yahoo.com Subject: Re: comments on newfs raw disk ? Safe ? (7 terabyte array) Message-ID: <200702121552.l1CFqi6q046650@lurza.secnetix.de> In-Reply-To: <506226.11053.qm@web58610.mail.re3.yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Arone Silimantia wrote: > Thank you for your detailed response - my own response is inline below: You should indent the quoted text, so it's easier to tell the quoted text from your own text. Most MUAs offer a function to indent the quoted text automatically. I tried to fix it below. > Oliver Fromme <olli@lurza.secnetix.de> wrote: > > Arone Silimantia wrote: > > > Big 3ware sata raid with 16 disks. First two disks are a mirror to > > > boot off of. I installed the system with sysinstall and created all the > > > partitions on the boot mirror, etc., and just didn't even touch the > > > 14-disk array that was also created. > > > [...] > > > newfs -m 0 /dev/da1 > > > > You didn't mention the size of the FS, but I guess it's at > > least 4 TB, probably more. > > Well, in the subject line I mentioned 7 TB, but I have > since rearranged some things and it will be 5.5 TB. Sorry, I didn't look at the subject line too closely. I expected that all important information was included in the mail body. > > You'll probably want to reduce the inode density (i.e. > > increase the bytes-per-inode ratio). With the default > > value, an fsck will be a royal pain, no matter whether you > > use background fsck (with snapshots) or not. It might even > > not work at all if you don't have a huge amount of RAM. > > Well, I have 4 GB of physical RAM, and 4 GB of swap - so does that > total of 8 GB satisfy the "1 GB per TB" requirement, or do I really > need >5.5 GB of actual swap space (in addition to the physical) ? That "1 GB per TB" requirement is just a rule of thumb. I don't know hoe accurate it is. Also note that it is desirable to avoid having fsck use swap, because it will be even slower then. A lot slower. > > If you increase the ratio to 64 K, it will lower the fsck > > time and RAM requirement by an order of magnitude, while > > there are still about 15 million inodes available per TB. > > If possible, increase the ratio (-i option) further. It > > depends on the expected average file size and the maximum > > number of files that you intend to store on the FS, of > > course. > > Ok, I will look into this. My data population uses a little less > than 5 million inodes per TB, so this may be workable to tune. So I > see the default is '4' - so I could run newfs with: The default is 4096 (one inode per 4 KB). > newfs -i 8 > > to do what you are suggesting ? I think you mean 8192. That will cut the number of inodes in half, but I think you can go even further. Try: # newfs -i 65536 That will leave room for about 15 million inodes per TB, which is plenty for your needs. By the way, reducing the inode density like that will also give your more space for actual file data. In UFS2, every inode takes 256 bytes. Reducing the bytes-per-inode ratio from 4 KB to 64 KB will give you additional 60 GB of space. _And_ it will reduce the memory and time requirements of fsck. > > Oh by the way, I also agree with Eric that you should have > > a look at gjournal. It pratically removes the fsck issues. > > At the moment it's only in -current, but I think Pawel > > provided a port for 6.x. > > Well, I don't mind a 24 hour fsck, and I would like to remove > complexity and not be so on the bleeding edge with things. Since I > am only using 5mil inodes per TB anyway, that ends up being 25-30 > million inodes in the 5 TB drive which I think could fsck in a day or > so. I suggest you test it before putting it into production, i.e. populate the file system with the expected number of files, then run fsck. > I just need to know if my 4+4 GB of memory is enough, and if this > option in loader.conf: > > kern.maxdsiz="2048000000" That will limit the process size to 2 GB. You might need to set it higher if fsck needs more than that. (I assume you're running FreeBSD/amd64, or otherwise you'll run into process size limitations anyway.) Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, USt-Id: DE204219783 Any opinions expressed in this message are personal to the author and may not necessarily reflect the opinions of secnetix GmbH & Co KG in any way. FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd Perl is worse than Python because people wanted it worse. -- Larry Wall
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200702121552.l1CFqi6q046650>