Date: Thu, 3 Nov 2011 10:41:30 +0100 From: Borja Marcos <borjam@sarenet.es> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: freebsd-fs@freebsd.org Subject: Re: Default inode number too low in FFS nowadays? Message-ID: <32C5CE2F-8C1C-442E-A1B4-9DD9FD47C691@sarenet.es> In-Reply-To: <20111102131311.GA56941@icarus.home.lan> References: <B888842A-7DB4-491B-93E3-A376745019F5@sarenet.es> <20111102131311.GA56941@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Nov 2, 2011, at 2:13 PM, Jeremy Chadwick wrote: > On Wed, Nov 02, 2011 at 12:57:33PM +0100, Borja Marcos wrote: >> Today I?ve come across an issue long ago forgotten :) Running out of i-nodes. > > The last time I dealt with this was about 3 months ago, on a Netapp > filer; hardly forgotten. :-) > >> The condition was triggered on a 8 GB /usr filesystem which includes the ports tree, on which I have compiled several ports and their dependencies. I know, maybe not the best practices, it's a machine being used to test a couple of thingies, but I wonder how many newbies can run into such a problem. I guess many. >> >> Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on >> /dev/ad1s1e 8245660 5308960 2277048 70% 507099 61219 89% /usr >> >> Now i-nodes aren't completely full, as I have deleted /usr/obj (turns out I had made a make world as well). >> >> Looking at the number of i-nodes per /usr subdirectory, I have noticed that, wow! /usr/ports consumes A LOT of them. >> >> freebsd9-borja# find . -print | wc -l >> 405481 > > "find -x /usr/ports | wc -l" on my system returns 146420. My find was on /usr/ports after having compiled the mentioned ports. It's using so many inodes after compiling the mentioned ports. > Your 8GB filesystem is only ~21% the capacity of mine. So if we do some > simple math, your /usr should really be showing (assuming yours is as > "bare bones" as mine) around 26% inodes used. Yet what you have is 89%. > > You probably just have a lot of other software on /usr that "abuses" > lots of individual files. You list off what ports you built from > source, which means you have lots of "build cruft" laying around en > masse (work/ directories). Yes, of course, I am aware of that. But a mother-of-all-stuff-and-not-too-large /usr filesystem can create problems if you build plenty of ports. The issue is: given that tasks such as port building now consume inordinate amounts of inodes, is the current default value reasonable? This is an issue on which I can imagine a newbie having problems, hence my suggestion to increase the default number. > Really? That isn't what the default installer does (re: single > filesystem), and hasn't since... well, as long as I can remember > (2.2.8). Sorry, I'm wrong on that. > I wouldn't recommend a /usr/ports filesystem. It solves nothing. > Imagine: /usr/ports can now run out of inodes while /usr won't. What > would this address? It would decrease impact to applications creating > new files on /usr. How often do you have that happening on your system? > For me it's extremely rare. For comparison, /var gets a lot more > activity. Yes, but there are ports that use /usr as far as I know. Anyway, it would be an issue if the user is just building ports. > Adding a /usr/ports filesystem also provides zero scalability. Meaning: > say someone decides "8GB for /usr/ports is enough" during the installer > and so on. Then down the road, we run out of space, or inodes, or > whatever else. Oops, what now? Symlinks all over the place? Yeah, not > going to happen. Good point. > We were already in this boat regarding the root filesystem size, which > for quite some time defaulted to 512MBytes, which was not enough. > Today, the default size is 1GBytes, which is enough for a kernel built > with debugging symbols and so on, but tends to reach ~55-58% used. > > There's a reason /usr on FreeBSD defaults to "all the remaining space on > the disk" if you pick the defaults/auto. Surprise. :-) > > The summarised version is: > > 1. You have control over this yourself: newfs(8) -i flag. You can even > input this flag during sysinstall when building a new system. > > 2. When you built the system, you chose to assume /usr should be 8GBytes. > Your usage habits may have changed, software gotten larger, etc.. It > may be time for you to re-evaluate how you create your filesystems and > what sizes you choose. You chose something that didn't scale. :-) I'm not complaining about the issue, I know well I have to clean up, etc. Been using FreeBSD since 2.0.5 ;) But I'm wondering if it could be an issue to others. And the /usr was small because I need a lot of room for /var. > 4. For a limited-space (not just size but inodes!) filesystem, consider > using packages (pkg_add -r) for anything that does not require > customisation during the build/configure phase. For example, for quite > some time now on my systems, I've always installed the package versions > of perl, python27, cmake, gettext, pcre, etc. solely because they are > "key softwares" that I hate building from source. Unfortunately there are several key ports that *do* need compile time customization. Moreover, I could be wrong, as I've been compiling ports for many, many years, but do we have a mechanism to document which compile-time options were used to build a package? I'm not aware of it. Just a suggestion anyway. The calculation for the default number of inodes was right many years ago, in my opinion the number is too low nowadays. And yes, I know this isn't an issue with ZFS, been using it for a long time as well. But FFS is still very widely used. Borja.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?32C5CE2F-8C1C-442E-A1B4-9DD9FD47C691>