Date: Thu, 3 Nov 2011 10:41:30 +0100 From: Borja Marcos <borjam@sarenet.es> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: freebsd-fs@freebsd.org Subject: Re: Default inode number too low in FFS nowadays? Message-ID: <32C5CE2F-8C1C-442E-A1B4-9DD9FD47C691@sarenet.es> In-Reply-To: <20111102131311.GA56941@icarus.home.lan> References: <B888842A-7DB4-491B-93E3-A376745019F5@sarenet.es> <20111102131311.GA56941@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Nov 2, 2011, at 2:13 PM, Jeremy Chadwick wrote: > On Wed, Nov 02, 2011 at 12:57:33PM +0100, Borja Marcos wrote: >> Today I?ve come across an issue long ago forgotten :) Running out of = i-nodes. >=20 > The last time I dealt with this was about 3 months ago, on a Netapp > filer; hardly forgotten. :-) >=20 >> The condition was triggered on a 8 GB /usr filesystem which includes = the ports tree, on which I have compiled several ports and their = dependencies. I know, maybe not the best practices, it's a machine = being used to test a couple of thingies, but I wonder how many newbies = can run into such a problem. I guess many. >>=20 >> Filesystem 1K-blocks Used Avail Capacity = iused ifree %iused Mounted on >> /dev/ad1s1e 8245660 5308960 2277048 70% 507099 61219 = 89% /usr >>=20 >> Now i-nodes aren't completely full, as I have deleted /usr/obj (turns = out I had made a make world as well). >>=20 >> Looking at the number of i-nodes per /usr subdirectory, I have = noticed that, wow! /usr/ports consumes A LOT of them.=20 >>=20 >> freebsd9-borja# find . -print | wc -l >> 405481 >=20 > "find -x /usr/ports | wc -l" on my system returns 146420. My find was on /usr/ports after having compiled the mentioned ports. = It's using so many inodes after compiling the mentioned ports. > Your 8GB filesystem is only ~21% the capacity of mine. So if we do = some > simple math, your /usr should really be showing (assuming yours is as > "bare bones" as mine) around 26% inodes used. Yet what you have is = 89%. >=20 > You probably just have a lot of other software on /usr that "abuses" > lots of individual files. You list off what ports you built from > source, which means you have lots of "build cruft" laying around en > masse (work/ directories). Yes, of course, I am aware of that. But a = mother-of-all-stuff-and-not-too-large /usr filesystem can create = problems if you build plenty of ports. The issue is: given that tasks such as port building now consume = inordinate amounts of inodes, is the current default value reasonable? = This is an issue on which I can imagine a newbie having problems, hence = my suggestion to increase the default number. > Really? That isn't what the default installer does (re: single > filesystem), and hasn't since... well, as long as I can remember > (2.2.8). Sorry, I'm wrong on that. > I wouldn't recommend a /usr/ports filesystem. It solves nothing. > Imagine: /usr/ports can now run out of inodes while /usr won't. What > would this address? It would decrease impact to applications creating > new files on /usr. How often do you have that happening on your = system? > For me it's extremely rare. For comparison, /var gets a lot more > activity. Yes, but there are ports that use /usr as far as I know. Anyway, it = would be an issue if the user is just building ports. > Adding a /usr/ports filesystem also provides zero scalability. = Meaning: > say someone decides "8GB for /usr/ports is enough" during the = installer > and so on. Then down the road, we run out of space, or inodes, or > whatever else. Oops, what now? Symlinks all over the place? Yeah, = not > going to happen. Good point.=20 > We were already in this boat regarding the root filesystem size, which > for quite some time defaulted to 512MBytes, which was not enough. > Today, the default size is 1GBytes, which is enough for a kernel built > with debugging symbols and so on, but tends to reach ~55-58% used. >=20 > There's a reason /usr on FreeBSD defaults to "all the remaining space = on > the disk" if you pick the defaults/auto. Surprise. :-) >=20 > The summarised version is: >=20 > 1. You have control over this yourself: newfs(8) -i flag. You can = even > input this flag during sysinstall when building a new system. >=20 > 2. When you built the system, you chose to assume /usr should be = 8GBytes. > Your usage habits may have changed, software gotten larger, etc.. It > may be time for you to re-evaluate how you create your filesystems and > what sizes you choose. You chose something that didn't scale. :-) I'm not complaining about the issue, I know well I have to clean up, = etc. Been using FreeBSD since 2.0.5 ;) But I'm wondering if it could be an issue to others. And the /usr was small because I need a lot of room for /var.=20 > 4. For a limited-space (not just size but inodes!) filesystem, = consider > using packages (pkg_add -r) for anything that does not require > customisation during the build/configure phase. For example, for = quite > some time now on my systems, I've always installed the package = versions > of perl, python27, cmake, gettext, pcre, etc. solely because they are > "key softwares" that I hate building from source. Unfortunately there are several key ports that *do* need compile time = customization. Moreover, I could be wrong, as I've been compiling ports = for many, many years, but do we have a mechanism to document which = compile-time options were used to build a package? I'm not aware of it. Just a suggestion anyway. The calculation for the default number of = inodes was right many years ago, in my opinion the number is too low = nowadays. And yes, I know this isn't an issue with ZFS, been using it for a long = time as well. But FFS is still very widely used. Borja.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?32C5CE2F-8C1C-442E-A1B4-9DD9FD47C691>