From owner-freebsd-arch Sat Dec 8 22:16:36 2001 Delivered-To: freebsd-arch@freebsd.org Received: from monorchid.lemis.com (monorchid.lemis.com [192.109.197.75]) by hub.freebsd.org (Postfix) with ESMTP id F214137B405; Sat, 8 Dec 2001 22:16:07 -0800 (PST) Received: by monorchid.lemis.com (Postfix, from userid 1004) id 43180786E6; Sun, 9 Dec 2001 16:46:06 +1030 (CST) Date: Sun, 9 Dec 2001 16:46:06 +1030 From: Greg Lehey To: Mike Smith Cc: arch@FreeBSD.ORG, Marko Zec , "Louis A. Mamakos" , Matthew Dillon , Sheldon Hearn , freebsd-arch@FreeBSD.ORG Subject: Getting rid of /usr file system (was: Using a larger block size on large filesystems) Message-ID: <20011209164606.C83634@monorchid.lemis.com> References: <20011208102658.B11428@dragon.nuxi.com> <200112082050.fB8Ko1T01347@mass.dis.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="nFreZHaLTZJo0R7j" Content-Disposition: inline In-Reply-To: <200112082050.fB8Ko1T01347@mass.dis.org> User-Agent: Mutt/1.3.23i Organization: The FreeBSD Project Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.FreeBSD.org/ X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Saturday, 8 December 2001 at 12:50:01 -0800, Mike Smith wrote: >>> Why would we want to do that? Putting unnecessary things on / is always a >>> bad idea, as root partition should remain small and as free of frequent RW >>> operations as possible. >> >> That is a *PERSONAL* opinion, so please mark it as such. I personally >> use 1.5GB / partitions. I _personally_ see zero reason for a /usr >> partition. > > PERSONALLY, I disagree. 1.5GB is too small these days; 2GB is a better > choice. 8) Well, since we're all being personal, here's my opinion. Yes, I agree with David that /usr has outlived its use-by date. It was originally developed because the root file system at the time went on a 512 kB disk, and it needed to be separate. It was kept for a long time because of superstition that it might help recover file systems badly broken as the result of a crash. In fact, it does pretty much the opposite: it means that a lot of tools are not available in single user mode unless you mount /usr, since the dynamic libraries are on /usr. It doesn't pose much of a crash or corruption danger, since it's hardly ever written to nowadays. And of course, it's not /usr at all any more, it's /system. I've been running systems like this for a few years. I disagree with both of you about size; 4 GB is the one true size. You can fit it on a DDS-2 tape, and it's big enough to keep /usr/local on the same file system. I've tried with 2 GB, and that's definitely too small. 3 GB might be OK, but in a few years' time that will be too small too. You can discount this all you want, but if you have good, solid arguments against doing it, speak now: otherwise this recommendation with be in the next edition (soon!) of "The Complete FreeBSD". I'm attaching an extract from the current draft; criticize all you want. > Then again, I work for a company that puts everything in > /System/Library, so I guess I should be quiet now. 8) Sounds ugly, but when you think of it, it makes a fair amount of sense. Greg -- See complete headers for address and phone numbers --nFreZHaLTZJo0R7j Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=sizing What size partitions? _____________________ In our example, we have 20 GB of space to divide up. How should we do it? You don't have to worry about this issue, since sysinstall will do it for you if you ask it, but we'll see below why this might not be the best choice. In this section we'll consider how UNIX file systems have changed over the years, and Page 90 install.mm,v v3.5 (2000/08/30 00:08:01) (modified 25 Mar, 2001 UTC) Chapter 6: Installing FreeBSD we'll look at the issues in file system layout nowadays. When UNIX was young, disks were tiny. By the time of the Third Edition of UNIX, in 1972, the root file system was on a Digital RF-11, a fixed head disk with 512 kB. It was no longer possible to keep the entire system on this disk, so a second file system became essential. It was mounted on a Digital RK03 with 2 MB of storage. To quote from a paper published in the Communications of the ACM in July 1974: In our installation, for example, the root directory resides on the fixed-head disk, and the large disk drive, which contains user's files, is mounted by the system initialization program... As time went on, UNIX got bigger, but so did the disks. By the early 80s, disks were large enough to put / and /usr on the same disk, and it would have been possible to merge / and /usr, but they didn't. Since that time, an additional file system, /var, has come into common use for frequently changed data. This is what sysinstall does by default: [Omitting PostScript image images/disk-label-default.ps 4i ] Figure 6-12: Default disk partition sizes It's relatively simple to estimate the size of the root file system, and sysinstall's value of 50 MB is fine. But what about /var? 20 MB isn't worth the trouble any more. In fact, both file systems put together would be lost in the 19 GB of /usr file system. Why do we still do things this way? Let's look at the advantages and disadvantages: o If you write to a file system, a system crash can have serious consequences for the data integrity of that file system. o If you have a crash and lose the root file system, recovery can be difficult. o If a file system fills up, it can cause lots of trouble. Most messages about file systems on the FreeBSD-questions mailing list are complaining about file systems filling up. If you have a large number of small file systems, the chances are higher that one will fill up while space remains on another. o In single-user mode, only the root file system is mounted. With the classical layout, this means that the only programs you can run are those in /bin and /sbin. In order to run any other programs, you must first mount the file system on which they are located. install.mm,v v3.5 (2000/08/30 00:08:01) (modified 25 Mar, 2001 UTC) Page 91 Creating space on disk o It's nice to keep your personal files separate from the system files. That way you can upgrade a system much more easily. o It's very difficult to calculate the sizes of some file systems. For example, on some systems /var can be very small, maybe only 2 or 3 MB. It's hardly worth making a separate file system for that much data. On the other hand, other systems, such as ftp or web servers, may have a /var system of 50 or 100 GB. How do you choose the correct size for your system? o When doing backups, it's a good idea to be able to get a file system on a single tape. In the early days of UNIX, system crashes were relatively common, and the damage they did to the file systems was relatively serious. Times have changed, and nowadays file system damage is relatively seldom, particularly on file systems that have little activity. On the other hand, disk drives have grown beyond most peoples' wildest expectations. The first edition of this book, only five years ago, showed how to install on a 200 MB drive. The smallest disk drives in current production are 10 GB in size, more than will fit on many tapes. As a result of these considerations, I have changed my recommendations. In earlier editions of this book, I recommended to put a small root file system and a /usr file system on the first (or only) disk on the system. /var was to be a symbolic link to /usr/var. This is still a valid layout, but it has a couple of problems: o In the example we're looking at, /usr is about 19 GB in size. Not many people have backup devices which can write this much data on a single medium. o Many people had difficulty with the symbolic link to /usr/var. As a result, I now recommend: o Make a single root file system of between 2 and 4 GB. o Do not have separate /usr or /var file systems. o Use the rest of the space on disk for a /home file system, as long as it's possible to back it up on a single tape. Otherwise make multiple file systems. /home is the normal directory for user files. This layout allows for easy backup of the file systems, and it also allows for Page 92 install.mm,v v3.5 (2000/08/30 00:08:01) (modified 25 Mar, 2001 UTC) Chapter 6: Installing FreeBSD easy upgrading to a new system version: you just need to replace the root file system. It's not perfect, though: on a web server you probably wouldn't want to put /var on the root file system. How much swap space? ____________________ Apart from files, you should also have at least one swap partition on your disk. It's very difficult to predict how much swap space you need. The automatic option gave us 522 MB, slightly more than twice the size of physical memory. Maybe you can get by with 64 MB. Maybe you'll need 2 GB. How do you decide? It's almost impossible to know in advance what your system will require. Here are some considerations: o Swap space is needed for all pages of virtual memory which contain data that is not locked in memory and which can't be recreated automatically. This is the majority of virtual memory in the system. o Some people use rules of thumb like ``2.5 times the size of physical memory, or 64 MB, whichever is bigger''. These rules work only by making assumptions about your workload. If you're using more than 2.5 times as much swap space as physical memory, performance will suffer. o Known memory hogs are X11 and integrated graphical programs such as NetScape and StarOffice. If you use these, you will probably need more swap space. Older UNIX-based hogs such as Emacs and the GNU C compiler (gcc) are not in the same league. o You can add additional swap partitions on other disks. This has the additional advantage of balancing the disk load if your machine swaps a lot. o About the only ways to change the size of a swap partition are to add another partition or to reinstall the system, so if you're not sure, a little bit more won't do any harm, but too little can really be a problem. o If your system panics, and memory dumping is enabled, it will write the contents of memory to the swap partition. This will obviously not work if your swap partition is smaller than main memory. Under these circumstances, the system refuses to dump, but it's not impossible that a bug might cause the dump to write beyond the bounds of the swap partition, probably causing irreperable damage to your /usr partition. install.mm,v v3.5 (2000/08/30 00:08:01) (modified 25 Mar, 2001 UTC) Page 93 Creating space on disk o Even with light memory loads, the virtual memory system slowly pages out data in preparation for a possible sudden demand for memory. This means that it can be more responsive to such requests. As a result, you should have at least as much swap as memory. A couple of examples might make this clearer: 1. I used to run X, StarOffice, Netscape and a whole lot of other memory-hungry applications on an old 486 with 16 MB. Sure, it's really slow, especially when changing from one application to another, but it works. Since there's not much memory, it uses a lot of swap. To view the current swap usage, use pstat. Here's a typical view of this machine's swap space: $ pstat -s Device 1024-blocks Used Avail Capacity Type /dev/da0s1b 122880 65148 57668 53% Interleaved 2. I now run much more stuff on an AMD Athlon with 256 MB of memory. I've got lots of swap space, but what I see is: $ pstat -s Device 1024-blocks Used Avail Capacity Type /dev/ad0s1b 524020 22328 501692 4% Interleaved It's not so important that the Athlon is using less swap: it's using 9% of its memory in swap, whereas the 486 is using 4 times its memory. In a previous edition of this book, I had the example of a Pentium with 96 MB of memory, which used 43 MB of swap. Look at it from a different point, and it makes more sense: swap makes up for the lack of real memory, so the 486 is using a total of 80 MB of memory, the Pentium is using 140 MB, and the Athlon is using 280 MB. In other words, there is a tendency to be able to say ``the more main memory you have, the less swap you need''. If, however, you look at it from the point of view of acceptable performance, you will hear things like ``you need at least one-third of your virtual memory in real memory''. That makes sense from a performance point of view, assuming all processes are relatively active. And, of course, it's another way of saying ``take twice as much swap as real memory''. In summary: be generous in allocating swap space. In this example, we have a Page 94 install.mm,v v3.5 (2000/08/30 00:08:01) (modified 25 Mar, 2001 UTC) Chapter 6: Installing FreeBSD tiny disk, so we're forced to use a small swap partition. If you have the choice, use more. If you really can't make up your mind, take 512 MB of swap space. For the file systems, the column Mount now shows the mount points, and Newfs contains the letters UFS for UNIX File System, and the letter Y, indicating that we need to create a new file system before we can use it. At this point, we have two choices: decide for ourselves what we want, or let the disk label editor do it for us. Let's look at both ways: --nFreZHaLTZJo0R7j-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message