Date: Thu, 18 Jul 2002 10:57:05 -0500 From: "Douglas K. Rand" <rand@meridian-enviro.com> To: freebsd-fs@freebsd.org Subject: Unexpected FS rot. Message-ID: <87y9c9hvvy.wl@delta.meridian-enviro.com>
next in thread | raw e-mail | index | archive | help
The other day we had a mildly embarrassing event when we closed up a case on one of our dual Athlon systems with out turning the case fans on first. (There is this helpful switch to allow you to "hot swap" the fans.) So the Athlon's essentially turned themselves off after getting too hot, which froze the system. After figuring out what we did wrong (lots of Duh! going around that afternoon) we brought the system back up. As is our habit, we came up into single-user mode and did a fsck by hand, and had only one file system (/project/Vault) with any serious problems. The filesystem had 167 INCORRECT BLOCK COUNT errors in phase 1, 590 UNREF FILE errors and 13 LINK COUNT DIR errors in phase 2, and 34 ALLOCATED FRAG [number] MARKED FREE errors in phase 5, along with the usual FREE BLK COUNT(S) WRONG IN SUPERBLK, SUMMARY INFORMATION BAD, and BLK(S) MISSING IN BIT MAPS. Certainly one of the dirtier filesystems that I've had to repair. And since I've had problems before with a single fsck not catching everything, I ran fsck again on all the filesytems, and this file system checked clean the second time. [On an (I think) unrelated note, the /usr file system didn't check clean the second time, it complained about FREE BLK COUNT(S) WRONG IN SUPERBLK, SUMMARY INFORMATION BAD, and ALLOCATED INODE 2103 MARKED FREE - BLK(S) MISSING IN BIT MAPS. A third fsck reported clean.] So we brought the system up and started using it. A day later we noticed a few of these problems: $ ls -l /project/Vault/tmp/raw/blobs/nids/27.3/7d27120e 0104.1d8bf700: Bad file descriptor 0702.2495bce8: Bad file descriptor 0c39.38a05298: Bad file descriptor total 154 -rw-r--r-- 1 weather wx 12538 Jul 18 09:19 122a.2dc7e110 -rw-r--r-- 1 weather wx 12480 Jul 18 09:25 1901.253e6c80 -rw-r--r-- 1 weather wx 12478 Jul 18 09:31 1e39.2198ad70 -rw-r--r-- 1 weather wx 12312 Jul 18 09:36 240e.16a25f60 -rw-r--r-- 1 weather wx 12468 Jul 18 09:42 2a0f.02dbe348 -rw-r--r-- 1 weather wx 12636 Jul 18 09:50 3016.2d588d10 -rw-r--r-- 1 weather wx 12622 Jul 18 09:54 3604.19f22e70 -rw-r--r-- 1 weather wx 12456 Jul 18 09:59 3b21.22af6500 Which didn't seem good at all. We made /project/Vault quiet, umounted it, and fsck'ed it, and lo and behold there were problems: It reported UNEXPECTED SOFT UPDATE INCONSISTENCY, and asked us to remove those files that reported a bad file descriptor. Everything has been working just fine since then, but what has me somewhat befuddled is that we had a clean fsck, and then the file system just started to rot. This is a 4.5-RELEASE-p8 system. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?87y9c9hvvy.wl>